I am new to Entity Framework so please bear with me.
I have a program that I want to select multiple records from a table and store it in a queue:
private Queue<RecordsToProcess> getRecordsToProcess()
{
Queue<RecordsToProcess> results = new Queue<RecordsToProcess>();
using (MyEntity context = new MyEntity())
{
var query = from v in context.RecordsToProcess
where v.Processed == false
select v;
foreach (RecordsToProcess record in query)
{
results.Enqueue(record);
}
}
}
Then I spin up multiple worker threads. Each worker thread takes one of the items in queue, processes it, and then saves it to the database.
private void processWorkerThread(object stateInfo)
{
while (workQueue.Count > 0)
{
RecordToProcess record = new RecordToProcess;
lock(workQueue)
{
if (workQueue.Count > 0)
RecordToProcess = workQueue.Dequeue();
else
break;
}
//Do the record processing here
//How do I save that record here???
}
}
My understanding is that to save changes back to the database you just call context.SaveChanges() but I can't do that in this situation can I?
Any help is appreciated.
Thanks!
Since you are disposing your MyEntity context in the first method (by wrapping it in a using statement), the entities that are enqueued will be in a "detached" state. That means, among other things, that changes done to the entity will not be tracked and you will not be able to lazy load navigation properties.
It is perfectly fine to dequeue these entities, "attaching" them to a different context, update them, and then call SaveChanges to persist the changes.
You can read about Attaching and Detaching Objects and Add/Attach and Entity States
It might be safer if you save off the primary key in the queue instead and retrieve the entities again. This way you are more likely avoid any data concurrency issues.
Related
Experiencing an issue about updating mysql DB through EF. It's not the first time I'm dealing with it, so I had some ideas about why isn't my data getting changed. I tried changing an element in goods array; tried editing an object, recieved through LINQ-request (seen some examples of this method); made some attempts on marking element found in the database before editing (like EntityState and Attach()). Nothing of these made any difference, so I tried removing <asp:UpdatePanel> from Site.Master to see what happens (responsive for postback blocking to prevent page shaking on update), but nothing changed (while btnRedeemEdit.IsPostBack having its default value).
Code below is the function I use for updates.
protected void btnRedeemEdit_Click(object sender, EventArgs e)
{
if (!string.IsNullOrEmpty(Request.QueryString["id"]))
{
var db = new GoodContext();
var goods = db.Goods.ToList();
Good theGood = goods.FirstOrDefault(x => x.Id == int.Parse(Request.QueryString["id"]));
//db.Goods.Attach(theGood);//No effect
//db.Entry(theGood).State = System.Data.Entity.EntityState.Modified; //No effect
if (theGood != default)
{
theGood.AmountSold = GetInput().AmountSold;
theGood.APF = GetInput().APF;
theGood.Barcode = GetInput().Barcode;
theGood.Description = GetInput().Description;
theGood.ImagesUrl = GetInput().ImagesUrl;//"https://i.pinimg.com/564x/2d/b7/d8/2db7d8c53b818ce838ad8bf6a4768c71.jpg";
theGood.Name = GetInput().Name;
theGood.OrderPrice = GetInput().OrderPrice;
theGood.Profit = GetInput().Profit;
theGood.RecievedOn = GetInput().RecievedOn;//DateTime.Parse(GetInput().RecievedOn).Date.ToString();
theGood.TotalAmount = GetInput().TotalAmount;
theGood.WeightKg = GetInput().WeightKg;
//SetGoodValues(goods[editIndex],GetInput());//Non-working
db.SaveChanges();
Response.Redirect("/AdminGoods");
}
else Response.Write($"<script>alert('Good on ID does not exist');</script>");
}
else Response.Write($"<script>alert('Unable to change: element selected does not exist');</script>");
}
Notice, that no alerts appear during execution, so object in database can be found.
Are there any more things, that can be responsible for blocking database updates?
A few things to update & check:
Firstly, DbContexts should always be disposed, so in your case wrap the DbContext inside a using statement:
using (var db = new GoodContext())
{
// ...
}
Next, there is no need to load all goods from the DbContext, just use Linq to retrieve the one you want to update:
using (var db = new GoodContext())
{
Good theGood = db.Goods.SingleOrDefault(x => x.Id == int.Parse(Request.QueryString["id"]));
if (theGood is null)
{
Response.Write($"<script>alert('Good on ID does not exist');</script>");
return;
}
}
The plausible suspect is what does "GetInput()" actually do, and have you confirmed that it actually has the changes you want? If GetInput is a method that returns an object containing your changes then it only needs to be called once rather than each time you set a property:
(Inside the using() {} scope...)
var input = GetInput();
theGood.AmountSold = input.AmountSold;
theGood.APF = input.APF;
theGood.Barcode = input.Barcode;
theGood.Description = input.Description;
// ...
db.SaveChanges();
If input has updated values but after calling SaveChanges you aren't seeing updated values in the database then there are two things to check.
1) Check that the database connection string at runtime matches the database that you are checking against. The easiest way to do that is to get the connection string from the DbContext instance's Database.
EF 6:
using (var db = new GoodContext())
{
var connectionString = db.Database.Connection.ConnectionString; // Breakpoint here and inspect.
EF Core: (5/6)
using (var db = new GoodContext())
{
var connectionString = db.Database.GetConnectionString();
Often at runtime the DbContext will be initialized with a connection string from a web.config / .exe.config file that you don't expect so you're checking one database expecting changes while the application is using a different database / server. (More common than you'd expect:)
2) Check that you aren't disabling tracking proxies. By default EF will enable change tracking which is how it knows if/when data has changed for SaveChanges to generate SQL statements. Sometimes developers will encounter performance issues and start looking for ways to speed up EF including disabling change tracking on the DbContext. (A fine option for read-only systems, but a pain for read-write)
EF6 & EF Core: (DbContext initialization)
Configuration.AutoDetectChangesEnabled = false; // If you have this set to false consider removing it.
If you must disable change tracking then you have to explicitly set the EntityState of the entity to Modified before calling SaveChanges():
db.Entry(theGood).State = EntityState.Modified;
db.SaveChanges();
Using change tracking is preferable to using EntityState because with change tracking EF will only generate an UPDATE statement if any values have changed, and only for the values that changed. With EntityState.Modified EF will always generate an UPDATE statement for all non-key fields regardless if any of them had actually changed or not.
I have a application that uses EF6 ORM.
I have a method that creates a new object. I also have a thread that is responsible of the app logging. It Enqueues LogObject and saves it to the database every 5 seconds..
So. A user calls the method to create the new object:
using(DataContext context = new DataContext() {
context.MyObjects.Add(new MyObject() { prop1 = "test" });
context.SaveChanges(); // save #1
Log("A new MyObject was created!"); // this method enqueues log info
}
The logger database saving method is as follow:
private void Flush() {
using (DataContext db = new DataContext() {
foreach(Log l in _logs) {
db.Logs.Add(new LogObject() { propX = "blabla" });
}
db.SaveChanges(); // save #2
}
}
The save #1 inserts the object properly.
The problem is that the save #2 also inserts an identical MyObject()....
I profiled the database and debug the processes.
Whenever you see a behavior like this, consider that the object is not being tracked by the context. You are creating a context and disposing it so the object is probably not tracked and for EF it's considered a new object, that's why it is inserted again.
You should attach the object again so EF tracks it and knows it's not a new object.
Put them in the same context maybe? In my understanding you would only log what is going in to the db.... that would not require a "timer function". Or choose another ORM where you have more control...
I need to insert some objects (about 4 million) in the database using C# and EF (using .NET 3.5). My method that adds the objects is in a for:
private DBModelContainer AddToContext(DBModelContainer db, tblMyTable item, int count)
{
db.AddTottblMyTable (item);
if ((count % 10000== 0) || (count == this.toGenerate))
{
try
{
db.SaveChanges();
}
catch (Exception e)
{
Console.WriteLine(e.StackTrace);
}
}
return db;
}
How to detach the added objects (of type tblMyTable) from the context object? I don't need them for a later use and when more than 300000 objects are added, the execution time between db saving ( db.SaveChanges()) increases considerably.
Regards
Entity Framework may not be the best tool for this type of operation. You may be better off with plain ADO.Net, some stored procedures... But if you had to use it, here are a number of suggestions:
Keep the active Context Graph small by using a new context for each
Unit of Work
Turn off AutoDetechChangesEnabled - context.Configuration.AutoDetectChangesEnabled = false;
Batching, in your loop, Call SaveChanges periodically
EDIT
using(var db = new DBModelContainer())
{
db.tblMyTable.MergeOption = MergeOption.NoTracking;
// Narrow the scope of your db context
db.AddTottblMyTable (item);
db.SaveChanges();
}
Keeping a long running db context is not advisable, so consider refactoring your Add method to not keep attempting to reuse the same context.
See Rick Strahl's post on bulk inserts for more details
AFAK EF does not support directly the BulkInsert so it will be tedious to do such thing manually.
try to consider EntityFramework.BulkInsert
using (var ctx = GetContext())
{
using (var transactionScope = new TransactionScope())
{
// some stuff in dbcontext
ctx.BulkInsert(entities);
ctx.SaveChanges();
transactionScope.Complete();
}
}
You may try Unit Of Work and dont save context (SaveChanges) on every record insert but save it at end
I have related tables.
For example -
Worker (id, mame, idWorkerType, ...) and WorkerType (id, code (unique key), ...)
Then I have same classes, that was mapped.
Then, i have list of generated objects like this
Worker1
Id1
Name1
WorkerType1
IdType1
CodeType1
Worker2
Id2
Name2
WorkerType1
IdType1
CodeType1
...
The logic is as follows:
foreach (var Worker in Workers)
{
var WorkerTypeFromDB = GetWorkerTypeByField("Code", Worker.WorkerType.Code).FirstOrDefault();
if (WorkerTypeFromDB == null)
{
session.Insert(Worker.WorkerType);
}
else
{
session.Update(Worker.WorkerType);
}
var WorkerFromDB = GetWorkerByField("Code", Worker.Code).FirstOrDefault();
if (WorkerFromDB == null)
{
session.Insert(Worker);
}
else
{
session.Update(Worker);
}
}
So, in first iteration I inserting WorkerType1 and Worker1.
In second iteration I updating WorkerType1 (because same unique Code) and trying insert Worker2... but I can't!
I get exception - object references an unsaved transient instance - save the transient instance before flushing or set cascade action for the property to something that would make it autosave.
I know, that, it happened because i don't insert WorkerType, but I dont need that. I already has WorkerType1 in db.
Сan somebody explain me how resolve this problem?
It is possible, but we have to fix the (in)appropriate reference handling. Once we know, that object exists in DB (has the same "CODE") and we have it loaded in a session, we should use that object reference.
So the change should be like:
var WorkerTypeFromDB = GetWorkerTypeByField("Code", Worker.WorkerType.Code).FirstOrDefault();
if (WorkerTypeFromDB == null)
{
session.Save(Worker.WorkerType); // Save/Insert not existing
}
else
{
Worker.WorkerType = WorkerTypeFromDB // assign the existing to our object
}
So now, once we have the existing object, we are assigning it as a property WorkerType
To handle the Worker, we can do even more: Merge() in case that Worker already exists.
var WorkerFromDB = GetWorkerByField("Code", Worker.Code).FirstOrDefault();
if (WorkerFromDB == null)
{
session.Save(Worker); // insert not existing
}
else
{
// here, we should assign the existing Worker ID to passed one
Worker.ID = WorkerFromDB.ID; // current worker has same identifier
session.Merge(Worker); // both will be merged, latest changes will be applied
}
And at the end, when session.Flush() is called, all changes will be persisted. (Merge could be used even for WorkerType... as required)*
Read more about Merge() here 9.4.2. Updating detached objects, An extract:
... using Merge(Object o). This method copies the state of the given object onto the persistent object with the same identifier. If there is no persistent instance currently associated with the session, it will be loaded. The method returns the persistent instance. If the given instance is unsaved or does not exist in the database, NHibernate will save it and return it as a newly persistent instance. Otherwise, the given instance does not become associated with the session. In most applications with detached objects, you need both methods, SaveOrUpdate() and Merge().
static Object LockEx=new Object();
public void SaveMyData(IEnumerable<MyData> list)
{
lock (LockEx)
{
using (PersistencyContext db = new PersistencyContext())
{
foreach (var el in list)
{
try
{
db.MyData.Add(el);
db.SaveChanges();
}
catch (DbUpdateException)
{
db.Entry(el).State = EntityState.Modified;
db.SaveChanges();
}
}
}
}
}
This methods is called from multiple threads. Right now I use a static lock to avoid 2 threads to save data at the same time. Though this is wrong because I only want to save data. The catch is used to create an update query in case the insert (Add) fails because the entry already exists.
What happens if I remove the lock. How will the SaveChanges work? How should my code look like? Thanks
I would remove the lock because the database already handles concurrency anyway by design, then I will also verify if the record exists before trying to add it, then I would do the add or update depending on this result. Just to avoid exceptions because they are performance killers.
Building on Davide's answer, you could also call SaveChanges once after you added all the new entities. That should be faster.