I have 2 problems.
First one is that Connection pool and (OracleConnectionImpl) is growing steadily over time, till it reaches out of memory exception. As you can see on the image, it has 90 MB over 3-4 hours of running.
I am using short lived contexts everywhere, but it keeps on growing and never clears itself. Is there any way for me to clear it?
Second one is that EF stores too much duplicates of string queries over time.
It mostly stores those which come from .Reload() function, because it is not paramterized, it hardcode the ID into the query.
And then there are strings like "ID" which is somewhere cached 2947x.
Is there any way for the .Reload() function to make it parametrized, or to clear all of those stored strings?
This app is refreshing warehouse jobs and palletes every few seconds, to keep it in sync for all machines and I am not aware of better way than .Reload() because of WPF bindings.
Edit 1
I have simple helper function for reloading many intities at once, even one extension. It doesnt matter that it is passed as object, because the same problem remains even with the last example.
public static void ReloadEntities(bool dispatch, params IEnumerable<object>[] entities)
{
using (var ctx = new eWMSEntities())
{
if (dispatch)
{
Application.Current.Dispatcher.Invoke(DispatcherPriority.Background, (SendOrPostCallback)delegate
{
entities.SelectMany(x => x.Select(s => s)).ToList().ForEach(entity =>
{
ctx.Set(entity.GetType()).Attach(entity);
ctx.Entry(entity).Reload();
ctx.Entry(entity).State = EntityState.Detached;
});
}, null);
}
else
{
entities.SelectMany(x => x.Select(s => s)).ToList().ForEach(entity =>
{
ctx.Set(entity.GetType()).Attach(entity);
ctx.Entry(entity).Reload();
ctx.Entry(entity).State = EntityState.Detached;
});
}
ctx.Dispose();
}
}
public static void ReloadEntity(this object entity, bool dispatch)
{
using (var ctx = new eWMSEntities())
{
ctx.Set(entity.GetType()).Attach(entity);
if (dispatch)
{
Application.Current.Dispatcher.Invoke(DispatcherPriority.Background, (SendOrPostCallback)delegate
{
ctx.Entry(entity).Reload();
}, null);
}
else
{
ctx.Entry(entity).Reload();
}
ctx.Entry(entity).State = EntityState.Detached;
ctx.Dispose();
}
}
while (true && JobLines.Contains(line))
{
using (var ctx = new eWMSEntities())
{
ctx.T_JOB_LINES.attach(line);
ctx.entry(line).Reload();
}
await Task.Delay(3000);
}
This is snapshot after 3 days of running
I'm not recommend but if there are no any other solution...
Maybe you can try
GC.Collect();
Let system force to collect something didn't important in your memory.
Use the "Unit of work" pattern - it will solve many problems. I did not find for WPF, but found for ASP.NET MVC
Related
I've been working on a hobby project being developed in C# + Xamarin Forms + Prism + EF Core + Sqlite, debugging in UWP app.
I've written the following code to store tick data received from broker to Sqlite.
First, the OnTick call back that receives the ticks (approx. 1 tick per sec per instrument):
private void OnTick(Tick tickData)
{
foreach (var instrument in IntradayInstruments.Where(i => i.InstrumentToken == tickData.InstrumentToken))
{
instrument.UpdateIntradayCandle(tickData);
}
}
And the UpdateIntradayCandle method is:
public void UpdateIntradayCandle(Tick tick)
{
if (LastIntradayCandle != null)
{
if (LastIntradayCandle.Open == 0m)
{
LastIntradayCandle.Open = tick.LastPrice;
}
if (LastIntradayCandle.High < tick.LastPrice)
{
LastIntradayCandle.High = tick.LastPrice;
}
if (LastIntradayCandle.Low == 0m)
{
LastIntradayCandle.Low = tick.LastPrice;
}
else if (LastIntradayCandle.Low > tick.LastPrice)
{
LastIntradayCandle.Low = tick.LastPrice;
}
LastIntradayCandle.Close = tick.LastPrice;
}
}
The LastIntradayCandle is a property:
object _sync = new object();
private volatile IntradayCandle _lastIntradayCandle;
public IntradayCandle LastIntradayCandle
{
get
{
lock (_sync)
{
return _lastIntradayCandle;
}
}
set
{
lock (_sync)
{
_lastIntradayCandle = value;
}
}
}
Now, the LastIntradayCandle is changed periodically, say, 5 minutes, and a new candle is put in place for updating, from a different thread coming from a System.Threading.Timer which is scheduled to run every 5m.
public void AddNewIntradayCandle()
{
if (LastIntradayCandle != null)
{
LastIntradayCandle.IsClosed = true;
}
var newIntradayCandle = new IntradayCandle { Open = 0m, High = 0m, Low = 0m, Close = 0m };
LastIntradayCandle = newIntradayCandle;
IntradayCandles.Add(newIntradayCandle);
}
Now, the problem is, I'm getting 0s in those Open, High or Low but not in Close, Open having the most number of zeroes. This is happening very randomly.
I'm thinking that if any of the Open, High, Low or Close values is getting updated, it means the tick is having a value to be grabbed, but somehow one or more assignments in UpdateIntradayCandle method are not running. Having zeroes is a strict NO for the purpose of the app.
I'm neither formally trained as a programmer nor an expert, but a self-learning hobbyist and definitely never attempted at multi-threading before.
So, I request you to please point me what I am doing wrong, or better still, what should I be doing to make it work.
Multithreading and EF Core is not compatible things. EF Core context is not a thread safe. You have to create new context for each thread. Also making your object thread safe is wasting time.
So, schematically you have to do the following and you can remove locks from your object.
private void OnTick(Tick tickData)
{
using var ctx = new MyDbContext(...);
foreach (var instrument in ctx.IntradayInstruments.Where(i => i.InstrumentToken == tickData.InstrumentToken))
{
instrument.UpdateIntradayCandle(tickData);
}
ctx.SaveChanges();
}
I am attempting to implement retries on my database connections, which I've gotten working, but it's broken the user-initiated transactions I have as well.
Specifically, I see errors like the following:
The configured execution strategy 'RetryExecutionStrategy' does not support user initiated transactions. See http://go.microsoft.com/fwlink/?LinkId=309381 for additional information.
I would very much like to implement the first workaround from Microsoft, but even the second one would be enough. Despite many attempts from different angles, I've been unable to get things working for transactions. I've tried searching existing questions on StackOverflow (and the rest of the internet, really), but to no avail. Perhaps the solutions I've found elsewhere work for single threads with only one connection at a time, but I'm working on a large project where there are both transactions and non-transactions occurring in no predictable order.
Ultimately, what I need is a way to turn off retries for transactions or a fix to make transactions not crash when retrying is turned on.
What I have for my DbContext:
[DbConfigurationType("MyNamespace.RetryConfiguration","MyNamespace")]
public partial class RepoContext : DbContext {
public RepoContext(string entityConnectionString) : base(entityConnectionString)
{
}
}
My RetryExecutionStrategy:
public class RetryExecutionStrategy : DbExecutionStrategy
{
public RetryExecutionStrategy(int maxRetries, TimeSpan maxDelay)
: base(maxRetries, maxDelay)
{
}
protected override bool ShouldRetryOn(Exception e)
{
return true;
}
}
My RetryConfiguration:
public class RetryConfiguration : DbConfiguration
{
public RetryConfiguration()
{
var executionStrategy = SuspendExecutionStrategy
? (IDbExecutionStrategy)new DefaultExecutionStrategy()
: new RetryExecutionStrategy(3, new TimeSpan(0, 0, 0, 3));
this.SetExecutionStrategy("Devart.Data.PostgreSql", () => executionStrategy);
}
public static bool SuspendExecutionStrategy
{
get
{
return (bool?)CallContext.LogicalGetData("SuspendExecutionStrategy") ?? false;
}
set
{
CallContext.LogicalSetData("SuspendExecutionStrategy", value);
}
}
}
Unless I am missing something, I believe I implemented Microsoft's example correctly. From my understanding of their example, EF should make a new instance of RetryConfiguration for every repo call. Otherwise, their example doesn't make any sense as it would only ever create a connection with RetryExecutionStrategy rather than DefaultExecutionStrategy, which is needed for user-initiated transactions. Contrary to that, I've stuck a breakpoint on the constructor of RetryConfiguration and discovered it's only ever instantiated once.
This has led to a lot of headaches and attempts at fulfilling their second workaround like the following:
var res = new RetryExecutionStrategy(0, new TimeSpan(0,0,0));
RetryConfiguration.SuspendExecutionStrategy = true;
res.Execute(() =>
{
using (var trans = _repoFactory.GetTransactableRepository())
{
trans.BeginTransaction();
var entry = trans.GetAll<MyTable>().First();
entry.Alive = true;
trans.Save();
trans.Commit();
}
});
RetryConfiguration.SuspendExecutionStrategy = false;
I even tried manually calling DefaultExecutionStrategy.Execute().
var des = new System.Data.Entity.Infrastructure.DefaultExecutionStrategy();
des.Execute(() =>
{
using (var trans = _repoFactory.GetTransactableRepository())
{
trans.BeginTransaction();
var entry = trans.GetAll<MyTable>().First();
entry.Alive = true;
trans.Save();
trans.Commit();
}
});
Even in that situation I get an exception saying that RetryConfiguration does not allow user-initiated transactions.
For what it's worth, I tried adding RetryConfiguration.SuspendExecutionStrategy = true/false to our TransactableRepository class's BeginTransaction(), Commit(), and Rollback() functions. The class itself is just a wrapper for the connection. I was unsurprised when it didn't work, given that Microsoft's example only ever shows it being read from on RetryConfiguration's constructor, but I figured it was worth a try since I'm not terribly familiar with CallContext.
In my WebApi controller I have the following (pseudo) code that receives update notifications from Instagrams real-time API:
[HttpPost]
public void Post(InstagramUpdate instagramUpdate)
{
var subscriptionId = instagramUpdate.SubscriptionId;
var lastUpdate = GetLastUpdate(subscriptionId);
// To avoid breaking my Instagram request limit, do not fetch new images too often.
if (lastUpdate.AddSeconds(5) < DateTime.UtcNow)
{
// More than 5 seconds ago since last update for this subscription. Get new images
GetNewImagesFromInstagram(subscriptionId);
UpdateLastUpdate(subscriptionId, DateTime.UtcNow);
}
}
This won't work very well if I receive two update notifications for the same subscription almost simultaneously, since lastUpdate won't have been updated until after the first request has been processed.
What would be the best way to tackle this problem? I'm thinking of using some kind of cache, but I'm not sure how. Is there some kind of best practices for these kind of things? I'm guessing it's a common problem: "receive notification, do something if something hasn't been done recently..."
Thanks to this answer I went with the following approach, using MemoryCache
[HttpPost]
public void Post(IEnumerable<InstagramUpdate> instagramUpdates)
{
foreach (var instagramUpdate in instagramUpdates)
{
if (WaitingToProcessSubscriptionUpdate(instagramUpdate.Subscription_id))
{
// Ongoing request, do nothing
}
else
{
// Process update
}
}
}
private bool WaitingToProcessSubscriptionUpdate(string subscriptionId)
{
// Check in the in memory cache if this subscription is in queue to be processed. Add it otherwise
var queuedRequest = _cache.AddOrGetExisting(subscriptionId, string.Empty, new CacheItemPolicy
{
// Automatically expire this item after 1 minute (if update failed for example)
AbsoluteExpiration = DateTime.Now.AddMinutes(1)
});
return queuedRequest != null;
}
I am afraid that it is awful idea, but ... Maybe it worth to add lock to this method ? Like
private List<int> subscriptions = new List<int>();
and then
int subscriptinId = 1;//add calculation here
int subscriptionIdIndex = subscriptions.IndexOf(subscriptinId);
lock (subscriptions[subscriptionIdIndex])
{
//your method code
}
Feel free to criticize this approach )
I gotta run a maybe one-time or potentially once every few months update on this table. Basically to allow other programs that reference to run faster. I really thought this update looked right but it did not change the data. Is what I am doing wrong?
public static void UpdateMFGtoID()
{
DataDataContext _db = new DataDataContext();
foreach (VINPatternDecode vin in _db.VINPatternDecodes)
{
vin.DivisionName = GetMfgID(vin.DivisionName.Replace("~",""));
_db.SubmitChanges();
}
}
It should work, but its better to Dispose your resources, and maybe, depending on the amount of data (if its small) just use one SubmitChanges():
using (DataDataContext _db = new DataDataContext())
{
foreach (VINPatternDecode vin in _db.VINPatternDecodes)
{
vin.DivisionName = GetMfgID(vin.DivisionName.Replace("~",""));
}
_db.SubmitChanges();
}
The using calls the Dispose function implicit.
This is driving me nuts. I'm not sure what else to try. This is my latest attempt to update a list of objects within another object using EF 4.3.
The scenario is that a user has added a new Task to an Application that already has one task in its Tasks property. The Application is not attached to the DB context because it was retrieved in a prior logic/DB call. This is the class and property:
public class Application : EntityBase
{
public ObservableCollection<TaskBase> Tasks { // typical get/set code here }
}
This is my attempt to update the list. What happens is that the new Task gets added and the association correctly exists in the DB. However, the first task, that wasn't altered, has its association removed in the DB (its reference to the Application).
This is the Save() method that takes the Application that the user modified:
public void Save(Application newApp)
{
Application appFromContext;
appFromContext = this.Database.Applications
.Include(x => x.Tasks)
.Single(x => x.IdForEf == newApp.IdForEf);
AddTasksToApp(newApp, appFromContext);
this.Database.SaveChanges();
}
And this is the hooey that's apparently necessary to save using EF:
private void AddTasksToApp(Application appNotAssociatedWithContext, Application appFromContext)
{
List<TaskBase> originalTasks = appFromContext.Tasks.ToList();
appFromContext.Tasks.Clear();
foreach (TaskBase taskModified in appNotAssociatedWithContext.Tasks)
{
if (taskModified.IdForEf == 0)
{
appFromContext.Tasks.Add(taskModified);
}
else
{
TaskBase taskBase = originalTasks.Single(x => x.IdForEf == taskModified.IdForEf); // Get original task
this.Database.Entry(taskBase).CurrentValues.SetValues(taskModified); // Update with new
}
}
}
Can anyone see why the first task would be losing its association to the Application in the DB? That first task goes through the else block in the above code.
Next, I'll need to figure out how to delete one or more items, but first things first...
After continual trial and error, this appears to be working, including deleting Tasks. I thought I'd post this in case it helps someone else. I'm also hoping that someone tells me that I'm making this more complicated than it should be. This is tedious and error-prone code to write when saving every object that has a list property.
private void AddTasksToApp(Application appNotAssociatedWithContext, Application appFromContext)
{
foreach (TaskBase taskModified in appNotAssociatedWithContext.Tasks)
{
if (taskModified.IdForEf == 0)
{
appFromContext.Tasks.Add(taskModified);
}
else
{
TaskBase taskBase = appFromContext.Tasks.Single(x => x.IdForEf == taskModified.IdForEf); // Get original task
this.Database.Entry(taskBase).CurrentValues.SetValues(taskModified); // Update with new
}
}
// Delete tasks that no longer exist within the app.
List<TaskBase> tasksToDelete = new List<TaskBase>();
foreach (TaskBase originalTask in appFromContext.Tasks)
{
TaskBase task = appNotAssociatedWithContext.Tasks.Where(x => x.IdForEf == originalTask.IdForEf).FirstOrDefault();
if (task == null)
{
tasksToDelete.Add(originalTask);
}
}
foreach (TaskBase taskToDelete in tasksToDelete)
{
appFromContext.Tasks.Remove(taskToDelete);
this.Database.TaskBases.Remove(taskToDelete);
}
}