I am trying to understand why my application has the following issues
after running for an hour or so, memory usage is over 2GB
after running for an hour or so event processing slows from ~100m/s to ~6-10m/s
I am using Dotnet 5 with Autofac. Here is the method that is fired by the Consumer_Received event
private async Task ProcessEvent(string eventName, string message)
{
if (_subsManager.HasSubscriptionsForEvent(eventName))
{
await using var scope = _autofac.BeginLifetimeScope(AUTOFAC_SCOPE_NAME);
var subscriptions = _subsManager.GetHandlersForEvent(eventName);
foreach (var subscription in subscriptions)
{
var handler = scope.ResolveOptional(subscription.HandlerType);
if (handler == null) continue;
var eventType = _subsManager.GetEventTypeByName(eventName);
var integrationEvent = JsonConvert.DeserializeObject(message, eventType);
var concreteType = typeof(IIntegrationEventHandler<>).MakeGenericType(eventType);
await Task.Yield();
await ((Task) concreteType.GetMethod("Handle")?
.Invoke(handler, new[] {integrationEvent}))
.ConfigureAwait(false);
}
}
}
This is the event
public class MPTEvent
{
public string M {get;set;}
public string Number {get;set;}
public string Type {get;set;}
public bool T {get;set;}
public bool A {get;set;}
public bool LB {get;set;}
}
The above method is fired for EVERY event stored inside the Queue. i have several "INotification" handlers that process
the events (IE. Handle), here is an example of one (Edited for brievety)
public async Task Handle(MPTEvent tEvent)
{
try
{
// try to get the tag and device
var contextD = await _repository.GetByNumber(tEvent.M);
if (contextD == null)
{
_logger.LogError("Detailed Error Message");
return;
}
var contextT = await _repository.GetByNumber(tEvent.Number);
if (contextT == null)
{
_logger.LogWarning("Detailed Warning Message");
return;
}
if (tEvent.T == 1 && tEvent.A == 0 && tEvent.LB == 0)
{
contextT?.TriggerTEvent(contextD, tEvent.R, tEvent.CreatedDate);
}
await _repository.UpdateAsync(contextT);
}
catch (Exception ex)
{
_logger.LogCritical(ex, "Detailed Exception Message");
}
}
The above handler is fired more than any others and i believe is partly where the biggest issue resides, the questions i have are
Should the handler (in this case) have a public async Task and return something like _repository.UpdateAsync(contextT)?
would it be smart to replace the _repository.UpdateAsync and _repository.AddAsync with mediator commands?
I am relatively new to Autofac/DI/rabbitMq so any advice is greatly appreciated
Related
I have a problem, and I am not quite sure how to solve this, except for making my Akka Actor not have async methods.
Here is my Actor Code:
public class AggregatorActor : ActorBase, IWithUnboundedStash
{
public IStash Stash { get; set; }
private AggregatorTimer _aggregatorTimer;
private IActorSystemSettings _settings;
private AccountSummary _accountResponse;
private ContactDetails _contactResponse;
private AnalyticDetails _analyticsResponse;
private FinancialDetails _financialResponse;
private ActorSelection _accountActor;
private ActorSelection _contactActor;
private ActorSelection _analyticsActor;
private ActorSelection _financialActor;
public AggregatorActor(IActorSystemSettings settings) : base(settings)
{
_accountActor = Context.System.ActorSelection(ActorPaths.AccountActorPath);
_contactActor = Context.System.ActorSelection(ActorPaths.ContactActorPath);
_analyticsActor = Context.System.ActorSelection(ActorPaths.AnalyticsActorPath);
_financialActor = Context.System.ActorSelection(ActorPaths.FinancialActorPath);
_settings = settings;
}
#region Public Methods
public override void Listening()
{
ReceiveAsync<ProfilerMessages.ProfilerBase>(async x => await HandleMessageAsync(x));
}
private void Busy()
{
Receive<ProfilerMessages.ProfilerBase>(x => Stash.Stash());
}
private void Aggregate()
{
try
{
Context.Sender.Tell(AggregatedSummaryResponse.Instance(_accountResponse, _contactResponse, _analyticsResponse, _financialResponse));
}
catch (Exception ex)
{
ExceptionHandler(ex);
}
}
public override async Task HandleMessageAsync(object msg)
{
//if is summary, generate new isntance of AggregatorTimer in _eventHandlerCollection.
if (msg is ProfilerMessages.GetSummary)
{
//Become busy. Stash
Become(Busy);
//Handle different requests
var clientId = (msg as ProfilerMessages.GetSummary).ClientId;
await HandleSummaryRequest(clientId);
}
}
private async Task HandleSummaryRequest(string clientId)
{
try
{
var accountMsg = new AccountMessages.GetAggregatedData(clientId);
_accountResponse = (await _accountActor.Ask<Messages.AccountMessages.AccountResponseAll>(accountMsg, TimeSpan.FromSeconds(_settings.NumberOfSecondsToWaitForResponse))).AccountDetails;
//Need to uncomment this
var contactMsg = new ContactMessages.GetAggregatedContactDetails(clientId);
_contactResponse = (await _contactActor.Ask<Messages.ContactMessages.ContactResponse>(contactMsg, TimeSpan.FromSeconds(_settings.NumberOfSecondsToWaitForResponse))).ContactDetails;
var analyticMsg = new AnalyticsMessages.GetAggregatedAnalytics(clientId);
_analyticsResponse = (await _analyticsActor.Ask<Messages.AnalyticsMessages.AnalyticsResponse>(analyticMsg, TimeSpan.FromSeconds(_settings.NumberOfSecondsToWaitForResponse))).AnalyticDetails;
var financialMsg = new FinancialMessages.GetAggregatedFinancialDetails(clientId);
_financialResponse = (await _financialActor.Ask<Messages.FinancialMessages.FinancialResponse>(financialMsg, TimeSpan.FromSeconds(_settings.NumberOfSecondsToWaitForResponse))).FinancialDetails;
//Start new timer
_aggregatorTimer = new AggregatorTimer(_settings.NumberOfSecondsToWaitForResponse);
_aggregatorTimer.TimeElapsed += _aggregatorTimer_TimeElapsed;
}
catch (Exception ex)
{
ExceptionHandler(ex);
}
}
//Event that is raised when an external timers time elapsed.
private async void _aggregatorTimer_TimeElapsed(object sender, ElapsedTimeHandlerArg e)
{
Aggregate();
_aggregatorTimer = null;
_accountResponse = null;
_contactResponse = null;
_analyticsResponse = null;
_financialResponse = null;
//Unstash
Stash.Unstash();
//Start listening again
Become(Listening);
}
#endregion
}
Inside the _aggregatorTimer_TimeElapsed event, I call the await Aggregate function, but the following exception is thrown.
There is no active ActorContext, this is most likely due to use of async operations from within this actor
I think this is caused by the fact that the Aggregate function tries to Tell() the Sender about the responses that are aggregated, but those Tasksare not yet completed? I might be completely wrong, but I have no idea why this is thrown.
I'm not sure what do you even need an AggregatorTimer for - in akka you have a Context.System.Scheduler object, which can be used to schedule events going to happen in the future.
Another thing is that you probably shouldn't execute logic inside event handlers. If you really need them in your code, it's better to limit them only to send a message, once an event gets triggered i.e.:
Receive<TimedOut>(_ => /* handle timeout message */);
var self = Self; // bind current self to a variable, so it won't change
_aggregatorTimer.TimeElapsed += (sender, e) => self.Tell(new TimedOut(), self);
I have an application which currently runs Tasks on a time interval, however I would like more control over that, to be able to stop a running task and restart it by clicking a UI.
There are 6 tasks at the moment, but I would want to keep things generic, to be able to easily more when required. I was hoping to be able to create a wrapper to control them, which I can pass a method into as a parameter.
As such I created an object, which I create as many of as there are tasks, I can get status updates from it as well as manage it
I want to:
- Start a method/Task
- Stop a method/Task
- Restart a method/Task
- Get feedback from it's log/updates/progress/errors that I record to updates List
Is this a good way to do this, is there a better way to achieve what I'm after?
public class ManagedTask
{
public ManagedTask()
{
CreateNewToken();
}
public int Id { get; set; }
public string DescriptiveName { get; set; }
public Action<CancellationToken> TheVoidToRun { private get; set; }
private CancellationTokenSource CTokenSource { get; set; }
private CancellationToken CToken { get; set; }
private Task TheRunningThing { get; set; }
public void StartIt()
{
if (TheRunningThing == null || TheTaskStatus() == TaskStatus.Canceled || TheTaskStatus() == TaskStatus.RanToCompletion)
{
CreateNewToken();
}
// Start up the Task
AddUpdate($"Starting Task at {DateTime.Now}");
TheRunningThing = Task.Run(() => TheVoidToRun?.Invoke(CToken), CToken);
AddUpdate($"Started Task at {DateTime.Now}");
}
public void EndIt()
{
AddUpdate($"Cancelling Task at {DateTime.Now}");
CTokenSource.Cancel();
// Do - If in progress try to stop (Cancellation Token)
// Do - Stop future repeats
}
private void CreateNewToken()
{
CTokenSource = new CancellationTokenSource();
CTokenSource.Token.ThrowIfCancellationRequested();
CToken = CTokenSource.Token;
}
public TaskStatus TheTaskStatus() => TheRunningThing.Status;
internal List<string> Updates { get; set; }
private void AddUpdate(string updates)
{
// Do stuff
}
}
So I have various methods which I'd like to pass into this such like:
public class AvailableTasks
{
public async void DoStuffThatIsCancelable(CancellationToken token)
{
DoTheLongStuffOnRepeat(token);
}
public async void DoAnotherThingThatIsCancelable(CancellationToken token)
{
DoTheLongStuffOnRepeat(token);
}
private async void DoTheLongStuffOnRepeat(CancellationToken token)
{
// Do stuff
for (int i = 0; i < 20; i++)
{
while (!token.IsCancellationRequested)
{
try
{
await Task.Delay(500, token);
}
catch (TaskCanceledException ex)
{
Console.WriteLine("Task was cancelled");
continue;
}
Console.WriteLine($"Task Loop at {(i + 1) * 500}");
}
}
}
}
Here is how I was thinking of calling it.
private static readonly List<ManagedTask> _managedTasks = new List<ManagedTask>();
public static void SetupManagedTasks()
{
var at = new AvailableTasks();
var mt1 = new ManagedTask
{
Id = 1,
DescriptiveName = "The cancelable task",
TheVoidToRun = at.DoStuffThatIsCancelable,
};
_managedTasks.Add(mt1);
var mt2 = new ManagedTask
{
Id = 2,
DescriptiveName = "Another cancelable task",
TheVoidToRun = at.DoAnotherThingThatIsCancelable,
};
_managedTasks.Add(mt2);
mt1.StartIt();
mt2.StartIt();
Console.WriteLine($"{mt1.DescriptiveName} status: {mt1.TheTaskStatus()}");
Console.WriteLine($"{mt2.DescriptiveName} status: {mt2.TheTaskStatus()}");
}
public static void CancelTask(int id)
{
var mt = _managedTasks.FirstOrDefault(t => t.Id == id);
if (mt != null)
{
mt.EndIt();
Console.WriteLine($"{mt.DescriptiveName} status: {mt.TheTaskStatus()}");
}
}
public static void GetTaskStatus(int id)
{
var mt = _managedTasks.FirstOrDefault(t => t.Id == id);
if (mt != null)
{
Console.WriteLine($"{mt.DescriptiveName} status: {mt.TheTaskStatus()}");
}
}
However even with all the above, I suffer from the Status only ever showing RanToCompletion.
How can I structure the above to achieve what I want?
Thanks,
David
I suffer from the Status only ever showing RanToCompletion.
This is because your methods are using async void. They should be async Task. As I describe in my async best practices article, you should avoid async void.
Other notes...
Start a method/Task
Restart a method/Task
You can start (or restart) a task on the thread pool by using Task.Run. However, if you have naturally asynchronous tasks, then you can represent them as Func<Task> and just invoke the Func<Task> to start them.
Stop a method/Task
The only appropriate way to do this is with a CancellationToken, which it looks like you're using correctly.
Get feedback from it's log/updates/progress/errors that I record to updates List
I recommend using IProgress<T> for any kind of progress updates.
I'm creating a generic loader and I want to kick off an HttpClient SendAsync request. However, some of these requests may take time, so I want to add the option to cancel, and notify upon completion.
This seems like a standard scenario imho.
I'm not sure if this is the correct way to go about this, but based on some examples I've looked at, here is where I'm at. If you look at the bottom of the code, my question is - at that point, do I check the response and raise a success or error event?
public bool StartFetch()
{
if (IsFetching) return false;
IsFetching = true;
mCancellationTokenSource = new CancellationTokenSource();
// this is not awaited, so execution should continue
StartTask(request, mCancellationTokenSource.Token);
return true;
}
public bool CancelFetch()
{
// send cancellation
if (mCancellationTokenSource != null)
mCancellationTokenSource.Cancel();
Cleanup();
return true;
}
private async Task StartTask(LFHttpRequest request, CancellationToken cancellationToken)
{
var message = new HttpRequestMessage(request.Method, request.Uri);
var response = await HttpClient.SendAsync(message, cancellationToken);
// at this point, do I take a look at response and raise a custom OnSuccess or OnError event???
// or do I want to grab the task from `SendAsync`, check for completed or faulted?
}
When you're looking at exposing task-related state like IsFetching, it's often cleaner and easier to just expose the Task itself.
Something like this:
public Task<T> FetchTask { get; private set; }
public bool StartFetch()
{
if (FetchTask != null) return false;
mCancellationTokenSource = new CancellationTokenSource();
FetchTask = FetchAsync(request, mCancellationTokenSource.Token);
return true;
}
public bool CancelFetch()
{
// send cancellation
if (mCancellationTokenSource != null)
mCancellationTokenSource.Cancel();
FetchTask = null;
return true;
}
private async Task<T> FetchAsync(LFHttpRequest request, CancellationToken cancellationToken)
{
var message = new HttpRequestMessage(request.Method, request.Uri);
var response = await HttpClient.SendAsync(message, cancellationToken);
response.EnsureSuccessStatusCode();
var ret = // Convert response.Content into T.
return ret;
}
I would recommend throwing InvalidOperationExceptions for the StartFetch and CancelFetch operations if the IsFetching state is invalid. This may seem annoying but it lets you catch programmer error and threading issues before they get to be a bigger, hidden problem.
As for your asynchronous approach, your method should return a result. So maybe something like private async Task<MyHttpResult> StartTask(...). Your result should contain a way determine success, failure, and cancellation.
For example:
public sealed class MyHttpResult
{
public HttpResponse Result { get; private set; }
public Exception Error { get; private set; }
public bool WasCancelled { get; private set; }
public MyHttpResult(HttpResponse result, Exception error, bool wasCancelled)
{
this.Result = result;
this.Error = error;
this.WasCancelled = wasCancelled;
}
}
Many async methods will throw a TaskCanceledException if they are cancelled, so you can catch that to signify, like so:
async Task<MyHttpResult> StartTask(LFHttpRequest request, CancellationToken cancellationToken)
{
var message = new HttpRequestMessage(new HttpMethod(request.Method), request.Uri);
HttpResponse response = null;
Exception lastError = null;
bool wasCancelled = false;
try
{
response = await MessageInvoker.SendAsync(message, cancellationToken);
}
catch(TaskCanceledException)
{
wasCancelled = true;
}
catch(Exception ex)
{
lastError = ex;
}
var result = new MyHttpResult(response, lastError, wasCancelled);
return result;
}
This is all assuming that your observers are also the callers, so they can await this method. If this is not the case, your idea of an EventHandler makes sense. Instead of returning the result, you could create a custom EventArgs class like so:
public delegate void TaskResultEventHandler<T>(object sender, TaskResultEventArgs<T> e);
public sealed class TaskResultEventArgs<T> : EventArgs
{
public T Result { get; private set; }
public Exception Error { get; private set; }
public bool WasCancelled { get; private set; }
public TaskResultEventArgs(T result, Exception error, bool wasCancelled)
{
this.Result = result;
this.Error = error;
this.WasCancelled = wasCancelled;
}
}
Then it's simply a matter of exposing a TaskResultEventHandler<HttpResponse> and your observers subscribing to it. You could invoke it like so:
var handler = this.HttpTaskCompleted;
if(handler != null)
handler(this, new TaskResultEventArgs<HttpResponse>(response, lastError, wasCancelled));
After you awaited the http call
var response = await HttpClient.SendAsync(message, cancellationToken);
You should test for cancellation:
if(cancellationToken.IsCancellationRequested)
//... do what you want, throw or return false or null, depending on how you want to handle this cancellation.
Or you can check and throw the Microsoft exception in one call :
cancel.ThrowIfCancellationRequested();
I'm facing difficulties understanding how to handle program control during asynchronous flow.
I have a SessionManager class which calls the initiates the session and we need to register
for the event OnStartApplicationSessionResponse and my control will return to the calling point. I will get the session id in the eventhandler after sometime or the error code if there is an error.
class SessionManager
{
public bool startUp(Object params)
{
try
{
serviceProvider = new ServiceProvider();
serviceProvider.OnStartApplicationSessionResponse += new StartApplicationSessionResponseHandler(ServiceProvider_OnStartApplicationSessionResponse);
serviceProvider.startUp(params);
}
}
public void ServiceProvider_OnStartApplicationSessionResponse(object sender, ServiceProvider.StartApplicationSessionResponseArgs e)
{
//e.getError
//I will get the session Id here or error code
}
}
How do I get sessionId or the error as my control is now at the calling position?
You could use TaskCompletionSource to make the Event awaitable.
class SessionManager
{
private ServiceProvider _serviceProvider;
public int SessionId
{
get;
private set;
}
public Task<bool> StartUp(Object param)
{
_serviceProvider = new ServiceProvider();
var tcs = new TaskCompletionSource<bool>();
_serviceProvider.OnStartApplicationSessionResponse += (sender, args) =>
{
// do your stuff
// e.g.
SessionId = 0xB00B5;
tcs.SetResult(true);
};
_serviceProvider.startUp(param);
return tcs.Task;
}
}
The call would look like:
private static async void SomeButtonClick()
{
var mgr = new SessionManager();
var success = await mgr.StartUp("string");
if (success)
{
Console.WriteLine(mgr.SessionId);
// update ui or whatever
}
}
note: This Feature is available in .Net 4.5.
With the C# feature async and await you are able to rewrite an asynchronous flow into something that is like a synchronous flow. You have only provided some fragments of your code so to provide a complete example I have created some code that resembles your code:
class StartEventArgs : EventArgs {
public StartEventArgs(Int32 sessionId, Int32 errorCode) {
SessionId = sessionId;
ErrorCode = errorCode;
}
public Int32 SessionId { get; private set; }
public Int32 ErrorCode { get; private set; }
}
delegate void StartEventHandler(Object sender, StartEventArgs e);
class ServiceProvider {
public event StartEventHandler Start;
public void Startup(Boolean succeed) {
Thread.Sleep(TimeSpan.FromSeconds(1));
if (succeed)
OnStart(new StartEventArgs(321, 0));
else
OnStart(new StartEventArgs(0, 123));
}
protected void OnStart(StartEventArgs e) {
var handler = Start;
if (handler != null)
handler(this, e);
}
}
The ServiceProvider.Startup method will delay for a second before firing an event that either signals success or failure depending on the succeed parameter provided. The method is rather silly but hopefully is similar to the behavior of your ServiceProvider.Startup method.
You can convert the asynchronous startup into a task using a TaskCompletionSource:
Task<Int32> PerformStartup(ServiceProvider serviceProvider, Boolean succeed) {
var taskCompletionSource = new TaskCompletionSource<Int32>();
serviceProvider.Start += (sender, e) => {
if (e.ErrorCode > 0)
throw new Exception(e.ErrorCode.ToString());
taskCompletionSource.SetResult(e.SessionId);
};
serviceProvider.Startup(succeed);
return taskCompletionSource.Task;
}
Notice how an error signaled by the Start event is converted into an Exception (in production code you should use a custom exception type instead).
Using the async and await feature of C# you can now write code that looks very much like synchronous code even though it actually is asynchronous:
async void Startup(Boolean succeed) {
var serviceProvider = new ServiceProvider();
try {
var sessionId = await PerformStartup(serviceProvider, succeed);
Console.WriteLine(sessionId);
}
catch (Exception ex) {
Console.WriteLine(ex);
}
}
If an error is reported by the Start event you can now deal with in the catch block. Also the session ID is simply a return value of the function. The "magic" is that using await on a Task will return the result of the task when it completes and if an exception is thrown in the task it can be caught on the thread awaiting the task.
Here is my async download reactive extension for WebClient.
What is the best way to recall "DownloadStringAsync" again and again till the operation succeeds?
Something like this but in reactive way:
while (true)
{
var result = DownloadStringAsync();
if (result)
{
return;
}
}
MY CODE:
[Serializable]
public class WebClientException : Exception
{
public WebClientResponse Response { get; set; }
public WebClientException()
{
}
public WebClientException(string message)
: base(message)
{
}
public WebClientException(string message, Exception innerException)
: base(message, innerException)
{
}
protected WebClientException(SerializationInfo info, StreamingContext context)
: base(info, context)
{
}
}
public class WebClientResponse
{
public WebHeaderCollection Headers { get; set; }
public HttpStatusCode StatusCode { get; set; }
public string Result { get; set; }
public WebException Exception { get; set; }
}
public static IObservable<WebClientResponse> DownloadStringAsync(this WebClient webClient, Uri address, WebHeaderCollection requestHeaders)
{
var asyncResult =
Observable.FromEventPattern<DownloadStringCompletedEventHandler, DownloadStringCompletedEventArgs>
(ev => webClient.DownloadStringCompleted += ev, ev => webClient.DownloadStringCompleted -= ev)
.ObserveOn(Scheduler.TaskPool)
.Select(o =>
{
var ex = o.EventArgs.Error as WebException;
if (ex == null)
{
var wc = (WebClient) o.Sender;
return new WebClientResponse {Headers = wc.ResponseHeaders, Result = o.EventArgs.Result};
}
var wcr = new WebClientResponse {Exception = ex};
var r = ex.Response as HttpWebResponse;
if (r != null)
{
wcr.Headers = r.Headers;
wcr.StatusCode = r.StatusCode;
var s = r.GetResponseStream();
if (s != null)
{
using (TextReader tr = new StreamReader(s))
{
wcr.Result = tr.ReadToEnd();
}
}
}
throw new WebClientException {Response = wcr};
})
.Take(1);
if (requestHeaders != null)
{
foreach (var key in requestHeaders.AllKeys)
{
webClient.Headers.Add(key, requestHeaders[key]);
}
}
webClient.DownloadStringAsync(address);
return asyncResult;
}
Your method produces a hot observable, which means that it has already started loading when it returns and each new subscription does not create a new request to the web server. You need to wrap your method in another and use Observable.Create (in order to create a cold observable which does create a new request upon each subscription):
public static IObservable<WebClientResponse> DownloadStringAsync(this WebClient webClient, Uri address, WebHeaderCollection requestHeaders)
{
return Observable
.Create(observer =>
{
DownloadStringAsyncImpl(webClient, address, requestHeaders)
.Subscribe(observer);
return () => { webClient.CancelAsync(); };
});
}
Here, DownloadStringAsyncImpl is your previous implementation of DownloadStringAsync, while the public method has been replaced.
Now you can retry the async method until it succeeds as follows:
myWebClient
.DownloadStringAsync( /* args... */)
.Retry()
.Subscribe(result => {
/* now I've got a result! */
});
I think you have at least one decent "here is some code" answer, so I will focus on a more general hand holding.
The first thing I would look at is the design guidelines for Rx. It is a short (34 page) PDF document that helps change paradigm from pull "subscriptions" to push, or moving from IEnumerable to IObservable.
If you want to go a bit further, there are PDF HOLs (hands on labs) for both .NET and JavaScript. You can find other resources on the Rx pages (start here).
If it is an async function. Doing a repetitive checking means you turned it into a sync function call. Is this something you really want to do?
You can have a dedicated thread calling this async function and block itself after calling this function. When create this thread, pass it a delegate that should be called after the async function returns. Upon completion, call the delegate with error code.
Hope this answers your question.