We Keep Coding

How to efficiently count HTTP Calls in asp.net core?

I have an abstract class called HttpHelper it has basic methods like, GET, POST, PATCH, PUT
What I need to achieve is this:
Store the url, time & date in the database each time the function is called GET, POST, PATCH, PUT
I don't want to store directly to the database each time the functions are called (that would be slow) but to put it somewhere (like a static queue-memory-cache) which must be faster and non blocking, and have a background long running process that will look into this cache-storage-like which will then store the values in the database.
I have no clear idea how to do this but the main purpose of doing so is to take the count of each calls per hour or day, by domain, resource and url query.
I'm thinking if I could do the following:
Create a static class which uses ConcurrentQueue<T> to store data and call that class in each function inside HttpHelper class
Create a background task similar to this: Asp.Net core long running/background task
Or use Hangfire, but that might be too much for simple task
Or is there a built-in method for this in .netcore?

Both Hangfire and background tasks would do the trick as consumers of the queue items.
Hangfire was there before long running background tasks (pre .net core), so go with the long running tasks for net core implementations.
There is a but here though.
How important is to you that you will not miss a call? If it is, then neither can help you.
The Queue or whatever static construct you have will be deleted the time your application crashes/machine restarts or just plain recycling of the application pools.
You need to consider some kind of external Queuing mechanism like rabbit mq with persistence on.
You can also append to a file, but that might also cause some delays as read/write.

I do not know how complex your problem is but I would consider two solutions.
First is calling Async Insert Method which will not block your main thread but will start task. You can return response without waiting for your log to be appended to database. Since you want it to be implemented in only some methods, I would do it using Attributes and Middleware.
Simplified example:
public IActionResult SomePostMethod()
{
LogActionAsync("This Is Post Method");
return StatusCode(201);
}
public static Task LogActionAsync(string someParameter)
{
return Task.Run(() => {
// Communicate with database (X ms)
});
}
Better solution is creating buffer which will not communicate with database each time but only when filled or at interval. It would look like this:
public IActionResult SomePostMethod()
{
APILog.Log(new APILog.Item() { Date = DateTime.Now, Item1 = "Something" });
return StatusCode(201);
}
public partial class APILog
{
private static List<APILog.Item> _buffer = null;
private cont int _msTimeout = 60000; // Timeout between updates
private static object _updateLock = new object();
static APILog()
{
StartDBUpdateLoopAsync();
}
private void StartDBUpdateLoopAsync()
{
// check if it has been already and other stuff
Task.Run(() => {
while(true) // Do not use true but some other expression that is telling you if your application is running.
{
Thread.Sleep(60000);
lock(_updateLock)
{
foreach(APILog.Item item in _buffer)
{
//Import into database here
}
}
}
});
}
public static void Log(APILog.Item item)
{
lock(_updateLock)
{
if(_buffer == null)
_buffer = new List<APILog.Item>();
_buffer.Add(item);
}
}
}
public partial class APILog
{
public class Item
{
public string Item1 { get; set; }
public DateTime Date { get; set; }
}
}
Also in this second example I would not call APILog.Log() each time but use Middleware in combination with Attribute

Why does Task.Run() in windows service OnStart() cause the service to refuse to stop?

I'm not sure why this service is refusing to stop.
I ran into this when trying to correct a TimeoutException thrown when starting the service. Using:
public void OnStart()
{
_startTask = Task.Run(DoWork, _cancelTokenSource.Token);
}
private void DoWork(){ [listen for things and operate on them] }
public void OnStop()
{
_cancelTokenSource.Cancel();
_startTask.Wait();
}
I understand that implementing a simple timer will solve this, but that's not the point of my question. Why does the use of Task.Run(() => action, _tokenSource.Token) resolve the TimeoutException but causes the service to not respond to control messages?
The issue Observed
After installing and starting the service (it's TopShelf BTW), I'm unable to stop the service by conventional methods.
First Attempt:
All Subsequent Attempts:
Edit: Still no joy
Here is my attempt after following the provided example.
public void Start()
{
var token = _cancelTokenSource.Token;
Task.Factory.StartNew(() => Setup(token), token);
}
public void Stop()
{
_cancelTokenSource.Cancel();
//TearDown(); <-- original implementation of stop
}
private void Setup(CancellationToken token)
{
_mailman.SendServiceStatusNotification(_buildMode, "Started");
... create some needed objects
token.Register(TearDown);
InitializeInboxWatcherProcess(...);
}
private void TearDown()
{
_inboxWatcher.Terminate();
_mailman.SendServiceStatusNotification(_buildMode, "Stopped");
}
private void InitializeInboxWatcherProcess(...)
{
// pre create and initiate stuff
_inboxWatcher = new LocalFileSystemWatcherWrapper(...);
_inboxWatcher.Initiate();
}
public class LocalFileSystemWatcherWrapper : IFileSystemWatcherWrapper
{
// do FileSystemWatcher setup and control stuff
}

This is most likely because you either don't have a cancellation method, or there are subprocesses inside of DoWork() that are still running when you call Cancel(). As #Damien_The_Unbeliever said cancellation is a cooperative task.
When you call _cancelTokenSource.Cancel() if you haven't registered a callback function all that happens is that a boolean value isCancellationRequested is set to true, the DoWork() method is then responsible for seeing this and stopping its execution on its own. There is a flaw here, though, as you can probably tell, that if you have a time consuming loop running in the DoWork() task when Cancel() is called, that loop will have to finish an iteration before it can check the value of isCancellationRequested which can lead to hanging.
The way around this is to insert cancellation callback functions INTO the DoWork() method, see here and then register them to the token, so that when you call the Cancel() method, ALL of the tasks running in the background are stopped without having to wait for them.
Hope this helps!

HttpContext.Current is null in an asynchronous Callback

Trying to access the HttpContext.Current in a method call back so can I modify a Session variable, however I receive the exception that HttpContext.Current is null. The callback method is fired asynchronously, when the _anAgent object triggers it.
I'm still unsure of the solution to this after viewing similar questions on SO.
A simplified version of my code looks like so:
public partial class Index : System.Web.UI.Page
protected void Page_Load()
{
// aCallback is an Action<string>, triggered when a callback is received
_anAgent = new WorkAgent(...,
aCallback: Callback);
...
HttpContext.Current.Session["str_var"] = _someStrVariable;
}
protected void SendData() // Called on button click
{
...
var some_str_variable = HttpContext.Current.Session["str_var"];
// The agent sends a message to another server and waits for a call back
// which triggers a method, asynchronously.
_anAgent.DispatchMessage(some_str_variable, some_string_event)
}
// This method is triggered by the _webAgent
protected void Callback(string aStr)
{
// ** This culprit throws the null exception **
HttpContext.Current.Session["str_var"] = aStr;
}
[WebMethod(EnableSession = true)]
public static string GetSessionVar()
{
return HttpContext.Current.Session["str_var"]
}
}
Not sure if necessary but my WorkAgent class looks like so:
public class WorkAgent
{
public Action<string> OnCallbackReceived { get; private set; }
public WorkAgent(...,
Action<string> aCallback = null)
{
...
OnCallbackReceived = aCallback;
}
...
// This method is triggered when a response is received from another server
public BackendReceived(...)
{
...
OnCallbackReceived(some_string);
}
}
What happens in the code:
Clicking a button calls the SendData() method, inside this the _webAgent dispatches a message to another server and waits for reply (in the mean time the user can still interact with this page and refer to the same SessionID). Once received it calls the BackendReceived() method which, back in the .aspx.cs page calls the Callback() method.
Question:
When the WorkAgent triggers the Callback() method it tries to access HttpContext.Current which is null. Why is that the case when if I continue on, ignoring the exception, I can still obtain the same SessionID and the Session variable using the ajax returned GetSessionVar() method.
Should I be enabling the aspNetCompatibilityEnabled setting?Should I be creating some sort of asynchronous module handler? Is this related to Integrated/Classic mode?

Here's a class-based solution that is working for simple cases so far in MVC5 (MVC6 supports a DI-based context).
using System.Threading;
using System.Web;
namespace SomeNamespace.Server.ServerCommon.Utility
{
/// <summary>
/// Preserve HttpContext.Current across async/await calls.
/// Usage: Set it at beginning of request and clear at end of request.
/// </summary>
static public class HttpContextProvider
{
/// <summary>
/// Property to help ensure a non-null HttpContext.Current.
/// Accessing the property will also set the original HttpContext.Current if it was null.
/// </summary>
static public HttpContext Current => HttpContext.Current ?? (HttpContext.Current = __httpContextAsyncLocal?.Value);
/// <summary>
/// MVC5 does not preserve HttpContext across async/await calls. This can be used as a fallback when it is null.
/// It is initialzed/cleared within BeginRequest()/EndRequest()
/// MVC6 may have resolved this issue since constructor DI can pass in an HttpContextAccessor.
/// </summary>
static private AsyncLocal<HttpContext> __httpContextAsyncLocal = new AsyncLocal<HttpContext>();
/// <summary>
/// Make the current HttpContext.Current available across async/await boundaries.
/// </summary>
static public void OnBeginRequest()
{
__httpContextAsyncLocal.Value = HttpContext.Current;
}
/// <summary>
/// Stops referencing the current httpcontext
/// </summary>
static public void OnEndRequest()
{
__httpContextAsyncLocal.Value = null;
}
}
}
To use it can hook in from Global.asax.cs:
public MvcApplication() // constructor
{
PreRequestHandlerExecute += new EventHandler(OnPreRequestHandlerExecute);
EndRequest += new EventHandler(OnEndRequest);
}
protected void OnPreRequestHandlerExecute(object sender, EventArgs e)
{
HttpContextProvider.OnBeginRequest(); // preserves HttpContext.Current for use across async/await boundaries.
}
protected void OnEndRequest(object sender, EventArgs e)
{
HttpContextProvider.OnEndRequest();
}
Then can use this in place of HttpContext.Current:
HttpContextProvider.Current
There may be issues as I currently do not understand this related answer. Please comment.
Reference: AsyncLocal (requires .NET 4.6)

When using threads or an async function, HttpContext.Current is not available.
Try using:
HttpContext current;
if(HttpContext != null && HttpContext.Current != null)
{
current = HttpContext.Current;
}
else
{
current = this.CurrentContext;
//**OR** current = threadInstance.CurrentContext;
}
Once you set current with a proper instance, the rest of your code is independent, whether called from a thread or directly from a WebRequest.

Please see the following article for an explanation on why the Session variable is null, and possible work arounds
http://adventuresdotnet.blogspot.com/2010/10/httpcontextcurrent-and-threads-with.html
quoted from the from the article;
the current HttpContext is actually in thread-local storage, which explains why child threads don’t have access to it
And as a proposed work around the author says
pass a reference to it in your child thread. Include a reference to HttpContext in the “state” object of your callback method, and then you can store it to HttpContext.Current on that thread

Error when I use the Task-based Asynchronous Pattern in the client of duplex WCF service

I develop download manager application that consists of two parts: 1) Duplex WCF service that performs downloading and sends downloading status data to client in real-time. 2) WPF Client that receives downloading status data from the service and displays in DataGrid. In my duplex WCF service there is a callback interface
[ServiceContract(CallbackContract = typeof(IDownloadManagerServiceCalback))]
public interface IDownloadManagerServiceCalback
{
/// <summary>
/// Returns changed downloading status to client.
/// </summary>
/// <returns>Downloading which has changed status</returns>
[OperationContract(IsOneWay = true)]
void UpdateSelectedDownload(DownloadStatus p_SelectedDownload);
}
On the client side I implement this interface:
class CallbackHandler : IDownloadManagerServiceCallback
{
/// <summary>
/// "Download status changed" event.
/// </summary>
public event EventHandler<DownloadStatusChangedEventArgs> DownloadStatusChanged;
public async Task UpdateSelectedDownload(DownloadStatus p_UpdatedDownload)
{
await Task.Run(() =>
{
// If handler was subscribed to event:
if (DownloadStatusChanged != null)
{
DownloadStatus updatedDownload = p_UpdatedDownload;
DownloadStatusChangedEventArgs updatedDownloadArgs = new DownloadStatusChangedEventArgs();
updatedDownloadArgs.Download = updatedDownload;
DownloadStatusChanged(this, updatedDownloadArgs);
}
});
}
}
When I build the solution I have the following error (text of error message i translate from Russian to English because my Visual Studio 2013 is Russianize):
DownloadManager_Client.CallbackHandler doesn’t implement member "DownloadManager_Client.DownloadManager_ServiceReference.IDownloadManagerServiceCallback.UpdateSelectedDownload(DownloadManager_Client.DownloadManager_ServiceReference.DownloadStatus)". "DownloadManager_Client.CallbackHandler.UpdateSelectedDownload(DownloadManager_Client.DownloadManager_ServiceReference.DownloadStatus)" can’t be implemented "DownloadManager_Client.DownloadManager_ServiceReference.IDownloadManagerServiceCallback.UpdateSelectedDownload(DownloadManager_Client.DownloadManager_ServiceReference.DownloadStatus)", because it doesn’t contain appropriate returned “void” type.
Here DownloadManager_Client is the name of WPF client project, DownloadManager_ServiceReference is the name of service reference to WCF service in the client project. How can I correct this error?

The interface should be defined as returning Task not void since your implementation is an async method returning a Task.
EDIT: You are in a pickle because you want to use async which require a Task to be returned however your method is marked as IsOneWay = true - you can't have both. Either IsOneWay = false and keep the async nature or keep one-way but remove the async.
Example 1 - Async method
[ServiceContract(CallbackContract = typeof(IDownloadManagerServiceCalback))]
public interface IDownloadManagerServiceCalback
{
/// <summary>
/// Returns changed downloading status to client.
/// </summary>
/// <returns>Downloading which has changed status</returns>
[OperationContract(IsOneWay = false)]
Task UpdateSelectedDownload(DownloadStatus p_SelectedDownload);
}
Then keep your original implementation returning Task
Example 2 - One-way method
[ServiceContract(CallbackContract = typeof(IDownloadManagerServiceCalback))]
public interface IDownloadManagerServiceCalback
{
/// <summary>
/// Returns changed downloading status to client.
/// </summary>
/// <returns>Downloading which has changed status</returns>
[OperationContract(IsOneWay = true)]
void UpdateSelectedDownload(DownloadStatus p_SelectedDownload);
}
In your implementation remove any await; async; tasks.
General
Async WCF methods should return Task or Task < T >. The only time you have an async void is during an event handler which is not applicable here.
As a general rule with async methods - avoid async void like the plague because an exception thrown in a try catch inside such a method can not be caught by a regular try-catch. The only exception (no pun intended is during event handlers).
Async void methods have different error-handling semantics. When an exception is thrown out of an async Task or async Task method, that exception is captured and placed on the Task object. With async void methods, there is no Task object, so any exceptions thrown out of an async void method will be raised directly on the SynchronizationContext that was active when the async void method started. More...
Would you like to know more?
Best Practices in Asynchronous Programming

WCF Calling a service property from a callback without deadlock

Can someone kindly show me how to call a property of a WCF service from within a callback without deadlock occurring?
I've tried adding [CallbackBehavior(ConcurrencyMode=ConcurrencyMode.Multiple)] to the class which implements the callback, but without success.
The service has the following attribute:
[ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Single, InstanceContextMode = InstanceContextMode.Single)]
public class SAPUploadService : ISAPUploadService
{
Thanks MM
Here's the code which calls the Callback method
foreach (var otherConnection in _users.Keys)
{
SAPUploadInstruction ins = AddMessageToInstruction(message);
ins.UserName = user.UserName;
Task.Factory.StartNew(() =>
{
otherConnection.ReceiveInstruction(ins);
});
and here's the callback implementation of the ReceiveInstruction
public void ReceiveInstruction(SAPUploadInstruction instruction)
{
// fire this objects call back....
if (OnReceiveInstruction != null) OnReceiveInstruction(this, instruction);
}
In the above, the event OnReceiveInstruction is attached to the UI. This is handled as follows:
public void ReceiveInstruction(object sender, SAPUploadInstruction instruction)
{
DispatchIfNecessary(() => {
ProcessInstruction(instruction);
});
}
The method above - ProcessInstruction - sets various controls according to the service properties/functions. It is this that is getting deadlocked i.e Label1.Content = myService.SomeProperty.
BTW, DispatchIfNecessary is implemented as :
public void DispatchIfNecessary(Action action)
{
if (!Dispatcher.CheckAccess())
Dispatcher.Invoke(action);
else
action.Invoke();
}

In DispatchIfNecessary use asynchronous version of Invoke, so your callback won't wait for finishing UI changes, that can't be done because UI thread is waiting for end of callback processing (hence we have deadlock):
Dispatcher.BeginInvoke(action);