We Keep Coding

How can I invoke a function at special position after receiving outer special message in C#?

Let us look at a segment of a code: csharp
public void TestMethod(){
//do something
string a = await WaitingObj.Read();
//continue to do something
}
And we have a message queue, which is receiving messages all the time. When it finds the message is what the TestMethod() needs, it will pass this message to WaitingObj.Read() which will return the value and pass the value to the string a.
However, We know we cannot invoke Read() twice (in the TestMethod() to pass the value and when the queue receives new messages to judge whether the message is the TestMethod() needs).
So, How can I solve this problem with Await/Async or design the programs.
The message queue in the problem is just a simple queue structure in basic data structure.
The function of WaitingObj.Read() is just indicating when the data is ready for passing to string a and string a can directly use it can continue carrying the rest of codes.

After reading through your post and all the comments, I noticed in particular where you said:
I wonder if I can just solve it by designing WaitingObj.Read()....
Let's entertain that thought by designing a Queue that provides some basic observability by implementing INotifyCollectionChanged and provides these features:
A ReadAsync method to await a "special" message that matches a specified predicate.
A SelfTest method that enqueues one message per second from a list of 10 messages.
An instance of var WaitingObj = new DesignedObservableQueue() can then be exercised in a console app to see whether or not this would satisfy your design specs.
Designed Queue (a.k.a. "WaitingObj")
class DesignedObservableQueue : Queue<MockMessage>, INotifyCollectionChanged
{
public new void Enqueue(MockMessage message)
{
base.Enqueue(message);
CollectionChanged?
.Invoke(
this,
new NotifyCollectionChangedEventArgs(
NotifyCollectionChangedAction.Add,
message));
}
public new MockMessage Dequeue()
{
var message = base.Dequeue();
CollectionChanged?
.Invoke(
this,
new NotifyCollectionChangedEventArgs(
NotifyCollectionChangedAction.Remove,
message));
return message;
}
public event NotifyCollectionChangedEventHandler? CollectionChanged;
Provide a way to detect that a special message has been enqueued.
public async Task ReadAsync(Predicate<MockMessage> condition)
{
var awaiter = new SemaphoreSlim(0, 1);
try
{
CollectionChanged += localOnCollectionChanged;
await awaiter.WaitAsync();
}
finally
{
awaiter.Release();
CollectionChanged -= localOnCollectionChanged;
}
void localOnCollectionChanged(object? sender, NotifyCollectionChangedEventArgs e)
{
switch (e.Action)
{
case NotifyCollectionChangedAction.Add:
var message = e.NewItems!.Cast<MockMessage>().First();
if(condition(message))
{
Console.WriteLine($"MATCH: {message.Message}");
awaiter.Release();
}
else
{
Console.WriteLine($"NO MATCH: {message.Message}");
}
break;
default:
break;
}
}
}
Mock a queue that "is receiving messages all the time" by self-enqueuing at one-second intervals.
public async Task SelfTest(CancellationToken token)
{
foreach (
var message in new[]
{
"occasion",
"twin",
"intention",
"arrow",
"draw",
"forest",
"special",
"please",
"shell",
"momentum",
})
{
if(token.IsCancellationRequested) return;
Enqueue(new MockMessage { Message = message });
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
}
Exercise TestMethod
Once the TestMethod shown in your post is changed to an async method, perform this minimal test:
static void Main(string[] args)
{
Console.Title = "Test Runner";
var stopwatch = new Stopwatch();
var WaitingObj = new DesignedObservableQueue();
// Local test method is expecting to match
// the predicate in ~6 seconds so allow 10.
var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
stopwatch.Start();
_ = WaitingObj.SelfTest(cts.Token);
try
{
TestMethod().Wait(cts.Token);
Console.WriteLine($"PASSED {stopwatch.Elapsed}");
}
catch (OperationCanceledException ex)
{
Console.WriteLine($"FAILED {stopwatch.Elapsed}");
}
// Local test method
async Task TestMethod()
{
// do something
await WaitingObj.ReadAsync((message) => message.Message == "special");
// continue to do something
}
Console.ReadKey();
}
Where:
class MockMessage
{
public string Message { get; set; } = string.Empty;
}

Custom producer consumer randomly stops while using blocking collection

I have following problem: Service is being run on the server. It has timer that ticks every 5 seconds. It is being modeled to produce items for a consumer. Items are being consumed in multi thread.
I've added logging but I cannot find out when and where any problem occurs and this just blocks. No exception or error. My goal is to keep getting requests to be processed from db and consume them. Timer is being a producer.
public class CustomProducerConsumer<T> : IDisposable
{
private readonly BlockingCollection<T> blockingCollection;
private readonly Action<T> consumeItem;
private readonly Task[] workers;
public CustomProducerConsumer(Action<T> consumeItem,
int degreeOfParallelism,
int capacity = 1024)
{
this.consumeItem = consumeItem;
this.blockingCollection = new BlockingCollection<T>(capacity);
this.workers = Enumerable.Range(1, degreeOfParallelism)
.Select(_ => Task.Factory.StartNew(Worker,
TaskCreationOptions.LongRunning))
.ToArray();
}
public void Dispose()
{
// Unblock all workers even if the client
// didn't call CompleteProcessing
if (!this.blockingCollection.IsAddingCompleted)
{
this.blockingCollection.CompleteAdding();
}
Task.WaitAll(this.workers);
this.blockingCollection.Dispose();
}
public void Process(T item)
{
this.blockingCollection.TryAdd(item);
}
private void Worker()
{
foreach (var item in this.blockingCollection.GetConsumingEnumerable())
{
this.consumeItem(item);
}
}
}
Here is my code from service:
private readonly BlockingCollection<StitchingRequestProcessingModel> requestsToBeProcessed =
new BlockingCollection<StitchingRequestProcessingModel>(10);
private readonly BlockingCollection<Dictionary<int, StitchingRequest[]>> pendingRequests =
new BlockingCollection<Dictionary<int, StitchingRequest[]>>(10);
private readonly Timer timer;
public Service()
{
InitializeComponent();
this.produceConsumer =
new CustomProducerConsumer<StitchingRequestModel>(this.ProcessItems,
Environment.ProcessorCount);
this.timer = new Timer(o =>
{
this.TimerElapsed();
this.timer.Change(TimeSpan.FromSeconds(5), Timeout.InfiniteTimeSpan);
}, null, TimeSpan.Zero, Timeout.InfiniteTimeSpan);
this.ConsumeRequests();
}
public void TimerElapsed()
{
try
{
//this just adds into the list an item to ping the db for pending requests when available
this.requestsToBeProcessed.Add(new StitchingRequestProcessingModel());
this.pendingRequests.TryTake(out Dictionary<int, Request[]> requests);
if (requests == null)
{
return;
}
foreach (KeyValuePair<int, Request[]> request in requests)
{
this.produceConsumer.Process(new StitchingRequestModel(request));
}
}
catch (Exception exception)
{
this.errorLogger.Error(exception.Message);
}
}
private void ConsumeRequests()
{
Task.Factory.StartNew(() =>
{
while (!this.requestsToBeProcessed.IsCompleted)
{
if (this.tokenSource.Token.IsCancellationRequested)
{
break;
}
StitchingRequestProcessingModel data = null;
try
{
data = this.requestsToBeProcessed.Take();
}
catch (InvalidOperationException)
{
}
if (data == null)
{
continue;
}
try
{
// this just executes sql query to get those request from db
var requests = this.requestService.GetPendingRequests();
this.pendingRequests.Add(requests);
}
catch (Exception exception)
{
this.errorLogger.Error(exception.Message, "Failed to get pending requests");
}
}
},
this.tokenSource.Token,
TaskCreationOptions.LongRunning, TaskScheduler.Current);
}
private void ProcessItems(StitchingRequestModel model)
{
foreach (StitchingRequest request in model.Requests)
{
this.requestsToBeProcessed.Add(new StitchingRequestProcessingModel(request);
}
}
Main reason why I placed consuming items into blocking collection is Nhibernate. It is giving me issues while doing multithreading. No ideas what else to try, nor why this approach is not working. I do not want to call CompleteAdding on blocking collection since I need requests to be added and just processed in first available thread.
Timer on each elapsed event will try to create an a pending request that will be added into blocking collection and processed on first available turn. Service runs for 2~3h and just stops. ProcessItems method can be long running. CPU is 8 core.
UPDATE
Added cancelation for consumer task.

Problem was solved with not working with entity object between consumer and producer. Created dto for info needed.

Awaitable task in not awaited task

I'm working on a little console application that selects messages from a database queue
and forwards the messages to a rest api (ASP.NET Web Api).
In general the application does the following steps:
Get the number of pending messages
Load the last pending messages
Post the message to the rest api
Remove the message from the database
To make the program more flexible and to have the ability to process every single message
in a separate database transcation the steps 2 - 3 will be executed as tasks.
This means if there are four messages in the database, we'll have four tasks that
should run nearly parallel and process the messages.
This is what the code looks like:
Database message
public class DatabaseMessage
{
public string Message { get; set; }
}
UnitOfWork (Interface IUnitOfWork)
public class UnitOfWork
{
// ... extermely simplified
public int GetNumberOfPendingMessages() { ... }
public DatabaseMessage GetNextPendingMessage() { ... }
public void DeleteMessage(DatabaseMessage message) { ... }
}
HttpService (Interface IHttpService)
public class HttpService
{
private readonly HttpClient _httpClient;
public HttpService()
{
_httpClient = new HttpClient();
/* Some initalization stuff for the HttpClient object */
}
public async Task<HttpResponse> PostMessage(DatabaseMessage message)
{
var content = /* Create content object */
return await _httpClient.PostAsync(..., content);
}
}
MessageProcessingService (Interface IMessageProcessingService)
public class MessageProcessingService
{
private readonly IHttpService _httpService;
private readonly Semaphore _databaseProcessingSemaphore;
public MessageProcessingService(IHttpService httpService)
{
_httpService = httpService;
}
public async Task ProcessDatabaseMessages()
{
var unitOfWork = new UnitOfWork();
var numberOfPendingMessages = unitOfWork.GetNumberOfPendingMessages();
var messageProcessingTasks = new List<Task>();
for(int t = 0; t < numberOfPendingMessages; t++)
{
messageProcessingTasks.Add(new Task(() => {
ProcessMessageAsTask();
}));
}
var continuationHandler = Task.WhenAll(messageProcessingTasks);
messageProcessingTasks.ForEach(e => e.Start());
await continuationHandler;
}
private void ProcessMessageAsTask()
{
// Single unit of work for each tasks
var unitOfWork = new UnitOfWork();
try{
// Starting a database transaction
unitOfWork.StartTransaction();
_databaseProcessingSemaphore.OnWait();
var message = unitOfWork.GetNextPendingMessage();
_databaseProcessingSemaphore.Release();
if(message != null)
{
var response = _httpService.PostMessage(message).Result;
if(response == HttpStatus.OK)
{
unitOfWork.DeleteMessage(message);
unitOfWork.Commit();
}
else
{
unitOfWork.Rollback();
}
}
else
{
unitOfWork.Commit();
}
}
catch(Exception ex)
{
unitOfWork.Rollback();
// Further error handling...
}
}
}
For better understanding, the HttpClient object is created and managed by Unity and is injected
into the MessageProcessingService object. The HttpClient is held as singleton in the container.
I'm facing now the problem that the call of the method _httpService.PostMessage(). For example, if there are five
messages in the message queue, the call fails five times with an exception that tells me that an task has been canceled.
My question is now what is the problem with PostAsync call of the .NET HttpClient? Is the issue caused by the .Result option or would
it be better to create a new instance of the HttpClient for each message processing task?
Or is there a general problem with the architecture with tasks and the processing of rest api calls?
Update 2018-04-04 - 08:09
I've now made the method ProcessMessageAsTask async and I'm awaiting now the call of the HttpService.
But now I don't get any exception at all. In the ressource monitor and while debugging I can see that all tasks reach the call of the HttpClient (return await _httpClient.PostAsync(..., content);)
But there is no exception nor will the messages be posted. But I don't get any exceptions. The program will be closed immediately after the calls of the HttpClient. All futher statements were not processed.
Changes:
public async Task ProcessDatabaseMessages()
{
var unitOfWork = new UnitOfWork();
var numberOfPendingMessages = unitOfWork.GetNumberOfPendingMessages();
var messageProcessingTasks = new List<Task>();
for(int t = 0; t < numberOfPendingMessages; t++)
{
messageProcessingTasks.Add(new Task(async () => {
await ProcessMessageAsTask();
}));
}
var continuationHandler = Task.WhenAll(messageProcessingTasks);
messageProcessingTasks.ForEach(e => e.Start());
await continuationHandler;
}
private async Task ProcessMessageAsTask()
{
// Single unit of work for each tasks
var unitOfWork = new UnitOfWork();
try{
// Starting a database transaction
unitOfWork.StartTransaction();
_databaseProcessingSemaphore.OnWait();
var message = unitOfWork.GetNextPendingMessage();
_databaseProcessingSemaphore.Release();
if(message != null)
{
var response = await _httpService.PostMessage(message);
if(response == HttpStatus.OK)
{
unitOfWork.DeleteMessage(message);
unitOfWork.Commit();
}
else
{
unitOfWork.Rollback();
}
}
else
{
unitOfWork.Commit();
}
}
catch(Exception ex)
{
unitOfWork.Rollback();
// Further error handling...
}
}

Async queue return info

Hello I am trying to create a logging system which sends logs to a WCF. In principal it has a Log(string text) method, which can be called multiple times before the actual logging action is made to reduce network chatter. To achieve this I've created a queue (a list) of logs and a timer, which performs the actual logging with a set frequency.
When I want to log something in my program I use the Log method. It looks a bit like this:
private readonly List<string> _currentLogQueue = new List<string>();
public void Log(string logText)
{
lock (_currentLogQueue)
{
_currentLogQueue.Add(logText);
}
}
The queue is then periodically sent to the WCF. The periodic sending is done like so:
private void SetUpQueue(TimeSpan queueFlushPeriod)
{
Task.Run(async () =>
{
while (true)
{
SendQueue(); // Does the actual communication and clears queue on success.
await Task.Delay(queueFlushPeriod);
}
});
}
How can I enable the program using this logger to react to errors during the SendQueue()? I can modify SendQueue to return some kind of error if needed. Right now I only can think of a callback in the form of a delegate passes to the Log() method, but it seems very passé and not fun in the age of async await.

To answer your question:
You can have a TaskCompletionSource indicating success/failure of the logged message:
private readonly List<Tuple<string, TaskCompletionSource<object>> _currentLogQueue = ...;
public Task LogAsync(string logText)
{
var tcs = new TaskCompletionSource<object>();
lock (_currentLogQueue)
{
_currentLogQueue.Add(Tuple.Create(logText, tcs));
}
return tcs.Task;
}
// (Within SendQueue)
var message = queueElement.Item1;
var tcs = queueElement.Item2;
try
{
SendMessage(message);
tcs.TrySetResult(null);
}
catch (Exception ex)
{
tcs.TrySetException(ex);
}
However, I don't think this would really be helpful. What meaningful action can the program take if logging failed?

Async requests to a web service

How to make async requests to a webservice from a Thread?

Here is the short answer without a load of explanations.
Before calling the Async method on your Client object make sure you are not running on the UI Thread:-
System.Threading.ThreadPool.QueueUserWorkItem( o =>
{
try
{
svc.SomeMethodAsync();
}
catch (err)
{
// do something sensible with err
}
});
Now the corresponding completed event will occur on a ThreadPool thread not the UI Thread.

Here is a solution using WCF.
Service Code FileService.svc
public class FileService
{
[OperationContract]
public byte[] GetFile(string filename)
{
byte[] File;
//do logic
return File;
}
}
Client Code
public int requested_file_count = 5;
public list<string> filenames;
public FileServiceClient svc
//Constructor
public Example()
{
svc = new FileServiceClient();
}
Public void GetFiles()
{
//Initialise the list of names and set the count of files received
filenames = new list<string>(5);
requested_file_count = filenames.Count;
svc.GetFileCompleted += new EventHandler<GetFileCompletedEventArgs>(GetFile_Completed);
//Call the Async Method passing it the file name and setting the userstate to 1;
svc.GetFileAsync(filenames[0],1);
}
void GetFile_Completed(object Sender, GetFileCompletedEventArgs e)
{
if (e.UserState == requested_file_count)
{
//All files have been downloaded
}
else
{
svc.GetFileAsync(filenames[e.UserState],++e.UserState);
}
//Do Something with the downloaded file
byte[] filedata = e.result;
}