Hello I am trying to create a logging system which sends logs to a WCF. In principal it has a Log(string text) method, which can be called multiple times before the actual logging action is made to reduce network chatter. To achieve this I've created a queue (a list) of logs and a timer, which performs the actual logging with a set frequency.
When I want to log something in my program I use the Log method. It looks a bit like this:
private readonly List<string> _currentLogQueue = new List<string>();
public void Log(string logText)
{
lock (_currentLogQueue)
{
_currentLogQueue.Add(logText);
}
}
The queue is then periodically sent to the WCF. The periodic sending is done like so:
private void SetUpQueue(TimeSpan queueFlushPeriod)
{
Task.Run(async () =>
{
while (true)
{
SendQueue(); // Does the actual communication and clears queue on success.
await Task.Delay(queueFlushPeriod);
}
});
}
How can I enable the program using this logger to react to errors during the SendQueue()? I can modify SendQueue to return some kind of error if needed. Right now I only can think of a callback in the form of a delegate passes to the Log() method, but it seems very passé and not fun in the age of async await.
To answer your question:
You can have a TaskCompletionSource indicating success/failure of the logged message:
private readonly List<Tuple<string, TaskCompletionSource<object>> _currentLogQueue = ...;
public Task LogAsync(string logText)
{
var tcs = new TaskCompletionSource<object>();
lock (_currentLogQueue)
{
_currentLogQueue.Add(Tuple.Create(logText, tcs));
}
return tcs.Task;
}
// (Within SendQueue)
var message = queueElement.Item1;
var tcs = queueElement.Item2;
try
{
SendMessage(message);
tcs.TrySetResult(null);
}
catch (Exception ex)
{
tcs.TrySetException(ex);
}
However, I don't think this would really be helpful. What meaningful action can the program take if logging failed?
Related
Let us look at a segment of a code: csharp
public void TestMethod(){
//do something
string a = await WaitingObj.Read();
//continue to do something
}
And we have a message queue, which is receiving messages all the time. When it finds the message is what the TestMethod() needs, it will pass this message to WaitingObj.Read() which will return the value and pass the value to the string a.
However, We know we cannot invoke Read() twice (in the TestMethod() to pass the value and when the queue receives new messages to judge whether the message is the TestMethod() needs).
So, How can I solve this problem with Await/Async or design the programs.
The message queue in the problem is just a simple queue structure in basic data structure.
The function of WaitingObj.Read() is just indicating when the data is ready for passing to string a and string a can directly use it can continue carrying the rest of codes.
After reading through your post and all the comments, I noticed in particular where you said:
I wonder if I can just solve it by designing WaitingObj.Read()....
Let's entertain that thought by designing a Queue that provides some basic observability by implementing INotifyCollectionChanged and provides these features:
A ReadAsync method to await a "special" message that matches a specified predicate.
A SelfTest method that enqueues one message per second from a list of 10 messages.
An instance of var WaitingObj = new DesignedObservableQueue() can then be exercised in a console app to see whether or not this would satisfy your design specs.
Designed Queue (a.k.a. "WaitingObj")
class DesignedObservableQueue : Queue<MockMessage>, INotifyCollectionChanged
{
public new void Enqueue(MockMessage message)
{
base.Enqueue(message);
CollectionChanged?
.Invoke(
this,
new NotifyCollectionChangedEventArgs(
NotifyCollectionChangedAction.Add,
message));
}
public new MockMessage Dequeue()
{
var message = base.Dequeue();
CollectionChanged?
.Invoke(
this,
new NotifyCollectionChangedEventArgs(
NotifyCollectionChangedAction.Remove,
message));
return message;
}
public event NotifyCollectionChangedEventHandler? CollectionChanged;
Provide a way to detect that a special message has been enqueued.
public async Task ReadAsync(Predicate<MockMessage> condition)
{
var awaiter = new SemaphoreSlim(0, 1);
try
{
CollectionChanged += localOnCollectionChanged;
await awaiter.WaitAsync();
}
finally
{
awaiter.Release();
CollectionChanged -= localOnCollectionChanged;
}
void localOnCollectionChanged(object? sender, NotifyCollectionChangedEventArgs e)
{
switch (e.Action)
{
case NotifyCollectionChangedAction.Add:
var message = e.NewItems!.Cast<MockMessage>().First();
if(condition(message))
{
Console.WriteLine($"MATCH: {message.Message}");
awaiter.Release();
}
else
{
Console.WriteLine($"NO MATCH: {message.Message}");
}
break;
default:
break;
}
}
}
Mock a queue that "is receiving messages all the time" by self-enqueuing at one-second intervals.
public async Task SelfTest(CancellationToken token)
{
foreach (
var message in new[]
{
"occasion",
"twin",
"intention",
"arrow",
"draw",
"forest",
"special",
"please",
"shell",
"momentum",
})
{
if(token.IsCancellationRequested) return;
Enqueue(new MockMessage { Message = message });
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
}
Exercise TestMethod
Once the TestMethod shown in your post is changed to an async method, perform this minimal test:
static void Main(string[] args)
{
Console.Title = "Test Runner";
var stopwatch = new Stopwatch();
var WaitingObj = new DesignedObservableQueue();
// Local test method is expecting to match
// the predicate in ~6 seconds so allow 10.
var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
stopwatch.Start();
_ = WaitingObj.SelfTest(cts.Token);
try
{
TestMethod().Wait(cts.Token);
Console.WriteLine($"PASSED {stopwatch.Elapsed}");
}
catch (OperationCanceledException ex)
{
Console.WriteLine($"FAILED {stopwatch.Elapsed}");
}
// Local test method
async Task TestMethod()
{
// do something
await WaitingObj.ReadAsync((message) => message.Message == "special");
// continue to do something
}
Console.ReadKey();
}
Where:
class MockMessage
{
public string Message { get; set; } = string.Empty;
}
currently I try to implement a MessagingClient, working with RabbitMQ and C#.
My idea is that every service uses its own queue for receiving messages and there is an own exchange for each message type. For example a message type could be "download/requested" or "download/started". These exchanges are of type "fanout". Now the clients, that want to listen on some message, bind their queue to the corresponding exchange. For example the download service binds its queue "download-bot" to the exchange "download/requested".
My problem is now that I can't really imagine, how to deal with different types of messages on the parsing site. I use Newtonsoft.Json for encoding/decoding the message objects. For each different message type, there is separate "handler" I want to execute. This handler should get the message (deserialized) as a parameter. My problem is, that with mulitple message types over one queue, there seems to be just one handler qer queue. How do I find out, which concrete handler to execute and how to parse the message, if I don't know the type at compile time? I know an important keyword will be "reflection", but I could not tinker something working together. I should say that i'm pretty new with C#. Could someone provide a working example of this ? Or is this even something, I should do like that?
As a workarround, currently I got the following (reduced) example (using one queue per message type and service)
namespace Lib.Message {
public class RabbitMqMessageClient : IMessageClient {
private static NLog.ILogger logger = Common.Logger.GetLogger ();
private RabbitMqConfig config;
private IModel model;
private const string DOWNLOAD_REQUESTED = "download/requested";
public RabbitMqMessageClient (RabbitMqConfig config) {
this.config = config;
var factory = new ConnectionFactory () { HostName = config.Hostname };
var connection = factory.CreateConnection ();
this.model = connection.CreateModel ();
}
public Task PublishDownloadRequested (string userRef, string id, string url) {
var message = new DownloadRequestMessage (userRef, id, url);
return this.publish (DOWNLOAD_REQUESTED, message);
}
public Task SubscribeToDownloadRequested (Action<DownloadRequestMessage> action) {
return this.subscribe<DownloadRequestMessage> (DOWNLOAD_REQUESTED, action);
}
public Task SubscribeToDownloadRequested (Func<DownloadRequestMessage, Task> action) {
return this.subscribe<DownloadRequestMessage> (DOWNLOAD_REQUESTED, action);
}
public Task Start () {
return Task.CompletedTask;
}
private Task subscribe<TPayloadType> (string topic, Action<TPayloadType> action) {
Func<TPayloadType, Task> wrappedAction = (TPayloadType args) => {
action (args);
return Task.CompletedTask;
};
return this.subscribe<TPayloadType> (topic, wrappedAction);
}
private Task subscribe<TPayloadType> (string topic, Func<TPayloadType, Task> action) {
var queue = $"{config.QueueName}--{topic}";
model.ExchangeDeclare (topic, ExchangeType.Fanout, true);
model.QueueDeclare (
queue,
durable : true,
exclusive : false,
autoDelete : false,
arguments : null);
model.QueueBind (config.QueueName, topic, "foo");
var consumer = new EventingBasicConsumer (model);
// this consumer will be shared for multiple queues right ?
// maybe even not here in this method, but in the "Start" method
consumer.Received += async (model, ea) => {
logger.Info ($"handling {ea.Exchange}");
var jsonString = Encoding.UTF8.GetString (ea.Body.Span.ToArray ());
// here I need to know how to deserialize the payload from the value of "ea.Exchange"
var message = JsonConvert.DeserializeObject<TPayloadType>(jsonString);
// I think i have to put the action in a map (?) instead and then find the concrete handler by the exchange name?
await action(message);
};
model.BasicConsume (config.QueueName, true, consumer);
return Task.CompletedTask;
}
private Task publish (string topic, object payload) {
logger.Info ($"publishing {topic}");
string message = JsonConvert.SerializeObject (payload);
var bytes = Encoding.UTF8.GetBytes (message);
model.BasicPublish (
exchange: topic,
routingKey: "foo",
basicProperties : null,
body : bytes
);
return Task.CompletedTask;
}
}
}
Environment
Windows 10 Professional
.NET Core Console Application
Code
I have an abstracted message receiver that looks like this. In this code the entity is the name of the Subscription (e.g. user).
public class AzureMessageReceiver : ITdlMessageReceiver
{
private readonly ServiceBusConnection serviceBusConnection;
private readonly ILogger<AzureMessageReceiver> logger;
public AzureMessageReceiver(ServiceBusConnection serviceBusConnection, ILogger<AzureMessageReceiver> logger)
{
this.serviceBusConnection = serviceBusConnection;
this.logger = logger;
}
public async Task<TdlMessage<T>> ReceiveAsync<T>(string topic, string entity) where T : class
{
try
{
var subscriptionPath = EntityNameHelper.FormatSubscriptionPath(topic, entity);
var messageReceiver = new MessageReceiver(serviceBusConnection, subscriptionPath, ReceiveMode.ReceiveAndDelete);
var message = await messageReceiver.ReceiveAsync();
if (message == null)
{
return null;
}
var messageString = Encoding.UTF8.GetString(message.Body);
return JsonConvert.DeserializeObject<TdlMessage<T>>(messageString);
}
catch (Exception ex)
{
logger.LogError(ex, "Error receiving Azure message.");
return null;
}
}
}
The injected ServiceBusConnection is constructed like this. NOTE: this same connection initialization works to write messages to the same Topic and Subscription.
services.AddSingleton(serviceProvider =>
new ServiceBusConnection(configuration[$"{DurableCommunicationKey}:AzureConnectionString"]));
UPDATE: here is the code that wraps the call to the receiver class and is the controller for receiving messages:
static async void Receive(ITdlMessageReceiver receiver, ILogger logger)
{
while (true)
{
var message = await receiver.ReceiveAsync<TdlMessage<object>>(topic, entity);
if (message != null)
{
logger.LogDebug($"Message received. Topic: {topic}. Action: {Enum.GetName(typeof(TopicActions), message.Action)}. Message: {JsonConvert.SerializeObject(message)}.");
}
Thread.Sleep(sleepTime);
}
}
Problem
Every time I execute this line var message = await messageReceiver.ReceiveAsync(); it just crashes the Console app. No Exception and nothing in Event Viewer.
What I've Tried
Using the Secondary Connection String from the ASB
Providing a timeout like messageReceiver.ReceiveAsync(TimeSpan.FromMinutes(1));
Changing the injected topic from just the name of the topic to the entire URL of the topic (e.g. https://{...}.servicebus.windows.net/{topicName})
Changing the ReceiveMode to PeekLock
Tacking on ConfigureAwait(false) to the ReceiveAsync call.
Changing the timeout to TimeSpan.Zero. NOTE: this does not crash the app but actually throws an Exception that gets logged.
async void should be converted to an async Task as well as you should be awaiting Task.Delay instead of invoking Thread.Sleep. If going async you need to go async all the way
static async Task Receive(ITdlMessageReceiver receiver, ILogger logger) {
while (true) {
var message = await receiver.ReceiveAsync<TdlMessage<object>>(topic, entity);
if (message != null) {
logger.LogDebug($"Message received. Topic: {topic}. Action: {Enum.GetName(typeof(TopicActions), message.Action)}. Message: {JsonConvert.SerializeObject(message)}.");
}
await Task.Delay(sleepTime);
}
}
Try making the code async all the way through, yes, but as a console application (single thread) you will be allowed to call Wait() on the Receive method in Main as it is not mixing calls that would cause problem with the async flow.
public static void Main(string[] args) {
//...
//...
//...
Receive(receiver, logger).Wait();
}
Reference Async/Await - Best Practices in Asynchronous Programming
In my service I currently have a few tasks and a ReportProgress method that continually updates a List. How can I return that list to my client host application?
Service side:
public async void BeginSync(string dbId)
{
var progressIndicator = new Progress<string>(ReportSyncProgress);
var output = await BeginSyncTaskAsync(dbId, progressIndicator);
}
...within the task I have a progress Report in a loop:
while ((output = process.StandardOutput.ReadLine()) != null)
{
progress.Report(output);
}
...and here is my report method:
public void ReportSyncProgress(string value)
{
// report by appending to global list
progressOutput.Add(value);
}
progressOutput is a List and I need my client to receive that in real time as it is updated.
Thank you!
Because Rest services don't have sessions you can't make normal WCF callback method. Instead what you will need to do is pass in some kind of token and then use that token to get the progress information.
private static ConcurrentDictionary<Guid, ConcurrentQueue<string>> _progressInfo;
//You should never do "async void" WCF can handle using tasks and having Async suffixes.
//see https://blogs.msdn.microsoft.com/endpoint/2010/11/12/simplified-asynchronous-programming-model-in-wcf-with-asyncawait/
public async Task BeginSyncAsync(string dbId, Guid progressKey)
{
if (!_progressInfo.TryAdd(progressKey, new ConcurrentQueue<string>()))
{
throw new InvalidOperationException("progress key is in use");
}
var progressIndicator = new Progress<string>((value) => ReportSyncProgress(value, progressKey));
try
{
var output = await BeginSyncTaskAsync(dbId, progressIndicator);
}
finally
{
//Remove progress list
ConcurrentQueue<string> temp;
_progressInfo.TryRemove(progressKey, out temp);
}
}
public List<string> GetSyncProgress(Guid progressKey)
{
ConcurrentQueue<string> progressOutput;
if (!_progressInfo.TryGetValue(progressKey, out progressOutput))
{
//the key did not exist, retun null;
return null;
}
//transform the queue to a list and return it.
return progressOutput.ToList();
}
private void ReportSyncProgress(string value, Guid progressKey)
{
ConcurrentQueue<string> progressOutput;
if (!_progressInfo.TryGetValue(progressKey, out progressOutput))
{
//the key did not exist, progress is being reported for a completed item... odd.
return;
}
//This is the requests specific queue of output.
progressOutput.Enqueue(value);
}
I expect there's probably a simple fix for this but I just can't see it.
I'm trying to insert data into an Azure Mobile Services DB from a C# Console program. However when the program is run from within VS (via F5), the data is not being inserted nor is an exception being thrown (that I can see) during the regular course of running the program. When I set a breakpoint to the await dataModel.InsertAsync(data) line and run that in the Immediate Window it throws a ThreadAbortException. Any help is appreciated.
Namespace TestApp {
class Program
{
public static MobileServiceClient MobileService = new MobileServiceClient(
"https://x.azure-mobile.net/",
"API key");
public static IMobileServiceTable<performance> dataModel = Program.MobileService.GetTable<performance>();
static void Main(string[] args)
{
try
{
var test = new performance("http://www.example.com");
var x = InsertItem(test);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.StackTrace);
}
}
static public async Task InsertItem(performance data)
{
await dataModel.InsertAsync(data).ConfigureAwait(false);
}
}
class performance
{
[JsonProperty(PropertyName = "id")]
string Id { get; set; }
[JsonProperty(PropertyName = "uri")]
string Uri { get; set; }
public performance(string uri)
{
Uri = uri;
}
}
}
Your problem comes from the fact that var x = InsertItem(test); is a non blocking call. When you get to await dataModel.InsertAsync(data).ConfigureAwait(false); the function InsertItem immediately returns with a Task.
Normally the correct approach would be do await InsertItem(test); however because your code is being called from Main you can't make the function async. So for this console application (it would not be the correct choice if running in WinForms or WPF app) You need to put a x.Wait() before the end of your try-catch block.
static void Main(string[] args)
{
try
{
var test = new performance("http://www.example.com");
var x = InsertItem(test);
//This makes the program wait for the returned Task to complete before continuing.
x.Wait();
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.StackTrace);
}
}
However if you where to do this in a WPF or WinForms app you would just make the calling function (Assuming the function was a event) async.
private async void Button1_OnClick(object sender, EventArgs e)
{
try
{
var test = new performance("http://www.example.com");
//The code now waits here for the function to finish.
await InsertItem(test);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.StackTrace);
}
}
Do not do async void function calls unless you are in a event delegate function
I created a small test to (somewhat) simulate what you're doing. When the awaited task in InsertItem takes very little or no time at all, the task returned by the var x = InsertItem(test) line returns a task in the RanToCompletion state and the debugger acts as expected.
However, when I make the awaited task do something substantial, such as Thread.Sleep(5000), then I get the behavior you're describing and the task returned by the var x = InsertItem(test) line returns a task in the WaitingForActivation state.
When I put Task.WaitAll(x) following the var x = InsertItem(test) line, then I get the behavior that I think we both expect and x.Status is RanToCompletion.