I've started using ETW and the out-of-process Semantic Logging Block from Entlib 6. When I use async/await, the CurrentThreadActivityId is not set on the continuation thread and the TPL framework does not log a transfer event. This makes it extremely difficult to perform end-to-end tracing.
According to Microsoft's documentation, the TPL framework is supposed to log a transfer event and generate a new activityid, but I can't seem to get it to work.
Documentation from Entlib 6
Here's a small example showing the problem:
To log TPL events, I'm using the following:
<eventSource name="System.Threading.Tasks.TplEventSource" level="Informational" matchAnyKeyword="1"/>
And here my test code:
using System;
using System.Collections.Generic;
using System.Diagnostics.Tracing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace AsyncContextTest
{
class Program
{
static Guid _activityId = Guid.NewGuid();
static void Main(string[] args)
{
EventSource.SetCurrentThreadActivityId(_activityId);
Console.WriteLine(EventSource.CurrentThreadActivityId);
Task t = Task.Run(async () => await DoStuffAsync());
Console.WriteLine(EventSource.CurrentThreadActivityId);
Console.Read();
}
public static async Task DoStuffAsync()
{
var x = "one,two,three".Split(',');
Console.WriteLine(EventSource.CurrentThreadActivityId);
await Task.Delay(1000);
Console.WriteLine(EventSource.CurrentThreadActivityId);
var y = String.Join(",", x);
Console.WriteLine("Done");
}
}
}
Results
334540cc-ccb1-4196-8587-815abf237e4c
334540cc-ccb1-4196-8587-815abf237e4c
00000000-0000-0000-0000-000000000000
00000000-0000-0000-0000-000000000000
Done
Does anyone have a simple example showing the proper way to do end to end tracing with ETW and async/await?
EDIT:
I was able to get this working properly with an in-process listener, but not with the out-of-process listener. Nothing is logged from TPL.
Change the configuration to use the event source's GUID instead of the name.
<eventSource id="2e5dba47-a3d2-4d16-8ee0-6671ffdcd7b5" level="Informational" />
Output will look like:
605e615a-c849-4ee7-95b8-e6677f945c3f
605e615a-c849-4ee7-95b8-e6677f945c3f
00000001-0001-0000-0c22-0000ffdcd7b5
00000002-0001-0000-0c22-0000ffdcd7b5
Done
Does anyone have a simple example showing the proper way to do end to
end tracing with ETW and async/await?
Still looking...
Related
I'm creating a console-app following this guide: https://learn.microsoft.com/en-us/dotnet/core/extensions/configuration
Do I put my Main method where it says "// Application code should start here.
"?
Do I use a Main method, or how do I solve this?
My existing structure:
namespace MyNameSpace
{
public class Program
{
public static async Task Main(string[] args)
{
//application logic......
}
}
}
You don't need to create an explicit Main method. C# top-level statements mean that the three lines of code from the tutorial:
using Microsoft.Extensions.Hosting;
using IHost host = Host.CreateDefaultBuilder(args).Build();
await host.RunAsync();
... already form a valid program. It's roughly equivalent to:
using Microsoft.Extensions.Hosting;
class Program
{
static async Task Main(string[] args)
{
using IHost host = Host.CreateDefaultBuilder(args).Build();
await host.RunAsync();
}
}
I'd encourage you to use the existing top-level statement approach unless you need more complexity in your initialization code... although you can write the class and Main method declarations explicitly if you want to.
Later examples in the same tutorial show larger amounts of application startup code, still using top-level statements. Or you can just create a new ASP.NET Core application which will show more.
I understand that .NET 4.5 comes with a bunch of features to make asynchronous database operations easier to implement. MSDN says that if the connection string is not set to work asynchronously none of the async methods of ADO.NET will work in an asynchronous way. Therefore SqlConnectionStringBuilder comes with a property called AsynchronousProcessing.
I am wondering if these async features will work with other database servers (e.g. mysql) as well? What should I do to make aync work with a no-SQL database that is not recognized by .NET? (e.g. RavenDB)?
The asynchronous methods for all drivers are defined in DbDataReader, eg DbDataReader.ReadAsync. It is up to the specific drivers to override these methods with specific implementations to take advantage of the asynchronous characteristics of each database and use eg. a naturally asynchronous operation instead of a synchronous operation wrapped in a thread.
That said, MySQL Connector/Net 6.8 adds support for asynchronous operations in Entity Framework 6 but the MySqlDataReader class does NOT provide a ReadAsync method. This is because Connector uses an old architecture (pre-2.0), implementing the IDataReader interface instead of deriving from the generic DbDataReader class introduced in .NET 2.0.
As Panagiotis Kanavos mentioned DbDataReader provides ReadAsync method signatures, however not all drivers support this. Some, like the MySql 6.9.5 driver, implement them synchronously.
To answer your more general question, if a (No)SQL driver does NOT inherently support *Async methods that are awaitable, but it does have "APM" IAsyncResult based methods (e.g. BeginRead.. EndRead...), then you can wrap those up using Task.Factory.FromAsync. Here is an example for MySql:-
public static class MySqlCommandExtension
{
public static Task<MySqlDataReader> MyExecuteReaderAsync(this MySqlCommand source, CommandBehavior behavior = CommandBehavior.Default)
{
return Task<MySqlDataReader>.Factory.FromAsync(source.BeginExecuteReader(behavior), source.EndExecuteReader);
}
}
This pattern is descibed in more detail on MSDN.
I was into the same problem today and I made a Console Application to test whether it's working or not.
Seems to me it's not working for my MySQL connector 6.9.4.
using System;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace MySQLTest
{
class Program
{
static void Main(string[] args)
{
Service service = new Service();
var task1 = service.GetCountries("1");
var task2 = service.GetCountries("2");
var task3 = service.GetCountries("3");
Console.WriteLine("bö");
Console.ReadLine();
}
}
public class Service
{
public async Task<List<country>> GetCountries(string param)
{
Console.WriteLine(String.Format("{0} started.", param));
using (worldEntities context = new worldEntities())
{
Console.WriteLine(String.Format("{0} awaiting.", param));
List<country> countries = await context.country.ToListAsync();
Console.WriteLine(String.Format("{0} done.", param));
return new List<country>();
}
}
}
}
This one outputs,
And When I change
List countries = await context.country.ToListAsync();
to
await Task.Delay(5000);
it outpus :
after 5 seconds.
So I say it's not supported yet.
I understand that .NET 4.5 comes with a bunch of features to make asynchronous database operations easier to implement. MSDN says that if the connection string is not set to work asynchronously none of the async methods of ADO.NET will work in an asynchronous way. Therefore SqlConnectionStringBuilder comes with a property called AsynchronousProcessing.
I am wondering if these async features will work with other database servers (e.g. mysql) as well? What should I do to make aync work with a no-SQL database that is not recognized by .NET? (e.g. RavenDB)?
The asynchronous methods for all drivers are defined in DbDataReader, eg DbDataReader.ReadAsync. It is up to the specific drivers to override these methods with specific implementations to take advantage of the asynchronous characteristics of each database and use eg. a naturally asynchronous operation instead of a synchronous operation wrapped in a thread.
That said, MySQL Connector/Net 6.8 adds support for asynchronous operations in Entity Framework 6 but the MySqlDataReader class does NOT provide a ReadAsync method. This is because Connector uses an old architecture (pre-2.0), implementing the IDataReader interface instead of deriving from the generic DbDataReader class introduced in .NET 2.0.
As Panagiotis Kanavos mentioned DbDataReader provides ReadAsync method signatures, however not all drivers support this. Some, like the MySql 6.9.5 driver, implement them synchronously.
To answer your more general question, if a (No)SQL driver does NOT inherently support *Async methods that are awaitable, but it does have "APM" IAsyncResult based methods (e.g. BeginRead.. EndRead...), then you can wrap those up using Task.Factory.FromAsync. Here is an example for MySql:-
public static class MySqlCommandExtension
{
public static Task<MySqlDataReader> MyExecuteReaderAsync(this MySqlCommand source, CommandBehavior behavior = CommandBehavior.Default)
{
return Task<MySqlDataReader>.Factory.FromAsync(source.BeginExecuteReader(behavior), source.EndExecuteReader);
}
}
This pattern is descibed in more detail on MSDN.
I was into the same problem today and I made a Console Application to test whether it's working or not.
Seems to me it's not working for my MySQL connector 6.9.4.
using System;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace MySQLTest
{
class Program
{
static void Main(string[] args)
{
Service service = new Service();
var task1 = service.GetCountries("1");
var task2 = service.GetCountries("2");
var task3 = service.GetCountries("3");
Console.WriteLine("bö");
Console.ReadLine();
}
}
public class Service
{
public async Task<List<country>> GetCountries(string param)
{
Console.WriteLine(String.Format("{0} started.", param));
using (worldEntities context = new worldEntities())
{
Console.WriteLine(String.Format("{0} awaiting.", param));
List<country> countries = await context.country.ToListAsync();
Console.WriteLine(String.Format("{0} done.", param));
return new List<country>();
}
}
}
}
This one outputs,
And When I change
List countries = await context.country.ToListAsync();
to
await Task.Delay(5000);
it outpus :
after 5 seconds.
So I say it's not supported yet.
I'm working on a project where I'll have one application in C# and another one in Python.
The C# application will continuously analyse stream of data and raise a flag each time something interesting is detected. So each time there will be an event, my Python application will have to read it and continues with it own process, while other flags will continue being sent.
As you understand the C# app won't wait for the Python one to finish its computation before sending another flag.
So I was wondering if it was possible to create a sub/pub (C# being the Publisher, and Python the Subscriber), if yes how can I do it, and do you think it's a good idea?
I'm pretty new in this field, so could you tell me if there are other possibilities?
Thx for your help.
Redis pub/sub is awesome... or ZeroMQ.
Simplest way is PIPE communication. another simple way that not suggested is SOCKET programming. Pipes and Named pipes are good solution to communicate between different processes (over different programming languages). SOCKET programming is like this but may need more Access Level and may be less security.
other type of IPCs seems be unusable.
see for more info:
C# - Pipes
python - Pipes
You may use our aktos-dcs and aktos-dcs-cs libraries. I have successfully used these libraries in production in order to make an RFID Reader (from Impinj) communicate with (actually, integrated in) our telemetry system. The RFID reader has a C# API and we are heavily using Python in our telemetry system.
The simplest test case is a pinger-ponger application, and here is how it looks like with these libraries:
pinger.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using aktos_dcs_cs;
namespace pinger
{
class Pinger : Actor
{
public void handle_PingMessage(Dictionary<string, object> msg)
{
Console.WriteLine("Pinger handled PingMessage: {0} ", msg["text"]);
string msg_ser = #"
{""PongMessage"":
{""text"": ""this is proper message from csharp implementation""}
}
";
System.Threading.Thread.Sleep(2000);
send(msg_ser);
}
}
class Program
{
static void Main(string[] args)
{
Pinger x = new Pinger();
Actor.wait_all();
}
}
}
ponger.py:
from aktos_dcs import *
class Pinger(Actor):
def handle_PingMessage(self, msg_raw):
msg = get_msg_body(msg_raw)
print "Pinger got ping message: ", msg['text'], (time.time() - msg_raw['timestamp'])
sleep(2)
self.send({'PongMessage': {'text': "Hello ponger, this is pinger 1!"}})
if __name__ == "__main__":
ProxyActor()
pinger = Pinger()
pinger.send({'PongMessage': {'text': "Hello ponger, this is STARTUP MESSAGE!"}})
wait_all()
Just out of personal interest, I decided to try to write a basic web server using Reactive Extensions (aka Rx). Almost immediately, I came up against the following problem:
HttpListener's async pattern doesn't match the normal async pattern signature, the EndGetContext returns a HttpListenerContext instead of an IAsyncResult!
So my question is: is there a way around this?
Here's some code - the compiler (rightly) complains about the second argument to Observable.FromAsyncPattern:
Update - As Jon rightly pointed out, it wasn't the pattern, it's complaining that I haven't provided the return type - this works:
var observable = Observable.FromAsyncPattern<HttpListenerContext>(listener.BeginGetContext, listener.EndGetContext);
The original code:
using System;
using System.Net;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace RxWebTest
{
class Program
{
static void Main(string[] args)
{
HttpListener listener = new HttpListener();
try
{
listener.Start();
var observable = Observable.FromAsyncPattern(listener.BeginGetContext, listener.EndGetContext);
Console.ReadLine();
}
finally
{
listener.Abort();
}
}
}
}
That is the normal pattern. It's BeginGetContext which returns IAsyncResult; EndGetContext takes an IAsyncResult and returns the result of the asynchronous operation - the context. That's exactly the same as (say) Stream.BeginRead/EndRead, where EndRead returns an int, the number of bytes read.
What happens when you try the code you've posted? I can't immediately see why it wouldn't work.
The trick to figuring out the template parameters for FromAsyncPattern, is to imagine you're creating a Func that is matching the synchronous version of the function.
In this case, the prototype is:
public HttpListenerContext GetContext()
Which means, my Func would look like:
Func<HttpListenerContext>
Then, just take your template parameter and copy-paste it to the FromAsyncPattern.