How to get a simple stream of string using ServiceStack Grpc? - c#

fighting with the ServiceStack library since a while to get a basic "stream" of string to work in C#.
In short, I'm trying to replicate the basic example from "native" gRPC.
Proto buf
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (stream HelloReply);
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings.
message HelloReply {
string message = 1;
}
Server
public override async Task SayHello(HelloRequest request, IServerStreamWriter<HelloReply> responseStream, ServerCallContext context)
{
foreach (var x in Enumerable.Range(1, 10))
{
await responseStream.WriteAsync(new HelloReply
{
Message = $"Hello {request.Name} {x}"
});
await Task.Delay(200);
}
}
Client
var replies = client.SayHello(new HelloRequest { Name = "Laurent" });
await foreach (var reply in replies.ResponseStream.ReadAllAsync())
{
Console.WriteLine(reply.Message);
}
Then with the ServiceStack library, I'm not able to get the server piece done correctly. I always get a message telling me my function 'SayHello' isn't defined.
Let me know, thx !

ServiceStack gRPC implementation adopts a code-first implementation where your existing ServiceStack Services can be called from gRPC endpoints.
So instead of manually authoring a .proto file you would instead create Services using standard Request / Response DTOs and Service implementation for normal Request/Reply gRPC Services.
For Server Stream gRPC Services you would need to implement IStreamService interface in addition to inheriting from ServiceStack's Service base class.
An example of this is covered in Implementing Server Stream Services in the docs:
public class StreamFileService : Service, IStreamService<StreamFiles,FileContent>
{
public async IAsyncEnumerable<FileContent> Stream(StreamFiles request,
CancellationToken cancel = default)
{
var i = 0;
var paths = request.Paths ?? TypeConstants.EmptyStringList;
while (!cancel.IsCancellationRequested)
{
var file = VirtualFileSources.GetFile(paths[i]);
var bytes = file?.GetBytesContentsAsBytes();
var to = file != null
? new FileContent {
Name = file.Name,
Type = MimeTypes.GetMimeType(file.Extension),
Body = bytes,
Length = bytes.Length,
}
: new FileContent {
Name = paths[i],
ResponseStatus = new ResponseStatus {
ErrorCode = nameof(HttpStatusCode.NotFound),
Message = "File does not exist",
}
};
yield return to;
if (++i >= paths.Count)
yield break;
}
}
}
You would also need to register your Stream Service implementation in RegisterServices:
Plugins.Add(new GrpcFeature(App) {
RegisterServices = {
typeof(StreamFileService)
}
});
If you're using the smart C# generic gRPC Service Client you can avoid .proto descriptors and protoc generated classes entirely as you can reuse the Server DTOs in your ServiceModel project to enable an end-to-end API without code-gen:
var request = new StreamFiles {
Paths = new List<string> {
"/js/ss-utils.js",
"/js/hot-loader.js",
"/js/not-exists.js",
"/js/hot-fileloader.js",
}
};
var files = new List<FileContent>();
await foreach (var file in client.StreamAsync(request))
{
files.Add(file);
}
An alternative to sharing your ServiceModel.dll you can use C# Add ServiceStack Reference to generate your C# DTOs on the client.
For protoc generated clients you can use the x dotnet tool to Generate protoc Dart gRPC Client
$ x proto-dart https://todoworld.servicestack.net -out lib
Where you can use the serverStreamFiles API stubs to invoke the server stream Service:
var stream = client.serverStreamFiles(StreamFiles()..paths.addAll([
'/js/ss-utils.js',
'/js/hot-loader.js',
'/js/hot-fileloader.js',
]));
await for (var file in stream) {
var text = utf8.decode(file.body);
print('FILE ${file.name} (${file.length}): ${text.substring(0, text.length < 50 ? text.length : 50)} ...');
}
The todo-world/clients repo contains a number of gRPC test examples in different langauges.

In the end, here's what I end up doing as a simple POC of a stream, if this can help anyone else. No more proto-file either !
Client Program.cs :
private async static Task GetBotStream()
{
var res = client.StreamAsync(new BotStreamRequest { });
await foreach (var textReceived in res)
{
Console.WriteLine(textReceived.Result);
}
}
Client dtos.cs
[DataContract]
public partial class BotStreamRequest : IReturn<BotStreamReply>
{
}
[DataContract]
public partial class BotStreamReply
{
[DataMember(Order = 1)]
public virtual string Result { get; set; }
[DataMember(Order = 2)]
public virtual ResponseStatus ResponseStatus { get; set; }
}
Server Program.cs
public async IAsyncEnumerable<BotStreamReply> Stream(BotStreamRequest request, [EnumeratorCancellation]CancellationToken cancel = default)
{
foreach (var x in Enumerable.Range(1, 10))
{
yield return new BotStreamReply { Result = $"My stream {x}" };
}
}

Related

C# .NET Core 3.1 Web API Post parameter is Null

I am trying to make a post request from WPF to Web API using the following code but the request parameter is always null.
Request Model
public class Document
{
public string FileName { get; set; }
public byte[] Buffer { get; set; }
}
public class Request
{
public string Uploader { get; set; }
public List<Document> Documents { get; set; }
}
WPF Client
var obj = new Request()
{
Uploader = "John Doe",
Documents = new List<Document>
{
new Document()
{
FileName ="I Love Coding.pdf",
Buffer = System.IO.File.ReadAllBytes(#"C:\Users\john.doe\Downloads\I Love Coding.pdf.pdf")
}
}
};
using (var http = new HttpClient())
{
var encodedJson = JsonConvert.SerializeObject(obj);
var conent = new StringContent(encodedJson, Encoding.UTF8, "application/json");
HttpResponseMessage response = await http.PostAsync("https://my-app.com/api/upload", conent);
response.EnsureSuccessStatusCode();
}
Web API
[Route("")]
public class AppController : ControllerBase
{
[HttpPost]
[Route("api/upload")]
public async Task<IActionResult> UploadDocumentsAsync([FromBody] Request request)
{
// request is always null when app is running in production
// https://my-app.com/api/upload
//request is not null when running on https://localhost:8080/api/upload
}
}
Please what am I missing in the above implementation?
The request parameter is not null on localhost but always null in production.
Please what am I missing in the above implementation? The request
parameter is not null on localhost but always null in production.
Well, not sure how are getting data on local server becuse, you are sending MultipartFormData means your POCO object and file buffer. As you may know we can send json object in FromBody but not the files as json. Thus, I am not sure how it working in local and getting null data is logical in IIS Or Azure.
what am I missing in the above implementation?
As explained above, for sending both POCO object and Files as byte or steam we need to use FromForm and beside that, we need to bind our request object as MultipartFormDataContent to resolve your null data on your UploadDocumentsAsync API action.
Required Change For Solution:
WPF:
In your WPF http request please update your request code snippet as following:
var obj = new Request()
{
Uploader = "John Doe",
Documents = new List<Document>
{
new Document()
{
FileName ="I Love Coding.pdf",
Buffer = System.IO.File.ReadAllBytes(#"YourFilePath")
}
}
};
var httpClient = new HttpClient
{
BaseAddress = new("https://YourServerURL")
};
var formContent = new MultipartFormDataContent();
formContent.Add(new StringContent(obj.Uploader), "Uploader");
formContent.Add(new StringContent(obj.Documents[0].FileName), "Documents[0].FileName");
formContent.Add(new StreamContent(new MemoryStream(obj.Documents[0].Buffer)), "Documents[0].Buffer", obj.Documents[0].FileName);
var response = await httpClient.PostAsync("/api/upload", formContent);
if (response.IsSuccessStatusCode)
{
var responseFromAzureIIS = await response.Content.ReadAsStringAsync();
}
Note: Class in WPF side would remain same as before. No changes required.
Asp.net Core Web API:
In asp.net core web API side you should use [FromForm] instead of [FromBody]
So your controller Action would as following:
[Route("")]
public class AppController : ControllerBase
{
[HttpPost]
[Route("api/upload")]
public async Task<IActionResult> UploadDocumentsAsync([FromForm] Request file)
{
if (file.Documents[0].Buffer == null)
{
return Ok("Null File");
}
return Ok("File Received");
}
}
Note: For remote debugging I have checked the logs and for double check I have used a simple conditionals whether file.Documents[0].Buffer == null. I have tested both in local, IIS and Azure and working accordingly.
Update POCO Class in API Project:
For buffer you have used byte for your WPF project but for Web API project update that to IFormFile instead of byte. It should be as following:
public class Document
{
public string FileName { get; set; }
public IFormFile Buffer { get; set; }
}
public class Request
{
public string Uploader { get; set; }
public List<Document> Documents { get; set; }
}
Output:
If you would like to know more details on it you could check our official document here

Sending request reponse message on Artemis using C#

I am trying to implement a request response pattern in C# with the ArtemisNetClient, but having a bit of trouble finding out how to do so in a more generic way in a real solution.
I was able to do something like this in two console applications based on some Java examples:
Sender
static async System.Threading.Tasks.Task Main(string[] args)
{
var connectionFactory = new ConnectionFactory();
var endpoint = Endpoint.Create("localhost", 5672, "guest", "guest");
var connection = await connectionFactory.CreateAsync(endpoint);
string guid = new Guid().ToString();
var requestAddress = "TRADE REQ1";
var responseAddress = "TRADE RESP";
Message message = new Message("BUY AMD 1000 SHARES");
message.SetCorrelationId(guid);
message.ReplyTo = responseAddress;
var producer = await connection.CreateProducerAsync(requestAddress, RoutingType.Anycast);
await producer.SendAsync(message);
var consumer = await connection.CreateConsumerAsync(responseAddress, RoutingType.Anycast);
var responseMessage = await consumer.ReceiveAsync();
Console.WriteLine(responseMessage.GetBody<string>());
}
Receiver
static async System.Threading.Tasks.Task Main(string[] args)
{
// Create connection
var connectionFactory = new ConnectionFactory();
var endpoint = Endpoint.Create("localhost", 5672, "guest", "guest");
var connection = await connectionFactory.CreateAsync(endpoint);
var requestAddress = "TRADE REQ1";
// Create consumer to receive trade request messages
var consumer = await connection.CreateConsumerAsync(requestAddress, RoutingType.Anycast);
var message = await consumer.ReceiveAsync();
Console.WriteLine($"Received message: {message.GetBody<string>()}");
// Confirm trade request and ssend response message
if (!string.IsNullOrEmpty(message.ReplyTo))
{
Message responseMessage = new Message("Confirmed trade request");
responseMessage.SetCorrelationId(message.CorrelationId);
var producer = await connection.CreateProducerAsync(message.ReplyTo);
await producer.SendAsync(responseMessage);
}
}
This worked as expected, but I'd like to have something more down the line of what is described in this article, except it doesn't have any examples of a request response pattern.
To elaborate, I currently have two services that I want to communicate across.
In service 1 I want to create and publish a message and then wait for a response to enrich the instance object and save it to a database. I currently have this, but it lacks the await response message.
public async Task<Instance> CreateInstance(Instance instance)
{
await _instanceCollection.InsertOneAsync(instance);
var #event = new InstanceCreated
{
Id = instance.Id,
SiteUrl = instance.SiteUrl
};
await _messageProducer.PublishAsync(#event);
return instance;
}
I figured I might need to setup a temporary queue/connection or something in the PublishAsync() and change it to e.g. Task<Message> to support returning a response message. But how would I go about doing that? Would I have to do a new connectionfactory + CreateConsumerAsync etc. like in the console app example?
public class MessageProducer
{
private readonly IAnonymousProducer _producer;
public MessageProducer(IAnonymousProducer producer)
{
_producer = producer;
}
public async Task PublishAsync<T>(T message, string replyTo = null, string correlationId = null)
{
var serialized = JsonSerializer.Serialize(message);
var address = typeof(T).Name;
var msg = new Message(serialized);
if (replyTo != null && correlationId != null)
{
msg.CorrelationId = correlationId;
msg.ReplyTo = replyTo;
}
await _producer.SendAsync(address, msg);
}
public async Task PublishAsync<T>(T message, string routeName, string replyTo = null, string correlationId = null)
{
var serialized = JsonSerializer.Serialize(message);
var address = routeName;
var msg = new Message(serialized);
if(replyTo != null && correlationId != null)
{
msg.CorrelationId = correlationId;
msg.ReplyTo = replyTo;
}
await _producer.SendAsync(address, msg);
}
}
In Service 2 I have a InstanceCreatedConsumer which receives messages, but again it lacks a way to return response messages.
public class InstanceCreatedConsumer : ITypedConsumer<InstanceCreated>
{
private readonly MessageProducer _messageProducer;
public InstanceCreatedConsumer(MessageProducer messageProducer)
{
_messageProducer = messageProducer;
}
public async Task ConsumeAsync(InstanceCreated message, CancellationToken cancellationToken)
{
// consume message and return response
}
}
I figured I might be able to extend the ActiveMqExtensions class with a ConsumeAsync and HandleMessage that handles the response message with a return value, but I haven't gotten as far yet.
public static IActiveMqBuilder AddTypedConsumer<TMessage, TConsumer>(this IActiveMqBuilder builder,
RoutingType routingType)
where TConsumer : class, ITypedConsumer<TMessage>
{
builder.Services.TryAddScoped<TConsumer>();
builder.AddConsumer(typeof(TMessage).Name, routingType, HandleMessage<TMessage, TConsumer>);
return builder;
}
private static async Task HandleMessage<TMessage, TConsumer>(Message message, IConsumer consumer, IServiceProvider serviceProvider, CancellationToken token)
where TConsumer : class, ITypedConsumer<TMessage>
{
try
{
var msg = JsonConvert.DeserializeObject<TMessage>(message.GetBody<string>());
using var scope = serviceProvider.CreateScope();
var typedConsumer = scope.ServiceProvider.GetService<TConsumer>();
await typedConsumer.ConsumeAsync(msg, token);
await consumer.AcceptAsync(message);
}
catch(Exception ex)
{
// todo
}
}
Am I totally wrong in what I am trying to achieve here, or is it just not possible with the ArtemisNetClient?
Maybe someone has an example or can confirm whether I am down the right path, or maybe I should be using a different framework.
I am new to this kind of communication through messages like ActiveMQ Artemis, so any guidance is appreciated.
I don't see anything in the ArtemisNetClient that would simplify the request/response pattern from your application's point of view. One might expect something akin to JMS' QueueRequestor, but I don't see anything like that in the code, and I don't see anything like that listed in the documentation.
I recommend you simply do in your application what you did in your example (i.e. manually create the consumer & producer to deal with the responses on each end respectively). The only change I would recommend is to re-use connections so you create as few as possible. A connection pool would be ideal here.
For what it's worth, it looks to me like the first release of ArtemisNetClient was just 3 months ago and according to GitHub all but 2 of the commits to the code-base came from one developer. ArtemisNetClient may grow into a very successful C# client implementation, but at this point it seems relatively immature. Even if the existing code is high quality if there isn't a solid community around the client then chances are it won't have the support necessary to get timely bug fixes, new features, etc. Only time will tell.
With version 2.7.0 ArtemisNetClient introduces IRequestReplyClient interface that can be used to implement a request-response messaging pattern. With ArtemisNetClient.Extensions.DependencyInjection this may look as follows:
Client Side:
First you need to register your typed request-reply client in DI:
public void ConfigureServices(IServiceCollection services)
{
/*...*/
var endpoints = new[] { Endpoint.Create(host: "localhost", port: 5672, "guest", "guest") };
services.AddActiveMq("bookstore-cluster", endpoints)
.AddRequestReplyClient<MyRequestReplyClient>();
/*...*/
}
MyRequestReplyClient is your custom class that expects the IRequestReplyClient to be injected via the constructor. Once you have your custom class, you can either expose the IRequestReplyClient directly or encapsulate sending logic inside of it:
public class MyRequestReplyClient
{
private readonly IRequestReplyClient _requestReplyClient;
public MyRequestReplyClient(IRequestReplyClient requestReplyClient)
{
_requestReplyClient = requestReplyClient;
}
public async Task<TResponse> SendAsync<TRequest, TResponse>(TRequest request, CancellationToken cancellationToken)
{
var serialized = JsonSerializer.Serialize(request);
var address = typeof(TRequest).Name;
var msg = new Message(serialized);
var response = await _requestReplyClient.SendAsync(address, msg, cancellationToken);
return JsonSerializer.Deserialize<TResponse>(response.GetBody<string>());
}
}
That's it regarding the client-side.
Worker side
To implement the worker side you can (as you suggested), change the ITypedConsumer interface to return the message that would be sent back, or you can provide the additional data (ReplyTo and CorrelationId headers) so you can send the response back as part of your consumer logic. I prefer the latter as it's a more flexible option in my opinion.
Modified ITypedConsumer might look like that:
public interface ITypedConsumer<in T>
{
public Task ConsumeAsync(T message, MessageContext context, CancellationToken cancellationToken);
}
Where MessageContext is just a simple dto:
public class MessageContext
{
public string ReplyTo { get; init; }
public string CorrelationId { get; init; }
}
HandleMessage extension method:
private static async Task HandleMessage<TMessage, TConsumer>(Message message, IConsumer consumer, IServiceProvider serviceProvider, CancellationToken token)
where TConsumer : class, ITypedConsumer<TMessage>
{
var msg = JsonSerializer.Deserialize<TMessage>(message.GetBody<string>());
using var scope = serviceProvider.CreateScope();
var typedConsumer = scope.ServiceProvider.GetService<TConsumer>();
var messageContext = new MessageContext
{
ReplyTo = message.ReplyTo,
CorrelationId = message.CorrelationId
};
await typedConsumer.ConsumeAsync(msg, messageContext, token);
await consumer.AcceptAsync(message);
}
MessageProducer has to be slightly changed as well, so you can explicitly pass address and CorrelationId:
public class MessageProducer
{
private readonly IAnonymousProducer _producer;
public MessageProducer(IAnonymousProducer producer)
{
_producer = producer;
}
public async Task PublishAsync<T>(string address, T message, MessageContext context, CancellationToken cancellationToken)
{
var serialized = JsonSerializer.Serialize(message);
var msg = new Message(serialized);
if (!string.IsNullOrEmpty(context.CorrelationId))
{
msg.CorrelationId = context.CorrelationId;
}
await _producer.SendAsync(address, msg, cancellationToken);
}
}
And finally, the exemplary consumer could work like that:
public class CreateBookConsumer : ITypedConsumer<CreateBook>
{
private readonly MessageProducer _messageProducer;
public CreateBookConsumer(MessageProducer messageProducer)
{
_messageProducer = messageProducer;
}
public async Task ConsumeAsync(CreateBook message, MessageContext context, CancellationToken cancellationToken)
{
var #event = new BookCreated
{
Id = Guid.NewGuid(),
Title = message.Title,
Author = message.Author,
Cost = message.Cost,
InventoryAmount = message.InventoryAmount,
UserId = message.UserId,
Timestamp = DateTime.UtcNow
};
await _messageProducer.PublishAsync(context.ReplyTo, #event, new MessageContext
{
CorrelationId = context.CorrelationId
}, cancellationToken);
}
}

Can't get a valid response from a REST web service using System.Net.Http.HttpClient

I am using this test method (and helper class) to verify the response from an external web service:
[TestMethod]
public void WebServiceReturnsSuccessResponse()
{
using (var provider = new Provider(new Info()))
using (var result = provider.GetHttpResponseMessage())
{
Assert.IsTrue(result.IsSuccessStatusCode);
}
}
private class Info : IInfo
{
public string URL { get; set; } =
"https://notreallythe.website.com:99/service/";
public string User { get; set; } = "somename";
public string Password { get; set; } = "password1";
}
I can't get this test to pass; I always get a 500 - Internal Server Error result. I have connected via an external utility (Postman) - so the web service is up and I can connect with the url & credentials that I have.
I think the problem is in my instantiation of the HttpClient class, but I can't determine where. I am using Basic authentication:
public class Provider : IProvider, IDisposable
{
private readonly HttpClient _httpClient;
public Provider(IInfo config){
if (config == null)
throw new ArgumentNullException(nameof(config));
var userInfo = new UTF8Encoding().GetBytes($"{config.User}:{config.Password}");
_httpClient = new HttpClient
{
BaseAddress = new Uri(config.URL),
DefaultRequestHeaders =
{
Accept = { new MediaTypeWithQualityHeaderValue("application/xml")},
Authorization = new AuthenticationHeaderValue(
"Basic", Convert.ToBase64String(userInfo)),
ExpectContinue = false,
},
};
}
public HttpResponseMessage GetHttpResponseMessage()
{
return _httpClient.GetAsync("1234").Result;
}
}
The response I get back appears to go to the correct endpoint; the RequestUri in the response looks exactly like I expect, https://notreallythe.website.com:99/service/1234.
You need to load up Fiddler and do a recording of the HTTP traffic when this operation succeeds (through the browser).
Then, load up your code, stand up another instance (or window) of Fiddler, and do the same thing with your code. Now, compare the two Fiddler windows to see what is different.
You only need to compare those things in Fiddler that are highlighted in blue. You can ignore the other communications.

View POST request body in Application Insights

Is it possible to view POST request body in Application Insights?
I can see request details, but not the payload being posted in application insights. Do I have to track this with some coding?
I am building a MVC core 1.1 Web Api.
You can simply implement your own Telemetry Initializer:
For example, below an implementation that extracts the payload and adds it as a custom dimension of the request telemetry:
public class RequestBodyInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
var requestTelemetry = telemetry as RequestTelemetry;
if (requestTelemetry != null && (requestTelemetry.HttpMethod == HttpMethod.Post.ToString() || requestTelemetry.HttpMethod == HttpMethod.Put.ToString()))
{
using (var reader = new StreamReader(HttpContext.Current.Request.InputStream))
{
string requestBody = reader.ReadToEnd();
requestTelemetry.Properties.Add("body", requestBody);
}
}
}
}
Then add it to the configuration either by configuration file or via code:
TelemetryConfiguration.Active.TelemetryInitializers.Add(new RequestBodyInitializer());
Then query it in Analytics:
requests | limit 1 | project customDimensions.body
The solution provided by #yonisha is in my opinion the cleanest one available. However you still need to get your HttpContext in there and for that you need some more code. I have also inserted some comments which are based or taken from code examples above. It is important to reset the position of your request else you will lose its data.
This is my solution that I have tested and gives me the jsonbody:
public class RequestBodyInitializer : ITelemetryInitializer
{
readonly IHttpContextAccessor httpContextAccessor;
public RequestBodyInitializer(IHttpContextAccessor httpContextAccessor)
{
this.httpContextAccessor = httpContextAccessor;
}
public void Initialize(ITelemetry telemetry)
{
if (telemetry is RequestTelemetry requestTelemetry)
{
if ((httpContextAccessor.HttpContext.Request.Method == HttpMethods.Post ||
httpContextAccessor.HttpContext.Request.Method == HttpMethods.Put) &&
httpContextAccessor.HttpContext.Request.Body.CanRead)
{
const string jsonBody = "JsonBody";
if (requestTelemetry.Properties.ContainsKey(jsonBody))
{
return;
}
//Allows re-usage of the stream
httpContextAccessor.HttpContext.Request.EnableRewind();
var stream = new StreamReader(httpContextAccessor.HttpContext.Request.Body);
var body = stream.ReadToEnd();
//Reset the stream so data is not lost
httpContextAccessor.HttpContext.Request.Body.Position = 0;
requestTelemetry.Properties.Add(jsonBody, body);
}
}
}
Then also be sure to add this to your Startup -> ConfigureServices
services.AddSingleton<ITelemetryInitializer, RequestBodyInitializer>();
EDIT:
If you also want to get the response body I found it useful to create a piece of middleware (.NET Core, not sure about Framework). At first I took above approach where you log a response and a request but most of the time you want these together:
public async Task Invoke(HttpContext context)
{
var reqBody = await this.GetRequestBodyForTelemetry(context.Request);
var respBody = await this.GetResponseBodyForTelemetry(context);
this.SendDataToTelemetryLog(reqBody, respBody, context);
}
This awaits both a request and a response. GetRequestBodyForTelemetry is almost identical to the code from the telemetry initializer, except using Task. For the response body I have used the code below, I also excluded a 204 since that leads to a nullref:
public async Task<string> GetResponseBodyForTelemetry(HttpContext context)
{
var originalBody = context.Response.Body;
try
{
using (var memStream = new MemoryStream())
{
context.Response.Body = memStream;
//await the responsebody
await next(context);
if (context.Response.StatusCode == 204)
{
return null;
}
memStream.Position = 0;
var responseBody = new StreamReader(memStream).ReadToEnd();
//make sure to reset the position so the actual body is still available for the client
memStream.Position = 0;
await memStream.CopyToAsync(originalBody);
return responseBody;
}
}
finally
{
context.Response.Body = originalBody;
}
}
Few days back, I got a similar requirement to log the request Body in Application insights with filtering out sensitive input user data from the payload. So sharing my solution. The below solution is developed for ASP.NET Core 2.0 Web API.
ActionFilterAttribute
I've used ActionFilterAttribute from (Microsoft.AspNetCore.Mvc.Filters namespace) which provides the Model via ActionArgument so that by reflection, those properties can be extracted which are marked as sensitive.
public class LogActionFilterAttribute : ActionFilterAttribute
{
private readonly IHttpContextAccessor httpContextAccessor;
public LogActionFilterAttribute(IHttpContextAccessor httpContextAccessor)
{
this.httpContextAccessor = httpContextAccessor;
}
public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
if (context.HttpContext.Request.Method == HttpMethods.Post || context.HttpContext.Request.Method == HttpMethods.Put)
{
// Check parameter those are marked for not to log.
var methodInfo = ((Microsoft.AspNetCore.Mvc.Controllers.ControllerActionDescriptor)context.ActionDescriptor).MethodInfo;
var noLogParameters = methodInfo.GetParameters().Where(p => p.GetCustomAttributes(true).Any(t => t.GetType() == typeof(NoLogAttribute))).Select(p => p.Name);
StringBuilder logBuilder = new StringBuilder();
foreach (var argument in context.ActionArguments.Where(a => !noLogParameters.Contains(a.Key)))
{
var serializedModel = JsonConvert.SerializeObject(argument.Value, new JsonSerializerSettings() { ContractResolver = new NoPIILogContractResolver() });
logBuilder.AppendLine($"key: {argument.Key}; value : {serializedModel}");
}
var telemetry = this.httpContextAccessor.HttpContext.Items["Telemetry"] as Microsoft.ApplicationInsights.DataContracts.RequestTelemetry;
if (telemetry != null)
{
telemetry.Context.GlobalProperties.Add("jsonBody", logBuilder.ToString());
}
}
await next();
}
}
The 'LogActionFilterAttribute' is injected in MVC pipeline as Filter.
services.AddMvc(options =>
{
options.Filters.Add<LogActionFilterAttribute>();
});
NoLogAttribute
In above code, NoLogAttribute attribute is used which should be applied on Model/Model's Properties or method parameter to indicate that value should not be logged.
public class NoLogAttribute : Attribute
{
}
NoPIILogContractResolver
Also, NoPIILogContractResolver is used in JsonSerializerSettings during serialization process
internal class NoPIILogContractResolver : DefaultContractResolver
{
protected override IList<JsonProperty> CreateProperties(Type type, MemberSerialization memberSerialization)
{
var properties = new List<JsonProperty>();
if (!type.GetCustomAttributes(true).Any(t => t.GetType() == typeof(NoLogAttribute)))
{
IList<JsonProperty> retval = base.CreateProperties(type, memberSerialization);
var excludedProperties = type.GetProperties().Where(p => p.GetCustomAttributes(true).Any(t => t.GetType() == typeof(NoLogAttribute))).Select(s => s.Name);
foreach (var property in retval)
{
if (excludedProperties.Contains(property.PropertyName))
{
property.PropertyType = typeof(string);
property.ValueProvider = new PIIValueProvider("PII Data");
}
properties.Add(property);
}
}
return properties;
}
}
internal class PIIValueProvider : IValueProvider
{
private object defaultValue;
public PIIValueProvider(string defaultValue)
{
this.defaultValue = defaultValue;
}
public object GetValue(object target)
{
return this.defaultValue;
}
public void SetValue(object target, object value)
{
}
}
PIITelemetryInitializer
To inject the RequestTelemetry object, I've to use ITelemetryInitializer so that RequestTelemetry can be retrieved in LogActionFilterAttribute class.
public class PIITelemetryInitializer : ITelemetryInitializer
{
IHttpContextAccessor httpContextAccessor;
public PIITelemetryInitializer(IHttpContextAccessor httpContextAccessor)
{
this.httpContextAccessor = httpContextAccessor;
}
public void Initialize(ITelemetry telemetry)
{
if (this.httpContextAccessor.HttpContext != null)
{
if (telemetry is Microsoft.ApplicationInsights.DataContracts.RequestTelemetry)
{
this.httpContextAccessor.HttpContext.Items.TryAdd("Telemetry", telemetry);
}
}
}
}
The PIITelemetryInitializer is registered as
services.AddSingleton<ITelemetryInitializer, PIITelemetryInitializer>();
Testing feature
Following code demonstrates the usage of above code
Created a controller
[Route("api/[controller]")]
public class ValuesController : Controller
{
private readonly ILogger _logger;
public ValuesController(ILoggerFactory loggerFactory)
{
_logger = loggerFactory.CreateLogger<ValuesController>();
}
// POST api/values
[HttpPost]
public void Post([FromBody, NoLog]string value)
{
}
[HttpPost]
[Route("user")]
public void AddUser(string id, [FromBody]User user)
{
}
}
Where User Model is defined as
public class User
{
[NoLog]
public string Id { get; set; }
public string Name { get; set; }
public DateTime AnneviseryDate { get; set; }
[NoLog]
public int LinkId { get; set; }
public List<Address> Addresses { get; set; }
}
public class Address
{
public string AddressLine { get; set; }
[NoLog]
public string City { get; set; }
[NoLog]
public string Country { get; set; }
}
So when API is invoked by Swagger tool
The jsonBody is logged in Request without sensitive data. All sensitive data is replaced by 'PII Data' string literal.
Update: I have put the logic below into a ready-to-use NuGet package. You can find more about the package here and about the topic itself here.
I choose the custom middleware path as it made things easier with HttpContext already being there.
public class RequestBodyLoggingMiddleware : IMiddleware
{
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
var method = context.Request.Method;
// Ensure the request body can be read multiple times
context.Request.EnableBuffering();
// Only if we are dealing with POST or PUT, GET and others shouldn't have a body
if (context.Request.Body.CanRead && (method == HttpMethods.Post || method == HttpMethods.Put))
{
// Leave stream open so next middleware can read it
using var reader = new StreamReader(
context.Request.Body,
Encoding.UTF8,
detectEncodingFromByteOrderMarks: false,
bufferSize: 512, leaveOpen: true);
var requestBody = await reader.ReadToEndAsync();
// Reset stream position, so next middleware can read it
context.Request.Body.Position = 0;
// Write request body to App Insights
var requestTelemetry = context.Features.Get<RequestTelemetry>();
requestTelemetry?.Properties.Add("RequestBody", requestBody);
}
// Call next middleware in the pipeline
await next(context);
}
}
And this is how I log the response body
public class ResponseBodyLoggingMiddleware : IMiddleware
{
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
var originalBodyStream = context.Response.Body;
try
{
// Swap out stream with one that is buffered and suports seeking
using var memoryStream = new MemoryStream();
context.Response.Body = memoryStream;
// hand over to the next middleware and wait for the call to return
await next(context);
// Read response body from memory stream
memoryStream.Position = 0;
var reader = new StreamReader(memoryStream);
var responseBody = await reader.ReadToEndAsync();
// Copy body back to so its available to the user agent
memoryStream.Position = 0;
await memoryStream.CopyToAsync(originalBodyStream);
// Write response body to App Insights
var requestTelemetry = context.Features.Get<RequestTelemetry>();
requestTelemetry?.Properties.Add("ResponseBody", responseBody);
}
finally
{
context.Response.Body = originalBodyStream;
}
}
}
Than add an extension method...
public static class ApplicationInsightExtensions
{
public static IApplicationBuilder UseRequestBodyLogging(this IApplicationBuilder builder)
{
return builder.UseMiddleware<RequestBodyLoggingMiddleware>();
}
public static IApplicationBuilder UseResponseBodyLogging(this IApplicationBuilder builder)
{
return builder.UseMiddleware<ResponseBodyLoggingMiddleware>();
}
}
...that allows for a clean integration inside Startup.cs
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
// Enable our custom middleware
app.UseRequestBodyLogging();
app.UseResponseBodyLogging();
}
// ...
}
Don't forget to register the custom middleware components inside ConfigureServices()
public void ConfigureServices(IServiceCollection services)
{
// ...
services.AddApplicationInsightsTelemetry(Configuration["APPINSIGHTS_CONNECTIONSTRING"]);
services.AddTransient<RequestBodyLoggingMiddleware>();
services.AddTransient<ResponseBodyLoggingMiddleware>();
}
I never got #yonisha's answer working so I used a DelegatingHandler instead:
public class MessageTracingHandler : DelegatingHandler
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
// Trace the request
await TraceRequest(request);
// Execute the request
var response = await base.SendAsync(request, cancellationToken);
// Trace the response
await TraceResponse(response);
return response;
}
private async Task TraceRequest(HttpRequestMessage request)
{
try
{
var requestTelemetry = HttpContext.Current?.GetRequestTelemetry();
var requestTraceInfo = request.Content != null ? await request.Content.ReadAsByteArrayAsync() : null;
var body = requestTraceInfo.ToString();
if (!string.IsNullOrWhiteSpace(body) && requestTelemetry != null)
{
requestTelemetry.Properties.Add("Request Body", body);
}
}
catch (Exception exception)
{
// Log exception
}
}
private async Task TraceResponse(HttpResponseMessage response)
{
try
{
var requestTelemetry = HttpContext.Current?.GetRequestTelemetry();
var responseTraceInfo = response.Content != null ? await response.Content.ReadAsByteArrayAsync() : null;
var body = responseTraceInfo.ToString();
if (!string.IsNullOrWhiteSpace(body) && requestTelemetry != null)
{
requestTelemetry.Properties.Add("Response Body", body);
}
}
catch (Exception exception)
{
// Log exception
}
}
}
.GetRequestTelemetry() is an extension method from Microsoft.ApplicationInsights.Web.
In Asp.Net core it looks like we dont have to use ITelemetryInitializer. We can use the middleware to log the requests to application insights. Thanks to #IanKemp https://github.com/microsoft/ApplicationInsights-aspnetcore/issues/686
public async Task Invoke(HttpContext httpContext)
{
var requestTelemetry = httpContext.Features.Get<RequestTelemetry>();
//Handle Request
var request = httpContext.Request;
if (request?.Body?.CanRead == true)
{
request.EnableBuffering();
var bodySize = (int)(request.ContentLength ?? request.Body.Length);
if (bodySize > 0)
{
request.Body.Position = 0;
byte[] body;
using (var ms = new MemoryStream(bodySize))
{
await request.Body.CopyToAsync(ms);
body = ms.ToArray();
}
request.Body.Position = 0;
if (requestTelemetry != null)
{
var requestBodyString = Encoding.UTF8.GetString(body);
requestTelemetry.Properties.Add("RequestBody", requestBodyString);
}
}
}
await _next(httpContext); // calling next middleware
}
I implemented a middleware for this,
Invoke method does,
if (context.Request.Method == "POST" || context.Request.Method == "PUT")
{
var bodyStr = GetRequestBody(context);
var telemetryClient = new TelemetryClient();
var traceTelemetry = new TraceTelemetry
{
Message = bodyStr,
SeverityLevel = SeverityLevel.Verbose
};
//Send a trace message for display in Diagnostic Search.
telemetryClient.TrackTrace(traceTelemetry);
}
Where, GetRequestBody is like,
private static string GetRequestBody(HttpContext context)
{
var bodyStr = "";
var req = context.Request;
//Allows using several time the stream in ASP.Net Core.
req.EnableRewind();
//Important: keep stream opened to read when handling the request.
using (var reader = new StreamReader(req.Body, Encoding.UTF8, true, 1024, true))
{
bodyStr = reader.ReadToEnd();
}
// Rewind, so the core is not lost when it looks the body for the request.
req.Body.Position = 0;
return bodyStr;
}
I can able to log the request message body in Application Insights using #yonisha method but I can't able to log the response message body. I am interested in logging the response message body. I am already logging the Post, Put, Delete Request message body using #yonisha method.
When I tried to access the response body in the TelemetryInitializer I keep getting an exception with an error message saying that "stream was not readable. When I researched more I found that AzureInitializer is running as part of HttpModule(ApplicationInsightsWebTracking) so by the time it gets control response object is disposed.
I got an idea from #Oskar answer. Why not have a delegate handler and record the response since the response object is not disposed at the stage of message handler. The message handler is part of the Web API life cycle i.e. similar to the HTTP module but confined to web API. When I developed and tested this idea, fortunately, It worked I recorded the response in the request message using message handler and retrieved it at the AzureInitializer (HTTP module whose execution happens later than the message handler). Here is the sample code.
public class AzureRequestResponseInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
var requestTelemetry = telemetry as RequestTelemetry;
if (requestTelemetry != null && HttpContext.Current != null && HttpContext.Current.Request != null)
{
if ((HttpContext.Current.Request.HttpMethod == HttpMethod.Post.ToString()
|| HttpContext.Current.Request.HttpMethod == HttpMethod.Put.ToString()) &&
HttpContext.Current.Request.Url.AbsoluteUri.Contains("api"))
using (var reader = new StreamReader(HttpContext.Current.Request.InputStream))
{
HttpContext.Current.Request.InputStream.Position = 0;
string requestBody = reader.ReadToEnd();
if (requestTelemetry.Properties.Keys.Contains("requestbody"))
{
requestTelemetry.Properties["requestbody"] = requestBody;
}
else
{
requestTelemetry.Properties.Add("requestbody", requestBody);
}
}
else if (HttpContext.Current.Request.HttpMethod == HttpMethod.Get.ToString()
&& HttpContext.Current.Response.ContentType.Contains("application/json"))
{
var netHttpRequestMessage = HttpContext.Current.Items["MS_HttpRequestMessage"] as HttpRequestMessage;
if (netHttpRequestMessage.Properties.Keys.Contains("responsejson"))
{
var responseJson = netHttpRequestMessage.Properties["responsejson"].ToString();
if (requestTelemetry.Properties.Keys.Contains("responsebody"))
{
requestTelemetry.Properties["responsebody"] = responseJson;
}
else
{
requestTelemetry.Properties.Add("responsebody", responseJson);
}
}
}
}
}
}
config.MessageHandlers.Add(new LoggingHandler());
public class LoggingHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return base.SendAsync(request, cancellationToken).ContinueWith(task =>
{
var response = task.Result;
StoreResponse(response);
return response;
});
}
private void StoreResponse(HttpResponseMessage response)
{
var request = response.RequestMessage;
(response.Content ?? new StringContent("")).ReadAsStringAsync().ContinueWith(x =>
{
var ctx = request.Properties["MS_HttpContext"] as HttpContextWrapper;
if (request.Properties.ContainsKey("responseJson"))
{
request.Properties["responsejson"] = x.Result;
}
else
{
request.Properties.Add("responsejson", x.Result);
}
});
}
}
The solution provided by yonisha is clean, but it does not work for me in .Net Core 2.0. This works if you have a JSON body:
public IActionResult MyAction ([FromBody] PayloadObject payloadObject)
{
//create a dictionary to store the json string
var customDataDict = new Dictionary<string, string>();
//convert the object to a json string
string activationRequestJson = JsonConvert.SerializeObject(
new
{
payloadObject = payloadObject
});
customDataDict.Add("body", activationRequestJson);
//Track this event, with the json string, in Application Insights
telemetryClient.TrackEvent("MyAction", customDataDict);
return Ok();
}
I am sorry, #yonisha's solution does not seem to work in .NET 4.7. The Application Insights part works OK, but there is actually no simple way to get the request body inside the telemetry initializer in .NET 4.7. .NET 4.7 uses GetBufferlessInputStream() to get the stream, and this stream is "read once". One potential code is like this:
private static void LogRequestBody(ISupportProperties requestTelemetry)
{
var requestStream = HttpContext.Current?.Request?.GetBufferlessInputStream();
if (requestStream?.Length > 0)
using (var reader = new StreamReader(requestStream))
{
string body = reader.ReadToEnd();
requestTelemetry.Properties["body"] = body.Substring(0, Math.Min(body.Length, 8192));
}
}
But the return from GetBufferlessInputStream() is already consumed, and does not support seeking. Therefore, the body will always be an empty string.

Send Message to other Process

I want to be able to communicate between a Server-Application and a Client-Application. Both applications are written in C#/WPF. Interfaces are located in a separate DLL where both applications have a reference to it.
In the interface-dll is the IDataInfo-Interface which looks like:
public interface IDataInfo
{
byte[] Header { get; }
byte[] Data { get; }
}
The Server-Application calls the client by the following code:
Serializer<IDataInfo> serializer = new Serializer<IDataInfo>();
IDataInfo dataInfo = new DataInfo(HEADERBYTES, CONTENTBYTES);
Process clientProcess = Process.Start("Client.exe", serializer.Serialize(dataInfo));
The Client-Applications gets the message from the server by:
Serializer<IDataInfo> serializer = new Serializer<IDataInfo>();
IDataInfo dataInfo = serializer.Deserialize(string.Join(" ", App.Args));
The Serializer-Class is just a generic class which uses the Soap-Formatter to serialize/deserialze. The code looks like:
public class Serializer<T>
{
private static readonly Encoding encoding = Encoding.Unicode;
public string Serialize(T value)
{
string result;
using (MemoryStream memoryStream = new MemoryStream())
{
SoapFormatter soapFormatter = new SoapFormatter();
soapFormatter.Serialize(memoryStream, value);
result = encoding.GetString(memoryStream.ToArray());
memoryStream.Flush();
}
return result;
}
public T Deserialize(string soap)
{
T result;
using (MemoryStream memoryStream = new MemoryStream(encoding.GetBytes(soap)))
{
SoapFormatter soapFormatter = new SoapFormatter();
result = (T)soapFormatter.Deserialize(memoryStream);
}
return result;
}
}
Until here everything works fine. The server creates the client and the client can deserialize it's argument to the IDataInfo-Object.
Now I want to be able to send a message from the server to a running client. I Introduced the IClient-Interface in the Interface-DLL with the method void ReceiveMessage(string message);
The MainWindow.xaml.cs is implementing the IClient-Interface.
My Question is now how can I get the IClient-Object in my server, when I just have the Process-Object. I thought about Activator.CreateInstance, but I have no clue how to do this. I'm pretty sure that I can get the IClient by the Handle of the Process, but I don't know how.
Any idea?
As the other posts mention a common way is to create a service,
too keep it more simple I would consider a look at ServiceStack. AFAIK ServiceStack is used on stackoverflow
There also as course about it on pluralsight
ServiceStack is really easy to host in any .net dll (without iis and so on) and doesn't have the configuration complexity of WCF.
Also endpoints are available as SOAP and REST without the need to configure anything
For Example this defines a hello world service
public class HelloService : IService<Hello>
{
public object Execute(Hello request)
{
return new HelloResponse { Result = "Hello, " + request.Name };
}
}
Here an example of the client code:
var response = client.Send<HelloResponse>(new Hello { Name = "World!" });
Console.WriteLine(response.Result); // => Hello, World
You can find more
complex examples and walk-throughs at: ServiceStack.Hello
Communication that between multi processed have many waies to implement.
Like socket, fileMapping, share memory, windows 32 message and so on.
Maybe sample way is you can use WCF.
There are many ways to do inter process communication,
but if you are looking for a quick and easy solution you may want to look at ZeroMQ.
WCF is also an option but it might be overkill in your situation.
You can find more information about ZeroMQ here: http://www.zeromq.org/
And you can install it into your project using NuGet.
A quick example with a server and a client:
The server listens for connections, expects a string, reverses the string and returns it:
public class Server
{
public Server()
{
}
public void Listen()
{
Task.Run(() =>
{
using (var context = new Context())
{
//Open a socket to reply
using (var socket = context.Socket(SocketType.REP))
{
socket.Bind("tcp://127.0.0.1:32500");
while (true)
{
//Again you could also receive binary data if you want
var request = socket.Recv(Encoding.UTF8);
var response = ReverseString(request);
socket.Send(response, Encoding.UTF8);
}
}
}
});
}
private string ReverseString(string request)
{
var chars = request.ToCharArray();
Array.Reverse(chars);
return new string(chars);
}
}
The client connects to the server (in this case the same machine):
public class Client
{
public Client()
{
}
public string ReverseString(string message)
{
using (var context = new Context())
{
//Open a socket to request data
using (var socket = context.Socket(SocketType.REQ))
{
socket.Connect("tcp://127.0.0.1:32500");
//Send a string, you can send a byte[] as well, for example protobuf encoded data
socket.Send(message, Encoding.UTF8);
//Get the response from the server
return socket.Recv(Encoding.UTF8);
}
}
}
}
To test it, the program might look like this:
public class Program
{
public static void Main()
{
new Program();
}
public Program()
{
var server = new Server();
server.Listen();
var client = new Client();
var input = String.Empty;
while (input != "/quit")
{
input = Console.ReadLine();
Console.WriteLine(client.ReverseString(input));
}
}
}
It's easy and it gets the job done.
Another alternative is to use named pipes for IPC: http://www.codeproject.com/Tips/492231/Csharp-Async-Named-Pipes

Categories