We Keep Coding

Process incoming FileStream asynchronously

I'm reading a file from user upload and it was working synchronously. I needed to change it in order to immediately send a "received" alert to the user, then read the file asynchronously while the user would periodically poll back to see if the read was finished.
Here is what my code looks like right now:
public FileUpload SaveFile(Stream stream)
{
FileUpload uploadObj = //instantiate the return obj
var task = Task.Run(async () => await ProcessFileAsync(stream));
return upload;
}
public async Task ProcessFileAsync(Stream stream)
{
StreamReader file = new StreamReader(stream);
CsvReader csv = new CsvReader(file, CultureInfo.InvariantCulture);
while (await csv.ReadAsync())
{
//read the file
}
}
the issue I'm having is that by the time I call the csv.ReadAsync() method, the Stream object has been disposed. How do I access the Stream when I want the SaveFile() method to return a value to the user, but the act of returning disposes the Stream object?

The point here is that you're working within the constraints of ASP.NET, which abstracts away a lot of the underlying HTTP stuff.
When you say you want to process a user-uploaded file asynchronously, you want to step out of the normal order of doing things with HTTP and ASP.NET. You see, when a client sends a request with a body (the file), the server receives the request headers and kicks off ASP.NET to tell your application code that there's a new request incoming.
It hasn't even (fully) read the request body at this point. This is why you get a Stream to deal with the request, and not a string or a filename - the data doesn't have to be arrived at the server yet! Just the request headers, informing the web server about the request.
If you return a response at that point, for all HTTP and ASP.NET care, you're done with the request, and you cannot continue reading its body.
Now what you want to do, is to read the request body (the file), and process that after sending a response to the client. You can do that, but then you'll still have to read the request body - because if you return something from your action method before reading the request, the framework will think you're done with it and dispose the request stream. That's what's causing your exception.
If you'd use a string, or model binding, or anything that involves the framework reading the request body, then yes, your code will only execute once the body has been read.
The short-term solution that would appear to get you going, is to read the request stream into a stream that you own, not the framework:
var myStream = new MemoryStream();
await stream.CopyTo(myStream);
Task.Run(async () => await ProcessFileAsync(myStream));
Now you'll have read the entire request body and saved it in memory, so ASP.NET can safely dispose the request stream and send a response to the client.
But don't do this. Starting fire-and-forget tasks from a controller is a bad idea. Keeping uploaded files in memory is a bad idea.
What you actually should do, if you still want to do this out-of-band:
Save the incoming file as an actual, temporary file on your server
Send a response to the client with an identifier (the temporarily generated filename, for example a GUID)
Expose an endpoint that clients can use to request the status using said GUID
Have a background process continuously scan the directory for newly uploaded files and process them
For the latter you could hosted services or third-party tools like Hangfire.

You'll need to either do this if the environment warrants:
var result = task.Result;
//do stuff
...or
public Task<FileUpload> SaveFile(Stream stream)
{
var uploadObj = //instantiate the return obj
await ProcessFileAsync(stream);
return uploadObj;
}
See here for a thorough discussion on fire-and-forget if you go that route:
Web Api - Fire and Forget

How to: c# XML InfoSet to JSON

So WCF takes a JSON, and for whatever reason translates that to an XML Infoset (see: https://stackoverflow.com/a/5684703/497745). It then reads back this XML Infoset internally using the JsonReaderDelegator (see: https://referencesource.microsoft.com/#System.Runtime.Serialization/System/Runtime/Serialization/Json/JsonReaderDelegator.cs,c0d6a87689227f04).
I am doing some very in-depth modification of the WCF execution flow, and I need to reverse the XML Infoset back to the original JSON.
Is there a library either in .NET or external that can take the XML Infoset generated by WCF and convert that back to its original JSON?

Found an answer that works in my scenario with some limited modifications.
This article: https://blogs.msdn.microsoft.com/carlosfigueira/2011/04/18/wcf-extensibility-message-inspectors/ explains how to log the JSON that originally came in to WCF, even though it is only exposed as an XML InfoSet. This was the key insight. Specifically, look at its MessageToString implementation.
Below is the relevant portion of my code, based on the implementation of MessageToString in the above link, running within a class internal class MessageFormatInspector : IDispatchMessageInspector, IEndpointBehavior that I have written to inject into the WCF stack:
public object AfterReceiveRequest(ref Message request, IClientChannel channel, InstanceContext instanceContext)
{
...
var s = GetJsonFromMessage(request);
...
}
private static string GetJsonFromMessage(ref Message request)
{
using (MemoryStream ms = new MemoryStream())
{
using (XmlDictionaryWriter writer = JsonReaderWriterFactory.CreateJsonWriter(ms))
{
request.WriteMessage(writer);
writer.Flush();
string json = Encoding.UTF8.GetString(ms.ToArray()); //extract the JSON at this point
//now let's make our copy of the message to support the WCF pattern of rebuilding messages after consuming the inner stream (see: https://msdn.microsoft.com/en-us/library/system.servicemodel.dispatcher.idispatchmessageinspector.afterreceiverequest(v=vs.110).aspx and https://blogs.msdn.microsoft.com/carlosfigueira/2011/05/02/wcf-extensibility-message-formatters/)
ms.Position = 0; //Rewind. We're about to make a copy and restore the message we just consumed.
XmlDictionaryReader reader = JsonReaderWriterFactory.CreateJsonReader(ms, XmlDictionaryReaderQuotas.Max); //since we used a JsonWriter, we read the data back, we need to use the correlary JsonReader.
Message restoredRequestCopy = Message.CreateMessage(reader, int.MaxValue, request.Version); //now after a lot of work, create the actual copy
restoredRequestCopy.Properties.CopyProperties(request.Properties); //copy over the properties
request = restoredRequestCopy;
return json;
}
}
}
Unfortunately, the above code only works within the context of a WCF message inspector.
However, it is able to take an XMLDictionary, which has the XML InfoSet, and to using WCF's own built in JsonWriter to reverse the conversion WCF did and emit the original JSON.
I hope this helps someone save some time in the future.

Recreating a Message in WCF

I am retrieving data from a server via WCF (I am the client). Unfortunately, the server (which I have no control of) is sometimes returning invalid XML.
In order to fix this, I plan to add a IClientMessageInspector (on the client), which modifies the Message before WCF has chance to parse the returned XML.
My first step was to implement IClientMessageInspector, but have it so it leaves the response unchanged (it should effectively be a no-op), but for some reason it causes the generated WCF method (client.getBar() below) to return a null object, rather than a populated object.
class UTF8Policer : IClientMessageInspector
{
public void AfterReceiveReply(ref Message reply, object correlationState)
{
Message revised = null;
var contents = new StringBuilder();
var writer = XmlWriter.Create(contents);
reply.WriteMessage(writer);
writer.Flush();
revised = Message.CreateMessage(reply.Version, reply.Headers.Action, XmlReader.Create(new StringReader(contents.ToString()));
revised.Headers.CopyHeadersFrom(reply);
revised.Properties.CopyProperties(reply.Properties);
reply = revised;
}
public object BeforeSendRequest(ref Message request, IClientChannel channel)
{
return null;
}
}
However, when running:
var client = new Foo_RPCClient();
var header = new header();
// This is what registers the inspector
client.Endpoint.EndpointBehaviors.Add(new FooEndpointBehaviour());
var response = client.getBar(ref header, new BarRequest());
... response is null. If I comment out the inspector registration, response works.
My conclusion is therefore that I'm somehow invalidating the message within AfterReceiveReply. Can anyone advise what the correct way is to re-create the message received?
Once I've got this working, I'm hoping it'll be trivial to fixup the XML within AfterReceiveReply, so that it actually does something useful.

Ref parameter wont work with WCF.
When you do the call to the service everything is pass as an input message serialized. Then the service deserialize it and do the work and finally it return a serialized response to the client that is deserialized on the client side.
The ref parameter might compile but in no way you retain the reference to a memory pointer in the client computer. You must use the response to return back the object.

Error updating Wcf message before hitting service using IDispatchOperationSelector

We are having to intercept a SOAP message before it hits our WCF service to perform the following steps:
Route the message to the correct method as the client is unable to provide us with a SOAPAction value.
Update the namespaces of the xml as the client is unable to add namespace information to the message.
The routing is not an issue, but we are having a problem with creating the message; once we recreate the message the body merely consists of "... Stream ...".
Before creating the message, the messageContent variable contains valid, correct xml.
private Message UpdateNamespaces(Message message, string methodName)
{
var memoryStream = new MemoryStream();
var xmlWriter = XmlWriter.Create(memoryStream);
message.WriteMessage(xmlWriter);
xmlWriter.Flush();
var messageContent = Encoding.UTF8.GetString(memoryStream.ToArray());
xmlWriter.Close();
// Update messageContent with corrected XML
memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(messageContent));
var xmlDictionaryReader = XmlDictionaryReader.CreateTextReader(memoryStream, new XmlDictionaryReaderQuotas());
var newMessage = Message.CreateMessage(xmlDictionaryReader, int.MaxValue, message.Version);
newMessage.Properties.CopyProperties(message.Properties);
return newMessage;
}
The messageContent is correct at the point at which we create the memoryStream, but as soon as I check the content of newMessage.ToString(), I'm getting the "... Stream ..." body content.
If anyone could help, I'd be very grateful as I'm out of ideas!
Many thanks

I think you are mixing behavior responsibilities.
IDispatchOperationSelector.SelectOperation is intended for routing decisions. It gives you a reference to the message, but the intention is to enable access to message properties. WCF does not expect the you to modify the message using this extension point. I can't say definitively, but the problem could be that simple.
If you want to alter the message namespace you should be using IDispatchMessageInspector. I suggest creating a second behavior for that task. Here's a nice Message Inspector example.

Get XML body of a 400 or 500 HTTP XML response

Long story short, I am sending an XML HTTP post request to an application server, and I am getting back a response, also in the form of XML HTTP.
I have a test site available to me which allows me to see what the server's actual response is, visually, in the form of XML, but I cannot access this XML from my C# code the way it is.
The XML coming back from the application server in my test case looks like this:
<Error><Message>StringErrorMessage</Message></Error>
However, I have had no luck accessing this basic XML to retrieve the value of "StringErrorMessage" for the creation of a detailed error report.
... More code above, all wrapped in a try{}...
_response = Serializer.DeserializeObject<T>(ObjectRequest.GetResponse().GetResponseStream());
}
catch (System.Net.WebException exceptionParameter)
{
var response = (HttpWebResponse)exceptionParameter.Response;
string webExceptionStatus = exceptionParameter.Message;
_exception = exceptionParameter;
return false;
}
I have consulted
C# - Getting the response body from a 403 error
and
Get response body on 400 HTTP response in Android?
The first link's solution doesn't seem to give me access to the basic XML as part of any response object's properties. I am almost positive that there must be a byte[] in there somewhere (in the response, or in the exception object) that can be converted into a char[], which can be converted to a string, which can be converted to my XML body, but I have not been able to find it. The second link's solution is not exactly viable for me because I have to get the response body back in the form of XML, as it might not be an error, but an object that must be deserialized. This particular side of things, I cannot change.
Any advice would be very much appreciated.
- Eli
EDIT: Just wanted to clarify that my basic code is working okay for non-error situations, and is deserializing the XML just fine. It's when my code encounters a HTTP 400 or an HTTP 500 error, where accessing the XML from the catch statement becomes a problem, because my code immediately throws an exception.

The body of a HTTP message (the XML in your case) can be retrieved with the GetResponseStream method of the HttpWebResponse object you have. And, since it's a stream, you can for instance read it with a StreamReader, like so:
HttpWebResponse myWebResponse; // Get this from whereever you want
Stream responseStream = myWebResponse.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
string niceStringForYou = reader.ReadToEnd();
...and from that point on, you can do whatever to it.
If you're absolutely sure it's always gonna be XML you get back from the service, you can probably even use an XmlReader to get XML directly from the stream:
XmlReader foo = XmlReader.Create(responseStream);
Comment to edit: As long as you have the HttpWebResponse object, reading it's response stream (GetResponseStream()) should work. And as you point out in your own code, you can get the HttpWebResponse by looking at (HttpWebResponse)exceptionParameter.Response.