The problem I have is that the client is sending me a string of data as a stream. WCF then normalizes (removes the CR part of CRLF) and I get a hash mismatch between server and client on that specific string.
public void SomeWcfContract(Stream input)
{
try
{
string processed = ReadStream(input);
WebOperationContext.Current.OutgoingResponse.StatusCode = HttpStatusCode.OK;
}
catch (Exception ex)
{
WebOperationContext.Current.OutgoingResponse.StatusCode = HttpStatusCode.InternalServerError;
}
}
private string ReadStream(Stream input)
{
string output;
using (var reader = new StreamReader(input, Encoding.UTF8))
{
output = reader.ReadToEnd();
}
return output;
}
I read a post about this here : XML deserialization 'standardising' line endings, how to stop it? (.NET)
It's the exact same problem I have but I use the standard XmlSerializer of WCF. Do I need to create my own XmlSerializer implementation or can I add the "fix" to the settings somehow?
This seems to be a VERY nasty bug with the WCF XmlDictionaryReader what happens is that when WCF serializes the incoming stream to a Message all instances of carriage return in (CRLF) is removed and replaced with LF. According to this it's a known bug in WCF.
EDIT I reported this as a bug to Microsoft : https://connect.microsoft.com/wcf/feedback/details/532196/xmldictionaryreader-is-normalizing-incoming-stream-removing-cr-of-crlf#details
This seems to fix that problem:
[WebInvoke(BodyStyle = WebMessageBodyStyle.Bare,
UriTemplate = UriTemplates.SomeWcfContract), OperationContract]
void SomeWcfContract(Stream vcard);
I guess it makes sense since this would cause the parameter not to be wrapped or something like that.
Related
I'm writing a Windows Service to listen and process messages from MSMQ. The listener has various error handling steps, but if all else fails, I want to save the body of the message to a text file so that I can look at it. However, I can't seem to extract the content of my messages when this condition is hit. The following code is a simple representation of the sections in question, and it always produces an empty text file even though I know the message I'm testing with is not empty. HOWEVER, if I comment-out the initial attempt to deserialize the XML, the fail safe does work and produces a text file with the message body. So I think the problem is something to do with how the deserialization attempt leaves the underlying Stream? Just to clarify, when the message contains valid XML that CAN be deserialized, the service all works fine and the fail-safe never comes into action.
MyClass myClass = null;
try
{
XmlSerializer serializer = new XmlSerializer(typeof(MyClass));
// Comment the following out and the fail safe works
// Let this run and fail and the text file below is always empty
myClass = (MyClass)serializer.Deserialize(m.BodyStream);
}
catch (Exception ex)
{
}
if (myClass == null)
{
string filePath = #"D:\path\file.txt";
m.Formatter = new ActiveXMessageFormatter();
StreamReader reader = new StreamReader(m.BodyStream);
File.WriteAllText(filePath, reader.ReadToEnd());
}
Depending on the formatter you are using:
For windows binary messages:
File.WriteAllText(<path>, (new UTF8Encoding()).GetString((byte[])msg.Body)); // for binary
For XML messages try this:
msg.Formatter = new XmlMessageFormatter(new String[] { "System.String, mscorlib" });
var text = msg.Body.ToString();
// write to file..
For neither binary or XML, use the native formatter:
msg.Formatter = new ActiveXMessageFormatter();
reader = new StreamReader(msg.BodyStream);
msgBody = reader.ReadToEnd();
// write to file..
I'm working with a 3rd-party developed Windows 8/8.1 App that relies on a Web service to retrieve data. We have the source code for the App but we don't have it for the web service, so I need to rebuild it. To complicate matters, I can't make changes to the App because it was sideloaded on a number of devices.
Even though I was able to decompile the web service's DLL, it didn't get me to a working point. The hurdle right now seems to be that XDocument.Load cannot process the returned stream.
Here's the AppCode:
public async Task<CustomerModel> ReadDataFromXml(string address)
{
var client = new HttpClient();
var response = await client.GetAsync(address);
// check that response was successful or throw exception
response.EnsureSuccessStatusCode();
var streamResponse = await response.Content.ReadAsStreamAsync();
var xDocumentObject = XDocument.Load(streamResponse);
var contents = from contact in xDocumentObject.Descendants(Constants.CUSTOMER_TAG)
select new {
...
I'm using VS 2013 to build the service with the built in MVC ASP.NET Web Application template which implements the ApiController.
The old code, after Serializing it with UTF8Encoding returned the results via
HttpContext.Current.Response.Write(strMessage);
The new code
public IEnumerable<Customer> Get(string activationcode)
{
return _handler.GetCustomerData(activationcode);
}
I believe I need this returned through the Get call which currently returns IEnumerable.
I believe I tried returning a string for the Get call
Another thing I tried was changing the Get statement to return a string and then return the XMLserialized version of the text.
private string Serialize(Customer cus)
{
try
{
XmlSerializer xmlSerializer = new XmlSerializer(typeof(Customer));
XmlTextWriter xmlTextWriter = new XmlTextWriter((Stream)new MemoryStream(), Encoding.UTF8);
xmlSerializer.Serialize((XmlWriter)xmlTextWriter, (object)cus);
return this.UTF8ByteArrayToString(((MemoryStream)xmlTextWriter.BaseStream).ToArray());
}
catch (Exception ex)
{
return string.Empty;
}
}
But that returned "Data at the root level is invalid, Line 1, Position 1"
TLDR; Is there a way to get the GET Statement return the result so that XDocument.Load in our web app will function correctly?
Finally, figured it out:
Instead of returning the object, change the Get function to return HttpReponseMessage.
public HttpResponseMessage Get(string activationcode)
Serialize the returned data your object
Customer customer = _handler.GetCustomerData(activationcode).First();
string serializedCustomer = Serialize(customer);
and use StringContent to build your return:
return new HttpResponseMessage
{
Content = new StringContent(serializedCustomer, Encoding.UTF8, "text/xml")
};
Here's the functions used to serialized the data.
private string Serialize(Customer cus)
{
try
{
XmlSerializer xmlSerializer = new XmlSerializer(typeof(Customer));
XmlTextWriter xmlTextWriter = new XmlTextWriter((Stream)new MemoryStream(), Encoding.UTF8);
xmlSerializer.Serialize((XmlWriter)xmlTextWriter, (object)cus);
return this.UTF8ByteArrayToString(((MemoryStream)xmlTextWriter.BaseStream).ToArray());
}
catch (Exception ex)
{
return string.Empty;
}
}
private string UTF8ByteArrayToString(byte[] characters)
{
return new UTF8Encoding().GetString(characters);
//return characters.ToString();
}
I am using Web API to receive XML data and convert it to an Object. Which is working fine.
public void Post([FromBody] trackermessages model)
{
try
{
How do I get the RAW XML data? Is there a way to get the XML data as the Request begins or inside this action?
I tried this:
public void Post([FromBody] trackermessages model, [FromBody] string rawText)
{
try
{
But this is not allowed.
I also tried this:
public void Post([FromBody] trackermessages model)
{
try
{
var strean = new StreamReader(HttpContext.Current.Request.InputStream).ReadToEnd();
But I get the Exception:
This method or property is not supported after
HttpRequest.GetBufferlessInputStream has been invoked.
EDIT:
I am getting the Exception:
var stream = await Request.Content.ReadAsStreamAsync();
stream.Seek(0, System.IO.SeekOrigin.Begin); // On this Line
StreamReader reader = new StreamReader(stream);
string text = reader.ReadToEnd();
This is how I did it, because I had to read the RAW data then convert to Object:
public void Post(HttpRequestMessage request)
{
// Reading data as XML string to log to files - In case message structure is changed
var xmlDoc = new XmlDocument();
xmlDoc.Load(request.Content.ReadAsStreamAsync().Result);
var str = xmlDoc.InnerXml;
// Convert to model
var model = XMLHelper.FromXml<trackermessages>(str);
}
And the XMLHelper was copied from another question on stackoverflow: https://stackoverflow.com/a/3187539/1910735
Yes, you can get the raw XML. You do need to seek back to the start of the stream since it will have been read to the end when processing the Model.
public async void Post([FromBody]TestModel value)
{
var stream = await this.Request.Content.ReadAsStreamAsync();
stream.Seek(0, System.IO.SeekOrigin.Begin);
StreamReader reader = new StreamReader(stream);
string text = reader.ReadToEnd();
Console.Write(text);
}
The problem then is that your application is using GetBufferlessInputStream to read uploads without buffering them. While that is good for memory usage on the server, it means that after you've read the stream once it will no longer be available.
Your stream is being read like this when populating your model. By default GetBufferedInputStream is used which is why it works for me.
I suggest that you take the raw XML as input into the action and then manually deserialize into the model. Alternatively you can switch back to accepting posted data into a buffer.
You probably followed something like this to turn it on : https://blogs.msdn.microsoft.com/kiranchalla/2012/09/04/receiving-request-file-or-data-in-streamed-mode-at-a-web-api-service/ and should undo that to turn it off.
Consider the following C# code:
using System.Xml.Linq;
namespace TestXmlParse
{
class Program
{
static void Main(string[] args)
{
var testxml =
#"<base>
<elem1 number='1'>
<elem2>yyy</elem2>
<elem3>xxx <yyy zzz aaa</elem3>
</elem1>
</base>";
XDocument.Parse(testxml);
}
}
}
I get a System.Xml.XmlException on the parse, of course, complaining about elem3. The error message is this:
System.Xml.XmlException was unhandled
Message='aaa' is an unexpected token. The expected token is '='. Line 4, position 59.
Source=System.Xml
LineNumber=4
LinePosition=59
Obviously this is not the real Xml (we get the xml from a third party) and while the best answer would be for the third party to clean up their xml before they send it to us, is there any other way I might fix this xml before I hand it off to the parser? I've devised a hacky way to fix this; catch the exception and use that to tell me where I need to look for characters which should be escaped. I was hoping for something a bit more elegant and comprehensive.
Any suggestions are welcome.
If this is a dupe, please point me to the other questions; I'll close this myself. I am more interested in an answer than any karma gain.
EDIT:
I guess I didn't make my question as clear as I had hoped. I know the "<" in elem3 is incorrect; I'm trying to find an elegant way to detect (and correct) any badly formed xml of that sort before I attempt the parse. As I say, I get this xml from a third-party and I can't control what they give me.
I would recommend that you do not manipulate the data you receive. If it is invalid it's your client's problem.
Editing the input so it is valid xml can cause serious problems, e.g. instead of throwing an error you may end up processing wrong data (because you tried your best to make the xml valid, but this may lead to different data).
[EDIT]
I still think it's not a good idea, but sometimes you have to do what you have to do.
Here is a very simple class that parses the input and replaces the invald opening tag. You could do this with a regex (which I am not good at) and this solution is not complete, e.g. depending on your requirements (or lets say the bad xml you get) you will have to adopt it (e.g. scan for complete xml elements instead of only the "<" and ">" brackets, put CDATA around the inner text of a node and so on).
I just wanted to illustrate how you could do it, so please don't complain if it is slow/has bugs (as I mentioned, I would not do it).
class XmlCleaner
{
public void Clean(Stream sourceStream, Stream targetStream)
{
const char openingIndicator = '<';
const char closingIndicator = '>';
const int bufferSize = 1024;
long length = sourceStream.Length;
char[] buffer = new char[bufferSize];
bool startTagFound = false;
StringBuilder writeBuffer = new StringBuilder();
using(var reader = new StreamReader(sourceStream))
{
var writer = new StreamWriter(targetStream);
try
{
while (reader.Read(buffer, 0, bufferSize) > 0)
{
foreach (var c in buffer)
{
if (c == openingIndicator)
{
if (startTagFound)
{
// we have 2 following opening tags without a closing one
// just replace the first one
writeBuffer = writeBuffer.Replace("<", "<");
// append the new one
writeBuffer.Append(c);
}
else
{
startTagFound = true;
writeBuffer.Append(c);
}
}
else if (c == closingIndicator)
{
startTagFound = false;
// write writebuffer...
writeBuffer.Append(c);
writer.Write(writeBuffer.ToString());
writeBuffer.Clear();
}
else
{
writeBuffer.Append(c);
}
}
}
}
finally
{
// unfortunately the streamwriter's dispose method closes the underlying stream, so e just flush it
writer.Flush();
}
}
}
To test it:
var testxml =
#"<base>
<elem1 number='1'>
<elem2>yyy</elem2>
<elem3>xxx <yyy zzz aaa</elem3>
</elem1>
</base>";
string result;
using (var source = new MemoryStream(Encoding.ASCII.GetBytes(testxml)))
using(var target = new MemoryStream()) {
XmlCleaner cleaner = new XmlCleaner();
cleaner.Clean(source, target);
target.Position = 0;
using (var reader = new StreamReader(target))
{
result = reader.ReadToEnd();
}
}
XDocument.Parse(result);
var expectedResult =
#"<base>
<elem1 number='1'>
<elem2>yyy</elem2>
<elem3>xxx <yyy zzz aaa</elem3>
</elem1>
</base>";
Debug.Assert(result == expectedResult);
We've noticed that UTF8 characters don't come out correctly when using UIDevice.CurrentDevice.Name in MonoTouch.
It comes out as "iPad 2 ??", if you use some of the special characters like holding down the apostrophe key on the iPad keyboard. (Sorry don't know the equivalent to show these characters in windows)
Is there a recommended workaround to get the correct text? We don't mind to convert to UTF8 ourselves. I also tried simulating this from a UITextField and it worked fine--no UTF8 problems.
The reason this is causing problems is we are sending this text off to a web service, and it's causing XML parsing issues.
Here is a snipped of the XmlWriter code (_parser.WriteRequest):
using (XmlWriter xmlWriter = XmlWriter.Create(textWriter, new XmlWriterSettings
{
#if DEBUG
Indent = true,
#else
Indent = false, NewLineHandling = NewLineHandling.None,
#endif
OmitXmlDeclaration = true
}))
{
xmlWriter.WriteStartDocument();
xmlWriter.WriteStartElement("REQUEST");
xmlWriter.WriteAttributeString("TYPE", "EXAMPLE");
xmlWriter.WriteEndElement();
xmlWriter.WriteEndDocument();
}
The TextWriter is passed in from:
public Response MakeRequest(Request request)
{
var httpRequest = CreateRequest(request);
WriteRequest(httpRequest.GetRequestStream(), request);
using (var httpResponse = httpRequest.GetResponse() as HttpWebResponse)
{
using (var responseStream = httpResponse.GetResponseStream())
{
var response = new Response();
ReadResponse(response, responseStream);
return response;
}
}
}
private void WriteRequest(Stream requestStream, Request request)
{
if (request.Type == null)
{
throw new InvalidOperationException("Request Type was null!");
}
if (_logger.Enabled)
{
var builder = new StringBuilder();
using (var writer = new StringWriter(builder, CultureInfo.InvariantCulture))
{
_parser.WriteRequest(writer, request);
}
_logger.Log("REQUEST: " + builder.ToString());
using (requestStream)
{
using (StreamWriter writer = new StreamWriter(requestStream))
{
writer.Write(builder.ToString());
}
}
}
else
{
using (requestStream)
{
using (StreamWriter writer = new StreamWriter(requestStream))
{
_parser.WriteRequest(writer, request);
}
}
}
}
_logger writes to Console.WriteLine, it is enabled in #if DEBUG mode. Request is just a storage class with properties, sorry easy to confuse with HttpWebRequest.
I'm seeing ?? in both XCode's console and MonoDevelop's console. I'm also assuming the server is receiving them strangely as well, as I get an error. Using UITextField.Text with the same strange characters instead of the device description works fine with no issues. It makes me think the device description is the culprit.
EDIT: this fixed it -
Encoding.UTF8.GetString (Encoding.ASCII.GetBytes(UIDevice.CurrentDevice.Name));
Okay, I think I know the problem. You're creating a StringWriter, which always reports its encoding as UTF-16 (unless you override the Encoding property). You're then taking the string from that StringWriter (which will start with <?xml version="1.0" encoding="UTF-16" ?>) and writing it to a StreamWriter which will default to UTF-8. That mixture of encodings is causing the problem.
The simplest approach would be to change your code to pass a Stream directly to the XmlWriter - a MemoryStream if you really want, or just requestStream. That way the XmlWriter can declare that it's using the exact encoding that it's actually writing the binary data in - you haven't got an intermediate step to mess things up.
Alternatively, you could create a subclass of StringWriter which allows you to specify the encoding. See this answer for some sample code.
MonoTouch simply calls NSString.FromHandle on the value it receive from the call on UIDevice.CurrentDevice.Name. That just like most string are created from NSString inside all bindings.
That should get you a string that you can see it MonoDevelop (no ?) so I can't rule out a bug.
Can you tell us exactly how the device is named ? if so then please open a bug report and we'll check this possibility.