I'm writing a Windows Service to listen and process messages from MSMQ. The listener has various error handling steps, but if all else fails, I want to save the body of the message to a text file so that I can look at it. However, I can't seem to extract the content of my messages when this condition is hit. The following code is a simple representation of the sections in question, and it always produces an empty text file even though I know the message I'm testing with is not empty. HOWEVER, if I comment-out the initial attempt to deserialize the XML, the fail safe does work and produces a text file with the message body. So I think the problem is something to do with how the deserialization attempt leaves the underlying Stream? Just to clarify, when the message contains valid XML that CAN be deserialized, the service all works fine and the fail-safe never comes into action.
MyClass myClass = null;
try
{
XmlSerializer serializer = new XmlSerializer(typeof(MyClass));
// Comment the following out and the fail safe works
// Let this run and fail and the text file below is always empty
myClass = (MyClass)serializer.Deserialize(m.BodyStream);
}
catch (Exception ex)
{
}
if (myClass == null)
{
string filePath = #"D:\path\file.txt";
m.Formatter = new ActiveXMessageFormatter();
StreamReader reader = new StreamReader(m.BodyStream);
File.WriteAllText(filePath, reader.ReadToEnd());
}
Depending on the formatter you are using:
For windows binary messages:
File.WriteAllText(<path>, (new UTF8Encoding()).GetString((byte[])msg.Body)); // for binary
For XML messages try this:
msg.Formatter = new XmlMessageFormatter(new String[] { "System.String, mscorlib" });
var text = msg.Body.ToString();
// write to file..
For neither binary or XML, use the native formatter:
msg.Formatter = new ActiveXMessageFormatter();
reader = new StreamReader(msg.BodyStream);
msgBody = reader.ReadToEnd();
// write to file..
Related
I am facing this exception
Error getting value from 'Position' on 'Amazon.Runtime.Internal.Util.MD5Stream'.
when trying to read file from aws s3 bucket.
Here is my c# code
try
{
var s3ObjectPath = $"users/{email.Id}/emailattachments/{item.AttachedFileName}";
var ifExists = await this.Exists(s3ObjectPath);
if (ifExists)
{
Stream attachment = await s3Client.GetObjectStreamAsync(attachmentS3BucketName, s3ObjectPath, dicData);
Attachment att = new Attachment(attachment, item.AttachedFileName);
attachments.Add(att);
}
}
catch (AmazonS3Exception ex)
{
}
However this is working sometimes. I searched everywhere but didn't find solution.
Thanks in advance!!
This is due to an issue with how Streams work. When you open a Stream you are not actually getting the content of the file, rather you are opening a connection enabling you to access the data.
The error you got is returned when your code is trying to access the stream but doesn't have the ability to do so (a connection being lost / no appropriate credentials in the code accessing the stream or any other reason).
The solution is either to solve the underlaying issue and make sure your code still as access to the Stream or to just read from the stream and return a string rather than a stream
The code addition would look something like this:
...
Stream attachment = await s3Client.GetObjectStreamAsync(attachmentS3BucketName, s3ObjectPath, dicData);
StreamReader reader = new StreamReader(attachment);
string attachmentText = reader.ReadToEnd();
Attachment att = new Attachment(attachmentText, item.AttachedFileName);
attachments.Add(att);
I'm trying to consume a third party webservice. I craft a request message, send it to the httpclient and receive the httpResponseMessage.
Below is a snippet from the method that handles the call. xml = true for my situation.
HttpResponseMessage response = await HttpClientInstance.SendAsync(requestMessage);
if (response.IsSuccessStatusCode)
{
if (xml)
{
try
{
//This fails with "the data at the root level is invalid"
XmlMediaTypeFormatter xmlFormatter = new XmlMediaTypeFormatter { UseXmlSerializer = true };
var content = response.Content.ReadAsAsync<T>(new Collection<MediaTypeFormatter> { xmlFormatter });
return content.Result;
}
catch (Exception tempe)
{
var content = response.Content.ReadAsAsync<T>();
return content.Result;
}
}
else
{
var content = response.Content.ReadAsAsync<T>();
return content.Result;
}
}
else
{
_logWriter.LogProcessFault(operationContext, null, GetCurrentMethod(), $"An error occurred while calling the API. URI = {endpoint}. StatusCode = {response.StatusCode}", null);
throw new Exception(response.StatusCode.ToString());
}
After running this code, an error is thrown when attempting to deserialize the XML data in the stream. It fails with an error "the data at the root level is invalid".
I commented out the XMLMediaTypeFormatter and response.Content.Read... and replaced it with this
var fileStream = File.Create("D:\\Extract\\test.txt");
await response.Content.CopyToAsync(fileStream);
fileStream.Close();
which writes out valid XML to a file.
In the immediate window I ran response.Content.ReadAsStringAsync() and the returned string value has extra backslashes escaping content.
For instance, this is what is in the generated test.txt file:
<?xml version="1.0" encoding="utf-8"?>
and this is from ReadAsStringAsync:
<?xml version=\"1.0\" encoding=\"utf-8\"?>
I believe this is what is causing the deserialization to fail. Is there a clean fix for this or perhaps I am doing something wrong elsewhere?
The issue is in the following block of code
await response.Content.CopyToAsync(fileStream);
fileStream.Close();
//This fails with "the data at the root level is invalid"
XmlMediaTypeFormatter xmlFormatter = new XmlMediaTypeFormatter { UseXmlSerializer = true };
var content = response.Content.ReadAsAsync<T>(new Collection<MediaTypeFormatter> { xmlFormatter });
HTTP responses can be consumed only once. In other words, response is not read into some buffer from which multiple reads can be serviced. It reads it directly off of the socket. So once read, it cannot be read again.
Above, you copy response into a file. That consumes the response and it is no longer available to be read again. So by the time you try to read it again when assigning to content, it reads nothing, so the XML parser (in formatter) throws xml syntax error b/c it basically receives an empty string.
Not sure why you're saving to file, but once saved to file, you can simply read the file and send its contents to xml parser, and your code should now work. And of course if you remove saving to file, you should be able to read and parse contents of the response just fine.
I wrote a quick class to validate an XML file at a FilePath against an XSD with .NET (see below).
I have large volumes of data files being generated by another machine on the LAN, but the files are not true XML, they are malformed, but in the same way every time and based on their structure I can make some global replaces on the content of the file to correct it. So I have to correct these before testing with XSD. I have to replace <\ with </ and so on. All the replaces are listed in the code.
When I point this to the LAN network share of the machine generating the files at a list of about 50k files, and this took about 15 minutes to complete. I'm wondering if this is just IO capped by the LAN, or if there's a better (quicker) way to correct the malformed XML than the replaces I do here.
class VCheck
{
private static XmlReaderSettings settings = new XmlReaderSettings();
private bool valid;
string message;
public string Message { get { return message; } }
public VCheck()
{
settings.ValidationType = ValidationType.Schema;
settings.ValidationFlags |= XmlSchemaValidationFlags.ReportValidationWarnings;
settings.ValidationEventHandler += new ValidationEventHandler(ValidationCallBack);
settings.Schemas.Add(null, "schema.xsd");
}
public bool CheckFile(string FileFullPath)
{
StreamReader file = new StreamReader(FileFullPath);
valid = true;
message = null;
try
{ //setup xml reader with settings
XmlReader xml = XmlReader.Create(new StringReader(#"<?xml version='1.0'?><root xmlns=""MYE"">" +
file.ReadToEnd().Replace(#"<\", #"</").Replace("&", "&").Replace("\"", """).Replace("'", "'") + "</root>"),
settings);
while (xml.Read()) ; //read in all xml, validating against xsd
}
catch
{
//problem reading the xml file in, bad path, disk error etc.
return false;
}
return valid;
}
void ValidationCallBack(object sender, ValidationEventArgs e) //called on failed validations
{
valid = false;
message = e.Message;
switch (e.Severity)
{
case XmlSeverityType.Error:
//Do stuff on validation error
break;
case XmlSeverityType.Warning:
//Do stuff on validation warning
break;
}
}
}
I'd call it from main like this:
static void Main(string[] args)
{
VCheck checker = new VCheck();
foreach (string file in files) //files is a List<string> of file paths/names
{
if (!checker.CheckFile(file))
{
//To do stuff if not valid
}
}
}
I don't think reading it all into memory - ReadToEnd - and performing String.Replace on the contents is a good choice, with regard to your performance concerns.
If I were you, I'd rather rewrite those files "piece by piece" - that is, buffering and replacing data on the fly.
Just create a new file, load some of the malformed file into the buffer (say 4 kb), do the replacements, flush the results into your newly created file; rinse and repeat.
Beware: it can happen that one buffer ends with < and next one starts with \. If you want not to miss any <\s (and the like), you need to handle such cases as well.
Another possible solution is that you could try and create your own implementation of a "more tolerant" XmlReader (this class is not sealed, so you can base on it and create your own), although personally I haven't done it and I'm not sure this would be a good approach. Rewriting the files will at least leave you with syntactically valid XML, which may come in useful at some point.
PS. On a side note:
catch
{
//problem reading the xml file in, bad path, disk error etc.
return false;
}
I wouldn't do that. It leaves the caller with no idea whatsoever as for why the operation failed.
The quickest processes are those which do not need to be performed at all. So I commend Michael Kay's comments on dealing with "non-well-formed XML" to your attention.
If the non-XML data you'd like to handle as XML is being generated by a machine, there's no reason that that machine could not be generating XML data instead of the non-XML data you're currently trying to fix. Worse, every minute of effort you put into dealing with the errors in the data-producing process is a minute you've put into persuading those responsible for that process that they are producing correct, well-formed XML. So it's not only yourself you're hurting here.
It appears that JSON.NET is writing invalid JSON, although I wouldn't be surprised if it was due to my misuse.
It appears that it is repeating the last few characters of JSON:
/* ... */ "Teaser":"\nfoo.\n","Title":"bar","ImageSrc":null,"Nid":44462,"Vid":17}]}4462,"Vid":17}]}
The repeating string is:
4462,"Vid":17}]}
I printed it out to the console, so I don't think this is a bug in Visual Studio's text visualizer.
The serialization code:
static IDictionary<int, ObservableCollection<Story>> _sectionStories;
private static void writeToFile()
{
IsolatedStorageFile storage = IsolatedStorageFile.GetUserStoreForApplication();
using (IsolatedStorageFileStream stream = storage.OpenFile(STORIES_FILE, FileMode.OpenOrCreate))
{
using (StreamWriter writer = new StreamWriter(stream))
{
writer.Write(JsonConvert.SerializeObject(_sectionStories));
}
}
#if DEBUG
StreamReader reader = new StreamReader(storage.OpenFile(STORIES_FILE, FileMode.Open));
string contents = reader.ReadToEnd();
JObject data = JObject.Parse(contents);
string result = "";
foreach (char c in contents.Skip(contents.Length - 20))
{
result += c;
}
Debug.WriteLine(result);
// crashes here with ArgumentException
// perhaps because JSON is invalid?
var foo = JsonConvert.DeserializeObject<Dictionary<int, List<Story>>>(contents);
#endif
}
Am I doing something wrong here? Or is this a bug? Are there any known workarounds?
Curiously, JObject.Parse() doesn't throw any errors.
I'm building a Silverlight app for Windows Phone 7.
When writing the file you specify
FileMode.OpenOrCreate
If the file exists and is 16 bytes longer than the data you intend to write to it (from an older version of your data that just happens to end with the exact same data) then that data will still be present when you're done writing your new data.
Solution:
FileMode.Create
From:
http://msdn.microsoft.com/en-us/library/system.io.filemode.aspx
FileMode.Create: Specifies that the operating system should create a new file. If the file already exists, it will be overwritten
I have an application that crunches a bunch of text files. Currently, I have code like this (snipped-together excerpt):
FileInfo info = new FileInfo(...)
if (info.Length > 0) {
string content = getFileContents(...);
// uses a StreamReader
// returns reader.ReadToEnd();
Debug.Assert(!string.IsNullOrEmpty(contents)); // FAIL
}
private string getFileContents(string filename)
{
TextReader reader = null;
string text = "";
try
{
reader = new StreamReader(filename);
text = reader.ReadToEnd();
}
catch (IOException e)
{
// File is concurrently accessed. Come back later.
text = "";
}
finally
{
if (reader != null)
{
reader.Close();
}
}
return text;
}
Why am I getting a failed assert? The FileInfo.Length attribute was already used to validate that the file is non-empty.
Edit: This appears to be a bug -- I'm catching IO exceptions and returning empty-string. But, because of the discussion around fileInfo.Length(), here's something interesting: fileInfo.Length returns 2 for an empty, only-BOM-marker text file (created in Notepad).
You might have a file which is empty apart from a byte-order mark. I think TextReader.ReadToEnd() would remove the byte-order mark, giving you an empty string.
Alternatively, the file could have been truncated between checking the length and reading it.
For diagnostic purposes, I suggest you log the file length when you get an empty string.
See that catch (IOException) block you have? That's what returns an empty string and triggers the assert even when the file is not empty.
If I remember well, a file ends with end of file, which won't be included when you call ReadToEnd.
Therefore, the file size is not 0, but it's content size is.
What's in the getFileContents method?
It may be repositioning the stream's pointer to the end of the stream before ReadToEnd() is called.