I have a class MyData which is Json serializable by using Json.Net JsonSerializer.Serialize(TextWriter, object). I want to send this data (as json) to a web service via HttpClient.PostAsync.
Because converting the json to string and then sending it as StringContent is (probably) not performant, I want to do it with streams.
I found the class StreamContent, which takes a stream in its constructor. And serializing json into streams should be possible as well. So I tried this:
MyData data = ...; // already filled
string uri = ...; // already filled
HttpClient client = new HttpClient();
JsonSerializer serializer = new JsonSerializer();
using (MemoryStream ms = new MemoryStream())
{
using (StreamWriter sw = new StreamWriter(ms))
using (JsonWriter jw = new JsonTextWriter(sw))
{
serializer.Serialize(sw, data);
ms.Flush();
ms.Position = 0;
}
HttpResponseMessage response = client.PostAsync(uri, new StreamContent(ms)).Result;
}
But running this code gives me two exceptions in the line HttpResponseMessage response = ...:
HttpRequestException: Error when copying content into a stream.
ObjectDisposedException: Could not access closed stream.
What am I doing wrong?
If you serialize the object into a MemoryStream, the entire JSON data will be written in the buffer, so there is no significant performance benefit over just serializing into a string and using StringContent.
Your StremWriter disposes the memory stream before the request is sent, that is why you get the exceptions.
You can either move your using statements to be in the same scope as the MemoryStream, or use the StreamWriter's constructor that accepts a boolean parameter to leave the stream open after the writer is disposed.
StreamWriter constructor:
Unless you set the leaveOpen parameter to true, the StreamWriter object calls Dispose() on the provided Stream object when StreamWriter.Dispose is called.
Related
I have azure function, which accepts huge bson object. It binds to http request and then try to deserialize it using stream with the following code:
using (var stream = new MemoryStream())
{
await request.Content.CopyToAsync(stream);
using (var reader = new BsonDataReader(stream))
{
var serializer = new JsonSerializer();
var readings =
serializer.Deserialize<IEnumerable<ProviderReading>>(reader);
}
}
readings object is always null.
I tested it using the standard ReadAsAsync method:
var test = await request.Content.ReadAsAsync<List<ProviderReading>>(
new[]{new BsonMediaTypeFormatter()});
in that case it deserialize the collection of readings correctly.
Any suggestions?
Using CopyTo (or its async variant) advances both the source's and target's position. That means that by the time you construct the BsonDataReader the input stream is already at its end.
You should reset the stream's position:
stream.Position = 0;
I am trying to serialize large object using below. But sometimes it throws out of memory exception.
Method 1:
var settings = new JsonSerializerSettings { ContractResolver = BaseFirstContractResolver.Instance };
string jsonString = JsonConvert.SerializeObject(listObj, Newtonsoft.Json.Formatting.Indented, settings);
So I have decided to serialize object to file like below,
Method 2:
string path = System.Web.Configuration.WebConfigurationManager.AppSettings["jsonPath"].ToString();
using (StreamWriter file = File.CreateText(path))
{
JsonSerializer serializer = new JsonSerializer();
serializer.Serialize(file, listObj);
}
}
Also i am returning json data from file as HttpResponseMessage,
string path = System.Web.Configuration.WebConfigurationManager.AppSettings["jsonPath"].ToString();
var stream = new System.IO.FileStream(path, System.IO.FileMode.Open);
var response = this.Request.CreateResponse(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
return response;
I am getting data fast in method 1 than method 2. Is there any performance difference between these two methods ?
The first method is using memory while the second is writing to disk and then reading from disk again. The first method will always be faster, but it will consume more memory. The difference in performance will rely heavily on IO speeds of both memory and disk. I personally would avoid method 2 at all costs. I would implement paging on method 1 if the object was some type of collection. If I was forced to send the entire collection in one request then use a yeild return for chunks of the collection. That would create a chunked transfer encoding stream. If the object was indeed just a huge single object then some type of stream deserializer to again a chunked transfer encoding stream would be needed.
I use System.Net.Http.HttpClient Object to login to a forum successfully and do many operations. Then the object may have some state, so I want to reuse the object when the program is been opened again, implementing the automatic login.
I use Json.net to serialize the HttpClient Object to the IsolatedStorage, but not available.
Can somebody help me? Thank you!!
//The code for initializing the HttpClient Object
HttpClientHandler handler = new HttpClientHandler();
handler.UseCookies = true;
HttpCliclient = new HttpClient(handler);
I first serialize the object using Json.net, get the json string:
{"DefaultRequestHeaders":],"BaseAddress":null,"Timeout":"00:01:40","MaxResponseContentBufferSize":2147483647}
Then deserialize the json string to get the object, but the object need logining again to do operations.
using (IsolatedStorageFile storeFolder = IsolatedStorageFile.GetUserStoreForApplication()) {
string jsonData = JsonConvert.SerializeObject(httpclient);
System.Diagnostics.Debug.WriteLine(jsonData);
using (IsolatedStorageFileStream stream = storeFolder.CreateFile(path))
using (StreamWriter writer = new StreamWriter(stream))
writer.Write(jsonData);
}
using (IsolatedStorageFile storeFolder = IsolatedStorageFile.GetUserStoreForApplication()) {
string jsonData;
using (IsolatedStorageFileStream stream = storeFolder.OpenFile(path, FileMode.Open))
using (StreamReader reader = new StreamReader(stream))
jsonData = reader.ReadToEnd();
HttpClient httpclient = JsonConvert.DeserializeObject<HttpClient>(jsonData);
//need login again to do some operations
}
Don't try and serialize HttpClient. The chances of it working are highly unlikely. Pretty much the only state you might be able to serialize from it are the default request headers. Create a HttpClient Factory method and just serialize out the header information that you want to preserve.
I use the following snippet of code, and I'm unsure whether I need to call the Flush methods (once on StreamWriter, once on MemoryStream):
//converts an xsd object to the corresponding xml string, using the UTF8 encoding
public string Serialize(T t)
{
using (var memoryStream = new MemoryStream())
{
var encoding = new UTF8Encoding(false);
using (var writer = new StreamWriter(memoryStream, encoding))
{
var serializer = new XmlSerializer(typeof (T));
serializer.Serialize(writer, t);
writer.Flush();
}
memoryStream.Flush();
return encoding.GetString(memoryStream.ToArray());
}
}
First of all, because the code is inside the using block, I think the automatically called dispose method might do this for me. Is this true, or is flushing an entirely different concept?
According to stackoverflow itself:
Flush meaning clears all buffers for a stream and causes any buffered data to be written to the underlying device.
What does that mean in the context of the code above?
Secondly, the flush method of the MemoryStream does nothing according to the api, so what's up with that? why do we call a method that does nothing?
You don't need to use Flush on the StreamWriter, as you are disposing it (by having it in a using block). When it's disposed, it's automatically flushed and closed.
You don't need to use Flush on the MemoryStream, as it's not buffering anything that is written to any other source. There is simply nothing to flush anywhere.
The Flush method is only present in the MemoryStream object because it inherits from the Stream class. You can see in the source code for the MemoryStream class that the flush method actually does nothing.
In general Streams will buffer data as it's written (periodically flushing the buffer to the associated device if there is one) because writing to a device, usually a file, is expensive. A MemoryStream writes to RAM so the whole concept of buffering and flushing is redundant. The data is always in RAM already.
And yes, disposing the stream will cause it to be flushed.
Commenting flush method returning empty byte[], Though I am Using Using block
byte[] filecontent = null;
using var ms = new MemoryStream();
using var sw = new StreamWriter(fs);
sw.WriteCSVLine(new[] { "A", "B" });//This is extension to write as CSV
//tx.Flush();
//fs.Flush();
fs.Position = 0;
filecontent = fs.ToArray();
Im trying to prompt a downloadable text file (.txt), but I get this error:
Cannot access a closed Stream.
I have looked at simular questions in here:
Cannot Access Closed Stream
But it was not very useful.
Here is my code:
private FileStreamResult Export()
{
string name = "filename.txt";
MemoryStream stream = new MemoryStream();
using (StreamWriter writer = new StreamWriter(stream))
{
StringBuilder sb = new StringBuilder();
sb.Append("A text...");
writer.WriteLine(sb.ToString());
}
return File(stream, "text/plain", name);
}
UPDATE (working copy):
This gives me an blank text file.
private FileResult Export()
{
string name = "filename.txt";
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
StringBuilder sb = new StringBuilder();
sb.Append("A text...");
writer.WriteLine(sb.ToString());
writer.Flush();
stream.Seek(0, SeekOrigin.Begin);
return File(stream, "text/plain", name);
}
That is correct, when you wrap a stream in another stream, calling .Close() or .Dispose() on any of them will dispose the whole stream. In this case, wrapping the MemoryStream in a StreamWriter means that when the using statement completes the StreamWriter and MemoryStream are both disposed.
My guess is since you are returning a FileStreamResult the encapsulating File will close the stream for you after the stream is no longer used. In this case, you do not want to use the using statement and will want to leave the stream open when returning it.
UPDATE
Since a stream is forward access you'll need to see the stream back to the beginning to allow the data to be read back out.
stream.Seek(0, SeekOrigin.Begin);
Just remove that using statement - you are passing disposed object reference to File method and you that's the reason why you get exception. From MSDN,
The StreamWriter object calls Dispose on the provided Stream object
when StreamWriter.Dispose is called.
I believe File will dispose stream by itself after usage (not verified by looking at source code).
UPDATE:
writer.Flush(); before return statement should help you
You have to set the position of the memorystream to 0 before using it in your FileStreamResult, otherwise it will be read from current position (IE the end of the stream).
stream.Position = 0;
return File(stream, "text/plain", name);
Just had the same thing.
I know this thread is ancient, just hoping to aid others having the same issue.
Replace the FileStreamResult type on your action with FileResult.