I am using a create method for a constructor of a converter.
public void loadData()
{
byte [] data = new byte [] {......}; // some byte data in here
var converter = GetDataConverter(data);
}
Now inside the GetDataConverter I need to create a memorystream from the binary data (the converter is 3rd party and takes a stream)
If I write the GetDataConverter like this I get an error telling me I didnt' dispose which I understand. I created a MemoryStream and I need to dispose.
public MyDataConverter GetDataConverter(byte [] data)
{
return new MyDataConverter(new MemoryStream(data));
}
So my solution would be this:
public MyDataConverter GetDataConverter(byte [] data)
{
using(var ms = new MemoryStream(data))
{
return new MyDataConverter(ms);
}
}
The question is, is my solution correct? Should I be using a 'using' here? isn't the 'using' going to destroy my memory stream once it's out of scope so the converter will have nothing to work on?
I need an answer AND an explanation please, I'm a bit vague on the whole 'using' thing here.
Thanks
If you have no access to the code of ´MyDataConverter´ and the type doesn't implements ´IDisposable´ you can do:
public void loadData()
{
byte[] data = new byte[] { 0 }; // some byte data in here
using (var stream = new MemoryStream(data))
{
var converter = new MyDataConverter(stream);
// do work here...
}
}
If you have to use this many times and want to reuse your code you can do something like this:
public void loadData()
{
byte[] data = new byte[] { 0 }; // some byte data in here
UsingConverter(data, x =>
{
// do work here...
});
}
void UsingConverter(byte[] data, Action<MyDataConverter> action)
{
using (var stream = new MemoryStream(data))
{
var converter = new MyDataConverter(stream);
action(converter);
}
}
It really depends on the implementation of MyDataConverter. If the MemoryStream is only used inside the constructor to retrieve some data from it, then your solution with using is OK.
If, OTOH, MyDataConverter keeps a reference to the MemoryStream to access it later, you must not dispose it here.
Related
With the new C# 8 Using Declaration Syntax, what is containing scope of a second consecutive using statement?
TL;DR
Previous to C# 8, having a consecutive using statement like:
using(var disposable = new MemoryStream())
{
using(var secondDisposable = new StreamWriter(disposable))
{}
}
would expand to something like the following (My Source):
MemoryStream disposable = new MemoryStream();
try {
{
StreamWriter secondDisposable = new StreamWriter(disposable);
try{
{}
}
finally {
if(secondDisposable != null) ((IDisposable)secondDisposable).Dispose();
}
}
}
finally {
if(disposable != null) ((IDisposable)disposable).Dispose();
}
I know that there are two other possible expansions but they all are roughly like this
After upgrading to C# 8, Visual studio offered a Code Cleanup suggestion that I'm not certain I believe is an equivalent suggestion.
It converted the above consecutive using statement to:
using var disposable = new MemoryStream();
using var secondDisposable = new StreamWriter(disposable);
To me this changes the second's scope to the same scope as the first. In this case, It would probably coincidentally dispose of the streams in the correct order, but I'm not certain I like to rely on that happy coincidence.
To be clear on what VS asked me to do: I first converted the inner (which made sense because the inner was still contained in the outer's scope). Then I converted the outer (which locally made sense because it was still contained in the method's scope). The combination of these two clean ups is what I'm curious about.
I also recognize that my thinking on this could be slightly (or even dramatically) off, but as I understand it today, this doesn't seem correct. What is missing in my assessment? Am I off base?
The only thing I can think of is that there is some sort of an implicit scope inserted in the expansion for everything following a declaration statement.
In this case, It would probably coincidentally dispose of the streams in the correct order, but I'm not certain I like to rely on that happy coincidence.
From the spec proposal:
The using locals will then be disposed in the reverse order in which they are declared.
So, yes, they already thought about it and do the disposal in the expected order, just as chained using statements would before it.
To Illustrate the Daminen's answer; When you have a method something like;
public void M()
{
using var f1 = new System.IO.MemoryStream(null,true);
using var f2 = new System.IO.MemoryStream(null,true);
using var f3 = new System.IO.MemoryStream(null,true);
}
IL converts it into;
public void M()
{
MemoryStream memoryStream = new MemoryStream(null, true);
try
{
MemoryStream memoryStream2 = new MemoryStream(null, true);
try
{
MemoryStream memoryStream3 = new MemoryStream(null, true);
try
{
}
finally
{
if (memoryStream3 != null)
{
((IDisposable)memoryStream3).Dispose();
}
}
}
finally
{
if (memoryStream2 != null)
{
((IDisposable)memoryStream2).Dispose();
}
}
}
finally
{
if (memoryStream != null)
{
((IDisposable)memoryStream).Dispose();
}
}
}
Which is same as nested using statements
you can check from here: https://sharplab.io/#v2:CYLg1APgAgTAjAWAFBQMwAJboMLoN7LpGYZQAs6AsgBQCU+hxTUADOgG4CGATugGZx0AXnQA7AKYB3THAB0ASQDysyuIC2Ae24BPAMoAXbuM5rqogK4AbSwBpD58bQDcTRkyKsOPfjGFipMgrKqpo6BkYmZla29o5Obu6eXLx8GCIS0lBySirqWnqGxqYW1nbcDs4JAL7IVUA===
I'd like to see the real function that's using this. The compiler won't change scope or sequence of allocations or disposals willy-nilly. If you have a method like:
void foo()
{
using(var ms = new MemoryStream())
{
using(var ms2 = new MemoryStream())
{
/// do something
}
}
}
Then the Dispose() order doesn't matter, so it's safe for the compiler to arrange things however it sees fit. There may be other cases where the order is important, and the compiler should be smart enough to recognize that. I wouldn't file that under "coincidence" so much as "good AST analysis."
I have an issue with the following code. I create a memory stream in the GetDB function and the return value is used in a using block. For some unknown reason if I dump my objects I see that the MemoryStream is still around at the end of the Main method. This cause me a massive leak. Any idea how I can clean this buffer ?
I have actually checked that the Dispose method has been called on the MemoryStream but the object seems to stay around, I have used the diagnostic tools of Visual Studio 2017 for this task.
class Program
{
static void Main(string[] args)
{
List<CsvProduct> products;
using (var s = GetDb())
{
products = Utf8Json.JsonSerializer.Deserialize<List<CsvProduct>>(s).ToList();
}
}
public static Stream GetDb()
{
var filepath = Path.Combine("c:/users/tom/Downloads", "productdb.zip");
using (var archive = ZipFile.OpenRead(filepath))
{
var data = archive.Entries.Single(e => e.FullName == "productdb.json");
using (var s = data.Open())
{
var ms = new MemoryStream();
s.CopyTo(ms);
ms.Seek(0, SeekOrigin.Begin);
return (Stream)ms;
}
}
}
}
For some unknown reason if I dump my objects I see that the MemoryStream is still around at the end of the Main method.
That isn't particuarly abnormal; GC happens separately.
This cause me a massive leak.
That isn't a leak, it is just memory usage.
Any idea how I can clean this buffer ?
I would probably just not use a MemoryStream, instead returning something that wraps the live uncompressing stream (from s = data.Open()). The problem here, though, is that you can't just return s - as archive would still be disposed upon leaving the method. So if I needed to solve this, I would create a custom Stream that wraps an inner stream and which disposes a second object when disposed, i.e.
class MyStream : Stream {
private readonly Stream _source;
private readonly IDisposable _parent;
public MyStream(Stream, IDisposable) {...assign...}
// not shown: Implement all Stream methods via `_source` proxy
public override void Dispose()
{
_source.Dispose();
_parent.Dispose();
}
}
then have:
public static Stream GetDb()
{
var filepath = Path.Combine("c:/users/tom/Downloads", "productdb.zip");
var archive = ZipFile.OpenRead(filepath);
var data = archive.Entries.Single(e => e.FullName == "productdb.json");
var s = data.Open();
return new MyStream(s, archive);
}
(could be improved slightly to make sure that archive is disposed if an exception happens before we return with success)
I'm extending BinaryWriter using a MemoryStream.
public class PacketWriter : BinaryWriter
{
public PacketWriter(Opcode op) : base(CreateStream(op))
{
this.Write((ushort)op);
}
private static MemoryStream CreateStream(Opcode op) {
return new MemoryStream(PacketSizes.Get(op));
}
public WriteCustomThing() {
// Validate that MemoryStream has space?
// Do all the stuff
}
}
Ideally, I want to use write using PacketWriter as long as there is space available (which is already defined in PacketSizes). If there isn't space available, I want an exception thrown. It seems like MemoryStream just dynamically allocates more space if you write over capacity, but I want a fixed capacity. Can I achieve this without needing to check the length every time? The only solution I thought of so far was to override all the Write methods of BinaryWriter and compare lengths, but this is annoying.
Just provide a buffer of the desired size to write into:
using System;
using System.IO;
class Test
{
static void Main()
{
var buffer = new byte[3];
var stream = new MemoryStream(buffer);
stream.WriteByte(1);
stream.WriteByte(2);
stream.WriteByte(3);
Console.WriteLine("Three successful writes");
stream.WriteByte(4); // This throws
Console.WriteLine("Four successful writes??");
}
}
This is documented behavior:
Initializes a new non-resizable instance of the MemoryStream class based on the specified byte array.
With a little help from the performance tips in the Json.NET docs, I put together a method for downloading/deserializing JSON from a remote resource:
public async Task<T> GetJsonAsync<T>(string url)
{
using (var stream = await new HttpClient().GetStreamAsync(url))
{
using (var sr = new StreamReader(stream))
{
using (var jr = new JsonTextReader(sr))
{
return new JsonSerializer().Deserialize<T>(jr);
}
}
}
}
I'd like to have a non-generic version that returns a dynamic. Calling the above method with GetJsonAsync<dynamic>(url) works, until you try to access a dynamic property on the result, at which point I get:
'Newtonsoft.Json.Linq.JObject' does not contain a definition for '[MyProperty]'
I have seen how to deserialize to a dynamic from a string, but have not seen a working example of doing it directly from a stream, which would be preferable as it is more memory efficient. Is this possible?
It turns out this had little to do with Json.NET and more to do with my understanding of dynamics (which I rarely use). Thanks to #Peter Richie, I found that GetJsonAsync<dynamic> does work if I explicitly cast MyProperty to a string. But I'd rather not have to do that. Using my original method and a real working endpoint, here are 3 scenarios; only the last one works:
var url = "http://echo.jsontest.com/MyProperty/MyValue"; // great testing site!
var x1 = await GetJsonAsync<dynamic>(url);
Assert.AreEqual("MyValue", x1.MyProperty); // fail!
dynamic x2 = await GetJsonAsync<dynamic>(url);
Assert.AreEqual("MyValue", x2.MyProperty); // fail!
dynamic x3 = await GetJsonAsync<ExpandoObject>(url);
Assert.AreEqual("MyValue", x3.MyProperty); // pass!
Armed with that knowledge, the non-generic overload of my original method looks like this:
public async Task<dynamic> GetJsonAsync(string url) {
dynamic d = await GetJsonAsync<ExpandoObject>(url);
return d;
}
And users can do this:
var x = await GetJsonAsync(url);
Assert.AreEqual("MyValue", x.MyProperty); // pass!
It sounds like there's some information you haven't provided. The following works fine for me:
private T ReadJson<T>(Stream stream)
{
using (var reader = new StreamReader(stream))
{
using (var jr = new JsonTextReader(reader))
{
dynamic d = new JsonSerializer().Deserialize(jr);
return d;
}
}
}
//...
var d = ReadJson<dynamic>(new MemoryStream(Encoding.UTF8.GetBytes("{'MyProperty' : 'MyValue'}")));
Debug.WriteLine((String)d.MyProperty);
I have a class DocumentGenerator which wraps a MemoryStream. So I have implemented IDisposable on the class.
I can't see how/where I can possibly dispose it though.
This is my current code, which performs a file download in MVC:
using (DocumentGenerator dg = DocumentGenerator.OpenTemplate(path))
{
/* some document manipulation with the
DocumentGenerator goes here ...*/
return File(dg.GetDocumentStream(), "text/plain", filename);
}
This errors as the stream is closed/disposed before the controller has finished with it. How can I make sure my resources are properly disposed in this situation?
EDIT: My implementation of IDisposable at the moment just disposes the MemoryStream. I know it's not a proper implementation, I just used it as a test. Is there something different I could do here to make it work?
public void Dispose()
{
_ms.Dispose();
_ms = null;
}
You don't need to dispose the stream. It will be disposed by the FileStreamResult.WriteFile method. Code excerpt from this class:
public FileStreamResult(Stream fileStream, string contentType) : base(contentType)
{
if (fileStream == null)
{
throw new ArgumentNullException("fileStream");
}
this.FileStream = fileStream;
}
protected override void WriteFile(HttpResponseBase response)
{
Stream outputStream = response.OutputStream;
using (this.FileStream)
{
byte[] buffer = new byte[0x1000];
while (true)
{
int count = this.FileStream.Read(buffer, 0, 0x1000);
if (count == 0)
{
return;
}
outputStream.Write(buffer, 0, count);
}
}
}
Notice the using. When you call File(dg.GetDocumentStream(), "text/plain", filename) from your controller this invokes the constructor which stores the stream into a public property which is disposed during the rendering.
Conclusion: you don't need to worry about disposing the stream obtain with dg.GetDocumentStream().
Just to add to what Darin has said, it's important to note this concept:
public Stream GetDownloadFile(...)
{
using (var stream = new MemoryStream()) {
return stream;
}
}
public Stream GetDownloadFile(...)
{
using (var generator = DocumentGenerator.OpenTemplate(path))
{
// Document manipulation.
return File(generator.GetDocumentStream(), "text/plain", filename);
}
}
Regardless of how you are using it in your method, the using block ensures that Dispose is always called, this is important when you consider to use the result of the using block as a return statement, it won't stop it from being disposed....