I am serializing a class with a binaryformatter and compressing the data with deflatestream. The save function is as follows and is called from a backgroundworker:
public static void save(System system, String filePath)
{
//Make filestream
FileStream fs = new FileStream(filePath, FileMode.Create);
try
{
//Serialize offerte
BinaryFormatter bf = new BinaryFormatter();
DeflateStream cs = new DeflateStream(fs, CompressionMode.Compress);
bf.Serialize(cs, system);
//Push through
fs.Flush();
cs.Flush();
cs.Close();
}
catch (Exception e)
{
var mess = e.Message;
}
finally
{
//Close
fs.Close();
}
}
The class has a number of 'users'. With 100 users it takes 10 seconds and the file is 2MB. With 1000 users it gives an out-of-memory exception (the estimated size is 16MB). Can anyone see a problem here, or give suggestions how to solve this?
(I was first thinking the time on a background thread was causing this, it takes to long. But I have other background threads that can run longer.)
You aren't disposing of your streams, which may be part of the problem, suggest:
public static void save(System system, String filePath)
{
//Make filestream
using(FileStream fs = new FileStream(filePath, FileMode.Create))
{
//Serialize offerte
BinaryFormatter bf = new BinaryFormatter();
using (DeflateStream cs = new DeflateStream(fs, CompressionMode.Compress)) {
bf.Serialize(cs, system);
//Push through
fs.Flush();
cs.Flush();
cs.Close();
}
}
}
This also removes your exception swallowing, which would probably be a good thing.
You use several objects of classes that implement System.IDisposable
If a designer implemented IDisposable he informs you that he might use scarce resources. You might get out of resources before the garbage collector collects the garbage.
In other words: whenever you use a class that implements System.IDisposable you should call Dispose() as soon as you don't need the class anymore. This is especially needed if you need the resources of the class for something else.
You use two Stream classes: FileStream and DeflateStream. They both implement IDisposable. If you don't call Dispose(), the garbage collector eventually will, but in the mean time the resources that these Streams use are not available for anyone else.
The most easy method to make sure that a Disposable object is disposed is by using the using statement:
using (var myStream = new FileStream(...))
{
... // use myStream
}
When the closing bracket is reached, myStream.Dispose() is called, effectively releasing all scarce resources it uses.
This works on every method that is used to leave the {...} block, including break, return, and even Exceptions.
Therefore using is a very safe method: Dispose() will always be called.
By the way: Dispose() will also take care that the Streams are Flushed and Closed, so at the end of the using statement you don't have to Flush() and Close().
Related
I'm trying to use streams to progressively display a jpeg as it loads. This used to work fine, but this morning I tried running my code and now no images can be loaded because of this error. The relevant code is as follows:
using (WebClient wc = new WebClient())
using (Stream streamRemote = wc.OpenRead(url))
using (Stream streamLocal = new MemoryStream((int)fileSize))
{
int byteSize = 0;
byte[] buffer = new byte[fileSize];
while ((byteSize = streamRemote.Read(buffer, 0, buffer.Length)) > 0)
{
streamLocal.Write(buffer, 0, byteSize); // Error is here.
bytesDownloaded += byteSize;
int progressPercentage = (int)(bytesDownloaded / buffer.Length) * 100;
if (progressPercentage % 10 == 0)
{
imageLoader.ReportProgress(progressPercentage, streamLocal);
}
}
}
// Exception thrown: 'System.ObjectDisposedException' in mscorlib.dll
// An exception of type 'System.ObjectDisposedException' occurred in mscorlib.dll but was not handled in user code
// Cannot access a closed Stream.
After using Console.WriteLine at the end of the that final using statement (after the while loop), I've found that the code seems to run through the loop a couple times before throwing that exception.
I don't understand why I would be trying to access a closed stream when the code is clearly happening within the using statement that the stream is declared in. I also don't understand why it worked the other day and doesn't now. Most of the code for this comes from here, so the rest of my method can be found there. Mine isn't much different apart from some variable name changes and other small changes. Can anyone help fix this?
Edit: My _ProgressChanged event:
private void ImageLoader_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
if (!fileFailed)
{
Dispatcher.BeginInvoke(System.Windows.Threading.DispatcherPriority.Normal,
new Action(delegate ()
{
try
{
using (MemoryStream stream = e.UserState as MemoryStream)
{
BitmapImage bmp = new BitmapImage();
bmp.BeginInit();
using (MemoryStream ms = new MemoryStream(stream.ToArray())) bmp.StreamSource = ms;
bmp.EndInit();
img_CurrentImage.Source = bmp;
}
}
// A few catch statements here - none of the exceptions here are being thrown anyway so I'll omit the catch statements
}));
}
}
As I suspected, it's because you're misusing the MemoryStream in your ProgressChanged handler.
This line:
using (MemoryStream stream = e.UserState as MemoryStream)
Is taking the stream object that was passed to ProgressChanged() and will later on in the delegate Dispose of it. But that's exactly the same stream object that you're wondering how it got disposed inside your DoWork method.
That stream doesn't "belong" to the ProgressChanged handler. It shouldn't be disposing of it.
It's somewhat subtle, but in the linked question, the only thing that's done with the passed in stream is to access its ToArray method. Which you need to also be careful to do since, because ProgressChanged (and Dispatcher.BeginInvoke) is asynchronous, you could easily be dealing with an already disposed object in this handler (but ToArray is safe to call on disposed MemoryStreams)
You might also consider extracting the byte[] array from the MemoryStream inside your DoWork method and making that the passed state instead of the MemoryStream. That would make it far less prone to misuse both now and in any future edits to this code.
I have a class that needs to keep an instance of a BinaryWriter open over several write function calls (data is packet based). It also has to create a new file once it has written a certain amount of data/packets.
Normally I would just close the Binary Writer and reinstantiate it with a new file path, but the overhead associated with that operation is too great for my application. I tried closing the writer in a seperate thread, but that interferes with the new instance I create later.
My last ditch attempt was not to close the Writer (and stream) at all, and simply create a new instance of it everytime I'd written the required packets. This seems to work, and doesn't cause any memory leaks, but I'd really like to know what goes on if you do this.
Here is my (simplified) code to illustrate:
class Writer
{
BinaryWriter binWriter;
int bytesWritten;
int filesWritten;
const int maxFilesize = 10E9;
Writer(string filepath)
{
binWriter = new BinaryWriter(File.Open(filepath, FileMode.Create));
bytesWritten = 0;
filesWritten = 0;
}
WritePacket(byte[] packet)
{
if(bytesWritten<maxFileSize)
{
binWriter.Write(packet);
bytesWritten += packet.Length;
}
else
{
// this is where I'd normally call Dispose(), but the overhead
// is too high, and disposing the stream in a seperate thread
// interferes with the new one
// what actually happens here? it's the only thing I've found to
//work...
filesWritten++;
binWriter = new BinaryWriter(File.Open(filepath + filesWritten, FileMode.Create));
}
}
It feels bad, but this is the only solution that works so far. Any insight would be great!
Consider the following piece of code:
public static Stream OpenWavStream(string path)
{
var stream = new FileStream(path, FileMode.Open);
try
{
stream.Seek(44, SeekOrigin.Current);
}
catch (Exception)
{
stream.Dispose();
throw;
}
return stream;
}
I'm opening a wav stream whose data always starts at offset 44. If seeking to that offset fails, the stream is disposed, otherwise it is returned. Considering that catch (Exception) is considered bad practice, is it appropriate in this case?
Should one rather research the concrete exceptions (even though the stream should be disposed if any kind of exception happens within the Stream.Seek call) or move it into a finally block?
Only if the Stream fails to load. Something I use:
string fileName = "C:\\PCM.wav";
if (!System.IO.File.Exists(fileName))
{
LogStatus("Wave file not found.");
return;
}
else
{
WaveFileByteArray = File.ReadAllBytes(fileName);
LogStatus("Wave file Loaded!" + Environment.NewLine);
}
This works fine.
then to play/Use:
System.Media.SoundPlayer soundPlayer;
soundPlayer.Stream.Seek(0, SeekOrigin.Begin);
soundPlayer.Stream.Write(WaveFileByteArray, 0, WaveFileByteArray.Length);
soundPlayer.Play();
Relying on an Exception to catch possible errors, like is presented, is not best practice, unless the error is unexpected. Dealing with possible Errors before they occur is best practice.
catch (Exception) is bad practice if you are using it to swallow exceptions and not handling them. You are immediately re-throwing the exception, and doing it properly (you're not doing throw ex; for example). You will need to dispose the stream for any exception, so you should not catch specific exceptions here.
Your code is perfectly fine. However, I am skeptical about the usefulness of the method. Without seeing the rest of the application, it might make sense for the callee to create the stream within a using block, even with a helper method.
//In your callee code
using (var stream = new FileStream(path, FileMode.Open))
{
ConfigureStream(steam);
//Other stuff..
}
public static void ConfigureStream(Stream stream)
{
stream.Seek(44, SeekOrigin.Current);
}
Or, you could check the length of the stream first, to avoid the exception entirely.
Why not use a using block in calling method and leave the closing of steam to the system. using will close the stream even if there is an exception.
public static Stream OpenWavStream(string path)
{
var stream = new FileStream(path, FileMode.Open);
stream.Seek(44, SeekOrigin.Current);
return stream;
}
public static void UseWaveStream()
{
try
{
using(Stream thisStream = OpenWavStream("C:\\myfile.txt"))
{
// do whatever
}
}
catch(Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
Writing Stringbuilder to file asynchronously. This code takes control of a file, writes a stream to it and releases it. It deals with requests from asynchronous operations, which may come in at any time.
The FilePath is set per class instance (so the lock Object is per instance), but there is potential for conflict since these classes may share FilePaths. That sort of conflict, as well as all other types from outside the class instance, would be dealt with retries.
Is this code suitable for its purpose? Is there a better way to handle this that means less (or no) reliance on the catch and retry mechanic?
Also how do I avoid catching exceptions that have occurred for other reasons.
public string Filepath { get; set; }
private Object locker = new Object();
public async Task WriteToFile(StringBuilder text)
{
int timeOut = 100;
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
while (true)
{
try
{
//Wait for resource to be free
lock (locker)
{
using (FileStream file = new FileStream(Filepath, FileMode.Append, FileAccess.Write, FileShare.Read))
using (StreamWriter writer = new StreamWriter(file, Encoding.Unicode))
{
writer.Write(text.ToString());
}
}
break;
}
catch
{
//File not available, conflict with other class instances or application
}
if (stopwatch.ElapsedMilliseconds > timeOut)
{
//Give up.
break;
}
//Wait and Retry
await Task.Delay(5);
}
stopwatch.Stop();
}
How you approach this is going to depend a lot on how frequently you're writing. If you're writing a relatively small amount of text fairly infrequently, then just use a static lock and be done with it. That might be your best bet in any case because the disk drive can only satisfy one request at a time. Assuming that all of your output files are on the same drive (perhaps not a fair assumption, but bear with me), there's not going to be much difference between locking at the application level and the lock that's done at the OS level.
So if you declare locker as:
static object locker = new object();
You'll be assured that there are no conflicts with other threads in your program.
If you want this thing to be bulletproof (or at least reasonably so), you can't get away from catching exceptions. Bad things can happen. You must handle exceptions in some way. What you do in the face of error is something else entirely. You'll probably want to retry a few times if the file is locked. If you get a bad path or filename error or disk full or any of a number of other errors, you probably want to kill the program. Again, that's up to you. But you can't avoid exception handling unless you're okay with the program crashing on error.
By the way, you can replace all of this code:
using (FileStream file = new FileStream(Filepath, FileMode.Append, FileAccess.Write, FileShare.Read))
using (StreamWriter writer = new StreamWriter(file, Encoding.Unicode))
{
writer.Write(text.ToString());
}
With a single call:
File.AppendAllText(Filepath, text.ToString());
Assuming you're using .NET 4.0 or later. See File.AppendAllText.
One other way you could handle this is to have the threads write their messages to a queue, and have a dedicated thread that services that queue. You'd have a BlockingCollection of messages and associated file paths. For example:
class LogMessage
{
public string Filepath { get; set; }
public string Text { get; set; }
}
BlockingCollection<LogMessage> _logMessages = new BlockingCollection<LogMessage>();
Your threads write data to that queue:
_logMessages.Add(new LogMessage("foo.log", "this is a test"));
You start a long-running background task that does nothing but service that queue:
foreach (var msg in _logMessages.GetConsumingEnumerable())
{
// of course you'll want your exception handling in here
File.AppendAllText(msg.Filepath, msg.Text);
}
Your potential risk here is that threads create messages too fast, causing the queue to grow without bound because the consumer can't keep up. Whether that's a real risk in your application is something only you can say. If you think it might be a risk, you can put a maximum size (number of entries) on the queue so that if the queue size exceeds that value, producers will wait until there is room in the queue before they can add.
You could also use ReaderWriterLock, it is considered to be more 'appropriate' way to control thread safety when dealing with read write operations...
To debug my web apps (when remote debug fails) I use following ('debug.txt' end up in \bin folder on the server):
public static class LoggingExtensions
{
static ReaderWriterLock locker = new ReaderWriterLock();
public static void WriteDebug(string text)
{
try
{
locker.AcquireWriterLock(int.MaxValue);
System.IO.File.AppendAllLines(Path.Combine(Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase).Replace("file:\\", ""), "debug.txt"), new[] { text });
}
finally
{
locker.ReleaseWriterLock();
}
}
}
Hope this saves you some time.
I have an issue from time to time, I have a few StreamReaders and StreamWriters in my program that read info and write it. They go right about 99% of the time, but once in a while I end up with a StreamWriter that won't close, on a piece of code I've run multiple times.
This tends to happen if I spam a function, but I am trying to find a safe way to guarantee a steam disposed. Anyone know how?
try a using statement MSDN
using (StreamWriter stream = new StreamWriter(Initialization)){
//your code
}
this can be useful:
Closing Stream Read and Stream Writer when the form exits
Also you could use a Try Block
try
{
//Declare your streamwriter
StreamWriter sw = new StreamWriter(Initialization);
}
catch
{
//Handle the errors
}
finally
{
sw.Dispose();
}
If the stream's scope is local, always use the following construct:
using (var stream = new Stream())
{
...do stream work here...
}
If on the other hand you are using the stream as a class field then implement the IDisposable pattern and dispose your stream objects when disposing your class: IDisposable
Wrapping the StreamWriter in a using statement is how I usually ensure it is disposed of.
using (var writer = new StreamWriter(#"C:\AFile.txt"))
{
//do some stuff with writer
}
An alternative would be to use a finally block.