We Keep Coding

MemoryStream throws ObjectDisposedException when trying to access it in its own using statement

I'm trying to use streams to progressively display a jpeg as it loads. This used to work fine, but this morning I tried running my code and now no images can be loaded because of this error. The relevant code is as follows:
using (WebClient wc = new WebClient())
using (Stream streamRemote = wc.OpenRead(url))
using (Stream streamLocal = new MemoryStream((int)fileSize))
{
int byteSize = 0;
byte[] buffer = new byte[fileSize];
while ((byteSize = streamRemote.Read(buffer, 0, buffer.Length)) > 0)
{
streamLocal.Write(buffer, 0, byteSize); // Error is here.
bytesDownloaded += byteSize;
int progressPercentage = (int)(bytesDownloaded / buffer.Length) * 100;
if (progressPercentage % 10 == 0)
{
imageLoader.ReportProgress(progressPercentage, streamLocal);
}
}
}
// Exception thrown: 'System.ObjectDisposedException' in mscorlib.dll
// An exception of type 'System.ObjectDisposedException' occurred in mscorlib.dll but was not handled in user code
// Cannot access a closed Stream.
After using Console.WriteLine at the end of the that final using statement (after the while loop), I've found that the code seems to run through the loop a couple times before throwing that exception.
I don't understand why I would be trying to access a closed stream when the code is clearly happening within the using statement that the stream is declared in. I also don't understand why it worked the other day and doesn't now. Most of the code for this comes from here, so the rest of my method can be found there. Mine isn't much different apart from some variable name changes and other small changes. Can anyone help fix this?
Edit: My _ProgressChanged event:
private void ImageLoader_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
if (!fileFailed)
{
Dispatcher.BeginInvoke(System.Windows.Threading.DispatcherPriority.Normal,
new Action(delegate ()
{
try
{
using (MemoryStream stream = e.UserState as MemoryStream)
{
BitmapImage bmp = new BitmapImage();
bmp.BeginInit();
using (MemoryStream ms = new MemoryStream(stream.ToArray())) bmp.StreamSource = ms;
bmp.EndInit();
img_CurrentImage.Source = bmp;
}
}
// A few catch statements here - none of the exceptions here are being thrown anyway so I'll omit the catch statements
}));
}
}

As I suspected, it's because you're misusing the MemoryStream in your ProgressChanged handler.
This line:
using (MemoryStream stream = e.UserState as MemoryStream)
Is taking the stream object that was passed to ProgressChanged() and will later on in the delegate Dispose of it. But that's exactly the same stream object that you're wondering how it got disposed inside your DoWork method.
That stream doesn't "belong" to the ProgressChanged handler. It shouldn't be disposing of it.
It's somewhat subtle, but in the linked question, the only thing that's done with the passed in stream is to access its ToArray method. Which you need to also be careful to do since, because ProgressChanged (and Dispatcher.BeginInvoke) is asynchronous, you could easily be dealing with an already disposed object in this handler (but ToArray is safe to call on disposed MemoryStreams)
You might also consider extracting the byte[] array from the MemoryStream inside your DoWork method and making that the passed state instead of the MemoryStream. That would make it far less prone to misuse both now and in any future edits to this code.

BinaryFormatter.Deserialize hangs the whole thread

I have two simple applications connected via named pipes. In the client side I have a method that checks incoming messages every n ms:
private void timer_Elapsed(Object sender, ElapsedEventArgs e)
{
IFormatter f = new BinaryFormatter();
try
{
object temp = f.Deserialize(pipeClient); //hangs here
result = (Func<T>)temp;
}
catch
{
}
}
In the beginning the pipe is empty, and f.Deserialize method hangs the whole application. And I can't even check that pipe's empty? Is there any solution to this problem?
UPD: tried XmlSerializer, everything's the same.

The thing that is hanging on you is the pipeClient.Read( call both formatters are making internally.
This is the expected behavior of a Stream, when you call Read:
Return Value
Type: System.Int32
The total number of bytes that are read into buffer. This might be less than the number of bytes
requested if that number of bytes is not currently available, or 0 if
the end of the stream is reached.
So the stream will block till data shows up or throw a timeout exception if it is the type of stream that supports timeouts. It will never just return without reading anything unless you are "at the end of the stream" which for a PipeStream (or similarly a NetworkStream) only happens when the connection is closed.
The way you solve the problem is don't use a timer to check if a new message arrives, just start up a background thread and have it sitting in a loop, it will block itself until a message shows up.
class YourClass
{
public YourClass(PipeStream pipeClient)
{
_pipeClient = pipeClient;
var task = new Task(MessageHandler, TaskCreationOptions.LongRunning);
task.Start();
}
//SNIP...
private void MessageHandler()
{
while(_pipeClient.IsConnected)
{
IFormatter f = new BinaryFormatter();
try
{
object temp = f.Deserialize(_pipeClient);
result = (Func<T>)temp;
}
catch
{
//You really should do some kind of logging.
}
}
}
}

Memory Mapped File gets deleted from memory

For some reason, when i read from a memory mapped file a couple of times it just gets randomly deleted from memory, i don't know what's going on. Is the kernel or GC deleting it from memory? If they are, how do i prevent them from doing so?
I am serializing an object to Json and writing it to memory.
I get an exception when trying to read again after a couple of times, i get FileNotFoundException: Unable to find the specified file.
private const String Protocol = #"Global\";
Code to write to memory mapped file:
public static Boolean WriteToMemoryFile<T>(List<T> data)
{
try
{
if (data == null)
{
throw new ArgumentNullException("Data cannot be null", "data");
}
var mapName = typeof(T).FullName.ToLower();
var mutexName = Protocol + typeof(T).FullName.ToLower();
var serializedData = JsonConvert.SerializeObject(data);
var capacity = serializedData.Length + 1;
var mmf = MemoryMappedFile.CreateOrOpen(mapName, capacity);
var isMutexCreated = false;
var mutex = new Mutex(true, mutexName, out isMutexCreated);
if (!isMutexCreated)
{
var isMutexOpen = false;
do
{
isMutexOpen = mutex.WaitOne();
}
while (!isMutexOpen);
var streamWriter = new StreamWriter(mmf.CreateViewStream());
streamWriter.WriteLine(serializedData);
streamWriter.Close();
mutex.ReleaseMutex();
}
else
{
var streamWriter = new StreamWriter(mmf.CreateViewStream());
streamWriter.WriteLine(serializedData);
streamWriter.Close();
mutex.ReleaseMutex();
}
return true;
}
catch (Exception ex)
{
return false;
}
}
Code to read from memory mapped file:
public static List<T> ReadFromMemoryFile<T>()
{
try
{
var mapName = typeof(T).FullName.ToLower();
var mutexName = Protocol + typeof(T).FullName.ToLower();
var mmf = MemoryMappedFile.OpenExisting(mapName);
var mutex = Mutex.OpenExisting(mutexName);
var isMutexOpen = false;
do
{
isMutexOpen = mutex.WaitOne();
}
while (!isMutexOpen);
var streamReader = new StreamReader(mmf.CreateViewStream());
var serializedData = streamReader.ReadLine();
streamReader.Close();
mutex.ReleaseMutex();
var data = JsonConvert.DeserializeObject<List<T>>(serializedData);
mmf.Dispose();
return data;
}
catch (Exception ex)
{
return default(List<T>);
}
}

The process that created the memory mapped file must keep a reference to it for as long as you want it to live. Using CreateOrOpen is a bit tricky for exactly this reason - you don't know whether disposing the memory mapped file is going to destroy it or not.
You can easily see this at work by adding an explicit mmf.Dispose() to your WriteToMemoryFile method - it will close the file completely. The Dispose method is called from the finalizer of the mmf instance some time after all the references to it drop out of scope.
Or, to make it even more obvious that GC is the culprit, you can try invoking GC explicitly:
WriteToMemoryFile("Hi");
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
ReadFromMemoryFile().Dump(); // Nope, the value is lost now
Note that I changed your methods slightly to work with simple strings; you really want to produce the simplest possible code that reproduces the behaviour you observe. Even just having to get JsonConverter is an unnecessary complication, and might cause people to not even try running your code :)
And as a side note, you want to check for AbandonedMutexException when you're doing Mutex.WaitOne - it's not a failure, it means you took over the mutex. Most applications handle this wrong, leading to issues with deadlocks as well as mutex ownership and lifetime :) In other words, treat AbandonedMutexException as success. Oh, and it's good idea to put stuff like Mutex.ReleaseMutex in a finally clause, to make sure it actually happens, even if you get an exception. Thread or process dead doesn't matter (that will just cause one of the other contendants to get AbandonedMutexException), but if you just get an exception that you "handle" with your return false;, the mutex will not be released until you close all your applications and start again fresh :)

Clearly, the problem is that the MMF loose its context as explained by Luaan. But still nobody explains how to perform it:
The code 'Write to MMF file' must run on a separate async thread.
The code 'Read from MMF' will notify once read completed that the MMF had been read. The notification can be a flag in a file for example.
Therefore the async thread running the 'Write to MMF file' will run as long as the MMF file is read from the second part. We have therefore created the context within which the memory mapped file is valid.

TcpClient.EndConnect throws NullReferenceException when socket is closed

I am trying to connect to my server with a TcpClient.BeginConnect / TcpClient.EndConnect combo. However, some things don't work as they should.
The scenario is as follows:
Call to the TcpClient.BeginConnect
Server is intentionally offline (for testing purposes) - thus no connection can be made.
I close the application (client.Close() gets called in the process which closes the socket which in turn stops the async operation)
TcpClient connection callback method happens giving IAsyncResult
Call to the TcpClient.EndConnect method with the given IAsyncResult
NullReferenceException happens on EndConnect (?)
Since the last form (window) was closed, the app should exit - however it does not, at least not until BeginConnect operation completes (which is strange, as callback has already been called).
What happens here is that a NullReferenceException is caught. As you can see from the picture above, neither client nor ar are null. The problem is that the MSDN documentation for the EndConnect does not mention the case in which this exception is thrown.
So basically, I have no idea what is going on. The problem is that I am forced to wait for the app to close (as if the connection operation still waits for a timeout). If a server is online, it connects and disconnects just fine.
What does NullReferenceException in this context mean? How to avoid BeginConnect operation to block the application closing in case the connection can't be established?
Additional notes (requested in comments):
Here is the code to create the client (client is a member variable:
public void Connect()
{
try
{
lock (connectionAccess)
{
if (State.IsConnectable())
{
// Create a client
client = new TcpClient();
client.LingerState = new LingerOption(false, 0);
client.NoDelay = true;
State = CommunicationState.Connecting;
client.BeginConnect(address, port, onTcpClientConnectionEstablished, null);
}
else
{
// Ignore connecting request if a connection is in a state that is not connectable
}
}
}
catch
{
Close(true);
}
}
Also the Close method:
public void Close(bool causedByError)
{
lock (connectionAccess)
{
// Close the stream
if (clientStream != null)
clientStream.Close();
// Close the gateway
if (client != null)
client.Close();
// Empty the mailboxes
incomingMailbox.Clear();
outgoingMailbox.Clear();
State = causedByError ? CommunicationState.CommunicationError : CommunicationState.Disconnected;
}
}

The NullReferenceException is probably due to TcpClient.Client being null.
If you were to follow the MSDN Example for TcpClient.BeginConnect and pass theTcpClient object as the state object:
private void onConnEst(IAsyncResult ar)
{
try
{
TcpClient client = (TcpClient)ar.AsyncState;
if(client!=null && client.Client!=null)
{
client.EndConnect(ar);
}
}
catch(Exception ex){...}
}
This should handle the case when Close() is called before the Callback.
Going back to your problem - how long does it take for the application to eventually close?

This obviously a bug inside the TcpClient class. I have also faced it. TcpClient.Dispose may set Client field to null but EndConnect does not expect that.

I had a similar error and ended up using this code. I am not sure if it will hold with the IASyncResult interface, but there may be a similar way to run this check. I do notice that your ar.AsyncState == null, so perhaps try starting there, i.e. is it null when you connect properly?
private void connConnectCompleted(AsyncCompletedEventArgs e)
{
if (e.Error != null)
{
// Something didn't work...abort captain
CloseSocket();
Console.WriteLine(this.GetType().ToString() + #":Error connecting socket:" + e.Error.Message);
return;
}
// Do stuff with your connection
}
EDIT: Sorry I didn't realise I didn't post what generated my AsyncCompletedEventArgs, which is more related to what you are doing. You will see the reason why I was wondering as to ar.AsyncState being null.
private void OnConnect(IAsyncResult asyncResult)
{
if (OnConnectCompleted == null) return; // Check whether something is using this wrapper
AsyncCompletedEventArgs args;
try
{
Socket outSocket = (Socket) asyncResult.AsyncState;
// Complete connection
outSocket.EndConnect(asyncResult);
args = new AsyncCompletedEventArgs(null);
OnConnectCompleted(this, args);
}
catch (Exception e)
{
args = new AsyncCompletedEventArgs(e.Message);
OnConnectCompleted(this, args);
}
}

This is a know bug.
You should be receiving 'ObjectDisposedException' instead of 'NullReferenceException'.

C# - Strange behavior while parsing an XElement

I've observed a strange behavior by parsing a string to a XElement.
First, here is the XML I want to parse :
<return value="0">
<resultset>
<meta>
<column type="char(30)"></column>
</meta>
<datarow>
<datacol>
<![CDATA[master]]>
</datacol>
</datarow>
</resultset>
</return>
Now the code :
try
{
xmlResult = XElement.Parse(
_UTF8Encoder.GetString(_responseBytes.GetBuffer(), 0, (int)_responseBytes.Length),
LoadOptions.PreserveWhitespace);
}
catch
{
xmlResult = XElement.Parse(
_UTF8Encoder.GetString(_responseBytes.GetBuffer(), 0, (int)_responseBytes.Length),
LoadOptions.PreserveWhitespace);
}
I do exactly the same thing in the try and catch block.
Sometime the try block raises an XmlException ("Root element is missing."), but the catch block (doing exactly the same thing) doesn't throw it and parse the string correctly.
Can someone tell me why?
Thanks !
[EDIT]
Here is the whole method code :
private TcpClient _client;
private NetworkStream _clientStream;
private MemoryStream _responseBytes;
private readonly UTF8Encoding _UTF8Encoder = new UTF8Encoding();
private const int BUFFER_SIZE = 1024;
private XElement Receive()
{
byte[] buffer = new byte[BUFFER_SIZE];
XElement xmlResult;
Encoding serverEncoding = this.Task.Server.Encoding;
// Reading result
while (true)
{
_responseBytes = new MemoryStream();
try
{
IAsyncResult e = _clientStream.BeginRead(buffer,
0, // Begin
BUFFER_SIZE, // Length
new AsyncCallback(OnBeginRead), // Callback used
new SocketAsyncState(_clientStream, buffer)); // Passing buffer to callback
e.AsyncWaitHandle.WaitOne(); // Wait until data are in pipe
if (((SocketAsyncState)e.AsyncState).HasError)
{
throw new ObjectDisposedException();
}
// Try to convert to a XElement, if fail, redo all process.
_responseBytes.Position = 0;
try
{
xmlResult = XElement.Parse(
_UTF8Encoder.GetString(_responseBytes.GetBuffer(), 0, (int)_responseBytes.Length),
LoadOptions.PreserveWhitespace);
}
catch
{
xmlResult = XElement.Parse(
_UTF8Encoder.GetString(_responseBytes.GetBuffer(), 0, (int)_responseBytes.Length),
LoadOptions.PreserveWhitespace);
}
// Result 100% retrieved : quit loop
break;
}
catch (Exception ex)
{
if (ex is ObjectDisposedException
|| ex is XmlException)
{
while (!IsConnected) { Wait(); } // Wait that the network comes back
SendSyn(); // Relaunch process
}
}
}
// Result 100% retrieved : send ACK to Socket
SendAck();
return xmlResult;
}
private void OnBeginRead(IAsyncResult ar)
{
SocketAsyncState state = ar.AsyncState as SocketAsyncState;
byte[] nextBuffer = new byte[BUFFER_SIZE];
int numberOfBytesReaded;
Encoding serverEncoding = this.Task.Server.Encoding;
try
{
numberOfBytesReaded = state.Stream.EndRead(ar);
}
catch(Exception)
{
((SocketAsyncState)ar.AsyncState).HasError = true;
// Quit
return;
}
// While data are available, read next buffer (recursive call to this method)
if (state.Stream.DataAvailable && state.Stream.CanRead)
{
state.Stream.BeginRead(nextBuffer,
0,
BUFFER_SIZE,
new AsyncCallback(OnBeginRead),
new SocketAsyncState(state.Stream, nextBuffer));
}
// Default C# strings are in UTF-8, so convert stream only if needed
if (serverEncoding.CodePage != _UTF8Encoder.CodePage)
{
byte[] buffer = Encoding.Convert(serverEncoding,
_UTF8Encoder,
state.Data.TakeWhile((b) => b != '\0').ToArray());
_responseBytes.Write(buffer, 0, buffer.Length);
}
else
{
_responseBytes.Write(state.Data, 0, numberOfBytesReaded);
}
}

You haven't shown us what _responseBytes is, but one suggestion springs to mind: if it's being populated asynchronously (e.g. via an async web request) then maybe the first attempt occurs before the data is present, but by the time the catch block executes, the data has arrived.
(Obviously if this is the case, the solution is not to use this sort of catch block but to fix the timing.)
EDIT: Okay, I think I possibly see the problem. It's here:
e.AsyncWaitHandle.WaitOne();
I suspect that will wait until the read has occurred at the socket level, but before the callback is invoked. Your code is assuming it waits until the callback has completed.
So what's happening (and this is still just a guess) is:
On your main thread, you kick off the operation and wait for it to complete
The data is read
WaitOne() returns in the main thread and the callback is invoked on the thread-pool thread at the same time
You try to parse the data from the memory stream in the main thread...
... and then the callback actually writes the data to the memory stream
... and then you have a second try at parsing, which succeeds because the data is now there