We Keep Coding

.net memory leaking with Thread.Abort() and using statement

Edit:I am trying to use Constrained Execution Regions as a new weapon against Abort(). I am still running test, I hope it work. Abort() is really a bad thing. I will report later.. If anyone has any argument against CER please note that.
I am having a problem about .NET memory leaking.
I have a project, quite complex, threads in threads in threads, makes it hard to debug.
I know Abort() is depreciated, but I have my reasons to use them:
My method is a long method, I/O, network related. There's no explicit time-consuming operation that I can put a flag inside. If I put flags everywhere the code will be a mess.
When needed, the thread has to be terminated at once, sooner is better. All the work inside is no longer needed.
When I run the program normally, by which threads finish their work and die naturally one by one, everything is ok (I have run my program for 2 years, no memory leaking).
But if I try to create new thread, Start() and Abort() them frequently (1-2 threads/second, just a bit faster than the case I don't abort()), the memory leaks.
And the more wired thing is that, after the operation is finished, the memory occupied will stay high for a few time, like minutes or 10 minutes, but it will finally return to normal level, like nothing happened.
In debug mode, I can see no active thread, but memory leaked.
So I used .NET memory profiler to trace. Most instances occupied the memory are many byte[] referenced by MemoryStream.
Yes, I do use MemoryStream, but ALL of my MemoryStreams are inside using blocks, no exception. Everything should be in using block correctly.
As far as I know, Thread.Abort() is throwing an exception to force it to close, but as I confirmed in debug mode, all the threads are closed expectedly. Why there are still reference? Why the memory is released after some time? (But still much longer than the situation when I don't abort the thread and let it finish the work.)
IMO using blocks can guarantee that even the ThreadAbortException is thrown inside the Dispose() can be executed correctly.
Edit:
public static byte[] Recover(byte[] png)
{
using (MemoryStream pngStream = new MemoryStream(png))
{
using (System.Drawing.Image image = System.Drawing.Image.FromStream(pngStream))
{
using (MemoryStream bmpStream = new MemoryStream())
{
image.Save(bmpStream, System.Drawing.Imaging.ImageFormat.Bmp);
bmpStream.Seek(0, System.IO.SeekOrigin.Begin);
byte[] BMP = new byte[bmpStream.Length];
bmpStream.Read(BMP, 0, (int)bmpStream.Length);
return BMP;
}
}
}
}
public xxxxMission(byte[] png, Server server, int no) //constructor
{
byte[] bmp = xxxBMP.Recover(png); //png is generated by getBlankBMP();
//....
}

Timers, Files and race conditions?

Hello and thanks for your help.
This time I am having a curious problem with a program (C#) I am writing and would like to hear your advice.
I am writing a normal program (not multithreaded) but then added a timer (System.Timers.Timer)
Also I am using a StreamWriter to write on a file. I open this like this
StreamWriter logStream=new StreamWriter(filename, true);
meaning that if the file exists, it appends , if not it creates.
Later I write in the file like this
logStream.WriteLine(message);
However, I write to the stream from both the main function and from the function that is called by the timer.
the problem symptoms
My program is throwing an error sometimes when I flush or write the stream saying that "Can not access a closed file" and other times "Can not access a closed TextWriter... (What is a "TextWriter"?)
However curiously, the file keeps being written without problem. (Even the "can not access a closed file" message is written in the supposed closed file)
I am not familiar with the inner workings of a Timer. (I suppose it runs a separate thread?)
My question is
Is it possible to use a StreamWriter from several threads? (in this case the main one and the Timer one)
Is it possible that there is happening a race condition or some problem like that?
One more thing: I made a logic mistake and close and reopen the file every time I want to write on it. Yes, it is a mistake and I should correct it. But maybe if I correct this the error I described above will disappear masking a more serious flaw.
My suspicions is that since I am closing and opening the file every time I write on it, maybe the both threads try to access them on a wrong time
Any help will be greatly appreciated

Closing and opening you file under this scenario will create a race condition like you suspect. You cannot keep the stream open and pass the object to the thread because you might end up with a similar issue if you call from different thread. Your best solution remain using a thread safe method that will write what you send to it.
the methods are static because the lock has to be accessible from all instance of the class.
private static ReaderWriterLockSlim readerWriterLockSlim = new ReaderWriterLockSlim();
public static void AppendToFile(string path, string text)
{
// Set to locked (other thread will freeze here until object is unlocked
readerWriterLockSlim.EnterWriteLock();
try
{
// Write that will append to the file
using (StreamWriter sw = File.AppendText(path))
{
// append the text
sw.WriteLine(text);
sw.Close();
}
}
finally
{
// Clear the lock
readerWriterLockSlim.ExitWriteLock();
}
}

How can I stop IIS 7 locking .XSLT file in C#

I have the following lines of code:
xslt.Load(XmlReader.Create(new FileStream(#"C:\website\TransList.xslt", System.IO.FileMode.Open)));
xslt.Transform(mydoc.CreateReader(),null, sw);
It works fine, if I stop the project and launch it again, I get the following error:
[System.IO.IOException] = {"The process cannot access the file 'C:\website\TransList.xslt' because it is being used by another process."}
I then have have to goto the command line and do a IISRESET to get, I can also reset the app pool, this is easiest at this time as this is just my dev box.
Now I do have the call in a try catch statement, but I cannot access the xslt object in the handler.
The xslt object doesn't seem to have a close or dispose method.
The garbage collector never gets a shot at it , it seems.
Any ideas?

You will need to close your FileStream and Reader, either explicitly using .Close() or via a using statement:
using (FileStream fs = new FileStream(#"C:\website\TransList.xslt", System.IO.FileMode.Open))
{
xslt.Load(XmlReader.Create(fs));
using (var reader = mydoc.CreateReader())
{
xslt.Transform(reader, null, sw);
}
}

There is no need to explicitly create a FileStream and an XmlReader, if you know the file location then you can simply pass that to the Load method, using this overload:
XslCompiledTransform xslt = new XslCompiledTransform();
xslt.Load(#"C:\website\Translist.xslt");
If you think you need to create a FileStream and an XmlReader then I agree with the suggestions already made, use the 'using' statement to properly close and dispose of those objects.

Filestream implements IDisposable and requires you to invoke Dispose to release external resources as well as implicit;y invoke close(). You should wrap your instantiation of Filestream in a using block as it ensures Dispose is invoked even if an exception is raised. To answer your question though, since you did not close the filestream, your process, presumably the w3wp.exe process still has a handle on the file stream and the only way you can release is it to reset iis or recycle the app pool. For future reference, just wrap the filestream in a using block to be safe.

Saving XML stream in C# says that it's being used by another process

I have this open-source library that I'm having some trouble fixing a problem... This library allows to easily create an XML file to store application settings. But I'm having an issue saving the changes.
I have another application where I'm using this library and every time that application window is done resizing, I call the Save() method of the library to save the window size/position to the XML file.
Most of times it works fine, everything is saved. Once in a while though, I get an exception saying the file is being used by another process.
I really need to make sure that changes are saved every time the Save() method is called, I need to handle this exception somehow or prevent it from happening.
What are you guys suggestions for best handling this situation?
The code for the Save() method is the following:
public void Save() {
// Create a new XML file if there's no root element
if(xDocument.DocumentElement == null) {
xDocument = new XmlDocument();
xDocument.LoadXml("<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n" +
"<" + XmlRootElement + ">\n</" + XmlRootElement + ">");
}
// OMITTED CODE WAS HERE (NOT IMPORTANT FOR THE PROBLEM)
// Create a new XML writer for the XML file
XmlWriter xWriter = XmlWriter.Create(XmlFilePath, new XmlWriterSettings() {
Indent = true,
IndentChars = "\t"
});
// Sort the XML file using the XSL sylesheet and save it
xslTransform.Transform(xDocument, xWriter);
// Clear the buffer and close the XML writer stream
xWriter.Flush();
xWriter.Close();
}

XmlWriter is IDisposable. You should wrap it in a using() clause.
http://msdn.microsoft.com/en-us/library/system.xml.xmlwriter.aspx

I have to go with a combination of the answers already given here.
Your XmlWriter should be in a using block for several reasons. You should dispose it so that your resources are freed as soon as possible. Also, what if you throw an exception while interacting with it. The file wouldn't be closed properly, at least until the finalizer kicks in and frees your resources.
Even with the using statement, you "might" have contention on the file and need to place the Save code in a lock statement. The method is non-reentrant by nature because the file is a shared resource. Putting a lock around it might be over kill if you don't have multiple threads, but you would ensure that you properly controlled access to the file.
The other thing to consider is that you might want to move the saving operation to a background thread to write the file out. If you get a large settings file you might cause strange UI interactions because you are waiting on the file to write every time the user resizes and this happens on the UI thread. If you did this you would definitely need to lock access to the file resource.

It could be the case that the window-resizing-completed events are firing so quickly, that the save function is being called, the called again before it finishes running the first time. This would result in the error you're describing (the other process using the file is... YOU!). Try surrounding your code with a lock, thusly:
lock(some_shared_object)
{
//Your code here
}

Also you might try using a lock statement. It could be that the methods are overrunning one another.

C# - Locking issues with Mutex

I've got a web application that controls which web applications get served traffic from our load balancer. The web application runs on each individual server.
It keeps track of the "in or out" state for each application in an object in the ASP.NET application state, and the object is serialized to a file on the disk whenever the state is changed. The state is deserialized from the file when the web application starts.
While the site itself only gets a couple requests a second tops, and the file it rarely accessed, I've found that it was extremely easy for some reason to get collisions while attempting to read from or write to the file. This mechanism needs to be extremely reliable, because we have an automated system that regularly does rolling deployments to the server.
Before anyone makes any comments questioning the prudence of any of the above, allow me to simply say that explaining the reasoning behind it would make this post much longer than it already is, so I'd like to avoid moving mountains.
That said, the code that I use to control access to the file looks like this:
internal static Mutex _lock = null;
/// <summary>Executes the specified <see cref="Func{FileStream, Object}" /> delegate on
/// the filesystem copy of the <see cref="ServerState" />.
/// The work done on the file is wrapped in a lock statement to ensure there are no
/// locking collisions caused by attempting to save and load the file simultaneously
/// from separate requests.
/// </summary>
/// <param name="action">The logic to be executed on the
/// <see cref="ServerState" /> file.</param>
/// <returns>An object containing any result data returned by <param name="func" />.
///</returns>
private static Boolean InvokeOnFile(Func<FileStream, Object> func, out Object result)
{
var l = new Logger();
if (ServerState._lock.WaitOne(1500, false))
{
l.LogInformation( "Got lock to read/write file-based server state."
, (Int32)VipEvent.GotStateLock);
var fileStream = File.Open( ServerState.PATH, FileMode.OpenOrCreate
, FileAccess.ReadWrite, FileShare.None);
result = func.Invoke(fileStream);
fileStream.Close();
fileStream.Dispose();
fileStream = null;
ServerState._lock.ReleaseMutex();
l.LogInformation( "Released state file lock."
, (Int32)VipEvent.ReleasedStateLock);
return true;
}
else
{
l.LogWarning( "Could not get a lock to access the file-based server state."
, (Int32)VipEvent.CouldNotGetStateLock);
result = null;
return false;
}
}
This usually works, but occasionally I cannot get access to the mutex (I see the "Could not get a lock" event in the log). I cannot reproduce this locally - it only happens on my production servers (Win Server 2k3/IIS 6). If I remove the timeout, the application hangs indefinitely (race condition??), including on subsequent requests.
When I do get the errors, looking at the event log tells me that the mutex lock was achieved and released by the previous request before the error was logged.
The mutex is instantiated in the Application_Start event. I get the same results when it is instantiated statically in the declaration.
Excuses, excuses: threading/locking is not my forté, as I generally don't have to worry about it.
Any suggestions as to why it randomly would fail to get a signal?
Update:
I've added proper error handling (how embarrassing!), but I am still getting the same errors - and for the record, unhandled exceptions were never the problem.
Only one process would ever be accessing the file - I don't use a web garden for this application's web pool, and no other applications use the file. The only exception I can think of would be when the app pool recycles, and the old WP is still open when the new one is created - but I can tell from watching the task manager that the issue occurs while there is only one worker process.
#mmr: How is using Monitor any different from using a Mutex? Based on the MSDN documentation, it looks like it is effectively doing the same thing - if and I can't get the lock with my Mutex, it does fail gracefully by just returning false.
Another thing to note: The issues I'm having seem to be completely random - if it fails on one request, it might work fine on the next. There doesn't seem to be a pattern, either (certainly no every other, at least).
Update 2:
This lock is not used for any other call. The only time _lock is referenced outside the InvokeOnFile method is when it is instantiated.
The Func that is invoked is either reading from the file and deserializing into an object, or serializing an object and writing it to the file. Neither operation is done on a separate thread.
ServerState.PATH is a static readonly field, which I don't expect would cause any concurrency problems.
I'd also like to re-iterate my earlier point that I cannot reproduce this locally (in Cassini).
Lessons learned:
Use proper error handling (duh!)
Use the right tool for the job (and have a basic understanding of what/how that tool does). As sambo points out, using a Mutex apparently has a lot of overhead, which was causing issues in my application, whereas Monitor is designed specifically for .NET.

You should only be using Mutexes if you need cross-process synchronization.
Although a mutex can be used for
intra-process thread synchronization,
using Monitor is generally preferred,
because monitors were designed
specifically for the .NET Framework
and therefore make better use of
resources. In contrast, the Mutex
class is a wrapper to a Win32
construct. While it is more powerful
than a monitor, a mutex requires
interop transitions that are more
computationally expensive than those
required by the Monitor class.
If you need to support inter-process locking you need a Global mutex.
The pattern being used is incredibly fragile, there is no exception handling and you are not ensuring that your Mutex is released. That is really risky code and most likely the reason you see these hangs when there is no timeout.
Also, if your file operation ever takes longer than 1.5 seconds then there is a chance concurrent Mutexes will not be able to grab it. I would recommend getting the locking right and avoiding the timeout.
I think its best to re-write this to use a lock. Also, it looks like you are calling out to another method, if this take forever, the lock will be held forever. That's pretty risky.
This is both shorter and much safer:
// if you want timeout support use
// try{var success=Monitor.TryEnter(m_syncObj, 2000);}
// finally{Monitor.Exit(m_syncObj)}
lock(m_syncObj)
{
l.LogInformation( "Got lock to read/write file-based server state."
, (Int32)VipEvent.GotStateLock);
using (var fileStream = File.Open( ServerState.PATH, FileMode.OpenOrCreate
, FileAccess.ReadWrite, FileShare.None))
{
// the line below is risky, what will happen if the call to invoke
// never returns?
result = func.Invoke(fileStream);
}
}
l.LogInformation("Released state file lock.", (Int32)VipEvent.ReleasedStateLock);
return true;
// note exceptions may leak out of this method. either handle them here.
// or in the calling method.
// For example the file access may fail of func.Invoke may fail

If some of the file operations fail, the lock will not be released. Most probably that is the case. Put the file operations in try/catch block, and release the lock in the finally block.
Anyway, if you read the file in your Global.asax Application_Start method, this will ensure that noone else is working on it (you said that the file is read on application start, right?). To avoid collisions on application pool restaring, etc., you just can try to read the file (assuming that the write operation takes an exclusive lock), and then wait 1 second and retry if exception is thrown.
Now, you have the problem of synchronizing the writes. Whatever method decides to change the file should take care to not invoke a write operation if another one is in progress with simple lock statement.

I see a couple of potential issues here.
Edit for Update 2: If the function is a simple serialize/deserialize combination, I'd separate the two out into two different functions, one into a 'serialize' function, and one into a 'deserialize' function. They really are two different tasks. You can then have different, lock-specific tasks. Invoke is nifty, but I've gotten into lots of trouble myself going for 'nifty' over 'working'.
1) Is your LogInformation function locking? Because you call it inside the mutex first, and then once you release the mutex. So if there's a lock to write to the log file/structure, then you can end up with your race condition there. To avoid that, put the log inside the lock.
2) Check out using the Monitor class, which I know works in C# and I'd assume works in ASP.NET. For that, you can just simply try to get the lock, and fail gracefully otherwise. One way to use this is to just keep trying to get the lock. (Edit for why: see here; basically, a mutex is across processes, the Monitor is in just one process, but was designed for .NET and so is preferred. No other real explanation is given by the docs.)
3) What happens if the filestream opening fails, because someone else has the lock? That would throw an exception, and that could cause this code to behave badly (ie, the lock is still held by the thread that has the exception, and another thread can get at it).
4) What about the func itself? Does that start another thread, or is it entirely within the one thread? What about accessing ServerState.PATH?
5) What other functions can access ServerState._lock? I prefer to have each function that requires a lock get its own lock, to avoid race/deadlock conditions. If you have many many threads, and each of them try to lock on the same object but for totally different tasks, then you could end up with deadlocks and races without any really easily understandable reason. I've changed to code to reflect that idea, rather than using some global lock. (I realize other people suggest a global lock; I really don't like that idea, because of the possibility of other things grabbing it for some task that is not this task).
Object MyLock = new Object();
private static Boolean InvokeOnFile(Func<FileStream, Object> func, out Object result)
{
var l = null;
var filestream = null;
Boolean success = false;
if (Monitor.TryEnter(MyLock, 1500))
try {
l = new Logger();
l.LogInformation("Got lock to read/write file-based server state.", (Int32)VipEvent.GotStateLock);
using (fileStream = File.Open(ServerState.PATH, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None)){
result = func.Invoke(fileStream);
} //'using' means avoiding the dispose/close requirements
success = true;
}
catch {//your filestream access failed
l.LogInformation("File access failed.", (Int32)VipEvent.ReleasedStateLock);
} finally {
l.LogInformation("About to released state file lock.", (Int32)VipEvent.ReleasedStateLock);
Monitor.Exit(MyLock);//gets you out of the lock you've got
}
} else {
result = null;
//l.LogWarning("Could not get a lock to access the file-based server state.", (Int32)VipEvent.CouldNotGetStateLock);//if the lock doesn't show in the log, then it wasn't gotten; again, if your logger is locking, then you could have some issues here
}
return Success;
}