Is there any way to get more detail at runtime about an OutOfMemoryException? Or, can this exception somehow not be caught by the enclosing try/catch and instead a try/catch higher up the call stack? I cannot reproduce using WinDBG so it must be something I can log from the application.
I apologize for the long explanation, but there are a lot of possible causes to eliminate, which I explain.
I have read up on all the possibilities for an OutofMemoryException and basically eliminated all of them. Normally, application runs great, but occasionally on only certain computers, I am getting an OutOfMemoryException. As these reports are in the field on not reproducible locally, I only have logs to go by. But I have a fair amount of detail.
What is strange:
Anything that might logically be allocating memory in the vicinity is in a try/catch, but the exception is treated as unhandled (and caught much higher up the call stack)
There are no StringBuffers in use
The exception happens even after rebooting and restarting the application.
The exception occurs after only a couple minutes, and only about 30MiB of memory allocated, in chunks no more than 1.5MiB.
Verified the application (built for "any" processor) is running as 64 bit.
no shortage of disk space (270Gb free) and pagefile is enabled.
does not appear to be a possible LoH fragmentation issue.
This has happened a couple times in different parts of the application recently. The first time, I concluded there was a corrupt .NET assembly, as the exception was occurring right when it would first load the System.Web.Serialization assembly. I could determine it was happening right during a method call where that assembly was used for the first time. Reimaging the computer (to be identical of original setup) and updating windows resolved this issue.
But, it seems highly unlikely to me that the second case, different client, happening within a few days, is also corruption. This one is happening in a location where no assemblies would be loaded. I'm rethinking the first error now. What I do know:
It's happening in a thread pool thread (System.Timers.Timer, [Statthread])
Small number of threads active (< 5)
It happens around the time a 1MiB file is downloaded. This is read into a MemoryStream, so that could be as big as 2MiB. That is then fed to a System.Drawing.Bitmap constructor, resulting in a Bitmap that would be about 8MiB. However, that is all in a try/catch that catches System.Exception. The only thing not in the try/catch is returning the byte[] reference, which should just be a reference copy, not any memory allocation.
No other significant memory allocations have been done before this time. Reviewing heap in my local version, which should be running identically, shows just the app icon and a couple dozen objects that would be on Small Object Heap.
it is repeatable on a specific system with specific input. However, these systems are cloned from one another. Only obvious variation would be the order of windows updates.
The assembly I'm running is signed. Isn't there a checksum that ensures it isn't corrupted? Same for all the system assemblies? I don't see how this instance could be explained by corrupted dlls, or even data.
Viewing the call stack at time of exception is surprisingly unhelpful. I indicate in the code below where exception is thrown.
There is some use of COM objects. However, under normal conditions we run the app for weeks without memory problems, and when we get these exceptions, they are almost immediate, after only using around 20 relatively lightweight COM objects (IUPnPDevice)
// Stack Trace indicates this method is throwing the OutOfMemoryException
// It isn't CAUGHT here, though, so not in the try/catch.
//
internal void Render(int w, int h)
{
if (bitmap != null)
{
bitmap.Dispose();
bitmap = null;
}
if (!String.IsNullOrEmpty(url))
{
// this information is printed successfully to log, with correct url
// exception occurs sometime AFTER this somehow.
Logger.Default.LogInfo("Loading {0}", url);
// when file contents changed (to go from 1MiB to 500MiB, the error went away)
byte[] data = DownloadBinaryFile(url);
if (data != null)
{
try
{
Bitmap bmp;
using (var ms = new MemoryStream(data))
{
bmp = new Bitmap(ms);
}
bitmap = bmp;
}
catch (Exception)
{
// We do not catch anything here.
Logger.Default.LogWarning("WARNING: Exception loading image {0}", url);
}
}
//
// if we had any errors, just skip this slide
//
if (bitmap == null)
{
return;
}
// QUESTION EDIT:
// in the problematic version, there was actually an unnecessary
// call here that turns out to be where the exception was raised:
using( Graphics g = Graphics.FromImage(bitmap)) {
}
}
}
// calling this would trigger loading of the System.Web assembly, except
// similar method has been called earlier that read everything. Only
// class being used first time is the BinaryReader, which is in System.IO
// and already loaded.
internal static byte[] DownloadBinaryFile(String strURL, int timeout = 30000)
{
try
{
HttpWebRequest myWebRequest = HttpWebRequest.Create(strURL) as HttpWebRequest;
myWebRequest.KeepAlive = true;
myWebRequest.Timeout = timeout;
myWebRequest.ReadWriteTimeout = timeout;
myWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
using (HttpWebResponse myWebResponse = myWebRequest.GetResponse() as HttpWebResponse)
{
if (myWebResponse.StatusCode != HttpStatusCode.OK)
{
Logger.Default.LogWarning("WARNING: Response {0} loading {1}", myWebResponse.StatusCode, strURL);
return null;
}
using (Stream receiveStream = myWebResponse.GetResponseStream())
{
using (BinaryReader readStream = new BinaryReader(receiveStream))
{
// this extension method uses MemoryStream, but seems irrelevant since we don't catch anything here.
return readStream.ReadAllBytes();
}
}
}
}
catch (Exception e)
{
// we do not catch anything here.
Logger.Default.LogError("ERROR: Exception {0} loading {1}", e.Message, strURL);
}
return null;
}
So, after all of that, I return to my opening question. Are there any known properties on the OutOfMemoryException object I can inspect, or calls I can make after the exception is thrown, to narrow this down?
And..is there any reason an OutOfMemoryException would not be caught by the first try/catch, but would be caught further up the call stack?
Thank you all. The answer is somewhat curious and I had some details wrong which made it hard to figure out. The error is here:
Bitmap bmp;
using (var ms = new MemoryStream(data))
{
bmp = new Bitmap(ms);
}
bitmap = bmp;
In the remarks in documentation on the Bitmap constructor, I found this:
You must keep the stream open for the lifetime of the Bitmap.
Obviously, closing the MemoryStream immediately after constructing was violating this. A garbage collection between this and when I actually used the Bitmap was apparently creating the error. (EDIT: actually, it seems that a boundary exists around 1MiB where the FromStream function will decompress only so much of a JPEG file initially. For JPEG < 1MiB, the entire image is decompressed and it doesn't actually use the stream after the initialization. For larger JPEG, it will not read beyond the first 1MiB until those pixels are needed)
It's hard for me to imagine why Microsoft did it this way. I wouldn't want to keep the original stream open, either (which is an http connection) so only solution I see is to clone the bitmap:
// Create a Bitmap object from a file.
using (var ms = new MemoryStream(data))
{
bmp = new Bitmap(ms);
Rectangle cloneRect = new Rectangle(0, 0, bmp.Width, bmp.Height);
System.Drawing.Imaging.PixelFormat format = bmp.PixelFormat;
this.bitmap = bmp.Clone(cloneRect, bmp.PixelFormat);
}
What led to my long frustrating search and the exclusion of a key piece of information was that the code executing on a client machine was a slightly older version, with only a subtle change. Graphics.FromImage() was being called on the bitmap in the previous version, but that had been removed. Still, that version functioned very well the vast majority of the time.
Related
Im using a Request System called xNet and it seems ObjectDisposedExceptions are occuring which on-occurence causes a HUGE cpu spike, sometimes continuously keeping CPU at 99-100% causing freezes and lag.
The script mentioned is the following:
https://github.com/PR4GM4/xNet-Ameliorated
An example code is:
using (HttpRequest httpRequest = new HttpRequest()) {
string url = "https://httpbin.org";
string[] proxysplit = proxy.Split(':');
httpRequest.Proxy = new InternalProxyClient(ProxyType.HTTP, proxysplit[0], int.Parse(proxysplit[1]), null, null);
httpRequest.UserAgent = Http.ChromeUserAgent();
httpRequest.KeepAlive = true;
httpRequest.ConnectTimeout = 15000;
httpRequest.AllowAutoRedirect = true;
HttpResponse hr = httpRequest.Start(HttpMethod.GET, new Uri(url, UriKind.Absolute), new StringContent(""));
if (hr == null) return "2";
string sr = hr.ToString();
if (sr == null) return "2";
}
(If a list of half/dead proxies are needed, I can provide it, just im not sure if linking to them is allowed or not.)
Big note here, it seems to only ever occur whenever theres some kind of other exception like failing to connect to the proxy, or a general bad response, so good connection proxies and/or no proxy at all never has this issue (unless again a general failed error).
If you loop this code, and give it a dead proxy (And to speed things up, multi-thread it to around 5 a time) it will eventually cause an exception like bad response or a timeout and eventually an objectdisposedexception.
I tried debugging in Visual Studio but it gives me almost no information, Historical Debugging gives no information just "Source not found".
Call Stack for the First Exception Thrown of the ObjectDisposedException in the screenshot above.
Seems to be related to line 1430 in ~Http/HttpRequest.cs or line 217 in ~Proxy/ProxyClient.cs as it's the only line I know to exist thats to do with EndConnect socket and also coincidentally can produce ObjectDisposedException. Just not sure how to properly handle the exception here without causing the rest of the script to fail. Also why does a simple exception here cause so much CPU spike?
Strangely enough, no matter how I wrap an exception handler for ObjectDisposedException it never gets triggered, no matter how much code or where I wrap? (On both scripts)
try
{
tcpClient.EndConnect(ar);
}
catch (Exception ex)
{
connectException = ex;
}
I found out why, it wasnt because of the .EndConnect on either of the 2 files, it was actually caused by the .Close() calls, since it does .EndConnect inside of that, thats why I couldnt see any "Source" etc.
So it was causeed because the socket connection wasnt connected, so doing .Close() would cause the Exception.
It was a simple fix.
(Where tcp = a TcpClient)
Do the following instead of just tcp.Close()
On Timeouts (Where it's most likely if never at all connected):
if (tcp.Client.Connected) {
tcp.GetStream().Close();
tcp.Close();
}
When it might be properly connected:
if (!tcp.Connected) {
if (tcp.Client.Connected) tcp.GetStream().Close();
tcp.Close();
}
I implemented a C# application that recevies frame RGB at framerate of 30fps.
The event of frame arrive is managed with this code:
void client_ColorFrameReady(object sender, ColorFrameReadyEventArgs e)
{
mycounter++;
Console.WriteLine("new frame received: " + mycounter);
if (writer != null)
{
count++;
if (count % 2== 0)
{
using (var frame = BitmapImage2Bitmap(e.ColorFrame.BitmapImage))
using (var thumb = ResizeBitmap(frame, 320, 240))
{
writer.WriteVideoFrame(thumb);
}
}
}
else
{
writer.Close();
}
}
with the if condition I manage only one of two frames.
When my code call BitmapImage2Bitmap I obtain this exception:
The exception in english should be:
A first chance exception of type 'System.NotSupportedException' occurred in `PresentationCore.dll`
Additional information: BitmapMetadata is not available on BitmapImage.
The strange thing is that my application works "well" because the frames are correctly inserted in the output file.
I've read this, so the problem seems a bug in WPF framework.
This is by design. A first-chance exception notification doesn't mean that there's a problem. The relevant code inside the Create() method looks like this:
try
{
metadata = source.Metadata as BitmapMetadata;
}
catch (NotSupportedException)
{
}
In other words, the exception is expected and simply swallowed. Which is certainly very annoying since these exceptions do make the debugger stop when you have the Thrown checkbox checked in the Debug + Exception dialog. But it certainly is not a bug, this was intentionally written this way. Sometimes it is a lot cheaper to just let an exception be thrown and swallowing it instead of writing the code that prevents the exception. Especially when it gets unpractical to avoid the exception, the case with bitmaps since there are so many different kind of bitmap types. Some of which don't support metadata. Wherever this is done inside the framework code, it is almost always done to make the code faster. Speed is also an important aspect of code.
Feature, not a bug. Untick the Thrown checkbox to avoid seeing these exceptions.
I hope my answer help you,
I had using same code, but BitmapFrame.cs (at PresetationCore.dll) occur Exception when we are using BitmapFrame.Create(source).
So, I just Using other create function below one, which is Inner function of BitmpaFrame.Create,
BitmapFrame.cs
public static BitmapFrame Create(
BitmapSource source,
BitmapSource thumbnail,
BitmapMetadata metadata,
ReadOnlyCollection<colorcontext> colorContexts
)
we can get same result BitmapFrame.Create(source, null, null, null).
in your case,
enc.Frames.Add(BitmapImage.Create(bitmap, null, null, null));
thanks.
Running with .NET Framework 4.5, I had to change the similar line in the Microsoft SDK WPF Photo Viewer sample from
_image = BitmapFrame.Create(_source);
to
_image = BitmapFrame.Create(_source, BitmapCreateOptions.None, BitmapCacheOption.None);
to avoid the ConfigurationErrorsException. Things seem to be drifting under the hood...
I've come across this issue lately when dealing with images included in projects as Resources (BuildType=Resource in file properties). It seems to be some build related issue which makes the resources corrupt and causes what seems like random issues as WPF are loading them. Simply performing a clean/rebuild makes the error(s) go away. They may reappear when adding new images though but the same fix obviously applies.
Here i am attaching the code snippet.
Error is: An error occurred while accessing IsolatedStorage.
public Boolean SaveImage(string filename, WriteableBitmap wrtbmp)
{
try
{
using (IsolatedStorageFile iSF = IsolatedStorageFile.GetUserStoreForApplication())
{
if (iSF.FileExists(filename))
{
iSF.DeleteFile(filename);
}
using (IsolatedStorageFileStream fstream = new IsolatedStorageFileStream(filename, FileMode.CreateNew, FileAccess.Write, iSF))
{
wrtbmp.SaveJpeg(fstream, wrtbmp.PixelWidth, wrtbmp.PixelHeight, 0, 100);
fstream.Close();
fstream.Dispose();
}
}
}
catch (Exception ex)
{
if (System.Diagnostics.Debugger.IsAttached)
{
System.Diagnostics.Debugger.Break();
}
return false;
}
return true;
}
This is the method i am using for saving Image, while executing when it reaches the portion of deleting the file if it already exists, it throws the error, but in some cases it executes perfectly without an error.
The stacktrace points to the DeleteFile. Are you sure the path-to-delete is valid? Are you sure that the file exists? I don't remember well, but I think the Delete file might throw if the file is not found - please check the example in the linked MSDN method description - they have an IF-exists there.
[EDIT: I'm sorry, I'm quite a bit tired today and I didn't read through your code properly. Obviously you have your code already guarded against file-nonexistence.]
Aside from the possible nonexistence problem, there is also a small possibility that ont here, but somewhere else in your code, there is something that has opened the file and did not close it properly. In such case, the system will think the file is in use (even if that "old" handle opened elsewhere is "forgotten" and wait to be GC'ed) and no operations on that file will succeed unles that handle actually dies.
Another thing is, that even with your using/autodispose, the operation can still fail if you try to invoke the method multiple times at once from different threads. With async patterns, it sometimes can be hard to notice - check thoroughly from what points of code this method is called and think whether it may happen ie. that at the same time GUI callback will invoke it, and maybe also some background operation? If so, try to wrap the using additionally with a lock-statement. If this helps, that means you have reentrancy problems.
edit: someone, at some point in distant future, will kill me for hundrets of edits.. but I'll write it anyways:) : I think that in general, you do not have to delete the file before writing to it. The CreateFile is not the only one to get access to the files. Check OpenFile with mode=OpenOrCreate|Truncate or even shorter: mode=Create (see FileMode flags for explanation)
First of all, there's no reason to call fstream.Close(); and fstream.Dispose(); when you're using a using statement, which automatically closes and disposes of the stream.
Secondly, your error isn't explicit enough.
What exception is thrown in the .NET Framework when trying to write a file but the disk is full?
Does Windows buffer file writes?
You will get an IO exception:
System.IO.IOException: There is not enough space on the disk.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
The System.IO library handles the actual tasks of reading and writing to disk and wraps them in managed code, so you shouldn't get any unmanaged exceptions using them.
It's also worth trying this (with one of those shoddy small USB sticks they give out everywhere) to see what happens with your code - that's usually the best way of finding out this sort of thing.
You will get an 'IOException'. But the problem is that 'IOException' is quite broad spectrum, not only disk full. I recommend to also check the 'HResult' inside to precisely filter out this case.
catch (IOException ioEx)
{
// Is it 'DISK_FULL'?
uint unsignedResult = (uint)ioEx.HResult;
if (unsignedResult.IsOneOf(0x80000027, 0x80000070, 0x80070027, 0x80070070))
{
// Remove the partial file
try
{
File.Delete(pathName);
}
catch { }
// do your special stuff here
}
else
throw;
}
Note that the Win32 error codes from region 8000 are mirrored in the region 8007, thats why there is a near duplicate in the checks.
You could check the Win 32 Exception after what you have fails to see if that gives more detail:
Win32Exception ex = new Win32Exception();
string low_level_error = ex.Message;
How are you writing to disk? For FileStream it says:
"Be sure to call the Dispose method on
all FileStream objects, especially in
environments with limited disk space.
Performing IO operations can raise an
exception if there is no disk space
available and the Dispose method is
not called before the FileStream is
finalized."
I have written an application which is getting the Icon info of current cursor using GetIconInfo function of user32.dll, It works fine for some time, but after some time it starts providing wrong information in ICONINFO.hbmMask (some negative value), and when on the next line I try to get Bitmap object from Bitmap.HBitmap(bitmask), it throws an exception:
A Generic error occured in GDI+.
From there onwords, it continuously gives this exception, as GetIconInfo always return negative value (all this code is working in a loop)..
Can any one tell me what this problem is? and how to avoid the next iteration exception?
Here is the code
while (true)
{
//DLLimport User32.dll
PlatformInvokeUSER32.ICONINFO temp;
//Get the current cursor
IntPtr curInfo = GetCurrentCursor();
Cursor cur;
Icon ic;
if (curInfo != null && curInfo.ToInt32() != 0 && isOSelected)
{
cur = CheckForCusrors(curInfo);
try
{
//Dllimport User32.dll
//after some time the temp.hbmMask begin to get -ive vlaue from following function call
PlatformInvokeUSER32.GetIconInfo(curInfo, out temp);
if (temp.hbmMask != IntPtr.Zero)
{
//due to negative value of hbmMask the following function generates an exception
Bitmap bitmask = Bitmap.FromHbitmap(temp.hbmMask);
ic = Icon.FromHandle(curInfo);
Bitmap bmpCur = ic.ToBitmap();
}
}
catch (Exception ee)
{
//Exception message is
//A Generic error occured in GDI+
//and this loop begins to throw exception continuously
}
}
}// while ended
How large is your loop? GDI+ resources are OS resources and are limited in availability.
You can find out if this is your problem by monitoring the HANDLEs allocated by your process. If GDI+ starts to complain when a certain handle count (HBITMAP or HICON) reaches a limit, then you know you have to deal with your resources more intelligently. You can start by using Task Manager to do this but might want to switch to more sophisticated software like Process Explorer.
If this is your problem, then you need to read about IDisposable and make sure you call Dispose on your objects after you're done with them (won't be rendered anymore). Bitmaps and Icons and most GDI+ objects implement IDisposable.
Furthermore, it's unclear to me, but you may need to call DeleteObject on some of the raw GDI objects themselves (all depends upon where you got their handles).
Check out this PInvoke sample, are you properly deleting the objects you are pulling in through unmanaged code?