We Keep Coding

Reading pictures from DB using memorystreams and images often crashes with Out of Memory

I'm trying to figure out if there is something seriously wrong with the following code. It reads the binary from the database, stores it as a picture and associates with an object of an Animal record.
For each row (record of an animal):
byte[] ba = (byte[])x.ItemArray[1]; //reading binary from a DB row
using (MemoryStream m=new MemoryStream(ba))
{
Image i = Image.FromStream(m); //exception thrown occassionally
c.Photo = i;
listOfAnimals.Add(c);
}
First of all, with 18 pictures loaded (the JPG files have 105 Mb in total), the running app uses 2 gb of memory. With no pictures loaded, it is only 500 Mb.
Often the exception gets raised in the marked point, the source of which is System Drawing.
Could anyone help me optimize the code or tell me what the problem is? I must have used some wrong functions...

According to Image.FromStream Method
OutOfMemoryException
The stream does not have a valid image format.
Remarks
You must keep the stream open for the lifetime of the Image.
The stream is reset to zero if this method is called successively with the same stream.
For more information see: Loading an image from a stream without keeping the stream open and Returning Image using Image.FromStream
Try the following:
Create a method to convert byte[] to image
ConvertByteArrayToImage
public static Image ConvertByteArrayToImage(byte[] buffer)
{
using (MemoryStream ms = new MemoryStream(buffer))
{
return Image.FromStream(ms);
}
}
Then:
byte[] ba = (byte[])x.ItemArray[1]; //reading binary from a DB row
c.Photo = ConvertByteArrayToImage(ba);
listOfAnimals.Add(c);

Checking the documentation, a possible reason for out of memory exceptions are that the stream is not a valid image. If this is the case it should fail reliably for a given image, so check if any particular source image is causing this issue.
Another possibility should be that you simply run out of memory. Jpeg typically gets a 10:1 compression level, so 105Mib of compressed data could use +1Gib of memory. I would recommend switching to x64 if at all possible, I see be little reason not to do so today.
There could also be a memory leak, the best way to investigate this would be with a memory profiler. This might be in just about any part of your code, so it is difficult to know without profiling.
You might also need to care about memory fragmentation. Large datablocks are stored in the large object heap, and this is not automatically defragmented. So after running a while you might still have memory available, just not in any continuous block. Again, switching to x64 would mostly solve this problem.
Also, as mjwills comments, please do not store large files in the database. I just spent several hours recovering a huge database, something that would have been much faster if images where stored as files instead.

C# .NET reading all of of a process troubles

I'm messing around with a scanning engine I'm working on and I'm trying to read the memory of a process. My code is below (it's a little messy) but for some reason if I read the memory of an application in different states, or after it has a lot of things loaded into memory, I get the same memory size no matter what. Are my entry point addresses and length incorrect?
If I use a memory editor I don't get the same results I do with this.
Process process = Process.GetProcessesByName(processName)[0];
List<Byte[]> moduleMemory = new List<byte[]>();
byte[] temp;
//MessageBox.Show(pm.FileName);
temp = new byte[pm.ModuleMemorySize];
int read;
if (ReadProcessMemory(process.Handle, pm.BaseAddress, temp, temp.Length, out read)) {
moduleMemory.Add(temp);
}
}
//string d = Encoding.Default.GetString(moduleMemory[0]);
MessageBox.Show("Size: " + moduleMemory[0].Length);

Your problem is probaly caused by the fact, that Process class caches values:
The process component obtains information about a group of properties
all at once. After the Process component has obtained information
about one member of any group, it will cache the values for the other
properties in that group and not obtain new information about the
other members of the group until you call the Refresh method.
Therefore, a property value is not guaranteed to be any newer than the
last call to the Refresh method. The group breakdowns are
operating-system dependent.
Therefore after target process loads some additional modules, process instance will still return old values. Calling process.Refresh() should update all cached values and fix the issue.

As I see this code does nothing more than reading the memory layout of the executable module (.exe file) which the process was created for. So no wonder you get the same size all the time.
I assume you are up to read the "operational" memory of the process. If so, you should have a look at this discussion.

How to display memory on a C# application

I'm trying to make an application that reads memory and displays it. The source of the data is coming from a byte[] array and I've tried converting it into text, but did not work. I also tried to use BinaryReader, but it seems to be only for reading files which isn't what I'm trying to do. If there's a way to read memory other than files with Binaryreader that will help too. I found ICSharpCode.TextEditor.dll that may work, but I don't know how to use it. My main goal is to edit the memory with or without it displaying on the application. If anyone can help me I'd appreciate it!
List of questions that's related to this topic
How to I display memory visually on the application? e.g. the picture below.
What can I use to read memory without streaming a file?
What .dll or c# function can I use to make a memory reader/editor that displays or not that reads a byte[].
This is how I want it to show on application.
Here are examples of what I'm trying to do
byte[] Memory = { }; // Source code
BinaryReader ReadMemory = new BinaryReader(Memory); // I need it to read byte[] , not a file
ReadMemory.ReadBytes(10);
//OR
byte[] Memorysource = { }; // Source code
richTextBox1.Text = Convert.ToString(Memorysource);

You can use String.Format("{0,10:X}", bitValue)
You can also use the BitConverter:
BitConverter.ToString(byteArray);

ActiveReports and Bitmaps using too much memory

Our software needs to produce variable-sized reports, which can easily move past 100 pages. Several of these pages contain large images/Bitmaps.
Is there a reliable way to prevent the overall report from consuming all available memory? Once we have enough pages being generated, the app almost never finishes creating the report without running out of memory. Most of the memory is consumed by Bitmaps that we cannot release. (Attempting to dispose of them before the report is complete causes the report generation to fail.)

John
Have you tried using the cache to disk with ActiveReports?
http://helpcentral.componentone.com/nethelp/AR7Help/OnlineEn/CacheToDiskAndResourceStorage.html
More details here:
http://helpcentral.componentone.com/nethelp/AR7Help/OnlineEn/GrapeCity.ActiveReports.Document.v7~GrapeCity.ActiveReports.Document.SectionDocument~CacheToDisk.html
Set this up prior to running the report. For example:
report.Document.CacheToDisk = true;
report.Run();

I think you could try spliting ur report into smaller chunks ,run them and then merge all the reports into one once all pages have been generated.

One more suggestion in addition to setting CacheToDisk property of ActiverReports to True would be to make use of Image.FromStream instead of Image.FromFile to access the images.
Image.FromFile leaves file handles open, and hence it may result in the Memory Exception.
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
using (Image original = Image.FromStream(fs))
{
...
}
}
Using an explicit Dispose(), a using() statement or setting the value to null on the bitmap doesn't generally solve the issue with Image.FromFile.
So if you App runs for a time and opens a lot of files consider using Image.FromStream() instead.
Regards,
Mohita

Simple Image Processing in WinRT

I'm working on a WinRT app that should require some image processing. I've done something similar so far but in Java, I would like to do some simple things in WinRT app also... But I can't seem to manage my way with the API...
Long story short, I have in xaml, on my page, an image that obtains an image with a file picker. Then when I click a "negativate" button, the image should get negativated.
Now, the method for the negativation button, I thought to look like this :
private void OnNegativateButtonClick(object sender, RoutedEventArgs e)
{
var imageToNegativate = ImagePanel.Source as WriteableBitmap ;
if (imageToNegativate == null) //Actually is ALWAYS null :(
{
//Wrong code here...
var bitmapSource = ImagePanel.Source as BitmapSource;
imageToNegativate = new WriteableBitmap(imageToNegativate.PixelWidth, imageToNegativate.PixelHeight);
}
imageToNegativate = ImageUtil.Negativate(imageToNegativate);
ImagePanel.Source = imageToNegativate;
}
This is very similar to this sample I found here but that sample project won't even load so I tried to open the files individually... My code is that method for negativation only there is something wrong with wb = new WriteableBitmap(bs); in his if (wb==null) { ... }.
What is the approach to take a WriteableBitmap from an image, do some pixel manipulation, and then set the source of the image with the new WriteableBitmap...
I'm saying about WriteableBitmap because my method for negativation uses one for input, does some processing and outputs it. (same type, WriteableBitmap.
Any suggestions or help is greatly appreciated, thank you!

The first problem is these lines of code:
var imageToNegativate = ImagePanel.Source as WriteableBitmap ;
if (imageToNegativate == null) //Actually is ALWAYS null :(
The null is telling you that ImagePanel.Source is not of type WriteableBitmap; your typecast failed. This is expected; the picker is going to give you something read-only, because read-only images are more performant (WinRT can do some optimizations if it knows the image's content isn't going to change). You only get a WriteableBitmap when you explicitly create one.
The body of your if block also doesn't make much sense -- you're trying to create a new, empty WriteableBitmap with the same size as the original image, and then you try to do an inverse-video on that empty image. Even if you got that far, you'd just get another empty image. You're not doing anything to keep the pixels from the original image.
You do need a WriteableBitmap to get access to the pixel buffer, but you need to make one that's a copy of the original image. Get rid of your cast and if block, and try this instead:
var imageToNegativate = new WriteableBitmap(ImagePanel.Source);

Joe is right in saying what is wrong with your code, but while his answer would probably work in WPF - the constructor he mentions indeed doesn't exist in WinRT/XAML. There is in fact no way to create a WriteableBitmap from a BitmapImage and the only way to attack it is to create a WriteableBitmap from the source of your BitmapImage.
One approach then would be to create a new 1x1-sized WriteableBitmap, then use its SetSource method on the source of your BitmapImage. Since the BitmapImage.UriSource property is a Uri and WriteableBitmap.SetSource() requires a Stream - you need to play with it a bit to find or download the actual image file and open a stream to read it. My toolkit has some extension methods (WriteableBitmap.FromBitmapImage()) to help you with it in case your images come from your application package and you could fairly easy expand it to also work with downloaded images.
The problem is - you get some inefficiencies that way - one is that you need to open and decode the file twice (which might be slow with large images on an ARM device - been there, done that), second is that you might need to re-download the file if it came from the network and third that there is a bug in WinRT/XAML that causes the UriSource property of a BitmapImage to become null if you set CacheMode of an associated Image control to BitmapCache, so you might need to track your sources separately anyway.
The best solution in many ways is to simply make sure you open the source image as a WriteableBitmap from the beginning, so you don't even have to create another bitmap if you want to change it anyway. In your case the user selects the image file, so there is only one file and it seems to have a pretty big chance of being processed afterwards so opening it as a WriteableBitmap sounds like the right thing to do.
Perhaps even if you want to keep both the original and processed version of the image - creating a new WriteableBitmap of same size and copying the pixels of the original might be faster (especially on an ARM device) than decoding a big image file twice.