We Keep Coding

Clearing EmguCV's image buffer

I'm working on a project using EmguCV which requires capturing images from a camera and processing them, looking for certain things in the image. The processing takes up about ~.2 seconds, and is the biggest time-sink in the application so we're looking into making the capturing/processing different threads to speed up the overall process.
We've already tried the VideoCapture.ImageGrabbed event handler, and calling Retrieve, as well as setting up a threaded loop of our own calling QueryFrame and any other capturing method we can find.
Most of the threaded solutions end up causing empty images (not caught by the .IsEmpty property of the Mat, though, for some reason), which end up as images saved with 0 bytes.
After this, we tried simplifying but ran into an issue where the camera is always a few seconds behind, due to the buffer internal to the library. This leads to my question: is there a way to refresh the buffer, or clear it of memory? We cannot dispose of the capture object because of the time overhead for creating the object. Any tips about threading the capturing and processing are welcome as well, though the other suggestions I've found around this site about similar situations have not led to much success.
A brief example of the code we're using
_capture.ImageGrabbed += GrabFrames;
...
public void GrabFrames()
{
Mat image = new Mat();
_capture.Retrieve(image);
ProcessThread = new Thread(new ParameterizedThreadStart(StartProcess));
ProcessThread.Start(image);
}
...
public void StartProcess(object image)
{
Mat img = (Mat)image;
Process(image);
img.Dispose();
}

How to save save high frame rate camera images with live preview

I have two high speed USB3 cameras (Ximea) and want to code an application for image recording. Framerates are up to 500fps at VGA resolution but I also want to use the 2Mpx resolution at 170fps.
Their .Net SDK tells me that I should simply "get" the images in a loop.
My problem is that I have no idea how to get the images and save them while still showing a live preview. Everytime I add some code to actually update the picturebox the frame rate drops drastically.
At the moment I utilize a recording function that is called with
Task.Run(() => Record());
and inside the Record() I have a loop getting the bitmaps
while(record == true)
{
Camera.GetImage(out myImage, timeout); //From Ximea .Net SDK
Info = Camera.GetLastImageParams();
Timestamp = Info.GetTimeStamp();
ThreadPool.QueueUserWorkItem(state => SaveImage(myImage, filepath, Timestamp));
}
with the SaveImage being
private void SaveImage(Bitmap myImage, string filepath, double Timestamp)
{
try
{
lock(myImage)
{
myImage.Save(filepath + Timestamp.ToString("0.00000") + ".tif");
}
}
catch{}
}
How can I show a live preview while recording and how can I make the entire code more stable (at the moment there are some dropped frames because of "object already in use"-errors or "generic error in GDI+" at the Image.Save() call, that I skip with the try/catch statement)?

I believe you can tell the Ximea API how many image buffers you want in the incoming queue... use XI_PRM_BUFFER_POLICY and XI_PRM_BUFFERS_QUEUE_SIZE appropriately to make that queue length somewhat long.
Then, have a thread that, when activated, copies an image out of a XI_IMG struct into your own buffer. Activate that thread every n frames (based on the size of the Ximea image buffer queue size)... but don't do any memory copies in the loop that actually calls xiGetImage. You should probably block in your thread to avoid tearing (because the Ximea code could get back around to using the same buffer again if you're not fast enough at copying the data out)... but you could then dynamically adjust the number of buffers so you can finish your copy within the time you have. Also, you may consider copying the image data to ANOTHER buffer if you're doing something that takes a long time...
pseudo-code (sorry, it's C-ish):
// sync objects and a global image buffer pointer
CRITICAL_SECTION cs;
void *buf;
HANDLE ev;
int CopyImageThreadProc(...)
{
while (true)
{
if (WaitOnSingleObject(ev) == WAIT_OBJ_0)
{
EnterCriticalSection(cs);
// copy the image data at buf where ever you want
LeaveCriticalSection(cs);
}
}
}
int main(...)
{
// set up ximea api with appropriate buffering
// create event and critsec, start thread
while (!done)
{
XI_IMG img;
xiGetImage(dev, 10, &img);
// every 15 frames, tell your thread to go...
// if you find that the critsec is causing a hiccup, you can adjust this
// but remember to adjust the queue length, too
// if you change this to TRY entercriticalsection, you can determine that
if ((img.acq_nframe % 15) == 0)
{
EnterCriticalSection(cs);
buf = img.bp;
SetEvent(ev);
LeaveCriticalSection(cs);
}
}
// clean up
}

Add each captured frame to a queue, then have a worker thread that takes those images, one at a time, and saves them. Trying to write multiple images to disk at the same time will most likely be slower. Also, always Dispose of any GDI objects or you will run into trouble really fast. I would think not doing that is what is giving you the exceptions.
As for showing the images, make sure you are not trying to display every image. Your monitor most likely runs at 60 Hz, so anything faster than that will be a waste. I also suspect that (with the performance of GDI), you won't necessarily be able to achieve even that. So I suggest you have a second queue with images to display, and if you see that queue getting too large, your program will need to slow down a bit and not push as many frames to the queue.
Edit: And of course, as #Franck mentions, if your disk can't keep up, your queue/buffer will fill up quickly. Compressing the images might help, assuming they have suitable content for compression and that your processor can keep up.
Edit: What you need is a producer-consumer pattern. There are many ways of doing this, but one might be something like:
// blocking collection
private BlockingCollection<Bitmap> m_Queue = ...
// camera thread
while( run )
{
var bitmap = GrabFrame();
m_Queue.Add( bitmap );
}
// worker thread
try
{
while( true )
{
// Take() will block if the queue is empty
var bitmap = m_Queue.Take();
bitmap.Save( ... );
bitmap.Dispose();
}
catch( InvalidOperationException )
{
// you'll end up here if you call `m_Queue.CompleteAdding()`
// (after the queue has been emptied, of course)
}
As for displaying the images, you could probably use something similar, with the addition of having some code that determines if it's time to push a new image or not.

How to avoid ReportViewer memory leak when navigating through many reports?

I am using Visual Studio 2013 to create a WPF Desktop application that have some report generation functionalities, I have about 30 report and the user can swich from a report to another. My problem is that each time I change ReportEmbeddedResource and then call the RefreshReport() methods, the memory increases, so if the user navigate through all the 30 report my app will consume about 130 Mb! I know that I have to release the Resources after each navigation, I googled about that but didn't find an answer; Here is my code
public MainWindow() // constructor
{
InitializeComponent();
this.reportViewer.ZoomMode = Microsoft.Reporting.WinForms.ZoomMode.PageWidth;
InitDataSources();
}
private void InitDataSources()
{
//manager data source
mangerDataSource = new ReportDataSource();
mangerDataSource.Name = "ManagerDataSet";
mangerDataSource.Value = uow.Members.GetAll().
ToList().Where((s) => s.MemberType == MemmberTypes.Manager);
reportViewer.LocalReport.DataSources.Add(mangerDataSource);
//adding 2 other data sources
}
public void RenderReport(string reportKey)
{
reportViewer.Reset();
string path = "Manager";
if (reportKey.Contains("tea")) path = "Teacher";
if (reportKey.Contains("stu")) path = "Student";
reportViewer.LocalReport.ReportEmbeddedResource = string.Format(
"Printers.Reports.{0}.{1}.rdlc", path,reportKey);
reportViewer.RefreshReport();
}
Is there a way to release the old report resource after Rendering a new report?

I don't have much experience with this but it seems that the best thing you can do is to use the Safe Handles to get your reports inside a manageable wrapper and then use the Dispose method and force the Garbage Collector to collect, while suppressing the Finalizer. Note that the memory usage you see in the Taskmanager is reserved memory, not actually memory in current use; it is possible that you release the report object and the taskmanager continues to report high memory values on the executable.
reportViewer.Dispose();
GC.SuppressFinalize(reportViewer);
The whole Disposing Method can become quite confusing so take your time and have a look here:
MSDN - Implementing a Dispose Method
MSDN - IDisposable.Dispose Method

I was having the same issue with .NET 4.5 VS 2013
I tried several things, but what finally made it work was:
Compiling the project in x64 and using LocalReport.ReleaseSandBoxAppDomain()
I got part of the solution from here: Very High Memory Usage in .NET 4.0

Problem is solved by MarkJ_KY's comment at https://connect.microsoft.com/VisualStudio/feedback/details/527451/ms-report-viewer-memory-leak-any-update-fix-winforms-application
It might look a little complex but it is not. The idea is to create an AppDomain, do your reporting stuff in that domain and then unload the domain. When unloading all memory are released :-)
I have used that solution which solves the problem for me.

Loading many large photos into a Panel efficiently

How do I load many large photos from a directory and its sub-directories in such a way as to prevent an OutOfMemoryException?
I have been using:
foreach(string file in files)
{
PictureBox pic = new PictureBox() { Image = Image.FromFile(file) };
this.Controls.Add(pic);
}
which has worked until now. The photos that I need to work with now are anywhere between 15 and 40MB's each, and there could be hundreds of them.

You're attacking the garbage collector with this approach. Loading 15-40mb objects in a loop will always invite an OutOfMemoryException. This is because the objects go straight onto the large object heap, all objects > 85K do. Large objects become Gen 2 objects immediately and the memory is not automatically compacted as of .Net 4.5.1 (you request it) and will not be compacted at all in earlier versions.
Therefore even if you get away with initially loading the objects and the app keeps running, there is every chance that these objects, even when dereferenced completely, will hang around, fragmenting the large object heap. Once fragmentation occurs and for example the user closes the control to do something else for a minute or two and opens the control again, it is much more likely all the new objects will not be able to slot in to the LOH - the memory must be contiguous when allocation occurs. The GC runs collections on Gen 2 and LOH much less often for performance reasons - memcpy is used by the GC in the background and this is expensive on larger blocks of memory.
Also, the memory consumed will not be released if you have all of these images referenced from a control that is in use as well, imagine tabs. The whole idea of doing this is misconceived. Use thumbnails or load full scale images as needed by the user and be careful with the memory consumed.
UPDATE
Rather than telling you what you should and should not do I have decided to try to help you do it :)
I wrote a small program that operates on a directory containing 440 jpeg files with a total size of 335 megabytes. When I first ran your code I got the OutOfMemoryException and the form remained unresponsive.
Step 1
The first thing to note is if you are compiling as x86 or AnyCpu you need to change this to x64. Right click project, go to Build tab and set the target platform to x64.
This is because the amount of memory that can be addressed on a 32 bit x86 platform is limited. All .Net processes run within a virtual address space and the CLR heap size will be whatever the process is allowed by the OS and is not really within the control of the developer. However, it will allocate as much memory as is available - I am running on 64 bit Windows 8.1 so changing the target platform gives me an almost unlimited amount of memory space to use - right up to the limit of physical memory your process will be allowed.
After doing this running your code did not cause an OutOfMemoryException
Step 2
I changed the target framework to 4.5.1 from the default 4.5 in VS 2013. I did this so I could use GCSettings.LargeObjectHeapCompactionMode, as it is only available in 4.5.1 . I noticed that closing the form took an age because the GC was doing a crazy amount of work releasing memory. Basically I would set this at the end of the loadPics code as it will allow the large object heap to not get fragmented on the next blocking garbage collection. This will be essential for your app I believe so if possible try to use this version of the framework. You should test it on earlier versions too to see the difference when interacting with your app.
Step 3
As the app was still unresponsive I made the code run asynchronously
Step 4
As the code now runs on a separate thread to the UI thread it caused a GUI cross thread exception when accessing the form, so I had to use Invoke which posts a message back to the UI thread from the code's thread. This is because UI controls can only be accessed from a UI thread.
Code
private async void button1_Click(object sender, EventArgs e)
{
await LoadAllPics();
}
private async Task LoadAllPics()
{
IEnumerable<string> files = Directory.EnumerateFiles(#"C:\Dropbox\Photos", "*.JPG", SearchOption.AllDirectories);
await Task.Run(() =>
{
foreach(string file in files)
{
Invoke((MethodInvoker)(() =>
{
PictureBox pic = new PictureBox() { Image = Image.FromFile(file) };
this.Controls.Add(pic);
}));
}
}
);
GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;
}

You can try resizing the image when you are putting on the UI.
foreach(string file in files)
{
PictureBox pic = new PictureBox() { Image = Image.FromFile(file).resizeImage(50,50) };
this.Controls.Add(pic);
}
public static Image resizeImage(this Image imgToResize, Size size)
{
return (Image)(new Bitmap(imgToResize, size));
}

How would I load photos in the background in a wpf desktop app so that it doesn't take a few seconds to load the next photo in the gallery?

Currently my program reads images and text in a record from an xml file, displays them on the screen, and then the click of the previous/next buttons moves to the next record. However, it seems to need a few seconds loading time between each photo and I'd like it to be instant, like how Windows Photo Gallery would...or Facebook photos (bear in mind this is not a web app).
I searched found a few similar situations to mine but none seemed to fit my situation. I tried making a class, based on my search, to deal with background loading and calling it in my program, but it's fraught with error and probably won't even do what I want it do:
//ImageManager.cs
class ImageManager
{
private Dictionary<string, Image> images = new Dictionary<string, Image>();
public Image get(string s)
{ // blocking call, returns the image
return load(s);
}
private Image load(string s)
{ // internal, thread-safe helper
lock (images)
{
if (!images.ContainsKey(s))
{
Image img = images.Add(s, img); //load the image s - ERROR cannot implicitly convert type void to image. Void??
return img;
}
return images[s];
}
}
public void preload(params string[] imgs)
{ // non-blocking preloading call
foreach (string img in imgs)
{
BackgroundWorker bw = new BackgroundWorker();
bw.DoWork += (s, e) => { load(img); }; // discard the actual image return
bw.RunWorkerAsync();
}
}
}
//MainWindow.cs
ImageManager im = new ImageManager();
im.preload("Data/Images"); // Errors - im is a field but used like a type/token '('
Many thanks in advance

Your ImageManager should work with ImageSources, not Images. Even if you get your current code to work you'll find that your UI still hangs because you have no choice but to perform the work on the UI thread. If you instead deal with ImageSources, you can load them on a background thread and then freeze them in order to use them from the UI thread. This frees you to pre-emptively load images, or to show a loading animation whilst they load.
BitmapFrame.Create is likely the method you want to be using to load the images.

Consider caching scaled down images - 1:1 of what you want to show, or even smaller. This way loading of preview will be much faster and if user looks at the image long enough you can load full image.
With modern photos original size of the image is usually way bigger than can be normally diaplayed. So if you always read original images you spend large amount of disk IO on something that will never be shown.
Usual note: it may not be case in your program. As with any performance issues measure, than optimize.