Displaying bitmaps rapidly - c#

I have a WPF application where I need to add a feature that will display a series of full screen bitmaps very fast. In most cases it will be only two images, essentially toggling the two images. The rate they are displayed should be constant, around 10-20ms per image. I tried doing this directly in WPF using a timer, but the display rate appeared to vary quit a bit. I also tried using SharpGL (a .Net wrapper on OpenGL), but it was very slow when using large images (I may not have been doing the best way). I will have all the bitmaps upfront, before compile time, so the format could be changed as long as the pixels are not altered.
What would be the best way to do this?
I'm already behind schedule so I don't have time to learn lots of APIs or experiment with lots of options.

"I tried doing this directly in WPF using a timer, but the display rate appeared to vary quit a bit."
Instead of using a Timer, use Thread.Sleep(20) as it wont hog as many system resources. This should give you an immediate improvement.
It also sounds as though there will be user interaction with the application while the images are rapidly toggling, in this case put the code that toggles the images in a background thread. Remember though that the UI- is not thread safe.
These are just quick wins, but you might need to use DirectX for Hardware Acceleration to get around HAL:
The Windows' Hardware Abstraction Layer (HAL) is implemented in
Hal.dll. The HAL implements a number of functions that are
implemented in different ways by different hardware platforms, which
in this context, refers mostly to the Chipset. Other components in the
operating system can then call these functions in the same way on all
platforms, without regard for the actual implementation.

Related

Multiple UI threads for page (speed up slow individual controls)

I am currently writing a kiosk style app in UWP (windows IoT Core), for use on embedded devices (e.g. pi3, etc.).
The device has several sensors, which are being output in real time into various graphs/charts in a single screen in the app, but I'm running into performance problems. The sensors are being read in separate threads (using Task.Run() => {}), but after analysis, this doesn't seem to cost that much cpu time at all.
It seems that updating the graphs take too much time and this isnt being distributed over the cores, since there is only a single UI thread. CPU usage doesnt get past 25%, the UI responsiveness just gets slow.
I tried several optimizations (e.g. reducing the amount of data points, etc.), which helps, but its not enough. Maybe there are faster chart components out there (currently using Telerik uwp component), but I was looking for another approach.
So summary of my question: Is there any way to have the charts each render in separate UI threads (and thus be distributed over the other cores somewhat)?
[UPDATE some time later]
Seems like the new win IoT release is a bit faster, as well as the newer chart component.
Can't speak strictly for UWP but from my experience of WPF having multiple UI threads is a hassle which you should avoid if at all possible (not to speak of the limitations they introduce).
When dealing with your sensor data what kind of polling rate are you looking at? Many sensors have a data rate which far outstrips what is useful to display to the user, would it be possible to have your worker thread compute an averaged value for a larger time frame and populate your graph with that? I doubt your users will see much of a benefit from updating them more than 20 times a second.
You can update the Graphs using Tasks too, just remember to use a dispatcher inside them, because changing UI from non UI threads is illegal.
For more info check this out:
https://learn.microsoft.com/en-us/windows/uwp/debug-test-perf/keep-the-ui-thread-responsive
and
https://learn.microsoft.com/en-us/uwp/api/windows.ui.core.coredispatcher

C++ AMP calculations and WPF rendering graphics card dual use performance

Situation:
In an application that has both the need for calculation as well as rendering images (image preprocessing and then displaying) I want to use both AMP and WPF (with AMP doing some filters on the images and WPF not doing much more than displaying scaled/rotated images and some simple overlays, both running at roughly 30fps, new images will continuously stream in).
Question:
Is there any way to find out how the 2 will influence each other?
I am wondering on whether I will see the hopefully nice speed-up I will see in an isolated AMP only environment in the actual application later on as well.
Additional Info:
I will be able and am going to measure the AMP performance separately, since it is low level and new functionality that I am going to set up in a separate project anyway. The WPF rendering part already exists though in a complex application, so it would be difficult to isolate that.
I am not planning on doing the filters etc for rendering only since the results will be needed in intermediate levels as well (other algorithms, e. g. edge detection, saving, ...).
There are a couple of things you should consider here;
Is there any way to find out how the 2 will influence each other?
Directly no, but indirectly yes. Both WPF and AMP are making use of the GPU for rendering. If the AMP portion of your application uses too much of the GPU's resources it will interfere with your frame rate. The Cartoonizer case study from the C++ AMP book uses MFC and C++ AMP to do exactly the way you describe. On slower hardware with high image processing loads you can see the application's responsiveness suffer. However in almost all cases cartoonizing images on the GPU is much faster and can achieve video frame rates.
I am wondering on whether I will see the hopefully nice speed-up
With any GPU application the key to seeing performance improvements is that the speedup from running compute on the GPU, rather than the CPU, must make up for the additional overhead of copying data to and from the GPU.
In this case there is additional overhead as you must also marshal data from the native (C++ AMP) to managed (WPF) environments. You need to take care to do this efficiently by ensuring that your data types are blitable and do not require explicit marshaling. I implemented an N-body modeling application that used WPF and native code.
Ideally you would want to render the results of the GPU calculation without moving it through the CPU. This is possible but not if you explicitly involve WPF. The N-body example achieves this by embedding a DirectX render area directly into the WPF and then renders data directly from the AMP arrays. This was largely because the WPF viewport3D really didn't meet my performance needs. For rendering images WPF may be fine.
Unless things have changed with VS 2013 you definitely want your C++ AMP code in a separate DLL as there are some limitations when using native code in a C++/CLI project.
As #stijn suggests I would build a small prototype to make sure that the gains you get by moving some of the compute to the GPU are not lost due to the overhead of moving data both to and from the GPU but also into WPF.

Efficiently creating many instances of a winrt metro control

I want to be able to create a few hundred instances of a winrt control (a search result control) in a C# project. The problem is that doing so takes too long (tenths of a second or worse) and must be done on the UI thread, creating stalls and delays in showing results.
For now I've bypassed the issue by pre-caching many instance of the control during startup. This approach works, but affects the startup time (profiling shows 40% of processor time near startup is spent caching these controls) and creates details to be managed, like the size of the cache.
I think the issue is that every time the control is instantiated redundant work, like re-parsing XAML, is done by the underlying framework. Maybe there's a way to avoid repeating this work? Maybe I can cheaply clone an existing control? Does anyone have ideas?
You could do the pre-caching in a parallel thread. Will make less of an impact on startup time on multicore processors
searchresult.memberwiseclone wil give you shallow copies. It might be faster, not sure
Could you use just one searchresult and fill it with the right data just before use? In that case there is no need to create a lot. Just use one as a re-usable container.
If the time is spent when adding the controls to the parent forms you could use
suspendlayout/resumelayout (this is win32)
setting parent to invisible and back to visible when you are done
Is there another way to do the same things faster? (competing control, 3d party etc)
Forenote: it's been awhile so Microsoft may have fixed the win8 app UI virtualization by now. I haven't checked.
What I ended up doing at the time was just hacking together my own UI virtualization to work around the issue. Basically: caching controls and re-using them to display the viewable data (using a binary search tree to efficiently query what can be seen). I wrote a blog post about it.

High performance graphics using the WPF Visual layer

I am creating a WPF mapping program which will potentially load and draw hundreds of files to the screen at any one time, and a user may want to zoom and pan this display. Some of these file types may contain thousands of points, which would most likely be connected as some kind of path. Other supported formats will include TIFF files.
Is it better for performance to have a single DrawingVisual to which all data is drawn, or should I be creating a new DrawingVisual for each file loaded?
If anyone can offer any advice on this it would be much appreciated.
You will find lots of related questions on Stack Overflow, however not all of them mention that one of the most high-performance ways to draw large amounts of data to the screen is to use the WriteableBitmap API. I suggest taking a look at the WriteableBitmapEx open source project on codeplex. Disclosure, I have contributed to this once, but it is not my library.
Having experimented with DrawingVisual, StreamGeometry, OnRender, Canvas, all these fall over once you have to draw 1,000+ or more "objects" to the screen. There are techniques that deal with the virtualization of a canvas (there' a million items demo with Virtualized Canvas) but even this is limited to the ~1000 visible at one time before slow down. WriteableBitmap allows you to access a bitmap directly and draw on that (oldskool style) meaning you can draw tens of thousands of objects at speed. You are free to implement your own optimisations (multi-threading, level of detail) but do note you don't get much frills with that API. You literally are doing the work yourself.
There is one caveat though. While WPF uses the CPU for tesselation / GPU for rendering, WriteableBitmap will use CPU for everything. Therefore the fill-rate (number of pixels rendered per frame) becomes the bottleneck depending on your CPU power.
Failing that if you really need high-performance rendering, I'd suggest taking a look at SharpDX (Managed DirectX) and the interop with WPF. This will give you the highest performance as it will directly use the GPU.
Using many small DrawingVisuals with few details rendered per visual gave better performance in my experience compared to less DrawingVisuals with more details rendered per visual. I also found that deleting all of the visuals and rendering new visuals was faster than reusing existing visuals when a redraw was required. Breaking each map into a number of visuals may help performance.
As with anything performance related, conducting timing tests with your own scenarios is the best way to be sure.

using C# for real time applications

Can C# be used for developing a real-time application that involves taking input from web cam continuously and processing the input?
You cannot use any main stream garbage collected language for “hard real-time systems”, as the garbage collect will sometimes stop the system responding in a defined time. Avoiding allocating object can help, however you need a way to prove you are not creating any garbage and that the garbage collector will not kick in.
However most “real time” systems don’t in fact need to always respond within a hard time limit, so it all comes down do what you mean by “real time”.
Even when parts of the system needs to be “hard real time” often other large parts of the system like the UI don’t.
(I think your app needs to be fast rather than “real time”, if 1 frame is lost every 100 years how many people will get killed?)
I've used C# to create multiple realtime, high speed, machine vision applications that run 24/7 and have moving machinery dependent on the application. If something goes wrong in the software, something immediately and visibly goes wrong in the real world.
I've found that C#/.Net provide pretty good functionality for doing so. As others have said, definitely stay on top of garbage collection. Break up to processing into several logical steps, and have separate threads working each. I've found the Producer Consumer programming model to work well for this, perhaps ConcurrentQueue for starters.
You could start with something like:
Thread 1 captures the camera image, converts it to some format, and puts it into an ImageQueue
Thread 2 consumes from the ImageQueue, processing the image and comes up with a data object that is put onto a ProcessedQueue
Thread 3 consumes from the ProcessedQueue and does something interesting with the results.
If Thread 2 takes too long, Threads 1 and 3 are still chugging along. If you have a multicore processor you'll be throwing more hardware at the math. You could also use several threads in place of any thread that I wrote above, although you'd have to take care of ordering the results manually.
Edit
After reading other peoples answers, you could probably argue my definition of "realtime". In my case, the computer produces targets that it sends to motion controllers which do the actual realtime motion. The motion controllers provide their own safety layers for things like timing, max/min ranges, smooth accel/decelerations and safety sensors. These controllers read sensors across an entire factory with a cycle time of less than 1ms.
Absolutely. The key will be to avoid garbage collection and memory management as much as possible. Try to avoid new-ing objects as much as possible, using buffers or object pools when you can.
Of course, someone has even developed a library to do that: AForge.NET
As with any real-time application and not just C#, you'll have to manage the buffers well as #David suggested.
Not only that, there're also the XNA Framework (for things like 3D games) and you can program DirectX using C# as well which are very real-time.
And did you know that, if you want, you can do pointer manipulations in C# too?
It depends on how 'real-time' it needs to be; ie, what your timing constraints are, and how quickly you need to 'do something'.
If you can handle 'doing something' maybe every 300ms or so in .NET, say on a timer event, I've found Windows to work okay. Note that this is something I found true on multiple systems of different ages and different speeds. As always, YMMV.
But that number is awfully long for a lot of applications. Maybe not for yours.
Do some research, make sure your app responds quickly enough for your application.

Categories