I currently have an application which displays Images in a grid. Each image has an effect. When the user moves the mouse while clicked, some input parameters for the effect are changed, leading to a different display of the image. However when I have about 8 or more images displayed, and each image is changed simultaneously, I experience a large drop in FPS.
Using the Perforator tool, I noticed that the HW IRTs get very large (32 per frame for 8 images). The amount of HW IRTs is proportional to the amount of images displayed. So it seems that for each frame change, the Image texture is loaded again and again to the GPU, although the actual texture never changes.
Is there a way to disable this behaviour? RenderOptions.CachingHint sounded like a candidate for this, but it does not have any effect when I set it on the Image.
Related
Like I said in the title when i built to mobile the letter became too small. I think it is because phone has more pixel for his little screen.
Well I tried to fix text by simply making text to images in photoshop. But how can I make InputFields text (not the placeholder).
At a guess, your Canvas' CanvasScaler's UI Scale Mode is probably set to Constant Pixel Size, and if your mobile device's resolution is too high, text will appear small.
Try changing the UI Scale Mode to Scale With Screen Size and change the properties it provides as needed.
(if your Canvas doesn't have a CanvasScaler component, click the Add Component button and add it)
Question:
What is a fast way to scale and/or crop a bitmap provided from a WritableBitmap for display in the UI?
Requirements:
Have Low CPU usage
Handle large images (5 Megapixel, abt 2500x2000 pixels)
Resize and/or crop to the same resolution/area as the UI element the bitmap is displayed in.
Use WPF
Specifically, it must allow a 14FPS 5 Megapixel camera image stream to be displayed in a WPF UI element at full speed.
Update:
I have been able to speed up the drawing quite a bit by painting to a Canvas control using an ImageBrush as follows, where m_bitmap is my WriteableBitmap:
ImageBrush brush = new ImageBrush();
brush.ImageSource = m_bitmap;
brush.Stretch = Stretch.Uniform;
canvas.Background = brush;
I'm now able to get the full 14FPS, though it still using about 20% CPU, so I'm not sure how well it preform if I add another camera or two (the plan is to have 4 or so running).
Another thing I think might be slowing down the drawing is the images are in a mono, Gray8, format, not the standard RGB32 (or is it bgra32 for WPF?) format. If I understand correctly, the image has to be converted to the standard format to be displayed, which would add significant overhead to each frame's drawing time.
Some background:
I'm currently working with a 5 Megapixel, 14 FPS, video camera and am trying to get the frames to render to the screen at full speed. I would like to do this using WPF.
I currently have an example in WinForms that runs full speed for an unscaled image, but (as I would expect) it has major trouble if I set the pictureBox.SizeMode = Zoom;. The example reads raw data directly from the camera stream to a buffer and then copies the data from the buffer into the bitmap set to the PictureBox control. The copy algorithm uses LockBits to speed things up.
I converted that example into WPF, rewriting the parts using Bitmap objects to instead use WritableBitmap objects and an Image control instead of PictureBox. Unforunately this is not able to render the stream to the screen at any decent rate, scaled or unscaled. Both have significant CPU load and very slow updates.
The performance when rendering to the screen is turned off is great. It is able to process the image stream at full speed and resolution while using around 3% CPU and less than 100MB memory.
Note: when I say rendering to the screen is turned off, the WritableBitmap is still being continuously updated, only is not set to the Image control.
I've seen a lot of discussion about getting fast bitmap updating in WPF, but have been unsuccessful in getting it to work at an reasonable speed/cpu load. Also I would like to have the image scaled in such a way that I can see the whole image.
I imagine the key will lie in some sort of scaling/crop combination that needs to be done so that WPF will not try to render(cache?) all 5 million pixels, but only those on the screen, and only at the current screen resolution. I imagine/hope this can be done fairly easily and without too much hit to memory or CPU, but currently have no idea how to do so. I have found the DecodePixelWidth and DecodePixelHeight properties, but those are only applicable when loading an image from a file to a BitmapImage.
Did you have a look at the following post?
Resizing WritableBitmap
If it does not solve your problem, I have more questions for you:
What is the resolution of your image?
Is the size of you UI element constant? What's its size?
Edit:
After your edit, I noticed that you want to display a BitmapImage in Gray8 PixelFormat, why don't you try to set this property when creating your BitmapImage (m_bitmap)?
m_bitmap.Format = PixelFormat.Gray8; // could not test
I am certain that taking your 8 bits/pixel and multiplying the amount of bits needed per pixel by 4 while not gaining any quality is slowing down your application. Especially because you run operations on 32 bits per pixel images when you could be running those operations on 8 bits per pixel images.
While its interface is a bit old-fashioned, I believe that convert (see http://en.wikipedia.org/wiki/ImageMagick) is very often used (and may in fact be the industry standard).
Edit: StackOverflow has about 2,300 question tagged with imagemagick here. See for example What is the difference for sample/resample/scale/resize/adaptive-resize/thumbnail operators in ImageMagick convert?
The OP for https://apple.stackexchange.com/a/41531 decided to go with ImageMagick. And the accepted answer to Efficient JPEG Image Resizing in PHP also suggests ImageMagick, with 19 votes.
However, I don't know whether ImageMagick is capable of meeting your requirements of 14FPS, 5 Megapixels.
The only answer to Recommendation for real time image processing tools on Linux suggests a fork graphicsmagick, which seems to also be available for Windows.
I am working on an Application that loads live video images from a camera and displays an overlay on top of said image. The Overlay does not change often so it can be considered as still. However it usually contains about 1,000 to 10,000 Lines.
When the video image is updated there is a notable impact to the CPU load depending on whether the overlay is visible or not. The overlay does neither get invalidated nor changed, just the image behind it is changing.
My setup is this:
<Canvas>
<Image/>
<Canvas>
<OverlayElement 1/>
<OverlayElement 2/>
<OverlayElement 3/>
<.../>
</Canvas>
</Canvas>
The Image's Source is a WriteableBitmap. Every time a new camera image (type byte[]) is available, the main Canvas' Dispatcher is invoked to write the image data by using WriteableBitmap.WritePixels().
The inner Canvas contains all Overlay Elements, being
- a contour (PolyLine)
- a circle (Path with EllipseGeometry) and
- a set of Rays (Path with one Figure containing LineSgements).
The number n of Points in the contour equals the number of line Segments in the last mentioned Path. n is usually around 1,000 - 3,000.
Depending on the count and length of Lines shown in the overlay the CPU load for showing a live image varies (increases if length or count go up) even if the overlay does not change. At some point this affects the frame rate and makes the program unusable. Line length is mostly correlated with line intersection, so maybe the Path is struggling to calculate it's fill area despite it is not painted?
So how could I improve the performance here?
What bugs me most is that even if the overlay does not change, the render time increases with it's primitive count. I would expect to have constant render time once the overlay has been drawn in it's last set state. What could I do to achieve that aside from rendering the whole overlay to a bitmap?
I am also open minded for suggestions on how to get the byte[] onto the screen more efficiently. Just keep in mind this problem is part of a bigger Application and i cannot change all paradigms concentrating on how to get the image drawn.
What I have tried so far:
Override the OnRender() method of the inner Canvas, drawing the overlay myself. This works fine but has the performance issue that brings me here ;)
Use Shapes (PolyLine, Ellipse, Path) as the inner Canvas' children to hold the overlay elements. This works, too. It is faster to redraw the overlay when it changes but on the other hand worsens the performance issue when updating the background image.
Like 2., but use Freeze() on Geometries wherever possible. Has no or little performance impact.
Thanks for your help in advance.
I want to make a control that acts as a time ruler with a wave form for sound wav files like that in movie maker or in camtasia studio video editor
i have a method to parse the sound wav file
i have a method to draw the wave of a specified time period
the problem is that the duration is long how to render all of it??one time or when the user is scrolling to a point if it isn't rendered I'll render it? what after its all rendered ? it will be so long user will zoom in and so on how to do it without effecting the performance
If you're dealing with a big image and lots of data I would sugest this:
render/display what user is currently looking at
render rest of if in a background thread into a memory image
Once you have the whole image rendered in an image kept in memory, any scrolling and/or zooming would be just to position your display window.
Render your image in an unzoomed scale into a Bitmap object. When you need to display part of if use Bitmap.Clone to copy section of that bitmap into a picture displayed to user. A bit difficult part will be to render part of your base Bitmap which is currently demanded to be viewed.
This could be memory demanding. It is effectively a caching mechanism and as always in cases like that you get speed by using more memory.
Something similar to how google maps works.
We do not know how you are rendering your graphic, but I presume you are using unsafe lockbits to generate directly into a bitmap. As Maciej says, just render the part you need to display - when zoomed, either render again (should be fast). You can preempt by rendering current, zoom in 1, zoom out 1, left 1/2 a screen, right 1/2 screen etc in the background, but then you need to manage these (they can be rendered in parallel of course using background worker threads). The key to memory is to keep the bitmaps to a minimum (you have the algorithm of the graph and the renderer) and the key to speed is to keep it low level and prefetch what you can and parallelism.
I am creating a custom control using C# GDI+.
Quick explanation...the control will be say 500 pixels on screen but will contain perhaps 500000 pixels of information. So although i'm only showing 500px at a time i need to obviously scroll in the horizontal plane (left & right). The tricky part is that each 500px chunk of bitmap takes a while (between 100ms - 1000ms) to render.
So my plan is to maintain a 1500px bitmap in memory. i.e. the 500px visible part and 500px either side of the visible area and draw the off-screen parts asynchronously as the user scrolls.
I would like some feedback, advice, criticism or examples of code to help me achieve this. It seems fairly straight forward but after a few initial test attempts its proving more difficult than one would imagine.
Thanks.
The effectiveness of this approach depends on, amongst other things, the amount of moving around the user will do. If the user is making small movement, then stopping to consider the new information, this could work. However, if the user is whizzing back and forth, you'll still have an issue.
Does your application lend itself to gradually improving the quality of the image - i.e. providing a quick usable image and then improving it as the user stops to consider it?
I had a similar problem a few years back. As dommer mentions, if you're processing chunks of the image before displaying them you're best off showing something and improving it later. If you're having problems blitting the original image, you've got something wrong with your method. GDI+ is very particular about pixel depth (you want 32bpp with alpha).
In our case, we processed in 500px tiles and padded out a tile around the visible view Ff the user scrolled outside the area we'd processed we blitted bits of the original image with a dark semi-opaque rectangle super-imposed on them. These chunks were queued for processing. As we processed chunks of the image (centre-out) they semi-opaque rectangles would disappear.
It worked reasonably well, was very responsive and very fast. It's very fast to blit the original bitmap onto the screen, and in our case the processing was usually very close behind. The effect of the tiles getting lighter was actually quite pretty.
Draw only the visible area.
Build a method
Bitmap DrawGraph(leftMargin, rightMargin) {...}
and then in OnPaint() do
Bitmap bmp = new Bitmap(e.ClipRectangle.Width, e.ClipRectangle.Height);
bmp = DrawGraph(e.ClipRectangle.Left, e.ClipRectangle.Right);
e.Graphics.DrawImageUnscaled(bmp, e.ClipRectangle.Location);