I am using D3DImage to display a sequence of frames that are rendered unto the same Direct3D Surface one after the other. My current logic is thus:
Display last rendered frame (i.e.D3DImage.Lock()/AddDirtyRect()/Unlock())
Start rendering next frame
Wait for next frame to be ready and that it's time to display it
Display last rendered frame
...
The problem with this approach is that when we are done calling Unlock() on D3DImage, the image isn't actually copied, it's only scheduled to be copied on the next WPF render. It's therefore possible that we render a new frame on the Direct3D surface before WPF has had the chance to display it. The net result is that we see missed frames on the display.
Right now I'm experimenting with using a separate Direct3D texture for rendering and performing a copy to a "display texture" just before display, which is giving better results but incurs substantial overhead. It would be preferrable to just be able to know when D3DImage is done refreshing and start rendering the next frame immediately after. Is this possible, if so how? Or do you have a better idea altogether?
Thanks.
The CompositionTarget.Rendering event is called when WPF is going to render, so that's when you should do your Lock() and Unlock(). After the Unlock(), you can kick off the next render.
You should also check the RenderingTime because the event may fire multiple times per frame. Try something like this:
private void HandleWpfCompositionTargetRendering(object sender, EventArgs e)
{
RenderingEventArgs rea = e as RenderingEventArgs;
// It's possible for Rendering to call back twice in the same frame
// so only render when we haven't already rendered in this frame.
if (this.lastRenderTime == rea.RenderingTime)
return;
if (this.renderIsFinished)
{
// Lock();
// SetBackBuffer(...);
// AddDirtyRect(...);
// Unlock();
this.renderIsFinished = false;
// Fire event to start new render
// the event needs to set this.renderIsFinished = true when the render is done
// Remember last render time
this.lastRenderTime = rea.RenderingTime;
}
}
Update to address comments
Are you sure that there's a race condition? This page says that the back buffer gets copied when you call Unlock().
And if there really is a race condition, how about putting Lock/Unlock around the render code? This page says that Lock() will block until the copy is finished.
It looks like the clean way of doing this, in order to render in parallel with the UI, is to render into a separate D3D surface, and copy it to the display surface (i.e. the one passed to SetBackBuffer) between the calls to Lock() and Unlock(). So the algorithm becomes:
Copy and display the last rendered frame, i.e.
Lock()
Copy from render to display surface
SetBackBuffer(displaySurface)
AddDirtyRect()
Unlock()
Schedule a new render to the render surface
Wait for it to complete and that the timing is ok to display it
Goto 1
The documentation for D3DImage explicitely states:
Do not update the Direct3D surface while the D3DImage is unlocked.
The sore point here is the copy, which is potentially costly (i.e. >2ms if the hardware is busy). In order to use the display surface while the D3DImage is unlocked (avoiding a potentially costly operation at render time), one would have to resort to disassembly and reflection to hook into D3DImage's own rendering...
Related
My question might seem silly to you, but I realized that moving my applications form makes the code inside it run slower. E.g. when I load a bitmap image and apply some image editing algorithms on it, it takes about 22 secs for the whole process to finish. But if I move the form during execution, it adds some 3-4 extra seconds to the elapsed time. I was able to spot the delay using a Stopwatch. So how can I get around this behaviour, if possible at all?
This is just an hypothesis that requires your investigation, as you didn't post any code and thus it is impossible to really know what is going on.
Most probably you move the boundaries of the image outside the screen. When you move in again, the windowing engine will do some draw calls on those rectangles to be redrawn. The same happens on resize when you enlarge but not when you shrink the window.
If this is the case, then you will not experience any extra draw calls as long as you don't cover/uncover areas of the image.
So this is not an answer but in your place I would override the Paint() method and log how many excess calls are made. Based on this, I'd search for a solution, such as suppress those calls like this:
public override void Paint()
{
if (algorithmRunning)
{
return; // suppress any further computations
}
base.Paint(); // do actual redraws
}
This code is just an example, you'll have to fix it according to the MSDN documentation.
What you should NOT do is just hook into the OnPaint() event, because then you'll still have the actual Paint() method called.
My app is supposed to take a photo, then stop there showing the photo, save it in a file and move on to another view.
I use the Microsoft.Devices.PhotoCamera and wait after a photo is taken for CaptureCompleted to take the image and for CaptureImageAvailable to store it and then move on.
Whan happens is: After a picture is taken, I get the camera preview again, which might be confusing for users (though a progress-bar is shown to tell them "wait pls". still not nice) and after 1-2 seconds it shows the taken picture again and moves on to the next view.
Is there a nice way to disable this preview-thing?
If I dispose the PhotoCamera after the first event, it hangs. If I dispose it after the 2nd event, it doesn't seem to make much difference..
Edit:
private void Cam_CaptureImageAvailable(object sender, Microsoft.Devices.ContentReadyEventArgs e)
{
this.imageStream = e.ImageStream;
Deployment.Current.Dispatcher.BeginInvoke(delegate
{
this.image = new BitmapImage();
this.image.SetSource(this.imageStream);
this.PreviewImage.Source = this.image;
// Part where I store the image into a file.
// Part where I navigate to the next view.
});
}
I need that Dispatcher-Thing because setting images and navigating needs the UI-thread but the method runs in a different one.
Also a strange point: it doesn't matter that the PreviewImage lies OVER the canvas with the camera-preview. It is still overwritten as soon as the picture is taken and being processed... ôo
The problem might be that I do many things and the UI is never refreshed. I could add a small sleep (1-5ms) but that would be async again. Also I can't do this completely serparated because the storing part makes a "screenshot" which should contain the final image instead of the camera-preview-thing.
Edit2:
I see the main problem in the capture itself. It takes a picture, shows it for a while, then switches to the camera preview and then back to my shown picture.
I am currently making a game with GDI+, I know it is not the optimal solution for developing a game, but since it is a school project I have no choice.
About every tenth time I run my game, the graphics gets rendered outside the form in the top left corner of the screen.
I'm using double buffering if that helps to narrow the problem down.
The rendering code looks like this:
while (true)
{
// Create buffer if it don't exist already
if (context == null)
{
context = BufferedGraphicsManager.Current;
this.buffer = context.Allocate(CreateGraphics(), this.DisplayRectangle);
}
// Clear the screen with the forms back color
this.buffer.Graphics.Clear(this.BackColor);
// Stuff is written to the buffer here, example of drawing a game object:
this.buffer.Graphics.DrawImage(
image: SpriteSheet,
destRect: new Rectangle(
this.Position.X
this.Position.Y
this.SpriteSheetSource.Width,
this.SpriteSheetSource.Height),
srcX: this.SpriteSheetSource.X,
srcY: this.SpriteSheetSource.Y,
srcWidth: this.SpriteSheetSource.Width,
srcHeight: this.SpriteSheetSource.Height,
srcUnit: GraphicsUnit.Pixel);
// Transfer buffer to display - aka back/front buffer swapping
this.buffer.Render();
}
It's easier to explain with a screenshot:
It was a bit of a design mistake in Winforms to make the BufferedGraphicsXxx classes public. They are an implementation detail of double-buffering support in Winforms and they are not terribly resilient to using them wrong.
You are definitely using the BufferedGraphics you get back from Allocate() wrong. You create buffers at a high rate, inside the game loop. But you forget to dispose the buffer you used at the end of the loop. This will consume device contexts (HDC's) at a high rate. That doesn't go on forever, if your program doesn't otherwise get the garbage collector running then Windows pulls the plug and will not let you create a new device context. The internal CreateCompatibleDC() call will fail and returns NULL. The BufferedGraphicsContext class otherwise misses the code to check for this error and plows on with the NULL handle. And starts painting to the desktop window instead of the form.
A fix will be to move the Allocate() call outside of the loop so you do it just once. But now you'll have a new problem when the user changes the window size, the buffer is no longer the correct size.
The better mousetrap is to just not use the BufferedGraphics class but leave it up to Winforms to get it right. There are several ways to get a gameloop in Winforms, but the simplest one is to just use the OnPaint() method to render the scene and immediately ask for another paint so it keeps getting called over and over again. Similar to this:
public partial class Form1 : Form {
public Form1() {
InitializeComponent();
this.DoubleBuffered = true;
this.ResizeRedraw = true;
}
protected override void OnPaint(PaintEventArgs e) {
RenderScene(e.Graphics);
this.Invalidate();
}
}
Where RenderScene() should draw the game objects, using the passed Graphics instance. Note that you no longer need to use Clear(), that was already done.
About every tenth time I run my game, the graphics gets rendered
outside the form in the top left corner of the screen.
From the screen shot and your description, you are occasionally drawing to the Window's Desktop device context (DC); Which is the effect of using a window handle of zero (IntPtr.Zero) when getting the DC.
This lead me to believe you could be starting the game loop before the form window has been created resulting in the graphics context to point to a zero'd window handle.
As confirmed in the commentary you are using a separate thread for your game loop resulting in the random behavior of this happening. Once dealing with threads, you don't always get the same result twice when it comes to timing of start up and completion of threads (especially when threads can run parallel, via a multi-core/cpu computer). Each time the game application is ran, there is a chance the game loop thread can start-up and execute before the form window on the UI thread can be created and shown.
I need to load 101 bitmaps from the filesystem (they can't be built into the app as a resource as they will be changed for each run of the program) into a windows form application picturebox sequentially based on short duration timer events (500ms give or take) .
Essentially it should work like a slow animation but it is critical that every image be shown.
I've roughed out a simple application in C# to do this however it seems that the image loading and displaying is taking longer than the 500ms so some images are never displayed.
Is there some way I can avoid this? Preloading or creating 101 pictureboxes and showing and hiding?
Anybody have any suggestions?
What's taking the longest? Image loading or displaying? Are you loading each image when it needs displaying? A look at your code would be really valuable.
If you are certain that it will always be exactly 101 images, load them all into an array of System.Drawing.Bitmap first then have an iterator variable that gets incremented on each call of the Tick event of a Timer. Have this Tick event load the image from the array into the PictureBox using PictureBox.Image = myBitmapArray[iterator] If you increment the iterator using ++i%=101; you won't get an OutOfBounds error and the animation will loop.
Populate an array of Bitmap objects before starting the animation.
A couple of options... pick one or combine them:
(1) Use a lock and a counter to guarantee that when the event fires it's loading the next image in line.
(2) Disable the timer in the Tick event, then re-enable it after you've loaded the image. The result is that the images arrive 500ms after the last one is drawn, so if a picture takes one second to load, the images are drawn at t=1000ms, t=1500ms, t=3000ms, etc. All the images are drawn, and 500ms is guaranteed to pass between pictures, but the animation might appear slow.
(3) Do the above, but track the time the event starts, and after the image is drawn, set the next timer tick to be 500 - (Now - eventStart)... so that if the image takes 250ms to draw, the next timer tick will fire in 250ms. If Now - eventStart < 0, the next timer tick should fire immediately. The animation will take the minimum amount of time possible, but images could potentially flash by, appearing only for a few milliseconds.
(4) Use PictureBox.LoadAsync() to give you some multi-threading... the next event can be loading the image while the previous event is drawing. But you'll need a Mutex that you release in the LoadCompleted event that you wait on before calling LoadAsync(), if you need to guarantee the images are all drawn.
(5) I'm not sure if the drawing of the picture falls under the Layout category, but you can try calling SuspendLayout() and ResumeLayout() before/after loading the image
(6) Use an array of images, lock a counter, and use the .Image property of the PictureBox to let you pre-load the images. I believe this was suggested in another answer, also
HTH,
James
I don't have a concrete answer for you, but I would approach this by first determining if it's image loading, image display, or both actions that are taking too much time.
I would imagine that image loading is going to be relatively quick, even with pretty large image sizes, provided that the images are local and you have relatively decent hardware. My first attempt would involve loading all images from a thread sequentially so that you application doesn't have to wait for image display to complete before the next image loads.
If image display is taking a long time (and even on my really powerful workstation it's not all that fast for large images), then is it possible for you to scale the images before the application displays everything? Does your application need to deal with full resolution images? With the megapixel cameras these days, I can't image that you'd want to have all of the data present in the image files anyway, since the sizes can easily exceed monitor resolutions by a factor of 6.
Another thing that may be a concern is the size of the images. If the are all 18 Megapixel Images, I can see it taking either a lot of time or a lot of room.
You may want to resize them to the size of your display area when you initially load them up so that you aren't using 2 gigs of ram or waiting to read off the disk depending on how you implement it.
I have a C# .NET application with which I've created a custom image display control. Each image display represents its own display context and draws the image using glDrawPixels (Yes I know it would be better to use textures, I plan to in the futures but this app is already too far along and my time is limited).
I am now trying to have both images pan simultaneously. That is, when one image is moved down ten pixels, the second image moves down ten pixels. Like so:
imageOne.YPan -= 10;
imageTwo.YPan -= 10;
imageOne.Invalidate(); //This forces a redraw.
imageTwo.Invalidate(); //This forces a redraw.
Alright so here is the problem I am having. Only one of the images displays is redrawing. If I place a pause in between the two Invalidate calls and make the pause duration at least 110 milliseconds both will redraw, but not simultaneously. So it looks as if the second image is always trying to catch up to the first. Plus, a 110 millisecond pause slows down the motion too much.
I have tried placing the updating and invalidating of each image in its own thread but this did not help.
At the beginning of drawing I make the appropriate context is current, and at the end I am calling swapbuffers(). I tried adding a glFinish to the end of the draw function, but there was no change.
Could it be that its the graphics card that is the problem? I am stuck using an integrated gpu that only has openGL 1.4.
Hopefully, I have provided enough detail that the answer to my problem can be found.
Its difficult telling what's wrong with what you do since you give so little detail. Here are some pointers which may help.
- before doing something in a context, make sure you make it the current one. If you want to pan two contexts, make the first one current, pan it and then make the second one current and pan it. These is no real reason why this should not work.
- If it looks like there is a timing problem, adding glFinish() at strategic places may help weed the problem out
- As should always be done, on occasions call glError() and see that everything went well.
- I'm not sure how this is done in the framework you're talking about but you should make sure that both contexts get a swapBuffers() call for every frame.
Invalidate doesn't force an immediate redraw. It marks the window invalid, and when the message queue runs out of other messages, a paint message will be created and processed. But that won't happen until you finish processing the current message and return to the main message loop, and it may be delayed even more than that.
Generally OpenGL animation is an exception to the rule of doing all drawing inside Control.OnPaint (or in a handler for the Control.Paint event).