We Keep Coding

C# Fastest image resize routine for streaming video?

I am trying to write a simple C# Windows App to display streaming video from a local USB webcam. I am using AForge.NET, FilterInfoCollection, VideoCaptureDevice etc to put video stream into a standard PictureBox. As is, it works fine. However, I need to downscale the videostream. When I try that, FPS drops significantly and becomes even lower as the app continues to run. My rescale routine is as follows (it aims to maintain aspect ratio and zoom in by preserving height):
public Image ResizeImage(Image img, int target_width, int height)
{
int width = (int)(height * img.Width / (float)img.Height);
Bitmap b = new Bitmap(width, height);
using (Graphics g = Graphics.FromImage(b))
{
g.DrawImage(img, (target_width - width)/2, 0, width, height);
}
return b;
}
And it is called by:
private void video_NewFrame(object sender, NewFrameEventArgs eventArgs)
{
Invoke(new Action(() =>
{
picVideo.Image = ResizeImage((Bitmap)eventArgs.Frame.Clone(), picVideo.Width, picVideo.Height); // rescaled stream
//picVideo.Image = (Bitmap)eventArgs.Frame.Clone(); // unscaled stream
}));
}
Setup code is this:
filterInfoCollection = new FilterInfoCollection(FilterCategory.VideoInputDevice);
videoCaptureDevice = new VideoCaptureDevice(filterInfoCollection[frmMain.Camera].MonikerString);
videoCaptureDevice.NewFrame += video_NewFrame;
videoCaptureDevice.Start();
However, I have this old 3rd party EXE that runs fine and is able to rescale the same webcam videostream as I resize its window. So it looks like this rescaling routine is too slow.
So, is there a faster rescaling method? I basically need to achieve downscaling from FullHD to 1024*768. I don't really need this to be high-quality.

Problem completely solved by adding
g.InterpolationMode = InterpolationMode.NearestNeighbor;
before g.DrawImage()

Convert Graphics object to Bitmap

I do have following issue with my graphics object.
EDIT:
I do have a picturebox_image (imageRxTx) which is a live stream from a camera. What I do in the paint event is to draw some lines on top of the image imageRxTx (not shown in the code below). This works so far without problem.
Now I need to check for circles in imageRxTx and therefore I have to use the method ProcessImage() which needs a Bitmap as parameter. Unfortunately I do not have the Bitmap image but rather the handle (hDC) to my imageRxTx.
Question: How can I get the imageRxTx from my graphics-object and "convert" it to a bitmap-image which I need to use in the method ProcessImage(Bitmap bitmap)? This method needs to be called continously in the paint-event in order to check the live-stream of my camera (imageRxTx).
Here my code:
private void imageRxTx_paint(object sender, PaintEventArgs e)
{
var settings = new Settings();
// Create a local version of the graphics object for the PictureBox.
Graphics Draw = e.Graphics;
IntPtr hDC = Draw.GetHdc(); // Get a handle to image_RxTx.
Draw.ReleaseHdc(hDC); // Release image_RxTx handle.
//Here I need to send the picturebox_image 'image_RxTx' to ProcessImage as Bitmap
AForge.Point center = ProcessImage( ?....? );
}
// Check for circles in the bitmap-image
private AForge.Point ProcessImage(Bitmap bitmap)
{
//at this point I should read the picturebox_image 'image_RxTx'
...
The video image is updated here:
private void timer1_Elapsed(object sender, EventArgs e)
{
// If Live and captured image has changed then update the window
if (PIXCI_LIVE && LastCapturedField != pxd_capturedFieldCount(1))
{
LastCapturedField = pxd_capturedFieldCount(1);
image_RxTx.Invalidate();
}
}

As the title suggests, your main problem is a (common) misconception about what a Graphics object is.
So far I can draw to my graphics object without problem
No! A 'Graphics' object does not contain any graphics. It is only the tool used to draw graphics onto a related Bitmap. So you don't draw onto the Graphics object at all; you use it to draw onto imageRxTx, whatever that is, probably the surface of some Control or Form..
This line is using an often confusing rather useless format of the Bitmap constructor:
Bitmap bmp = new Bitmap(image_RxTx.Width, image_RxTx.Height, Draw);
The last parameter is doing next to nothing; its only function is to copy the Dpi setting. In particular it does not clone or copy any content from 'Draw', which, as you know now, a Graphics object doesn't have anyway, nor any of its other settings. So yes, the bmp Bitmap is still empty after that.
If you want to draw into bmp you need to use a Graphics object that is actually bound to it:
using (Graphics G = Graphics.FromImage(bmp)
{
// draw now..
// to draw an Image img onto the Bitmap use
G.DrawImage(img, ...);
// with the right params for source and destination!
}
None of this should probably happen in the Paint event! But all the preparation code is unclear as to what you really want to do. You should explain just what is the source of the drawing and what is the target!
If instead you want to get the stuff you draw onto image_RxTx into a Bitmap you can use this method somwhere outside (!) the Paint event:
Bitmap bmp = new Bitmap(image_RxTx.Width, image_RxTx.Height);
image_RxTx.DrawToBitmap(bmp, image_RxTx.ClientRectangle);
This will use the Paint event to draw the control into a Bitmap. Not that the result includes the whole PictureBox: The BackgroundImage, the Image and the surface drawing!
Update: To get the combined content of the PictureBox, that is both its Image and what you have drawn onto the surface, you should use the code above (the last 2 lines) in the Tick event of a Timer or right after the line that triggers the Paint event. (You didn't show us how that happens.) You can't acutally put it in the Paint event itself, as it will use the Paint event and therefore would cause an infinite loop!

The method Graphics.CopyFromScreen is probably what you're looking for.
var rect = myControl.DisplayRectangle;
var destBitmap = new Bitmap(rect.Width, rect.Height, PixelFormat.Format24bppRgb);
using (var gr = Graphics.FromImage(destBitmap))
{
gr.CopyFromScreen(myControl.PointToScreen(new Point(0, 0)), new Point(0, 0), rect.Size);
}

How to know what is the length of an edge in a resized image?

Lets say I want to click on an image and enlarge the part above where I clicked. I do that with the simplest code, but at the bottom of the image there is an annoying edge because of the enlargement.
There is no Graphics involved, so I can't use SetWrapMode(WrapMode.TileFlipXY); or graphic.CompositingMode = CompositingMode.SourceCopy;
I can crop the annoying edge part (its not important) but I don't know what is its height.
How can I know what is its height so I can crop it? Better yet - is there a better method to enlarge the image (I need a method that returns a bitmap so I can save it later)?
The simple code:
private void button1_Click(object sender, EventArgs e)
{
try
{
Bitmap img = new Bitmap(imageLoc);
Rectangle cropArea = new Rectangle(0, 0, img.Width, yLoc);
Bitmap bmpImage = new Bitmap(img);
Bitmap bmpCrop = bmpImage.Clone(cropArea, bmpImage.PixelFormat);
pictureBox1.Image = resizeImage(bmpCrop, new Size(bmpCrop.Width, newHeight));
}
catch
{
}
}
public static Bitmap resizeImage(Image imgToResize, Size size)
{
return new Bitmap(imgToResize, size);
}
private void pictureBox1_MouseClick(object sender, MouseEventArgs e)
{
xLoc = e.X;
yLoc = e.Y;
}
The annoying edge (look at the bottom):

I think this is casued by the resizing algorithm using a bicubic or linear interpolation. These alhgorithms calculate pixels value after resizing by sampling the values of the surrounding pixels and interpolating them. In the case of the original edge pixels, they dont have a value for the pixel below them, so tend towards transparency (i.e. get lighter) since the average of the surrounding pixels takes into account that their next pixel down is missing.
To eliminate sizing artefacts like this you probably need to investigate resizing using the Graphics object and the various quality and algorithm settings available through that.

Get/set pixel from Graphics object GDI+

I'm trying to find an alternative solution to getting/setting pixels at certain position in a Graphics object.
Right now I'm using GDI functions:
[DllImport("gdi32.dll")]
public static extern int GetPixel(System.IntPtr hdc, int nXPos, int nYPos);
[DllImport("gdi32.dll")]
public static extern uint SetPixel(IntPtr hdc, int X, int Y, int crColor);
I couldn't find any for GDI+. GetPixel/SetPixel seems to be on Bitmap object only.
The alternatives from gdi32.dll work well when the Graphics is backed by the screen but when using Graphics with a bitmap it doesn't work anymore (GetPixel returns black, since this works on another bitmap not the actual one): http://support.microsoft.com/kb/311221
Some sample code:
private void ChangeImage(Graphics g)
{
IntPtr gDC = IntPtr.Zero;
try
{
gDC = g.GetHdc();
// get the pixel color
Color pixel = ColorTranslator.FromWin32(GetPixel(gDC, x, y));
//change pixel object and persist
SetPixel(gDC, x, y, ColorTranslator.ToWin32(pixel));
}
catch (Exception ex)
{
Trace.WriteLine(ex.Message);
}
finally
{
if (gDC != IntPtr.Zero)
g.ReleaseHdc(gDC);
}
}
Is there any way to parse the the Graphics object per pixel basis?
The Graphics object represents a surface where the user is free to add any objects including hand drawing. On top of this I need to apply filters (like blur or pixelation) on some screen parts so I need to access what has been painted already in the graphics.
I've also tried to find a way to persist the current graphics to a Bitmap and use GetPixel from the bitmap object but I've also couldn't find a way to save Graphics content.
Thx

No, it's not possible to read pixels from a Graphics object directly. You would need access to the underlying HDC or Bitmap object to do this.

Dim bmap As New Bitmap(550, 100)
Dim g As Graphics = Graphics.FromImage(bmap)
' Dont use the Graphics interface, use the BitMap interface
DIM col as color = bmap.GetPixel(x,y)

Drawing in Winforms

I have written this code, however, it doesn't work. Not only will this not work, but none of the methods I have tried for drawing have worked. I've spent more than an hour or two trying to solve this, but to no success. Ive tried simple programs where all it does is display a small line, but it wont work no matter what i do :c
What am I doing wrong, or what could cause this?
private void pictureBox1_MouseDown(object sender,
MouseEventArgs m,
EventArgs e,
PaintEventArgs q)
{
if (m.Button == System.Windows.Forms.MouseButtons.Left)
{
Point currpoint = System.Windows.Forms.Cursor.Position;
Point origin = new Point(0, 0);
decimal sizee = nud.Value;
int size = Convert.ToInt32(sizee);
Random randonGen = new Random();
Color randomColor = Color.FromArgb(randonGen.Next(255),
randonGen.Next(255),
randonGen.Next(255));
Pen selPen = new Pen(randomColor, size);
Graphics g = Graphics.FromImage(pictureBox1.Image);
g.DrawLine(selPen, 3, 3, 133, 133);
}
}

Try adding a
pictureBox1.Invalidate();
call.

This is not the right way to draw to a picture box:
private void pictureBox1_MouseDown(object sender,
MouseEventArgs m,
EventArgs e,
PaintEventArgs q)
{
if (m.Button == System.Windows.Forms.MouseButtons.Left)
{
Point currpoint = System.Windows.Forms.Cursor.Position;
Point origin = new Point(0, 0);
decimal sizee = nud.Value;
int size = Convert.ToInt32(sizee);
Random randonGen = new Random();
Color randomColor = Color.FromArgb(randonGen.Next(255),
randonGen.Next(255),
randonGen.Next(255));
Pen selPen = new Pen(randomColor, size);
using(Graphics g = pictureBox1.CreateGraphics()) // Use the CreateGraphics method to create a graphic and draw on the picture box. Use using in order to free the graphics resources.
{
g.DrawLine(selPen, 3, 3, 133, 133);
}
}
}
Btw, this method will create a temporary image which is reseted when the control is invalidated. For a more persistent drawing, you need to listen to the Paint event of the picture box and draw your graphics there.

You must draw it from image first. then attach it to pictureBox1
Bitmap canvas = new Bitmap(pictureBox1.Width, pictureBox1.Height);
Graphics g = Graphics.FromImage(canvas);
Point currpoint = System.Windows.Forms.Cursor.Position;
Point origin = new Point(0, 0);
decimal sizee = nud.Value;
int size = Convert.ToInt32(sizee);
Random randonGen = new Random();
Color randomColor = Color.FromArgb(randonGen.Next(255),
randonGen.Next(255),
randonGen.Next(255));
Pen selPen = new Pen(randomColor, size);
g.DrawLine(selPen, 3, 3, 133, 133);
pictureBox1.image = canvas;

This is an old question and if anyone else has a similar problem. See below. First let's examine the Ops code.
(1) See code: The first recommended change is to keep the Pen's format simple until we have a better understanding about how the Pen actually works when drawing to graphics. Look at the Op's line where we create graphics from image which is a perfectly good example of how to directly draw ("which means to write") to the supplied bitmap by use of the bitmap's graphics context. Next, the Op provides an excellent example of the Graphics DrawLine method which can draw the defined line to the supplied bitmap.
(2) Due to missing details we have to make the following assumptions about the Op's supplied bitmap and about their method for drawing a line to the bitmap. Assuming there already exists an image inside this pictureBox1; if an image is not set the graphics we get from image will be from a null image or that each pixel will be black just as a footnote:
(a) Is the Pen's color unique to the existing bitmap and is the alpha component of the color high enough to actually see the resultant color when it's drawn (when in doubt use a unique solid color or at least set the alpha channel to 255)?
(b) This line the Op wants to draw is starting Left 3, Top 3 to Left 133 and that is 3-pixels to the right of bitmap's left side where this line has a height of 133 and as such the Pen's line size was changed to a width = 3 for demonstration purposes.
(c) The final consideration, is the pictureBox1.Size sufficient for us to see this drawn line? The line's geometry forms a rectangle similar to this RectangleF(3, 3, 3, 133) structure, so if the pictureBox1 Bounds rectangle intersects with the derived line's rectangle then the area of that intersection is where the line could be drawn and considered visible.
Before we can draw to the pictureBox1 image from graphics we must first convert the pictureBox1 image data back to a usable image type like a bitmap for example. The reason is the picture box stores only pixel data in array format and is not directly usable by GDI/GDI+ without conversion to an image type ie. bitamp, jpeg, png etc..
One can avoid this messy conversion if you handle you own painting by the way of a custom user control and by properly handling the PaintEventArgs OnPaint implementation and/or by using related graphics screen buffer context scenarios.
For those who just want the answer about what's missing:
private void button1_Click(Object sender, EventArgs e)
{
Pen selPen = new Pen(Color.Red, 2); // The Op uses random color which is not good idea for testing so we'll choose a solid color not on the existing bitmap and we'll confine our Pen's line size to 2 until we know what we're doing.
// Unfortionately the picture box "image" once loaded is not directly usable afterwords.
// We need tp recreate the pictureBox1 image to a usable form, being the "newBmp", and for efficiency the bitmap type is chosen
Bitmap newBmp = new Bitmap(pictureBox1.Width, pictureBox1.Height, PixelFormat.Format32bppArgb); // Tip: Using System.Drawing.Imaging for pixel format which uses same pixel format as screen for speed
// We create the graphics from our new and empty bitmap container
Graphics g = Graphics.FromImage(newBmp);
// Next we draw the pictureBox1 data array to our equivelent sized bitmap container
g.DrawImage(pictureBox1.Image, 0, 0);
g.DrawLine(selPen, 3, 3, 3, 133); // Format: (pen, x1, y1, x2, y2)
pictureBox1.Image = newBmp;
// Don't forget to dispose of no longer needed resources
g.Dispose();
selPen.Dispose();
newBmp.Dispose(); // or save newBmp to file before dispose ie. newBmp.Save("yourfilepath", ImageFormat.Jpeg) or in whatever image type you disire;
}
The Op's code so far only draws a line to the bitmap's surface next if we are to "see" this change we must either save bitmap to file to be viewed later in an image viewer or we must draw the updated bitmap to our display monitor, the screen.
There are several methods with which to draw to your monitor's screen. The most common graphics contexts one could use are Control.CreateGraghics, graphics to screen method from (PaintEventArgs) and/or by using a graphics screen buffer sometimes called and used as a manual double buffered graphics context in which all is implemented by the way of DrawImage method from graphics.
The simplest solution, in this case based upon the Op's own code, is to display this newly updated bitmap using the pictureBox1 control. We'll simply update the control's image with the newly updated bitmap of course once first converted to a usage graphics image as seen in the above code descriptions.
Happy coding!