I am writing a program similar to TeamViewer. But I have a problem that the screen resolution is too big. Below is a how I am generating the image from the screen.
byte[] ScreenShut()
{
Bitmap bmp = new Bitmap(Screen.PrimaryScreen.Bounds.Width,Screen.PrimaryScreen.Bounds.Height);
Graphics gr = Graphics.FromImage(bmp);
bmp.SetResolution(96.0F,96.0F);
gr.CopyFromScreen(0, 0, 0, 0, new Size(bmp.Width, bmp.Height));
MemoryStream ms = new MemoryStream();
bmp.Save(ms, ImageFormat.Png);
return ms.GetBuffer();
}
How can I reduce the quality of the incoming picture?
Save it as jpg 8 bit using this
public Bitmap(
int width,
int height,
PixelFormat format
)
I am writing a program similar to TeamViewer
I'll assume you are referring to the RDP/desktop sharing aspects.
Your problem is that you are not taking into account any prior frames so there is much erroneous data being transmitted. Generally, not a great deal of the screen changes from moment to moment. You need to compare prior frames to the current frame to determine what has changed and only send the deltas. Therefore your problem is essentially that of how to stream moving images or consecutive frames in a reasonably fast fashion.
The problem can be solved with any streaming video solution. Perhaps H.264?
You will find that video codecs don't just work on the current frame but also prior frames. Thus you can think of the screen being a slice moving through time of a much larger rectangular prism. So simply to solve it in a 2D fashion like trying to reduce the bit-depth; spacial resolution won't be sufficient.
Related
I have a Windows Application project that deals with Image editing (Cropping & Resizing). Unfortunately these image processings consume a lot of Memory and CPU resources (easily reaches 600MB or 50% cpu) and it is all about cropping and resizing just one gif image that weighs 2.5MB (2300*5400px). More than that, due to large resource consumption, the program gets stuck while resizing...
public static Image Resize(Image imgToResize, Size size)
{
Bitmap b = new Bitmap(size.Width, size.Height);
Graphics g = Graphics.FromImage((Image)b);
g.InterpolationMode = InterpolationMode.Default;
g.SmoothingMode = SmoothingMode.HighSpeed;
g.PixelOffsetMode = PixelOffsetMode.Default;
g.DrawImage(imgToResize, 0, 0, size.Width, size.Height);
g.Dispose();
return (Image)b;
}
public static Image Crop(Image img, Point p1, Point p2)
{
Rectangle cropArea = new Rectangle(p1.X, p1.Y, p2.X - p1.X, p2.Y - p1.Y);
return (img as Bitmap).Clone(cropArea, img.PixelFormat);
}
What methods should I use to avoid this?
I've already tried compressing it to memory stream in several formats but it didn't help (even made it worse)
NOTE: I use the standard .NET Drawing libraries: System.Drawing, System.Drawing.Imaging
Your code is creating copies of the image so you should expect unmanaged memory usage to rise when you call these methods. What matters a great deal is what you do with the original. You would be wise to get rid of it so it no longer takes up memory. You have to call its Dispose() method to do so. Waiting for the garbage collector to do it takes too long. The Bitmap class takes very little managed memory but oodles of unmanaged memory.
From an earlier version of this question: http://snippets.dzone.com/posts/show/4336
Also, AForge.net has several resize functions
This is a tricky one, and one I've run into before. You could split the image into x pieces, depending on the size of the file, then save each one to disk to ensure that the memory is clean.
Next, you resize the component images one at a time, each time, ensuring that the component is disposed before proceeding to the next one. Once you're all done, you stitch them back together, and then crop.
A MAJOR PROBLEM with this approach - if you're resizing upward, this approach will put seams in your image, as the interpolation won't have the surrounding pixels to guess at. But I would think that this approach would work well with resizing downward.
HTH.
Another hint again:
For example in resize, like #Hans pointed out, you create a new Bitmap, which one is your bottlenecks.
But, what if you just draw image resized, the same image which was loaded originally (obviously you made a backup file of original image before on the disk).
After crop what if you just draw a portion of a bitmap that user cropped, so user will see only that rectangle. ?
I mean in general, operate over image you already have, and try (as much as it possible) do not initialize new object for it.
Regards.
write Application.DoEvents(); in your functions, at least it wont get stuck
I'm taking a screenshot of the screen, serializing the bitmap and sending it over the network. Overall this ends up being ~26KB of data transferred.
I'm trying to make this smaller. One thing I'm trying to do is converting the bitmap to greyscale. This is the function I'm using.
Public Function ConvertGreyscale(original As Bitmap) As Bitmap
Dim NewBitmap As New Bitmap(original.Width, original.Height)
Dim g As Graphics = Graphics.FromImage(NewBitmap)
Dim attributes As New ImageAttributes
attributes.SetColorMatrix(New ColorMatrix(New Single()() {New Single() {0.3F, 0.3F, 0.3F, 0, 0}, New Single() {0.59F, 0.59F, 0.59F, 0, 0}, New Single() {0.11F, 0.11F, 0.11F, 0, 0}, New Single() {0, 0, 0, 1, 0}, New Single() {0, 0, 0, 0, 1}}))
g.DrawImage(original, New Rectangle(0, 0, original.Width, original.Height), 0, 0, original.Width, original.Height, GraphicsUnit.Pixel, attributes)
g.Dispose()
Return NewBitmap
End Function
This works fine, and i end up getting a greyscale image. Problem is, that the size of the bitmap doesn't change. It's still 26KB, even though it's greyscale. I'm thinking that the new bitmap that's being created is just a regular 32bppargb bitmap with a greyscale image stuck into it.
I tried doing:
Dim NewBitmap As New Bitmap(original.Width, original.Height, PixelFormat.Format16bppgreyscale)
but i end up getting an "out of memory error".
What am i doing wrong? Also, are there any other ways to minimize the size of my bitmap?
EDIT:
So in an effort to take baby steps to tackle this problem, I'm using this code to convert the 32bpp bitmap to a 16bpp bitmap
Dim clone = New Bitmap(tmpImg.Width, tmpImg.Height, Imaging.PixelFormat.Format16bppRgb565)
Using gr = Graphics.FromImage(clone)
gr.DrawImage(tmpImg, New Rectangle(0, 0, clone.Width, clone.Height))
End Using
I tried doing Format16bbpGreyscale or Format16bppRgb555, but both of those cause "Our of memory errors". The only one that seems to work is the Format16bppRgb256
Regardless, I'm doing my packet sniffing again, and changing the format to 16bppRgb265 INCREASES the size of the image packet from ~26KB to 29KB. So changing to this format seems to increase size. I don't understand ;_;
EDIT2:
I've found multiple ways to convert the image to greyscale now and/or changing the pixelformat of the bitmap to something smaller than 32bpp. Unfortunately none of this seems to decrease the size of the serialized bitmap when it's being sent over the network. Some things seem to even increase the size. Not sure what i can do.
I recommend checking out Aforge's AForge.Imaging.ColorReduction.ColorImageQuantizer .
It reduced a screenshot of a SO homepage from 96kB to 33kB (going to 16 colors) while maintaining readabilty much better that an equally reduced jpg. Reducing to 32 or 64 colors left almost no artifacts, other than color changes while still staying at 48kB.
It does take a few seconds for processing, though..
Here is a piece of code that uses the Aforge libraries.
using AForge.Imaging.ColorReduction;
void reduceColors(string inFile, string outFile, int numColors)
{
using (Bitmap image = new Bitmap(inFile) )
{
ColorImageQuantizer ciq = new ColorImageQuantizer(new MedianCutQuantizer());
Color[] colorTable = ciq.CalculatePalette(image, numColors);
using (Bitmap newImage = ciq.ReduceColors(image, numColors))
newImage.Save(outFile);
}
}
If you're interested I also have a home-grown piece of code, that results in 40% of the original size with perfext text, albeit a little color shift; it is very fast.
Converting to greyscale doesnt do much by itself because all you are doing is changing the RGB values of the pixels. Unfortunately many of the greyscale formats are not fully supported, though there are some opensource image libraries which will do this.
Significant reduction can be gotten using JPG and some quality reduction. 26kb for a full size (?) screenshot doesn't sound all that large (or it is only part of a screen?), and we dont know what your desired target size is. Here is how to reduce quality via JPG.
Dim jpgEncoder As ImageCodecInfo = GetJPGEncoder()
Dim myEncoder As System.Drawing.Imaging.Encoder =
System.Drawing.Imaging.Encoder.Quality
Dim jEncoderParams As New EncoderParameters(1)
' set the quality (100& here)
jEncoderParams.Param(0) = New EncoderParameter(myEncoder, 100&)
' dont do this...creates a false baseline for size tests
'Dim bmp As Bitmap = My.Resources.testimage
Using ms As New System.IO.MemoryStream(),
fs As New FileStream("C:\Temp\zser.bin", FileMode.Create),
bmp As New Bitmap(My.Computer.Screen.WorkingArea.Width,
My.Computer.Screen.WorkingArea.Height),
g As Graphics = Graphics.FromImage(bmp)
' get screen in (BMP format)
g.CopyFromScreen(0, 0, 0, 0, My.Computer.Screen.WorkingArea.Size)
' save image to memstream in desired format
bmp.Save(ms, Imaging.ImageFormat.Png)
' use jpgEncoder to control JPG quality/compression
'bmp.Save(ms, jpgEncoder , jEncoderParams)
ms.Position = 0
Dim bf As New BinaryFormatter
bf.Serialize(fs, ms) ' serialize memstr to file str
jEncoderParams.Dispose()
End Using
Metrics from a screen capture (ACTUAL size depends on screen size and what is on it; the size differences are what is important):
Method memstr size file size after BF
BMP 5,568,054 5438 (same)
PNG 266,624 261k
JPG 100 634,861 1025
JPG 90 277,575 513
The content of the image plays a role in determining the sizes etc. In this case, PNG seems best size/quality balance; you'd have to compress JPG quite a bit to get the same size but with much less quality.
An actual photo type image will result in much larger sizes: 19MB for a 2500x1900 image and almost 13MB for a PNG, so test using actual images.
So eventually I've figured out the problem.
Essentially i had to binary serialize a bitmap and transmit it over a network stream. And i was trying to decrease the size of the bitmap to make transfer faster.
.NET's "image" class seems to only support bitmaps in certain pixelformats. So no matter what i did to the image (greyscaled, lossy compression, whatever) the size would be the same because i wasn't change the pixelformat of the image, s i was just moving pixels around and changing their colors.
From what i know, there is no native class for JPGs or PNGs that i could serialize, so i was forced to use the image class with it's pixel formats.
One thing i tried was to convert the compressed, greyscaled image into a jpeg, and then convert that into a byte(), and then gzip and serialize that byte(). Problem is that resulted in a 2x increase in network data being transmitted for some reason.
An interesting quick note, is that when you serialize an image object, it is converted to PNG format (compressed), according to the network traffic i sniffed anyway. So there is some optimization that has been done by microsoft to make image serialization efficient.
So i was pretty much forced to use the image class, and just somehow figure out how to convert my image into the 1bpp or 8bpp pixelformats. The 16bpp formats (greyscale for instance) are randomly "unsupported" by microsoft, and just "don't work", and apparently never will. Of course MSDN doesn't mention any of this.
Converting my image to 8bpp or lower was impossible, because i would get "out of memory" errors for unknown reasons, or something about not being allowed to draw on indexed images.
The solution i finally found was the CopyToBpp() function from here:
http://www.wischik.com/lu/programmer/1bpp.html
That function, along with the many API's in it, allowed me to quickly convert my 32bpp Image into an 8bpp or even 1bbp image. Which could then be easily and efficiently serialized a binary formatter and sent over my network stream.
So now it works.
I have a Windows Application project that deals with Image editing (Cropping & Resizing). Unfortunately these image processings consume a lot of Memory and CPU resources (easily reaches 600MB or 50% cpu) and it is all about cropping and resizing just one gif image that weighs 2.5MB (2300*5400px). More than that, due to large resource consumption, the program gets stuck while resizing...
public static Image Resize(Image imgToResize, Size size)
{
Bitmap b = new Bitmap(size.Width, size.Height);
Graphics g = Graphics.FromImage((Image)b);
g.InterpolationMode = InterpolationMode.Default;
g.SmoothingMode = SmoothingMode.HighSpeed;
g.PixelOffsetMode = PixelOffsetMode.Default;
g.DrawImage(imgToResize, 0, 0, size.Width, size.Height);
g.Dispose();
return (Image)b;
}
public static Image Crop(Image img, Point p1, Point p2)
{
Rectangle cropArea = new Rectangle(p1.X, p1.Y, p2.X - p1.X, p2.Y - p1.Y);
return (img as Bitmap).Clone(cropArea, img.PixelFormat);
}
What methods should I use to avoid this?
I've already tried compressing it to memory stream in several formats but it didn't help (even made it worse)
NOTE: I use the standard .NET Drawing libraries: System.Drawing, System.Drawing.Imaging
Your code is creating copies of the image so you should expect unmanaged memory usage to rise when you call these methods. What matters a great deal is what you do with the original. You would be wise to get rid of it so it no longer takes up memory. You have to call its Dispose() method to do so. Waiting for the garbage collector to do it takes too long. The Bitmap class takes very little managed memory but oodles of unmanaged memory.
From an earlier version of this question: http://snippets.dzone.com/posts/show/4336
Also, AForge.net has several resize functions
This is a tricky one, and one I've run into before. You could split the image into x pieces, depending on the size of the file, then save each one to disk to ensure that the memory is clean.
Next, you resize the component images one at a time, each time, ensuring that the component is disposed before proceeding to the next one. Once you're all done, you stitch them back together, and then crop.
A MAJOR PROBLEM with this approach - if you're resizing upward, this approach will put seams in your image, as the interpolation won't have the surrounding pixels to guess at. But I would think that this approach would work well with resizing downward.
HTH.
Another hint again:
For example in resize, like #Hans pointed out, you create a new Bitmap, which one is your bottlenecks.
But, what if you just draw image resized, the same image which was loaded originally (obviously you made a backup file of original image before on the disk).
After crop what if you just draw a portion of a bitmap that user cropped, so user will see only that rectangle. ?
I mean in general, operate over image you already have, and try (as much as it possible) do not initialize new object for it.
Regards.
write Application.DoEvents(); in your functions, at least it wont get stuck
I am loading a JPG image from hard disk into a byte[]. Is there a way to resize the image (reduce resolution) without the need to put it in a Bitmap object?
thanks
There are always ways but whether they are better... a JPG is a compressed image format which means that to do any image manipulation on it you need something to interpret that data. The bimap object will do this for you but if you want to go another route you'll need to look into understanding the jpeg spec, creating some kind of parser, etc. It might be that there are shortcuts that can be used without needing to do full intepretation of the original jpg but I think it would be a bad idea.
Oh, and not to forget there are different file formats for JPG apparently (JFIF and EXIF) that you will ened to understand...
I'd think very hard before avoiding objects that are specifically designed for the sort of thing you are trying to do.
A .jpeg file is just a bag o' bytes without a JPEG decoder. There's one built into the Bitmap class, it does a fine job decoding .jpeg files. The result is a Bitmap object, you can't get around that.
And it supports resizing through the Graphics class as well as the Bitmap(Image, Size) constructor. But yes, making a .jpeg image smaller often produces a file that's larger. That's an unavoidable side-effect of Graphics.Interpolation mode. It tries to improve the appearance of the reduced image by running the pixels through a filter. The Bicubic filter does an excellent job of it.
Looks great to the human eye, doesn't look so great to the JPEG encoder. The filter produces interpolated pixel colors, designed to avoid making image details disappear completely when the size is reduced. These blended pixel values however make it harder on the encoder to compress the image, thus producing a larger file.
You can tinker with Graphics.InterpolationMode and select a lower quality filter. Produces a poorer image, but easier to compress. I doubt you'll appreciate the result though.
Here's what I'm doing.
And no, I don't think you can resize an image without first processing it in-memory (i.e. in a Bitmap of some kind).
Decent quality resizing involves using an interpolation/extrapolation algorithm; it can't just be "pick out every n pixels", unless you can settle with nearest neighbor.
Here's some explanation: http://www.cambridgeincolour.com/tutorials/image-interpolation.htm
protected virtual byte[] Resize(byte[] data, int width, int height) {
var inStream = new MemoryStream(data);
var outStream = new MemoryStream();
var bmp = System.Drawing.Bitmap.FromStream(inStream);
var th = bmp.GetThumbnailImage(width, height, null, IntPtr.Zero);
th.Save(outStream, System.Drawing.Imaging.ImageFormat.Jpeg);
return outStream.ToArray(); }
I am messing around with Conway's Game of Life - http://en.wikipedia.org/wiki/Conway's_Game_of_Life
I started out coding algorithmns for winforms and now want to port my work onto windows mobile 6.1 (compact framework). I came across an article by Jon Skeet where he compared several different algorithmns for calculating next generations in the game. He used an array of bytes to store a cells state (alive or dead) and then he would copy this array to an 8bpp bitmap. For each new generation, he works out the state of each byte, then copies the array to a bitmap, then draws that bitmap to a picturebox.
void CreateInitialImage()
{
bitmap = new Bitmap(Width, Height, PixelFormat.Format8bppIndexed);
ColorPalette palette = bitmap.Palette;
palette.Entries[0] = Color.Black;
palette.Entries[1] = Color.White;
bitmap.Palette = palette;
}
public Image Render()
{
Rectangle rect = new Rectangle(0, 0, Width, Height);
BitmapData bmpData = bitmap.LockBits(rect, ImageLockMode.ReadWrite, bitmap.PixelFormat);
Marshal.Copy(Data, 0, bmpData.Scan0, Data.Length);
bitmap.UnlockBits(bmpData);
return bitmap;
}
His code above is beautifully simple and very fast to render. Jon is using Windows Forms but now I want to port my own version of this onto Windows Mobile 6.1 (Compact Framework) but . . . .there is no way to format a bitmap to 8bpp in the cf.
Can anyone suggest a way of rendering an array of bytes to a drawable image in the CF. This array is created in code on the fly (it is NOT loaded from an image file on disk). I basically need to store an array of cells represented by bytes, they are either alive or dead and I then need to draw that array as an image. The game is particularly slow on the CF so I need to implement clever optimised algoritmns but also need to render as fast as possible and the above solution would be pretty dam perfect if only it was available on the compact framework.
Many thanks for any help
Any suggestions?
You could have a look at GDI+ for CF. It's basically a wrapper for most of the GDI implemented in WinCE. Here's a link to the source code and a writeup: http://community.opennetcf.com/articles/cf/archive/2007/10/31/using-gdi-on-windows-mobile.aspx
I think ImagingFactoryClass.CreateBitmapFromBuffer() looks like a good place to start.
Ok, how about this:
use the Bitmap.Save() method to save to a MemoryStream instead of a file;
when you save to the MemoryStream, you get to name the ImageFormat as "GIF" (this is equivalent to 8bpp in .Net, according to this: http://support.microsoft.com/kb/318343)
use MemoryStream.Write() to change whatever data you want in the image, or copy the data using MemoryStream.ToArray() if that jives better.
After you change the MemoryStream, you'll probably have to copy it back into the Bitmap, or make a new Bitmap. If you do make a new Bitmap, be sure to Dispose() the old one, to avoid memory leaks.
Hi Rocjoe and thanks again for the help, I have tried the following
Image bmp = new Bitmap(10, 10);
byte[] array = ImageToByteArray(bmp);
public byte[] ImageToByteArray(Image img)
{
MemoryStream ms = new MemoryStream();
img.Save(ms, System.Drawing.Imaging.ImageFormat.Gif );
return ms.ToArray();
}
The array coming back has over 870 bytes in it, It seems to hold all sorts of header info, padding and what have you. so again it does not work...