We Keep Coding

C# - Working with high resolution screenshots

I need to capture an area within my desktop. But I need this area to be very high resolution (like, at least few thousand's pixels horizontal, same goes for vertical). Is it possible to get a screen capture that has high density of pixels? How can I do this? I tried capturing the screen with some AutoIt script, and got some very good results (images that were 350MB big), now I would like to do the same using C#.
Edit:
I am doing my read/write of a .tif file like that, and it already loses most of the data:
using (Bitmap bitmap = (Bitmap)Image.FromFile(#"ScreenShot.tif")) //this file has 350MB
{
using (Bitmap newBitmap = new Bitmap(bitmap))
{
newBitmap.Save("TESTRES.TIF", ImageFormat.Tiff); //now this file has about 60MB, Why?
}
}
I am trying to capture my screen like that, but the best I can get from this is few megabytes (nowhere near 350MB):
using (var bmpScreenCapture = new Bitmap(window[2], window[3], PixelFormat.Format32bppArgb))
{
using (var i = Graphics.FromImage(bmpScreenCapture))
{
i.InterpolationMode = InterpolationMode.High;
i.CopyFromScreen(window[0], window[1], 0, 0, bmpScreenCapture.Size, CopyPixelOperation.SourceCopy);
}
bmpScreenCapture.Save("test2.tif", ImageFormat.Tiff);
}

You can't gather more information than the source has.
This is a basic truth and it does apply here, too.
So you can't capture more the your 1920x1080 pixels at their color depth.
OTOH, since you want to feed the captured image into OCR, there a few more things to consider and in fact to do..
OCR is very happy if you help it by optimizing the image. This should involve
reducing colors and adding contrast
enlarging to the recommended dpi resolution
adding even more contrast
Funnily, this will help OCR although the real information cannot increase above the original source. But a good resizing algorithm will add invented data and these often will be just what the OCR software needs.
You should also take care to use a good i.e. non lossy format when you store the image to a file like png or tif and never jpg.
The best way will have to be adjusted by trial and error until the OCR results are good enough.
Hint: Due to font antialiasing most text on screenshots is surrounded by a halo of colorful pixels. Getting rid of it by the reducing or even removing saturation is one way; maybe you want to turn it off in your display properties? (Check out ClearType!)

Do something similar to Auto Tone of Photoshop with Aforge.net or c#

Im developing an image skin detection app.
But there is a problem with my camera, that try to compensate the light and the result image is bad, in most of cases i have a cold or warm effect on the image.
When i use photoshop there is the AutoTone function that normalize an image and reduce this problem.
With aforge i want to use HistogramEqualization() filter but the result is very bad:
// create filter
HistogramEqualization filter = new HistogramEqualization( );
// process image
filter.ApplyInPlace( sourceImage );
So my question is:
There is a function in Accord or Aforge to have the same result of the autotone of Photoshop?
If not, there is some library or script that let to do this?
Thank you all.

I use the LevelsLinear filter and base it on image stats:
ImageStatistics stats = new ImageStatistics(sourceImage);
LevelsLinear levelsLinear = new LevelsLinear {
InRed = stats.Red.GetRange( 0.90 ),
InGreen = stats.Green.GetRange( 0.90 ),
InBlue = stats.Blue.GetRange( 0.90 )
};
levelsLinear.ApplyInPlace(sourceImage);
You can play with the range to tweak the result.

You probably don't want to equalize the histogram, because as you see, a photo that wouldn't normally have much red, would have alot of red created and make it look nasty. Instead you probably want to examine for a bias to a hue that occurs almost everywhere. For example, your original photo probably had a bias towards blue in almost every pixel, and thus probably shouldn't be there. Look for a minimum bias and remove that amount everywhere.
A more practical solution is to experiment with the white balance setting on your camera to see what gives you the best result. Choosing the right preset, will leverage an algorithm that's probably as good as what you would write by hand. But maybe you are doing this as a learning experience.

How to apply image effects like edge detection oncamera stream in Windows 8 app?

I am trying to apply image manipulation effects in Windows 8 app on camera feeds directly.
I have tried a way using canvas and redrawing images after applying effects getting from webcam directly. But this approach works fine for basic effects but for effects like edge detection its creating large lag and flickering while using canvas approach.
Other way is to create MFT(media foundation transform) but it can be implemented in C about which i have no idea.
Can anyone tell me how can i achieve my purpose of applying effects on webcam stream directly in Windows 8 metro style app either by improving canvas approach so that large effects like edge detection don not have any issues or how can i apply MFT in C# since i have worked on C# language or by some other approach?

I have just played quite a bit in this area the last week and even considered writing a blog post about it. I guess this answer can be just as good.
You can go the MFT way, which needs to be done in C++, but the things you would need to write would not be much different between C# and C++. The only thing of note is that I think the MFT works in YUV color space, so your typical convolution filters/effects might behave a bit differently or require conversion to RGB. If you decide to go that route On the C# application side the only thing you would need to do is to call MediaCapture.AddEffectAsync(). Well that and you need to edit your Package.appxmanifest etc., but let's go with first things first.
If you look at the Media capture using webcam sample - it already does what you need. It applies a grayscale effect to your camera feed. It includes a C++ MFT project that is used in an application that is available in C# version. I had to apply the effect to a MediaElement which might not be what you need, but is just as simple - call MediaElement.AddVideoEffect() and your video file playback now applies the grayscale effect. To be able to use the MFT - you need to simply add a reference to the GrayscaleTransform project and add following lines to your appxmanifest:
<Extensions>
<Extension Category="windows.activatableClass.inProcessServer">
<InProcessServer>
<Path>GrayscaleTransform.dll</Path>
<ActivatableClass ActivatableClassId="GrayscaleTransform.GrayscaleEffect" ThreadingModel="both" />
</InProcessServer>
</Extension>
</Extensions>
How the MFT code works:
The following lines create a pixel color transformation matrix
float scale = (float)MFGetAttributeDouble(m_pAttributes, MFT_GRAYSCALE_SATURATION, 0.0f);
float angle = (float)MFGetAttributeDouble(m_pAttributes, MFT_GRAYSCALE_CHROMA_ROTATION, 0.0f);
m_transform = D2D1::Matrix3x2F::Scale(scale, scale) * D2D1::Matrix3x2F::Rotation(angle);
Depending on the pixel format of the video feed - a different transformation method is selected to scan the pixels. Look for these lines:
m_pTransformFn = TransformImage_YUY2;
m_pTransformFn = TransformImage_UYVY;
m_pTransformFn = TransformImage_NV12;
For my sample m4v file - the format is detected as NV12, so it is calling TransformImage_NV12.
For pixels within the specified range (m_rcDest) or within the entire screen if no range was specified - the TransformImage_~ methods call TransformChroma(mat, &u, &v).
For other pixels - the values from original frame are copied.
TransformChroma transforms the pixels using m_transform. If you want to change the effect - you can simply change the m_transform matrix or if you need access to neighboring pixels as in an edge detection filter - modify the TransformImage_ methods to process these pixels.
This is one way to do it. I think it is quite CPU intensive, so personally I prefer to write a pixel shader for such operations. How do you apply a pixel shader to a video stream though? Well, I am not quite there yet, but I believe you can transfer video frames to a DirectX surface fairly easily and call a pixel shader on them later. So far - I was able to transfer the video frames and I am hoping to apply the shaders next week. I might write a blog post about it. I took the meplayer class from the Media engine native C++ playback sample and moved it to a template C++ DirectX project converted to a WinRTComponent library, then used it with a C#/XAML application, associating the swapchain the meplayer class creates with the SwapChainBackgroundPanel that I use in the C# project to display the video. I had to make a few changes in the meplayer class. First - I had to move it to a public namespace that would make it available to other assembly. Then I had to modify the swapchain it creates to a format accepted for use with a SwapChainBackgroundPanel:
DXGI_SWAP_CHAIN_DESC1 swapChainDesc = {0};
swapChainDesc.Width = m_rcTarget.right;
swapChainDesc.Height = m_rcTarget.bottom;
// Most common swapchain format is DXGI_FORMAT_R8G8B8A8-UNORM
swapChainDesc.Format = m_d3dFormat;
swapChainDesc.Stereo = false;
// Don't use Multi-sampling
swapChainDesc.SampleDesc.Count = 1;
swapChainDesc.SampleDesc.Quality = 0;
//swapChainDesc.BufferUsage = DXGI_USAGE_BACK_BUFFER | DXGI_USAGE_RENDER_TARGET_OUTPUT;
swapChainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT; // Allow it to be used as a render target.
// Use more than 1 buffer to enable Flip effect.
//swapChainDesc.BufferCount = 4;
swapChainDesc.BufferCount = 2;
//swapChainDesc.Scaling = DXGI_SCALING_NONE;
swapChainDesc.Scaling = DXGI_SCALING_STRETCH;
swapChainDesc.SwapEffect = DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL;
swapChainDesc.Flags = 0;
Finally - instead of calling CreateSwapChainForCoreWindow - I am calling CreateSwapChainForComposition and associating the swapchain with my SwapChainBackgroundPanel:
// Create the swap chain and then associate it with the SwapChainBackgroundPanel.
DX::ThrowIfFailed(
spDXGIFactory.Get()->CreateSwapChainForComposition(
spDevice.Get(),
&swapChainDesc,
nullptr, // allow on all displays
&m_spDX11SwapChain)
);
ComPtr<ISwapChainBackgroundPanelNative> dxRootPanelAsSwapChainBackgroundPanel;
// Set the swap chain on the SwapChainBackgroundPanel.
reinterpret_cast<IUnknown*>(m_swapChainPanel)->QueryInterface(
IID_PPV_ARGS(&dxRootPanelAsSwapChainBackgroundPanel)
);
DX::ThrowIfFailed(
dxRootPanelAsSwapChainBackgroundPanel->SetSwapChain(m_spDX11SwapChain.Get())
);
*EDIT follows
Forgot about one more thing. If your goal is to stay in pure C# - if you figure out how to capture frames to a WriteableBitmap (maybe by calling MediaCapture.CapturePhotoToStreamAsync() with a MemoryStream and then calling WriteableBitmap.SetSource() on the stream) - you can use WriteableBitmapEx to process your images. It might not be top performance, but if your resolution is not too high or your frame-rate requirements are not high - it might just be enough. The project on CodePlex does not officially support WinRT yet, but I have a version that should work that you can try here (Dropbox).

As far as I know, MFTs need to be implemented in C++. I believe that there is a media transform SDK sample which shows implementing some straightforward transforms from a metro style application.

How to get coordinates of cropped image from original [duplicate]

I have 2 bmp images.
ImageA is a screenshot (example)
ImageB is a subset of that. Say for example, an icon.
I want to find the X,Y coordinates of ImageB within ImageA (if it exists).
Any idea how I would do that?

Here's a quick sample but it is slow take around 4-6 seconds, but it does exactly what you looking for and i know this post is old but if anyone else visiting this post recently
you can look this thing
you need .NET AForge namespace or framework google it and install it
include AForge name space in your project and that's it
it finds the pictiure with another and gives out the coordinates.
System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\1.jpg");
System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\2.jpg");
// create template matching algorithm's instance
// (set similarity threshold to 92.1%)
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
// find all matchings with specified above similarity
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings
BitmapData data = sourceImage.LockBits(
new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{
Drawing.Rectangle(data, m.Rectangle, Color.White);
MessageBox.Show(m.Rectangle.Location.ToString());
// do something else with matching
}
sourceImage.UnlockBits(data);

So is there any warping of ImageB in ImageA?
How "exact" are the images, as in, pixel-for-pixel they will be the same?
How much computational power do you have for this?
If the answers to the first two questions are No and Yes, then you have a simple problem. It also helps to know the answer to Q3.
Update:
The basic idea's this: instead of matching a window around every pixel in imageB with every pixel in imageA and checking the correlation, let's identify points of interest (or features) in both images which will be trackable. So it looks like corners are really trackable since the area around it is kinda similar (not going into details) - hence, let's find some really strong corners in both images and search for corners which look most similar.
This reduces the problem of searching every pixel in B with A to searching for, say, 500 corners in B with a 1000 corners in A (or something like that) - much faster.
And the awesome thing is you have several such corner detectors at your disposal in OpenCV. If you don't feel using emguCV (C# varriant), then use the FAST detector to find matching corners and thus locate multiple features between your images. Once you have that, you can find the location of the top-left corner of the image.

If image B is an exact subset of image A (meaning, the pixel values are exactly the same), this is not an image processing problem, it's just string matching in 2D. In 99% of the cases, taking a line form the middle of B and matching it against each line of A will do what you want, and super fast &mdhas; I guess C# has a function for that. After you get your matches (normally, a few of them), just check the whole of B against the appropriate part of A.
The only problem I can see with this is that in some cases you can get too many matches. E.g. if A is your desktop, B is an icon, and you are unlucky enough to pick a line in B consisting of background only. This problem is easy to solve (you have to choose lines from B a bit more carefully), but this depends on the specifics of your problem.

Finding sub images in an imageFind an image in an ImageCheck if an image exists within another image

PCL raster color graphics distorted. Code included: what am I missing?

The function below prints a color raster image to a PCL-5 printer. The function was adapted from a 2-color (1bpp) printing function we had that worked perfectly, except for the grainy 2-color printing. The problem is that the image comes out with a large black bar extending from the right of the image to the edge of the page like this:
IMAGE#########################################
IMAGE#########AREA COMPLETELY BLACK###########
IMAGE#########################################
The image itself looks perfect, otherwise.
Various PCL-to-PDF tools don't show the image at all, which leads me to believe I've forgotten do to something. Appropriate resets (\u001bE\u001b%-12345X) were sent before, and page-feeds after.
Any PCL experts out there? I've got the PCL 5 Color Technical Reference Manual, and it's gotten me this far. This last thing is driving me crazy though.
*Edit:
I now know what command is causing the problem, but I don't know why:
stream("\u001b*r0F");
This should keep the image rotated along with the page (portrait, landscape). If I remove this, the problem goes away. I can compensate by rotating the bitmap beforehand, but I really want to know what caused this!
static void PrintImage()
{
// Get an image into memory
Image original = Image.FromFile("c:\\temp\\test.jpg");
Bitmap newBitmap = new Bitmap(original, original.Width, original.Height);
stream(String.Format("\u001b*p{0:d}x*p{1:d}Y", 1000, 1000));// Set cursor.
stream("\u001b*t300R"); // 300 DPI
stream(String.Format("\u001b*r{0:d}T", original.Height)); // Height
stream(String.Format("\u001b*r{0:d}S", original.Width)); // Width
stream("\u001b*r3U"); // 8-bit color palette
stream("\u001b*r0F"); // Follow logical page layout (landscape, portrait, etc..)
// Set palette depth, 3 bytes per pixel RGB
stream("\u001b*v6W\u0000\u0003\u0000\u0008\u0008\u0008");
stream("\u001b*r1A"); // Start raster graphics
stream("\u001b*b0M"); // Compression 0 = None, 1 = Run Length Encoding
// Not fast, but fast enough.
List<byte> colors = new List<byte>();
for (int y2 = 0; y2 < original.Height; y2++)
{
colors.Clear();
for (int x2 = 0; x2 < original.Width; x2++)
{
Color c = newBitmap.GetPixel(x2, y2);
colors.Add(c.R);
colors.Add(c.G);
colors.Add(c.B);
}
stream(String.Format("\u001b*b{0}W", colors.Count)); // Length of data to send
streamBytes(colors.ToArray()); // Binary data
}
stream("\u001b*rB"); // End raster graphics (also tried *rC -- no effect)
}

There are a few problems with your code. First off your cursor position code is incorrect, it should read:
"\u001b*p{0:d}x1:d}Y", 1000, 1000
This equates to:
<esc>*p1000x1000Y
you had:
<esc>*p1000x*p1000Y
When joining PCL commands together you match up the same parameterized value and group and then simply add value + parametrized character + value + parametrized character etc. Ensure that the final parametrized character is a capital letter which signifies the end of the PCL command.
Also when defining an image I recommend you also specify the width & hight in decipoints, this should help with the scaling of the image (*r3A) on the page so add this (just after your resolution command should be an okay place for it):
Int32 deciHeight = original.Height / (int)original.HorizontalResolution * 720;
Int32 deciWidth = original.Width / (int)original.VerticalResolution * 720;
stream("\u001b*t{0:d}h{1:d}V", deciHeight, deciWidth));
The other recommendation is to write all of this to file (watch your encodings) and use one of the handful of PCL viewers to view your data vs. always printing it. Should save you some time and a forest or two! I've tried all of them and wold recommend spending the $89 and purchasing pclWorks. They also have a complete SDK if you are going to do a lot of PCL. We don't use that as we hardcode all PCL ourselves but it does look good.
As for rotation, we've had problems on some device, You could just rotate the the jpg first (original.RotateFlip) an then write it out.
I don't have much time today but hope that my comments assist. I can test your code on Monday or Tuesday and work with it and post any further comments.
Keep in mind that even though PCL is a standard its support from manufacturer to manufacturer and device to device can be a problem and vastly different. When doing basic things most devices seem okay; however, if you get into macros or complex graphics you will find difference.