We Keep Coding

What type of image filtering/processing do mobile PDF scanners use to convert a captured image into a monochrome/black and white image?

I am trying to implement my own monochrome/black and white filter in C# to scan text documents. My approach is to apply a threshold filter on the captured image. However, I often run into the problem that the varying brightness on the image causes a ''shadowing effect'' on the processed image. Refer to the link below (it is pretty blurry but it should suffice). The image to the far left is the original image. When I apply my threshold filter, I get the same result as the image in the middle; some of the text becomes unreadable because the brightness of the image varies, so some portions become really black or really white. However, with the right filter, you can obtain the processed image to the right where everything looks crystal clear.
https://www.google.dk/search?q=monochrome+image+processing&espv=2&biw=1706&bih=859&source=lnms&tbm=isch&sa=X&ved=0ahUKEwir8vXlhIzPAhUFiywKHeSBC1wQ_AUIBigB#imgrc=4UTzoIpyqTkwrM%3A
I would like to know what the process is to obtain the image to the far right. Another example can be seen in the image below. It shows a sample mobile PDF scanner in use. Scanning the image results in a very nice black and white image, where the text can be easily read and no ''shadowing'' occurs on the image. Does anyone know what this process is or what it is called? It is very often used in mobile PDF scanning applications. Thank you in advance.
EDIT: The filter is called ''Adaptive Thresholding''. You can use the BradleyLocalThresholding class to implement the filter, or you can write it yourself (which is what I did). Please refer to my response to the comment by Yves Daoust down below.

You need two ingredients.
One is "background reconstruction", i.e. retrieving the intensity of the white sheet "under the characters", for instance by morphological opening.
The other is "shading correction", i.e. compensating the unevenness of the background illumination by comparing to the reconstructed background, for instance by subtraction.
This will "flatten" the image, making it perfectly amenable to global thresholding.

A simple method is to convert the image to grayscale and then convert it to B/W using an error diffusion algorithm such as Floyd–Steinberg dithering.

Overlaying a 'dot' on to an image on click

I am trying to write a simple program that lets me overlay a dot on top of an image when the image is clicked. I can save the X and Y data back to my database but then I will want to be able to call that information back at a later date and overlay the dots again via code unlike the first time when the user had to click the image.
I got as far as capturing the X and Y of the click no problem but I am having trouble finding examples specifically for what I am trying to do. All of the examples online seem to be for saving the image with the added graphic but I do not need to do that as it will be the same image every time.
Once I can do this, I also need to work out a way that I can detect what area of the image has been clicked. The areas I need to mark out vary in shape and size so I need to try and work out a way to 'map' these areas and then cross reference with the co-ordinates of the users click (I assume that I may need to do some clever geometry stuff for that?)
If anyone has any suggestions of what subjects/classes/methods etc. to research for either of my queries, I would very grateful.
Thanks in advance

You can use the System.Drawing namespace to achieve this.
Create a control and override OnPaint and OnPaintBackground. Store your clicks in a List
In OnPaintBackground, draw the image using DrawImageUnscaled using the graphics object which is passed to you as a parameter.
In OnPaint, loop through your points array and call graphics.FillElipse or similar to draw a little dot.
Because this isnt a retained mode graphics system, you need to keep drawing these items so this may not suit a large number of dots. In that case, you can create an in memory bitmap and get a graphics drawing object using graphics.FromImage.

screen color filtering

I want to create a filter over a specific area of the screen to perform filtering opertions.
Examples what a filtering opertion might be:
    - inverting (e.g. change black pixel to white pixels, red to cyan)
    - masking pixels (e.g. mask = ff0000; input c79001 -> c70000)
    - operations like photoshop's layer effects
Here is an example of what it should look like:
http://img443.imageshack.us/img443/1462/overlayk.png
Does anyone know how to perform this under Windows OS.
(my prefered language is C#)
Thanks!

Depending on how fast you need the "filter" to update, a quick and hacky way is just to get a screenshot using CopyFromScreen while your filter window is invisible, apply the filter to the image data, and then set the filter window to display the image data.
If you want to do it without having to hide the window first, you'll probably need to do something like http://www.codeproject.com/KB/system/snapshot.aspx where you capture individual windows.
An even trickier but potentially faster thing to do, and requiring nearly complete use of p/invoke win32 calls, would be to not have a window at all, get the required capture windows based on their coordinates, capture the images as above, and then draw directly to the screen DC.

To clarify: you want an area of the desktop, not just within the bounds of your window, to be under your control allowing you to apply a per-pixel filter. If that's the case, I think what you need is DirectDraw using the XNA libraries. WPF MAY get you what you need, but WinForms will most likely not. There are third party tools as well.
If you want this capability only within the bounds of your application's window, for instance in a drawing application, it gets far easier. Every class in the Windows.Forms namespace that inherits from Control exposes a CreateGraphics() method. This method returns an object representing a drawing surface covering the screen area of the control, and is the basis for just about anything you want to do on a window involving custom graphics (and internally, it's used to draw the controls in the first place).
Once you have the Graphics object, you can draw Images on it. A popular method of drawing custom graphics like animations or games is to do the actual drawing on a Bitmap object (derived from the abstract Image) and then when you're done, draw the Bitmap on the Graphics area. This is done to reduce flicker; if the graphics area is shown to the user while it is being drawn on, the user will only see the complete image for a split second before it is "wiped" and redrawn, and shapes drawn halfway through will be there one moment and gone the next as they wait to be drawn. Drawing to a bitmap, then showing the Bitmap on the screen when you're done, means the user sees a complete image at a time.
You can extend this using transparency features to create multi-layered images. Have a Bitmap for every layer you wish to manipulate. Work on them seperately, then draw each of them, in their proper order from back to front, onto a master Bitmap, and draw that Bitmap on the screen. This allows you those PhotoShop-type manipulations where a part of the image is one layer, and can be manipulated independently of all others.
As for per-pixel filtering, Bitmap objects expose GetPixel() and SetPixel() methods, which allow you to grab the color of a single pixel, perform a filter calculation, and re-draw it. This process will be totally unaccelerated, and so limited by your CPU speed, but allow very fine control of your image, or repetitive tasks like your filters.

Taking screenshots in Windows Vista, Windows 7, with transparent areas outside the app region

I am trying to take a screenshot of an application and I would like to make the parts of the rectangle that are not part of the applications region be transparent. So for instance on a standard windows application I would like to make the rounded corners transparent.
I wrote a quick test application which works on on XP (or vista/windows 7 with aero turned off):
protected override void OnPaint(PaintEventArgs e)
{
base.OnPaint(e);
Graphics g = e.Graphics;
// Just find a window to test with
IntPtr hwnd = FindWindowByCaption(IntPtr.Zero, "Calculator");
WINDOWINFO info = new WINDOWINFO();
info.cbSize = (uint)Marshal.SizeOf(info);
GetWindowInfo(hwnd, ref info);
Rectangle r = Rectangle.FromLTRB(info.rcWindow.Left, info.rcWindow.Top, info.rcWindow.Right, info.rcWindow.Bottom);
IntPtr hrgn = CreateRectRgn(info.rcWindow.Left, info.rcWindow.Top, info.rcWindow.Right, info.rcWindow.Bottom);
GetWindowRgn(hwnd, hrgn);
// fill a rectangle which would be where I would probably
// write some mask color
g.FillRectangle(Brushes.Red, r);
// fill the region over the top, all I am trying to do here
// is show the contrast between the applications region and
// the rectangle that the region would be placed in
Region region = Region.FromHrgn(hrgn);
region.Translate(info.rcWindow.Left, info.rcWindow.Top);
g.FillRegion(Brushes.Blue, region);
}
When I run this test app on XP (or Vista/Windows 7 with Aero off), I get something like this, which is great because I can eek an xor mask out of this that can be used later with BitBlt.
removed dead Imageshack link - Screenshot
Here is the problem, on Vista or Windows 7 with Aero enabled, there isn't necessarily a region on the window, in fact in most cases there isn't. Can anybody help me figure out some way to build a mask like this on these platforms?
Here are some of the approaches I have already tried...
1. Using the PrintWindow function: This doesn't work because it gives back a screenshot taken of the window with Aero off and this window is a different shape from the window returned with Aero on
2 Using the Desktop Window Manager API to get a full size thumbnail: This didn't work because it draws directly to the screen and from what I can tell you can't get a screenshot directly out of this api. Yeah, I could open a window with a pink background, show the thumbnail, take a screenshot then hide this temporary window but thats a horrible user experience and a complete hack I would rather not have my name on.
3. Using Graphics.CopyFromScreen or some other pinvoke variant of this: This doesn't work because I can't assume that the window I need information from is at the top of the z-order on the screen.
Right now, the best solution I can think of is to special case Aero on Windows 7 and Vista to manually rub out the corners by hard coding some graphics paths I paint out but this solution would suck since any application that performs custom skinning will break this.
Can you think of another or better solution?
If you are here, thanks for taking time to read this post, I appreciate any help or direction that you can offer!

If you are looking for a finished application, there is 7capture, which captures also the translucency, so images can be saved to PNG format for later compositing.
EDIT:
The original question and comments indicate you are looking to produce a region on Windows Vista/7 that you can then use to mask out parts the captured image, as is done with Windows XP and non-Aero UIs. Using a region is not going to give you the result you are looking for, since the window outline is not computed as a region, but as an image with variable transparency - RGBA. The Alpha channel in that image is your mask, but it's not an on-off mask like a region, but a gradual mask with a range of values from pixels being fully included to being fully masked out.
Although it uses undocumented APIs, the code at http://spazzarama.wordpress.com/2009/02/12/screen-capture-with-vista-dwm/ will capture to a RGBA buffer which you can then use to render or save the image with the shadow and other translucency effects intact.
In DwmCapture.cs Change
BackBufferFormat = Format.X8R8G8B8
to
BackBufferFormat = Format.A8R8G8B8
(X8->A8)
And you should then be able to access both the usual RGB data plus transparency from the captured buffer. This can then be saved as a PNG or other format with alpha-channel for composing.

Removed idea that is terrible but would have been awesome back in the '90s
You say that using the DWM API only allows you to capture directly to the screen... could you create a window offscreen (say, X = -100000px, Y = -100000px) but visible (maybe even hidden?) and draw the screenshot to it? Since when using the DWM each window has a backing texture, I'm thinking it might still get drawn fine even though the target isn't directly onscreen.
Also, if you want to go the DirectX route and access the actual DX texture backing the window, I found a few leads that might help (especially the first link):
http://spazzarama.wordpress.com/2009/02/12/screen-capture-with-vista-dwm/
http://channel9.msdn.com/forums/TechOff/251261-Help-Getting-the-shared-window-texture-out-of-DWM-/
http://web.archive.org/web/20080625183653/http://www.aeroxp.org/board/index.php?showtopic=6286
http://www.youtube.com/watch?v=hRTgFTMnT_U

Using Graphics.CopyFromScreen or some other pinvoke variant of this:
This doesn't work because I can't
assume that the window I need
information from is at the top of the
z-order on the screen.
If you know which window you need the information from, can you bring it to the front, call Graphics.CopyFromScreen, and then reset its z-index? I know from experience that Aero does odd things when items are in the background in order to make their glass interface work correctly (partial rendering etc). This may not be great UX; however, it would be a special case and used only when Aero is turned on.

You can take a look at the source code of AeroShot, as described on the main page, it can capture rounded edges and with the Aero Glass transparency effect and save it to a PNG file. It's written in C#.

How do you draw a string to a Bitmap in Silverlight?

In normal C# it is easy to draw to a bitmap using the Grpahics.DrawString() method. Silverlight seems to have done away with Bitmap objects and Graphics is no longer available either. So...How am I meant to manipulate/create a bitmap when using Silverlight? If it helps, I am using Silverlight 3.
Let me tell you what I am doing. I am being given a template, basically a pre-rendered image. The user is then able to select from multiple images and enter the deisred text. I then render it to the image, adjusting size etc... within bounds and centering it in the pre-defined area of the image. If I can calculate the size (as in the MeasureString method) and then draw the string (as in the Graphics.DrawString method) that would be fine. The real question, no matter why I want to be able to do this, is can it be done?

The question is: why do you want to? Why not just use a TextBlock?
If you are trying to dynamically generate an image, use standard Silverlight/WPF controls (including TextBlock) and render them to a WritableBitmap.
Edit:
Ok, you've updated and expanded, which gives me more to go on. Unfortunately, you're not going to like the answer. First, keep in mind that Silverlight and WPF in general are vector based, and intended to be used as such. Although the Canvas allows you to do pseudo-pixel manipulations, you cannot be nearly as pixel-accurate as old-school GDI. This is a factor of your medium. If you absolutely have to measure things the way you want to measure them, I suggest you build your images on a remote server and transmit them to your Silverlight app.
You can calculate the size on-screen of the text rendered via a TextBlock using the ActualWidth and ActualHeight properties. But it only works on an already rendered control. Something like MeasureString is simply not available in Silverlight. Based on your description of your app, some user interaction could accomplish what you want. The user selects the image, enters the text, and is shown a preview. The user can then adjust the width and height of the various text areas until satisfied, at which point you can take a snapshot using the render method I liked to above.

The following may work, its a bit nebulous because I haven't tried yet myself.
The object you are looking for is the WritableBitmap.
You create a Visual tree, for example create your self a Grid or Canvas (you're not adding this to the UI). Add to it the selected image and a TextBlock positioned and sized as you prefer.
Create a new WritableBitmap either of a specific size or using the selected image to initialize it.
Use the WritableBitmap Render method passing the above root Grid or Canvas to it.
Now you have a bitmap which you should able to use to do whatever its you needed to do that required all this hoop jumping in the first place.