We Keep Coding

StartCoroutine() to fix targetTexture.ReadPixels error

As the title suggests I have a problem with the error occurring at the row
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
Error:
ReadPixels was called to read pixels from system frame buffer, while
not inside drawing frame. UnityEngine.Texture2D:ReadPixels(Rect,
Int32, Int32)
As I have understood from other posts one way to solve this issue is to make a Ienumerator method which yield return new WaitForSeconds or something, and call it like: StartCoroutine(methodname) so that the frames gets to load in time so that there will be pixels to read-ish.
What I don't get is where in the following code this method would make the most sense. Which part does not get to load in time?
PhotoCapture photoCaptureObject = null;
Texture2D targetTexture = null;
public string path = "";
CameraParameters cameraParameters = new CameraParameters();
private void Awake()
{
var cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);
// Create a PhotoCapture object
PhotoCapture.CreateAsync(false, captureObject =>
{
photoCaptureObject = captureObject;
cameraParameters.hologramOpacity = 0.0f;
cameraParameters.cameraResolutionWidth = cameraResolution.width;
cameraParameters.cameraResolutionHeight = cameraResolution.height;
cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
});
}
private void Update()
{
// if not initialized yet don't take input
if (photoCaptureObject == null) return;
if (Input.GetKey("k") || Input.GetKey("k"))
{
Debug.Log("k was pressed");
VuforiaBehaviour.Instance.gameObject.SetActive(false);
// Activate the camera
photoCaptureObject.StartPhotoModeAsync(cameraParameters, result =>
{
if (result.success)
{
// Take a picture
photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
}
else
{
Debug.LogError("Couldn't start photo mode!", this);
}
});
}
}
private static string FileName(int width, int height)
{
return $"screen_{width}x{height}_{DateTime.Now:yyyy-MM-dd_HH-mm-ss}.png";
}
private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
private void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result)
{
// Shutdown the photo capture resource
VuforiaBehaviour.Instance.gameObject.SetActive(true);
photoCaptureObject.Dispose();
photoCaptureObject = null;
}
Sorry if this counts as a duplicate to this for example.
Edit
And this one might be useful when I get to that point.
Is it so that I don't need these three lines at all?
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
As written in the comments the difference between using these three lines and not is that the photo saved has a black background + the AR-GUI. Without the second line of code above is a photo with the AR-GUI but with the background is a live stream of my computer webcam. And really I don't wanna see the computer webcam but what the HoloLens sees.

Your three lines
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
make not much sense to me. Texture2D.ReadPixels is for creating a Screenshot so you would overwrite the texture you just received from PhotoCapture with a screenshot? (Also with incorrect dimensions since camera resolution very probably != screen resolution.)
That's also the reason for
As written in the comments the difference between using these three lines and not is that the photo saved has a black background + the AR-GUI.
After doing
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
you already have the Texture2D received from the PhotoCapture in the targetTexture.
I think you probably confused it with Texture2D.GetPixels which is used to get the pixel data of a given Texture2D.
I would like to crop the captured photo from the center in the end and am thinking that maybe that is possible with this code row? Beginning the new rect at other pixels than 0, 0)
What you actually want is cropping the received Texture2D from the center as you mentioned in the comments. You can do that using GetPixels(int x, int y, int blockWidth, int blockHeight, int miplevel) which is used to cut out a certain area of a given Texture2D
public static Texture2D CropAroundCenter(Texture2D input, Vector2Int newSize)
{
if(input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!", this);
return null;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Create a new texture with newSize
var output = new Texture2D(newSize.x, newSize.y);
output.SetPixels(pixels);
output.Apply();
return output;
}
for (hopefully) better understanding this is an illustration what that GetPixels overload with the given values does here:
and than use it in
private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
// for example take only half of the textures width and height
targetTexture = CropAroundCenter(targetTexture, new Vector2Int(targetTexture.width / 2, targetTexture.height / 2);
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
Or you could make it an extension method in an apart static class like
public static class Texture2DExtensions
{
public static void CropAroundCenter(this Texture2D input, Vector2Int newSize)
{
if (input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!");
return;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Resize the texture (creating a new one didn't work)
input.Resize(newSize.x, newSize.y);
input.SetPixels(pixels);
input.Apply(true);
}
}
and use it instead like
targetTexture.CropAroundCenter(new Vector2Int(targetTexture.width / 2, targetTexture.height / 2));
Note:
UploadImageDataToTexture: You may only use this method if you specified the BGRA32 format in your CameraParameters.
Luckily you use that anyway ;)
Keep in mind that this operation will happen on the main thread and therefore be slow.
However the only alternative would be CopyRawImageDataIntoBuffer and generate the texture in another thread or external, so I'ld say it is ok to stay with UploadImageDataToTexture ;)
and
The captured image will also appear flipped on the HoloLens. You can reorient the image by using a custom shader.
by flipped they actually mean that the Y-Axis of the texture is upside down. X-Axis is correct.
For flipping the Texture vertically you can use a second extension method:
public static class Texture2DExtensions
{
public static void CropAroundCenter(){....}
public static void FlipVertically(this Texture2D texture)
{
var pixels = texture.GetPixels();
var flippedPixels = new Color[pixels.Length];
// These for loops are for running through each individual pixel and
// write them with inverted Y coordinates into the flippedPixels
for (var x = 0; x < texture.width; x++)
{
for (var y = 0; y < texture.height; y++)
{
var pixelIndex = x + y * texture.width;
var flippedIndex = x + (texture.height - 1 - y) * texture.width;
flippedPixels[flippedIndex] = pixels[pixelIndex];
}
}
texture.SetPixels(flippedPixels);
texture.Apply();
}
}
and use it like
targetTexture.FlipVertically();
Result: (I used FlipVertically and cropp to the half of size every second for this example and a given Texture but it should work the same for a taken picture.)
Image source: http://developer.vuforia.com/sites/default/files/sample-apps/targets/imagetargets_targets.pdf
Update
To your button problem:
Don't use
if (Input.GetKey("k") || Input.GetKey("k"))
First of all you are checking the exact same condition twice. And than GetKey fires every frame while the key stays pressed. Instead rather use
if (Input.GetKeyDown("k"))
which fires only a single time. I guess there was an issue with Vuforia and PhotoCapture since your original version fired so often and maybe you had some concurrent PhotoCapture processes...

C# WinForms Out of Memory Exception on Bitmap Clone

I've written a small "watermark" program to add a custom watermark to an image. There are two watermarks a white one and a black one. The watermark is always placed on the left bottom side of the image. I clone that region of the image to determine which watermark should be placed based on that spot (the black watermark on light regions and the white watermark on dark regions).
When I use the application (either in debug or normal) on my machine - no problem. All images are processed and watermarks are added on the correct location.
However, on the client machine, the program breaks on all images throwing a OutOfMemory Exception on the clone part.
I know, ussually, the OutOfMemory Exception will also be thrown when I specify a region out of bounds, but since the function is working like a charm on my machine I can't imagine that is the case. Besides that, the program doesn't break after a few tries, it breaks on all attempts to clone.
Doesn't matter if there is text (DrawString method) or not. It breaks on the Clone.
The images being processed are big, but not "huge" (6016 x 4000 pixels at most) but even with smaller images (3264 x 2448 pixels) the client will break.
Variables:
bmOriginal : original bitmap
processImage : original image (pictureBox) - bmOriginal is a bitmap copy of this
watermarkText : textbox for extra information below watermark
black and white : pictureboxes containing the watermark images
watermarkCombo : combobox for selecting automatic, white or black (automatic fails)
Code:
using (Graphics gWatermark = Graphics.FromImage(bmOriginal))
{
gWatermark.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
System.Drawing.SolidBrush drawBrush = new System.Drawing.SolidBrush(System.Drawing.Color.Black);
// position watermark - watermark should be 10% of the image height
int watermarkHeight = (int)(processImage.Image.Height * 0.1);
int watermarkPadding = (int)(watermarkHeight * 0.1); // not completely true, but asume watermark is square
Rectangle watermarkArea = new Rectangle(watermarkPadding, processImage.Image.Height - (watermarkPadding + (watermarkText.Text.Length == 0 ? 0 : watermarkPadding) + watermarkHeight), watermarkHeight, watermarkHeight);
// determine color watermark
bmWatermark = (Bitmap)black.Image;
if (watermarkCombo.SelectedIndex == 0)
{
using (Bitmap watermarkClone = bmOriginal.Clone(watermarkArea, bmOriginal.PixelFormat))
{
var pixels = Pixels(watermarkClone);
if (pixels.Average((Func<Color, decimal>)Intensity) < 110) // human eye adoption; normal threshold should be 128
{
bmWatermark = (Bitmap)white.Image;
drawBrush = new System.Drawing.SolidBrush(System.Drawing.Color.White);
}
}
}
else if (watermarkCombo.SelectedIndex == 1)
{
bmWatermark = (Bitmap)white.Image;
drawBrush = new System.Drawing.SolidBrush(System.Drawing.Color.White);
}
// draw the watermark
gWatermark.DrawImage(bmWatermark, watermarkArea.X, watermarkArea.Y, watermarkArea.Width, watermarkArea.Height);
// draw the text (if needed)
if (watermarkText.Text.Length > 0)
{
System.Drawing.Font drawFont = new System.Drawing.Font("Tahoma", (float)watermarkPadding);
gWatermark.DrawString(watermarkText.Text, drawFont, drawBrush, watermarkPadding, bmOriginal.Height - (watermarkPadding * 2));
}
}
bmOriginal.Save(System.IO.Path.Combine(diWatermarked.FullName, fileName), System.Drawing.Imaging.ImageFormat.Jpeg);
line of error: using (Bitmap watermarkClone = bmOriginal.Clone(watermarkArea, bmOriginal.PixelFormat))
Now the big question: how do I get rid of that OutOfMemory exception.. anyone an idea?
EDIT When I choose not to determine the color of the watermark automatically and just add a watermark (let's say the white one) the program functions normally. I've seen the stack trace in an error log (on the catch of the function I output the exception and -if any- inner exceptions).
I know a lot of the OOM exceptions using the Clone function occur when you specify a region out of bounds; but that is not the case here.
When I look at my memory when using the app in debug mode, I start at 5.36 Gb program started and a normalized 5.39 Gb (with a max spike of 5.42 Gb) when running the execution I mentioned, it's not guzzling memory like crazy.
The code I use the determine the average "color" (it's from someone on StackOverflow - I just copied that from some other answer, can't find the link though);
// functions used to determine watermark color
private static decimal ComponentAverage(decimal a, decimal b)
{
return Math.Min(a, b) + Math.Abs(a - b) / 2M;
}
private static decimal Intensity(Color color)
{
decimal result = color.A;
result = ComponentAverage(result, color.R);
result = ComponentAverage(result, color.G);
result = ComponentAverage(result, color.B);
return result;
}
private static IEnumerable<Color> Pixels(Bitmap bitmap)
{
for (int x = 0; x < bitmap.Width; x++)
for (int y = 0; y < bitmap.Height; y++)
yield return bitmap.GetPixel(x, y);
}
SOURCE There is a test project here: http://hotpepper.nu/oomtestapp.zip

Leon, I've changed your code you've uploaded to not lock any resources.
I've notices that if I keep your application open, I cannot delete the output folders because some files are in use. This usually means that you DID NOT release all file handles, basically it is always the last file.
I couldn't reproduce the out of memory issue on my computer before and after my changes, seems to be an issue with very big files maybe?
Ok anyways, I found that you use ImageBox to load the white and black resource and to load the image from disc. This is not needed at all, intead use the resources directly
Bitmap white = OomTestApp.Properties.Resources.white;
Bitmap black = OomTestApp.Properties.Resources.black;
Then to load an image from disc, simply use Bitmap.FromFile
I have added some lines to release your resources properly .Dispose where no using blocks are used.
And I also removed the Clone() call because it is absolutely not needed I think, because you are just computing the pixels of the original image and you do not draw something into that image at that point. So what was the need to have the image cloned?
Here is the full code (starting after you created the folders)
if (errors == 0)
{
this.Height = 323;
goButton.Enabled = false;
stopButton.Enabled = true;
Bitmap white = OomTestApp.Properties.Resources.white;
Bitmap black = OomTestApp.Properties.Resources.black;
Bitmap bmWatermark = black;
Bitmap processImage = null;
progressBar1.Maximum = filesToProcess.Count;
foreach (string handleFile in filesToProcess)
{
string fileName = System.IO.Path.GetFileName(handleFile);
fileNameLabel.Text = "File: " + System.IO.Path.GetFileName(handleFile);
try
{
// create backup if checked
if (diOriginal != null)
{
System.IO.File.Move(handleFile, System.IO.Path.Combine(diOriginal.FullName, fileName));
processImage = (Bitmap)Bitmap.FromFile(System.IO.Path.Combine(diOriginal.FullName, fileName));
}
else
{
processImage = (Bitmap)Bitmap.FromFile(handleFile);
}
double aspectRatio = (double)processImage.Width / (double)processImage.Height;
using (Graphics gWatermark = Graphics.FromImage(processImage))
{
gWatermark.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
System.Drawing.SolidBrush drawBrush = new System.Drawing.SolidBrush(System.Drawing.Color.Black);
// position watermark - watermark should be 10% of the image height
int watermarkHeight = (int)(processImage.Height * 0.1);
int watermarkPadding = (int)(watermarkHeight * 0.1); // not completely true, but asume watermark is square
// calculate rectangle. if there is extra text, add extra padding below
Rectangle watermarkArea = new Rectangle(watermarkPadding, processImage.Height - (watermarkPadding + (watermarkText.Text.Length == 0 ? 0 : watermarkPadding) + watermarkHeight), watermarkHeight, watermarkHeight);
// determine color watermark
bmWatermark = black;
if (watermarkCombo.SelectedIndex == 0)
{
/*using (Bitmap watermarkClone = processImage.Clone(watermarkArea, processImage.PixelFormat))
{*/
var pixels = Pixels(processImage);
if (pixels.Average((Func<Color, decimal>)Intensity) < 110) // human eye adoption; normal threshold should be 128
{
bmWatermark = white;
drawBrush = new System.Drawing.SolidBrush(System.Drawing.Color.White);
}
//}
}
else if (watermarkCombo.SelectedIndex == 1)
{
bmWatermark = white;
drawBrush = new System.Drawing.SolidBrush(System.Drawing.Color.White);
}
// draw the watermark
gWatermark.DrawImage(bmWatermark, watermarkArea.X, watermarkArea.Y, watermarkArea.Width, watermarkArea.Height);
// draw the text (if needed)
if (watermarkText.Text.Length > 0)
{
System.Drawing.Font drawFont = new System.Drawing.Font("Tahoma", (float)watermarkPadding);
gWatermark.DrawString(watermarkText.Text, drawFont, drawBrush, watermarkPadding, processImage.Height - (watermarkPadding * 2));
drawFont.Dispose();
}
// disposing resources
drawBrush.Dispose();
}
// save the watermarked file
processImage.Save(System.IO.Path.Combine(diWatermarked.FullName, fileName), System.Drawing.Imaging.ImageFormat.Jpeg);
// stop button pressed?
Application.DoEvents();
if (stopProcess) break;
// update exection progress
progressBar1.Value++;
percentLabel.Text = ((int)((progressBar1.Value * 100) / filesToProcess.Count)).ToString() + "%";
fileCountLabel.Text = "File " + progressBar1.Value.ToString() + "/" + filesToProcess.Count.ToString();
}
catch (Exception ex)
{
try
{
using (System.IO.StreamWriter sw = new System.IO.StreamWriter(System.IO.Path.Combine(folderText.Text, "errorlog.txt"), true))
{
sw.WriteLine("File: " + fileName);
while (ex != null)
{
sw.WriteLine("Message: " + ex.Message);
sw.WriteLine(ex.StackTrace);
sw.WriteLine(ex.Source);
ex = ex.InnerException;
}
sw.WriteLine();
}
}
catch
{
// nothing to do - it already failed
}
errors++;
}
finally
{
if (processImage != null) processImage.Dispose();
}
}
// dispose resources
white.Dispose();
black.Dispose();
bmWatermark.Dispose();
if (!stopProcess)
{
// set status to complete
fileCountLabel.Text = "File " + filesToProcess.Count.ToString() + "/" + filesToProcess.Count.ToString();
percentLabel.Text = "100%";
fileNameLabel.Text = "Completed...";
}
else
{
fileNameLabel.Text = "Aborted...";
}
fileNameLabel.Text += errors.ToString() + " error(s) encountered";
// defaults to screen
progressBar1.Value = progressBar1.Maximum;
stopProcess = false;
goButton.Enabled = true;
stopButton.Enabled = false;

Well, I didn't read everything on the page, but I wonder if anyone mentioned the "stride" of the Bitmap? Basically, your bitmap has to be a multiple of 4 bytes.
I had this problem taking apart a grid to make tiles. The last tile would error because the Bitmap was not evenly divisible by 4.
The stride is the width of a single row of pixels (a scan line),
rounded up to a four-byte boundary. If the stride is positive, the
bitmap is top-down. If the stride is negative, the bitmap is
bottom-up.
https://softwarebydefault.com/2013/04/11/bitmap-color-balance/

Selenium WebDriver C# Full Website Screenshots With ChromeDriver and FirefoxDriver

When I take screenshots with ChromeDriver I get screens with the size of my viewport.
When I take screenshots with FirefoxDriver I get what I want, which is a full screen print of a website.
ChromeDriver is declared like this:
IWebDriver driver = new ChromeDriver();
FirefoxDriver is declared like this:
IWebDriver driver = new FirefoxDriver();
Both drivers execute identical code:
driver.Manage().Window.Maximize();
driver.Navigate().GoToUrl(url);//url is a string variable
ITakesScreenshot screenshotDriver = driver as ITakesScreenshot;
Screenshot screenshot = screenshotDriver.GetScreenshot();
screenshot.SaveAsFile("c:/test.png", ImageFormat.Png);
ChromeDriver's test.png is of 1920x1099 resolution and contains only the browser viewport.
FirefoxDriver's test.png is of 1903x16559 resolution and contains the whole page.
I know that GetScreenshot() method doesn't return identical resolution sizes because it has slightly different implementations in IEDriver, FirefoxDriver, OperaDriver, ChromeDriver.
My questions are:
Why is there such difference between ChromeDriver's and FirefoxDriver's .GetScreenshot() method, even tho they use an identical interface (ITakesScreenshot)?
Is there a way to make ChromeDriver's GetScreenshot() method return the whole webpage screen instead of just the viewport?

we can't get the entire page screenshot with ChromeDriver2, we need to go for manual implementation.I have modified a method with is available in a blog which works fine with ChromeDriver.
use this method as following :
private IWebDriver _driver = new ChromeDriver(CHROME_DRIVER_PATH);
screenshot.SaveAsFile(saveFileName, ImageFormat.Jpeg);
public Bitmap GetEntereScreenshot()
{
Bitmap stitchedImage = null;
try
{
long totalwidth1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return document.body.offsetWidth");//documentElement.scrollWidth");
long totalHeight1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return document.body.parentNode.scrollHeight");
int totalWidth = (int)totalwidth1;
int totalHeight = (int)totalHeight1;
// Get the Size of the Viewport
long viewportWidth1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return document.body.clientWidth");//documentElement.scrollWidth");
long viewportHeight1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return window.innerHeight");//documentElement.scrollWidth");
int viewportWidth = (int)viewportWidth1;
int viewportHeight = (int)viewportHeight1;
// Split the Screen in multiple Rectangles
List<Rectangle> rectangles = new List<Rectangle>();
// Loop until the Total Height is reached
for (int i = 0; i < totalHeight; i += viewportHeight)
{
int newHeight = viewportHeight;
// Fix if the Height of the Element is too big
if (i + viewportHeight > totalHeight)
{
newHeight = totalHeight - i;
}
// Loop until the Total Width is reached
for (int ii = 0; ii < totalWidth; ii += viewportWidth)
{
int newWidth = viewportWidth;
// Fix if the Width of the Element is too big
if (ii + viewportWidth > totalWidth)
{
newWidth = totalWidth - ii;
}
// Create and add the Rectangle
Rectangle currRect = new Rectangle(ii, i, newWidth, newHeight);
rectangles.Add(currRect);
}
}
// Build the Image
stitchedImage = new Bitmap(totalWidth, totalHeight);
// Get all Screenshots and stitch them together
Rectangle previous = Rectangle.Empty;
foreach (var rectangle in rectangles)
{
// Calculate the Scrolling (if needed)
if (previous != Rectangle.Empty)
{
int xDiff = rectangle.Right - previous.Right;
int yDiff = rectangle.Bottom - previous.Bottom;
// Scroll
//selenium.RunScript(String.Format("window.scrollBy({0}, {1})", xDiff, yDiff));
((IJavaScriptExecutor)_driver).ExecuteScript(String.Format("window.scrollBy({0}, {1})", xDiff, yDiff));
System.Threading.Thread.Sleep(200);
}
// Take Screenshot
var screenshot = ((ITakesScreenshot)_driver).GetScreenshot();
// Build an Image out of the Screenshot
Image screenshotImage;
using (MemoryStream memStream = new MemoryStream(screenshot.AsByteArray))
{
screenshotImage = Image.FromStream(memStream);
}
// Calculate the Source Rectangle
Rectangle sourceRectangle = new Rectangle(viewportWidth - rectangle.Width, viewportHeight - rectangle.Height, rectangle.Width, rectangle.Height);
// Copy the Image
using (Graphics g = Graphics.FromImage(stitchedImage))
{
g.DrawImage(screenshotImage, rectangle, sourceRectangle, GraphicsUnit.Pixel);
}
// Set the Previous Rectangle
previous = rectangle;
}
}
catch (Exception ex)
{
// handle
}
return stitchedImage;
}

I cleaned up #Selvantharajah Roshanth's answer and added a check so that it won't try to stitch together screenshots that already fit in the viewport.
public Image GetEntireScreenshot()
{
// Get the total size of the page
var totalWidth = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return document.body.offsetWidth"); //documentElement.scrollWidth");
var totalHeight = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return document.body.parentNode.scrollHeight");
// Get the size of the viewport
var viewportWidth = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return document.body.clientWidth"); //documentElement.scrollWidth");
var viewportHeight = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return window.innerHeight"); //documentElement.scrollWidth");
// We only care about taking multiple images together if it doesn't already fit
if (totalWidth <= viewportWidth && totalHeight <= viewportHeight)
{
var screenshot = driver.TakeScreenshot();
return ScreenshotToImage(screenshot);
}
// Split the screen in multiple Rectangles
var rectangles = new List<Rectangle>();
// Loop until the totalHeight is reached
for (var y = 0; y < totalHeight; y += viewportHeight)
{
var newHeight = viewportHeight;
// Fix if the height of the element is too big
if (y + viewportHeight > totalHeight)
{
newHeight = totalHeight - y;
}
// Loop until the totalWidth is reached
for (var x = 0; x < totalWidth; x += viewportWidth)
{
var newWidth = viewportWidth;
// Fix if the Width of the Element is too big
if (x + viewportWidth > totalWidth)
{
newWidth = totalWidth - x;
}
// Create and add the Rectangle
var currRect = new Rectangle(x, y, newWidth, newHeight);
rectangles.Add(currRect);
}
}
// Build the Image
var stitchedImage = new Bitmap(totalWidth, totalHeight);
// Get all Screenshots and stitch them together
var previous = Rectangle.Empty;
foreach (var rectangle in rectangles)
{
// Calculate the scrolling (if needed)
if (previous != Rectangle.Empty)
{
var xDiff = rectangle.Right - previous.Right;
var yDiff = rectangle.Bottom - previous.Bottom;
// Scroll
((IJavaScriptExecutor) driver).ExecuteScript(String.Format("window.scrollBy({0}, {1})", xDiff, yDiff));
}
// Take Screenshot
var screenshot = driver.TakeScreenshot();
// Build an Image out of the Screenshot
var screenshotImage = ScreenshotToImage(screenshot);
// Calculate the source Rectangle
var sourceRectangle = new Rectangle(viewportWidth - rectangle.Width, viewportHeight - rectangle.Height, rectangle.Width, rectangle.Height);
// Copy the Image
using (var graphics = Graphics.FromImage(stitchedImage))
{
graphics.DrawImage(screenshotImage, rectangle, sourceRectangle, GraphicsUnit.Pixel);
}
// Set the Previous Rectangle
previous = rectangle;
}
return stitchedImage;
}
private static Image ScreenshotToImage(Screenshot screenshot)
{
Image screenshotImage;
using (var memStream = new MemoryStream(screenshot.AsByteArray))
{
screenshotImage = Image.FromStream(memStream);
}
return screenshotImage;
}

It appears as though full-screen screenshots are not yet implemented in the ChromeDriver, due to some inaccuracies in its previous implementation.
Source: https://code.google.com/p/chromedriver/issues/detail?id=294
I have recently written a Selenium based application to test an Internet Explorer UI and found that:
Taking screenshots with selenium was not as quick as using .NET, and
Selenium is unable to take screenshots when dialog boxes are present. This was a major drawback, as I needed to identify unexpected dialogs during interaction with the pages.
Investigate using the Graphics.CopyFromScreen method in System.Drawing as an alternative solution until the feature is implemented in Chrome. Once you have tried .the Net approach however, I don't think you will look back =]

I stumbled accross the same problem and ChromeDriver2 just does not support it.
So I created a little script which scrolls thru the page, takes screenshots and stitches everything together.
You can find the script in my blog post here:
http://dev.flauschig.ch/wordpress/?p=341

Two image compare using win32 api c#

i have two images and i want to compare two image and want to get difference. i search google and found a link from where i copy paste the code for image comparison using win32 api.
so this is the url
http://blog.bobcravens.com/2009/04/create-a-remote-desktop-viewer-using-c-and-wcf/
here i am pasting the code.
private void button1_Click(object sender, EventArgs e)
{
Bitmap _prevBitmap = new Bitmap(#"d:\prev.jpg");
Bitmap _newBitmap = new Bitmap(#"d:\current.jpg");
Rectangle bounds = GetBoundingBoxForChanges(_prevBitmap, _newBitmap);
if (bounds == Rectangle.Empty)
{
}
Bitmap diff = new Bitmap(bounds.Width, bounds.Height);
Graphics g = Graphics.FromImage(diff);
g.DrawImage(_newBitmap, 0, 0, bounds, GraphicsUnit.Pixel);
g.Dispose();
// Set the current bitmap as the previous to prepare
// for the next screen capture.
//
diff.Save(#"d:\diff.bmp");
//return diff;
}
private Rectangle GetBoundingBoxForChanges(Bitmap _prevBitmap, Bitmap _newBitmap)
{
// The search algorithm starts by looking
// for the top and left bounds. The search
// starts in the upper-left corner and scans
// left to right and then top to bottom. It uses
// an adaptive approach on the pixels it
// searches. Another pass is looks for the
// lower and right bounds. The search starts
// in the lower-right corner and scans right
// to left and then bottom to top. Again, an
// adaptive approach on the search area is used.
//
// Note: The GetPixel member of the Bitmap class
// is too slow for this purpose. This is a good
// case of using unsafe code to access pointers
// to increase the speed.
//
// Validate the images are the same shape and type.
//
if (_prevBitmap.Width != _newBitmap.Width ||
_prevBitmap.Height != _newBitmap.Height ||
_prevBitmap.PixelFormat != _newBitmap.PixelFormat)
{
// Not the same shape...can't do the search.
//
return Rectangle.Empty;
}
// Init the search parameters.
//
int width = _newBitmap.Width;
int height = _newBitmap.Height;
int left = width;
int right = 0;
int top = height;
int bottom = 0;
BitmapData bmNewData = null;
BitmapData bmPrevData = null;
try
{
// Lock the bits into memory.
//
bmNewData = _newBitmap.LockBits(
new Rectangle(0, 0, _newBitmap.Width, _newBitmap.Height),
ImageLockMode.ReadOnly, _newBitmap.PixelFormat);
bmPrevData = _prevBitmap.LockBits(
new Rectangle(0, 0, _prevBitmap.Width, _prevBitmap.Height),
ImageLockMode.ReadOnly, _prevBitmap.PixelFormat);
// The images are ARGB (4 bytes)
//
int numBytesPerPixel = 4;
// Get the number of integers (4 bytes) in each row
// of the image.
//
int strideNew = bmNewData.Stride / numBytesPerPixel;
int stridePrev = bmPrevData.Stride / numBytesPerPixel;
// Get a pointer to the first pixel.
//
// Note: Another speed up implemented is that I don't
// need the ARGB elements. I am only trying to detect
// change. So this algorithm reads the 4 bytes as an
// integer and compares the two numbers.
//
System.IntPtr scanNew0 = bmNewData.Scan0;
System.IntPtr scanPrev0 = bmPrevData.Scan0;
// Enter the unsafe code.
//
unsafe
{
// Cast the safe pointers into unsafe pointers.
//
int* pNew = (int*)(void*)scanNew0;
int* pPrev = (int*)(void*)scanPrev0;
// First Pass - Find the left and top bounds
// of the minimum bounding rectangle. Adapt the
// number of pixels scanned from left to right so
// we only scan up to the current bound. We also
// initialize the bottom & right. This helps optimize
// the second pass.
//
// For all rows of pixels (top to bottom)
//
for (int y = 0; y < _newBitmap.Height; ++y)
{
// For pixels up to the current bound (left to right)
//
for (int x = 0; x < left; ++x)
{
// Use pointer arithmetic to index the
// next pixel in this row.
//
if ((pNew + x)[0] != (pPrev + x)[0])
{
// Found a change.
//
if (x < left)
{
left = x;
}
if (x > right)
{
right = x;
}
if (y < top)
{
top = y;
}
if (y > bottom)
{
bottom = y;
}
}
}
// Move the pointers to the next row.
//
pNew += strideNew;
pPrev += stridePrev;
}
// If we did not find any changed pixels
// then no need to do a second pass.
//
if (left != width)
{
// Second Pass - The first pass found at
// least one different pixel and has set
// the left & top bounds. In addition, the
// right & bottom bounds have been initialized.
// Adapt the number of pixels scanned from right
// to left so we only scan up to the current bound.
// In addition, there is no need to scan past
// the top bound.
//
// Set the pointers to the first element of the
// bottom row.
//
pNew = (int*)(void*)scanNew0;
pPrev = (int*)(void*)scanPrev0;
pNew += (_newBitmap.Height - 1) * strideNew;
pPrev += (_prevBitmap.Height - 1) * stridePrev;
// For each row (bottom to top)
//
for (int y = _newBitmap.Height - 1; y > top; y--)
{
// For each column (right to left)
//
for (int x = _newBitmap.Width - 1; x > right; x--)
{
// Use pointer arithmetic to index the
// next pixel in this row.
//
if ((pNew + x)[0] != (pPrev + x)[0])
{
// Found a change.
//
if (x > right)
{
right = x;
}
if (y > bottom)
{
bottom = y;
}
}
}
// Move up one row.
//
pNew -= strideNew;
pPrev -= stridePrev;
}
}
}
}
catch (Exception ex)
{
int xxx = 0;
}
finally
{
// Unlock the bits of the image.
//
if (bmNewData != null)
{
_newBitmap.UnlockBits(bmNewData);
}
if (bmPrevData != null)
{
_prevBitmap.UnlockBits(bmPrevData);
}
}
// Validate we found a bounding box. If not
// return an empty rectangle.
//
int diffImgWidth = right - left + 1;
int diffImgHeight = bottom - top + 1;
if (diffImgHeight < 0 || diffImgWidth < 0)
{
// Nothing changed
return Rectangle.Empty;
}
// Return the bounding box.
//
return new Rectangle(left, top, diffImgWidth, diffImgHeight);
}
when GetBoundingBoxForChanges() call then i am getting error and error message is Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
error occur at this code if ((pNew + x)[0] != (pPrev + x)[0])
so i am not being able to find out the reason. how to fix this error. please guide. thanks

bmNewData = _newBitmap.LockBits(...., _newBitmap.PixelFormat);
This algorithm implicitly assumes that a pixel has 4 bytes and can be addressed with an int*. It however fails to provide that guarantee. Asking for _newBitmap.PixelFormat in LockBits() is not sufficient, that just asks for the same format that the original image used. You'll get a hard crash if the images are 24bpp for example, very common.
Explicitly ask for 32bppArgb instead.

Instead of the Win32 API you could use a managed image processing library such as AForge.NET. In the documentation look for the AForge.Imaging.Filters.Difference class. It works with Bitmap objects so you will have to make minimal changes to your program.
Bitmap overlayImage;
Bitmap sourceImage;
//ToDo: Load the two images.
// Create filter.
Difference filter = new Difference(overlayImage);
// Apply the filter and return a new bitmap that is the difference between the source and overlay images.
Bitmap resultImage = filter.Apply(sourceImage);
// If you don't want a new image the you can apply the filter directly to the source image.
filter.ApplyInPlace(sourceImage);

This is the mechanism that I use to calculate image differences in C#. Note that it need to be compiled with the unsafe directive. Hope it helps:
using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using System.Drawing.Imaging;
using log4net;
namespace ImageDiff
{
public class ImageDifferences
{
private static ILog mLog = LogManager.GetLogger("ImageDifferences");
public static unsafe Bitmap PixelDiff(Image a, Image b)
{
if (!a.Size.Equals(b.Size)) return null;
if (!(a is Bitmap) || !(b is Bitmap)) return null;
return PixelDiff(a as Bitmap, b as Bitmap);
}
public static unsafe Bitmap PixelDiff(Bitmap a, Bitmap b)
{
Bitmap output = new Bitmap(
Math.Max(a.Width, b.Width),
Math.Max(a.Height, b.Height),
PixelFormat.Format32bppArgb);
Rectangle recta = new Rectangle(Point.Empty, a.Size);
Rectangle rectb = new Rectangle(Point.Empty, b.Size);
Rectangle rectOutput = new Rectangle(Point.Empty, output.Size);
BitmapData aData = a.LockBits(recta, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
BitmapData bData = b.LockBits(rectb, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
BitmapData outputData = output.LockBits(rectOutput, ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
try
{
byte* aPtr = (byte*)aData.Scan0;
byte* bPtr = (byte*)bData.Scan0;
byte* outputPtr = (byte*)outputData.Scan0;
int len = aData.Stride * aData.Height;
for (int i = 0; i < len; i++)
{
// For alpha use the average of both images (otherwise pixels with the same alpha won't be visible)
if ((i + 1) % 4 == 0)
*outputPtr = (byte)((*aPtr + *bPtr) / 2);
else
*outputPtr = (byte)~(*aPtr ^ *bPtr);
outputPtr++;
aPtr++;
bPtr++;
}
return output;
}
catch (Exception ex)
{
mLog.Error("Error calculating image differences: " + ex.Message);
return null;
}
finally
{
a.UnlockBits(aData);
b.UnlockBits(bData);
output.UnlockBits(outputData);
}
}
}
}

ASP.NET the best way to handle thumbnail images after uploading

I've read about creating a HttpHandler to call each time I want to show a thumbnail where it will perfom the resizing for me.
I've also heard about some other solutions, but I was wondering which solution would be the best for a social networking website where thumbnails are shown all over the place on each page and everywhere.
Would it be good to resize and save the image on the disk after the origianl file has been uploaded? What's the best way to reference these images?
Does anyone have any advice for me?
Thank you.

Would it be good to resize and save the image on the disk after the
origianl file has been uploaded? What's the best way to reference
these images?
Definitely, that's what Twitter does, for example, and most websites when thumbnails need to be displayed. This is time consuming. You don't want your user to sit idle while you do this on every image every time.
Store the thumbnails on disk and keep a reference to the thumbnails on the database. Or store them on the db itself. I don't want to get into that debate about disk vs DB but just don't resize them every time. It should be done ONCE.

Here is some resize Code if you need it, the first you set a max height and width and it created a thumbnail with the same aspect retio as the original that do now violate either max.
The second, if you know the actual size of the final image and don't need to worry about aspect ratios is fatr simpler.
public Image Thumbnail(Image FullsizeImage, int MaxHeight, int MaxWidth)
{
try
{
// This has to be here or for some reason this resize code will
// resize an internal Thumbnail and wil stretch it instead of shrinking
// the fullsized image and give horrible results
FullsizeImage.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipNone);
FullsizeImage.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipNone);
System.Drawing.Image NewImage;
if (!((MaxWidth < FullsizeImage.Width) || (MaxHeight < FullsizeImage.Height)))
NewImage = FullsizeImage;
else
{
float HeightRatio = 1;
float WidthRatio = 1;
HeightRatio = (float)FullsizeImage.Width / FullsizeImage.Height;
WidthRatio = (float)FullsizeImage.Height / FullsizeImage.Width;
float DrawHeight = (float)FullsizeImage.Height;
float DrawWidth = (float)FullsizeImage.Width;
if (MaxHeight < FullsizeImage.Height)
{
DrawHeight = (float)MaxHeight;
DrawWidth = MaxHeight * HeightRatio;
}
if (MaxWidth < DrawWidth)
{
DrawWidth = MaxWidth;
DrawHeight = DrawWidth * WidthRatio;
}
NewImage = FullsizeImage.GetThumbnailImage((int)(DrawWidth),
(int)(DrawHeight), null, IntPtr.Zero);
}
return NewImage;
// To return a byte array for saving in a db
//ms = new MemoryStream();
//NewImage.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
//NewImage.Dispose();
//FullsizeImage.Dispose();
//return ms.ToArray();
}
catch
{
return null;
}
finally
{
}
}
public Image Resize(Image OrigImage, int NewHeight, int NewWidth)
{
if (OrigImage != null)
{
Bitmap bmp = new Bitmap(OrigImage, new Size(NewWidth, NewHeight));
bmp.SetResolution(this.ImageResolution, this.ImageResolution);
Graphics g = Graphics.FromImage(bmp);
return bmp;
}
else
{
return null;
}
}