I'm using Webdriver with C#, the issue is that Webdriver can't test design, so i trying to make something like screenshot comparison, because i working on big site and every day it has new futures, new changes so i don't know where or on which browser design didn't broke. My question is do you know any tool, framework, project or some ideas?
I already wrote something but its not so perfect, my code makes screenshots over site on different browsers and versions, after that it's cutting them and compare it with demo-screen. If there are more than 5% for wrong pixels test is falling.
P.S. I'm a junior)
Some of my code if someone will need it in the future:
var screenshot = ((ITakesScreenshot)Global.Driver).GetScreenshot();
screenshot.SaveAsFile(dirHomePage + "homepage.png", ImageFormat.Png);
var mainImg = new Bitmap(File.OpenRead(dirHomePage + "homepage.png"));
var section = new Rectangle(new Point(0, 0), new Size(mainImg.Width, 153));
this.header = this.CropImage(mainImg, section);
this.header = this.ClearImage(this.header, Color.White, 264, mainImg.Width, 30, 60);
ImageComparison(this.header, this.demoHeader);
public Bitmap ClearImage(Bitmap source, Color color, int fromW, int toW, int fromH, int toH)
{
Bitmap bmp = source;
for (int i = fromW; i < toW; i++)
{
for (int j = fromH; j < toH; j++)
{
bmp.SetPixel(i, j, color);
}
}
return bmp;
}
public Bitmap CropImage(Bitmap source, Rectangle section)
{
Bitmap bmp = new Bitmap(section.Width, section.Height);
Graphics g = Graphics.FromImage(bmp);
g.DrawImage(source, 0, 0, section, GraphicsUnit.Pixel);
return bmp;
}
public void ImageComparison(Bitmap firstImg, Bitmap secondImg)
{
bool match = false;
int nrBadPixels = 0;
if (firstImg.Width.Equals(secondImg.Width) && firstImg.Height.Equals(secondImg.Height))
{
double acceptedProcent = (firstImg.Width * firstImg.Height) * 0.05;
for (int i = 0; i < firstImg.Width; i++)
{
for (int j = 0; j < firstImg.Height; j++)
{
string firstImgRef = firstImg.GetPixel(i, j).ToString();
string secondImgRef = secondImg.GetPixel(i, j).ToString();
if (firstImgRef != secondImgRef)
{
nrBadPixels++;
if (nrBadPixels > acceptedProcent)
{
match = false;
break;
}
match = true;
}
}
}
}
Assert.IsTrue(match);
}
I think you can give Sikuli a try.
Sikuli Script automates anything you see on the screen. It uses image
recognition to identify and control GUI components. It is useful when
there is no easy access to a GUI's internal or source code.
However, this is just for layout and stuff, if you need to test functionality based on DOM, Selenium WebDriver is still the best way to go. Moreover, it's more like Selenium IDE, rather than Selenium WebDriver C#. You might find this wiki page interesting, SikuliWebDriver.
I have wrote a small code snippet that solve Your problem. It scrolls to the element plus width and height and then cutout only the area of the element.
Look at my Blog
https://modern-coding-today.blogspot.com/2018/11/c-net-make-partial-screenshots-in.html
public static Bitmap GetElementScreenshot(IWebElement element) {
IWebDriver driver = ((IWrapsDriver)element).WrappedDriver;
Actions actions = new Actions(driver);
actions.MoveToElement(element);
actions.Perform();
IJavaScriptExecutor executor = (IJavaScriptExecutor)driver;
int yOffset = Convert.ToInt32(executor.ExecuteScript($"window.scrollBy({element.Size.Width}, {element.Size.Height}); return window.pageYOffset;"));
Screenshot sc = ((ITakesScreenshot)driver).GetScreenshot();
var img = Image.FromStream(new MemoryStream(sc.AsByteArray)) as Bitmap;
return img.Clone(new Rectangle(element.Location.X, element.Location.Y - yOffset, element.Size.Width, element.Size.Height), img.PixelFormat);
}
In my Blog You will find this code as a IWebDriver Extension also.
Related
I think I have a simple problem that seems very hard for me to figure out - how to check if an image I get from Clipboard.GetImage() uses transparency. If it does then I will show it in a PictureBox with the transparent background.
I copy the picture directly from the application - e.g. the browser or one of the Windows image viewers. Pasting the picture to e.g. Word will get the transparent background.
I am using this code:
// Check if the picture in clipboard has any transparency (alpha channel != 255)
Bitmap img = new Bitmap(Clipboard.GetImage());
for (int y = 0; y < img.Height; ++y)
{
for (int x = 0; x < img.Width; ++x)
{
if (img.GetPixel(x, y).A != 255)
{
Debug.WriteLine("Picture is transparent - set breakpoint here");
}
}
}
...
// Show the picture with transparent background - this works fine
img.MakeTransparent(img.GetPixel(0,0));
myPictureBox.Image = (Image)img;
I am trying with various pictures found on the net and I can copy/paste those pictures with the transparent background so I know they are transparent but no pictures will trigger the Debug.WriteLine and all values equals 255?
Though I recognize this has been asked before then I must be doing something wrong since this simple example does not work? Also they are old so maybe there is a new and better way? I have tried to find other solutions besides these:
Detecting if a PNG image file is a Transparent image?
Check to see if image is transparent
.. and more also not from StackOverflow. I have seen both really simple solutions and horrofying complex ones - but still none of them seems to work.
is this because the clipboard object cannot see the transparency or .. ?
I ended up this solution, based on the comment from #Jeff, CopyTransparentImages. This will get the (real?) image from the clipboard (which will also include the alpha channel) and then I will check if the image contains any transparency afterwards. If it does then I will make the image background color transparent, according to my original question, before I show it in a PictureBox.
// Get the image formats from clipboard and check if it is transparent
Image imgCopy = GetImageFromClipboard();
bool isClipboardImageTransparent = IsImageTransparent(imgCopy);
if (isClipboardImageTransparent)
{
...
}
// Get the real image from clipboard (this supports the alpha channel)
private Image GetImageFromClipboard()
{
if (Clipboard.GetDataObject() == null) return null;
if (Clipboard.GetDataObject().GetDataPresent(DataFormats.Dib))
{
// Sometimes getting the image data fails and results in a "System.NullReferenceException" error - probably because clipboard handling also can be messy and complex
byte[] dib;
try
{
dib = ((System.IO.MemoryStream)Clipboard.GetData(DataFormats.Dib)).ToArray();
}
catch (Exception ex)
{
return Clipboard.ContainsImage() ? Clipboard.GetImage() : null;
}
var width = BitConverter.ToInt32(dib, 4);
var height = BitConverter.ToInt32(dib, 8);
var bpp = BitConverter.ToInt16(dib, 14);
if (bpp == 32)
{
var gch = GCHandle.Alloc(dib, GCHandleType.Pinned);
Bitmap bmp = null;
try
{
var ptr = new IntPtr((long)gch.AddrOfPinnedObject() + 40);
bmp = new Bitmap(width, height, width * 4, System.Drawing.Imaging.PixelFormat.Format32bppArgb, ptr);
return new Bitmap(bmp);
}
finally
{
gch.Free();
if (bmp != null) bmp.Dispose();
}
}
}
return Clipboard.ContainsImage() ? Clipboard.GetImage() : null;
}
// Check if the image contains any transparency
private static bool IsImageTransparent(Image image)
{
Bitmap img = new Bitmap(image);
for (int y = 0; y < img.Height; ++y)
{
for (int x = 0; x < img.Width; ++x)
{
if (img.GetPixel(x, y).A != 255)
{
return true;
}
}
}
return false;
}
At least this is working for me :-)
I am using imagecomparer in my mobile test project and I am able to compare a baseline image to a current screenshot, but the problem comes in where there is a section of the screenshot that is always changing and I would like to exclude that part from being compared. Here is my code:
private bool RunVisualCheck(string screen, string resultsPath, string baseline = "baseline.jpeg", string screenshot = "screenshot.jpeg")
{
GetScreenshot(resultsPath + screenshot);
var baselineImage = Image.FromFile(resultsPath + baseline);
var actualImage = Image.FromFile(resultsPath + screenshot);
Image diffImage;
int ignoreTop = 64;
var compareArea = new List<ToleranceRectangle>
{
new ToleranceRectangle()
{
Rectangle = new Rectangle(0,ignoreTop,baselineImage.Width, baselineImage.Height - ignoreTop),
Difference = new ColorDifference()
}
};
bool goodCompare = ImageComparer.Compare(actualImage, baselineImage, compareArea, out diffImage);
if (!goodCompare)
{
diffImage.Save(resultsPath + "diffImage.jpeg");
}
return goodCompare;
}
private void GetScreenshot(string pathFile)
{
System.Threading.Thread.Sleep(2000); // Temp fix to wait until page loads
var srcFiler = ((ITakesScreenshot)mobileDriver).GetScreenshot();
srcFiler.SaveAsFile(pathFile, ImageFormat.Jpeg);
}
Here is an example (not the app being tested) where I would like to exclude the area inside the red rectangle from the overall screenshot from being compared.
Mobile Screenshot Example
Is there an easy way to do this?
Found a better approach than trying to exclude a section from being compared. Thanks to a coworkers suggestion, I am blacking out the sections that do not need comparing and then saving this image. Doing this on the baseline image and the screenshot will have the same effect as excluding it altogether. Here is the code:
Image image = Image.FromFile(#"C:\Screenshots\Screenshot.jpeg");
using (Graphics g = Graphics.FromImage(image))
{
SolidBrush brush = new SolidBrush(Color.Black);
Size size = new Size(image.Width, 64);
Point point = new Point(0, 0);
Rectangle rectangle;
rectangle = new Rectangle(point, size);
g.FillRectangle(brush, rectangle);
}
image.Save(#"C:\Screenshots\Screenshot.jpeg");
It seems, that if image, that is used for icon for Windows shortcuts, doesn't have aspect ratio 1:1, it will look stretched.
Left one is how it actually looks, and right one is how it should look.
I'm creating shortcut and icon programmatically from image, so I want to fix image, so it will have correct aspect ratio, but image will not look stretched. This can be achieved by adding some padding to image.
As for now, I'm simply copying image to new bitmap with correct aspect ratio, but filling new area with transparent pixels
public static Bitmap FixBitmapAspectRatio(Bitmap sourceBitmap)
{
if (sourceBitmap.Width.Equals(sourceBitmap.Height))
return sourceBitmap;
int size;
bool horizontallyOriented;
if (sourceBitmap.Width > sourceBitmap.Height)
{
horizontallyOriented = true;
size = sourceBitmap.Width;
}
else
{
horizontallyOriented = false;
size = sourceBitmap.Height;
}
var sizeDifference = Math.Abs(sourceBitmap.Width - sourceBitmap.Height);
var newBitmap = new Bitmap(size, size);
var transparentColor = Color.FromArgb(0, 0, 0, 0);
for (int i = 0; i < size; i++)
{
for (int j = 0; j < size; j++)
{
if (horizontallyOriented)
{
if (i < sizeDifference / 2 || i >= sizeDifference / 2 + sourceBitmap.Height)
{
newBitmap.SetPixel(j, i, transparentColor);
}
else
{
var originalPixel = sourceBitmap.GetPixel(j, i - sizeDifference / 2);
newBitmap.SetPixel(j, i, originalPixel);
}
}
else
{
if (i < sizeDifference / 2 || i >= sizeDifference / 2 + sourceBitmap.Width)
{
newBitmap.SetPixel(i, j, transparentColor);
}
else
{
var originalPixel = sourceBitmap.GetPixel(i - sizeDifference / 2, j);
newBitmap.SetPixel(i, j, originalPixel);
}
}
}
}
return newBitmap;
}
But I don't know, if I'm inventing a wheel. Is there any way to do this by means by standard libraries, or maybe easier way to achieve what I need?
You really don't want to set individual pixels :)
Instead, have a look at the Graphics class, in particular Graphics.FromImage (that's where you paint to) and Graphics.DrawImage (that's how you paint the scaled image).
Below is the code for a method which I am using for detecting faces from Kinect Feed and then setting the pixels from the face into a new image. . It is triggered by a Gesture which is done by GestureFlag . The Detect method which I am calling from the FaceDetection class is taken from the EMGU CV sample.
public string Detect(WriteableBitmap colorBitmap, int GestureFlag)
{
if(GestureFlag!=0)
{
List<Rectangle> faces = new List<Rectangle>();
Bitmap bitface = BitmapFromSource(colorBitmap);
Image<Bgr, Byte> image = new Image<Bgr, Byte>(bitface);
FaceDetection.Detect(image, "haarcascade_frontalface_default.xml",faces);
Bitmap img = new Bitmap(#"C:\Users\rama\Downloads\kinect-2-background-removal-master\KinectBackgroundRemoval\Assets\emptyimage.png");
img = ResizeImage(img, 1540, 1980);
int high = image.Height;
int width = image.Width;
for (int i = 0; i < width; i++)
{
for (int j = 0; j < high; j++)
{
Bgr pixel = image[j,i];
System.Drawing.Point p = new System.Drawing.Point(j,i);
if (faces[0].Contains(p) && i<1540 && j<1980)
{
img.SetPixel(i, j, System.Drawing.Color.FromArgb(255, (int)pixel.Blue, (int)pixel.Green,(int) pixel.Red));
}
}
}
count++;
key = count.ToString() + "rich.jpg";
image.Save(#"C:\Users\rama\Downloads\kinect-2-background-removal-master\KinectBackgroundRemoval\Assets\"+key);
img.Save(#"C:\Users\rama\Downloads\kinect-2-background-removal-master\KinectBackgroundRemoval\"+key);
bool status = UploadToS3("nitish2", key, #"C:\Users\rama\Downloads\kinect-2-background-removal-master\KinectBackgroundRemoval\"+key);
var FBClient = new FacebookClient();
var UploadToFacebook = new FacebookUpload(FBClient, key);
UploadToFacebook.Show();
GestureFlag = 0;
}
return key;
}
The problem I'm running into is that an entirely different set of pixels gets printed on the new image which I'm saving.
Basically i think that the problem is here:
FaceDetection.Detect(image, "haarcascade_frontalface_default.xml",faces);
for (int i = 0; i < width; i++)
{
for (int j = 0; j < high; j++)
{
Bgr pixel = image[j,i];
System.Drawing.Point p = new System.Drawing.Point(j,i);
if (faces[0].Contains(p) && i<1540 && j<1980)
{
img.SetPixel(i, j, System.Drawing.Color.FromArgb(255, (int)pixel.Blue, (int)pixel.Green,(int) pixel.Red));
}
}
}
So can someone please point out where I'm going wrong?
Thank you very much.
EDITS : I've tried putting flags like faces[0]!=null which should take care of whether FaceDetection.Detect is actually returning anything but still I'm getting the same result.
I've also tried saving the ColorBitmap and testing it against the EMGU CV sample and it detects the faces in the image easily.
EDIT 2: So I've cross checked the co-ordinates of the rectangle which is being printed with the co-ordinates of the face being detected and the values of the Rectangle() being populated. They turn out to be almost same as far as I can see. So no luck there.
I don't know what else I can try for debugging.
If someone could point that out that would be great.
Thanks!
Ok I got it to work finally.
The problem was the Point[j,i] which should have been Point[i,j] . Also my FaceDetection method proved to be a little erratic and I had to fix that too to finally properly debug my code.
When I take screenshots with ChromeDriver I get screens with the size of my viewport.
When I take screenshots with FirefoxDriver I get what I want, which is a full screen print of a website.
ChromeDriver is declared like this:
IWebDriver driver = new ChromeDriver();
FirefoxDriver is declared like this:
IWebDriver driver = new FirefoxDriver();
Both drivers execute identical code:
driver.Manage().Window.Maximize();
driver.Navigate().GoToUrl(url);//url is a string variable
ITakesScreenshot screenshotDriver = driver as ITakesScreenshot;
Screenshot screenshot = screenshotDriver.GetScreenshot();
screenshot.SaveAsFile("c:/test.png", ImageFormat.Png);
ChromeDriver's test.png is of 1920x1099 resolution and contains only the browser viewport.
FirefoxDriver's test.png is of 1903x16559 resolution and contains the whole page.
I know that GetScreenshot() method doesn't return identical resolution sizes because it has slightly different implementations in IEDriver, FirefoxDriver, OperaDriver, ChromeDriver.
My questions are:
Why is there such difference between ChromeDriver's and FirefoxDriver's .GetScreenshot() method, even tho they use an identical interface (ITakesScreenshot)?
Is there a way to make ChromeDriver's GetScreenshot() method return the whole webpage screen instead of just the viewport?
we can't get the entire page screenshot with ChromeDriver2, we need to go for manual implementation.I have modified a method with is available in a blog which works fine with ChromeDriver.
use this method as following :
private IWebDriver _driver = new ChromeDriver(CHROME_DRIVER_PATH);
screenshot.SaveAsFile(saveFileName, ImageFormat.Jpeg);
public Bitmap GetEntereScreenshot()
{
Bitmap stitchedImage = null;
try
{
long totalwidth1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return document.body.offsetWidth");//documentElement.scrollWidth");
long totalHeight1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return document.body.parentNode.scrollHeight");
int totalWidth = (int)totalwidth1;
int totalHeight = (int)totalHeight1;
// Get the Size of the Viewport
long viewportWidth1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return document.body.clientWidth");//documentElement.scrollWidth");
long viewportHeight1 = (long)((IJavaScriptExecutor)_driver).ExecuteScript("return window.innerHeight");//documentElement.scrollWidth");
int viewportWidth = (int)viewportWidth1;
int viewportHeight = (int)viewportHeight1;
// Split the Screen in multiple Rectangles
List<Rectangle> rectangles = new List<Rectangle>();
// Loop until the Total Height is reached
for (int i = 0; i < totalHeight; i += viewportHeight)
{
int newHeight = viewportHeight;
// Fix if the Height of the Element is too big
if (i + viewportHeight > totalHeight)
{
newHeight = totalHeight - i;
}
// Loop until the Total Width is reached
for (int ii = 0; ii < totalWidth; ii += viewportWidth)
{
int newWidth = viewportWidth;
// Fix if the Width of the Element is too big
if (ii + viewportWidth > totalWidth)
{
newWidth = totalWidth - ii;
}
// Create and add the Rectangle
Rectangle currRect = new Rectangle(ii, i, newWidth, newHeight);
rectangles.Add(currRect);
}
}
// Build the Image
stitchedImage = new Bitmap(totalWidth, totalHeight);
// Get all Screenshots and stitch them together
Rectangle previous = Rectangle.Empty;
foreach (var rectangle in rectangles)
{
// Calculate the Scrolling (if needed)
if (previous != Rectangle.Empty)
{
int xDiff = rectangle.Right - previous.Right;
int yDiff = rectangle.Bottom - previous.Bottom;
// Scroll
//selenium.RunScript(String.Format("window.scrollBy({0}, {1})", xDiff, yDiff));
((IJavaScriptExecutor)_driver).ExecuteScript(String.Format("window.scrollBy({0}, {1})", xDiff, yDiff));
System.Threading.Thread.Sleep(200);
}
// Take Screenshot
var screenshot = ((ITakesScreenshot)_driver).GetScreenshot();
// Build an Image out of the Screenshot
Image screenshotImage;
using (MemoryStream memStream = new MemoryStream(screenshot.AsByteArray))
{
screenshotImage = Image.FromStream(memStream);
}
// Calculate the Source Rectangle
Rectangle sourceRectangle = new Rectangle(viewportWidth - rectangle.Width, viewportHeight - rectangle.Height, rectangle.Width, rectangle.Height);
// Copy the Image
using (Graphics g = Graphics.FromImage(stitchedImage))
{
g.DrawImage(screenshotImage, rectangle, sourceRectangle, GraphicsUnit.Pixel);
}
// Set the Previous Rectangle
previous = rectangle;
}
}
catch (Exception ex)
{
// handle
}
return stitchedImage;
}
I cleaned up #Selvantharajah Roshanth's answer and added a check so that it won't try to stitch together screenshots that already fit in the viewport.
public Image GetEntireScreenshot()
{
// Get the total size of the page
var totalWidth = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return document.body.offsetWidth"); //documentElement.scrollWidth");
var totalHeight = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return document.body.parentNode.scrollHeight");
// Get the size of the viewport
var viewportWidth = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return document.body.clientWidth"); //documentElement.scrollWidth");
var viewportHeight = (int) (long) ((IJavaScriptExecutor) driver).ExecuteScript("return window.innerHeight"); //documentElement.scrollWidth");
// We only care about taking multiple images together if it doesn't already fit
if (totalWidth <= viewportWidth && totalHeight <= viewportHeight)
{
var screenshot = driver.TakeScreenshot();
return ScreenshotToImage(screenshot);
}
// Split the screen in multiple Rectangles
var rectangles = new List<Rectangle>();
// Loop until the totalHeight is reached
for (var y = 0; y < totalHeight; y += viewportHeight)
{
var newHeight = viewportHeight;
// Fix if the height of the element is too big
if (y + viewportHeight > totalHeight)
{
newHeight = totalHeight - y;
}
// Loop until the totalWidth is reached
for (var x = 0; x < totalWidth; x += viewportWidth)
{
var newWidth = viewportWidth;
// Fix if the Width of the Element is too big
if (x + viewportWidth > totalWidth)
{
newWidth = totalWidth - x;
}
// Create and add the Rectangle
var currRect = new Rectangle(x, y, newWidth, newHeight);
rectangles.Add(currRect);
}
}
// Build the Image
var stitchedImage = new Bitmap(totalWidth, totalHeight);
// Get all Screenshots and stitch them together
var previous = Rectangle.Empty;
foreach (var rectangle in rectangles)
{
// Calculate the scrolling (if needed)
if (previous != Rectangle.Empty)
{
var xDiff = rectangle.Right - previous.Right;
var yDiff = rectangle.Bottom - previous.Bottom;
// Scroll
((IJavaScriptExecutor) driver).ExecuteScript(String.Format("window.scrollBy({0}, {1})", xDiff, yDiff));
}
// Take Screenshot
var screenshot = driver.TakeScreenshot();
// Build an Image out of the Screenshot
var screenshotImage = ScreenshotToImage(screenshot);
// Calculate the source Rectangle
var sourceRectangle = new Rectangle(viewportWidth - rectangle.Width, viewportHeight - rectangle.Height, rectangle.Width, rectangle.Height);
// Copy the Image
using (var graphics = Graphics.FromImage(stitchedImage))
{
graphics.DrawImage(screenshotImage, rectangle, sourceRectangle, GraphicsUnit.Pixel);
}
// Set the Previous Rectangle
previous = rectangle;
}
return stitchedImage;
}
private static Image ScreenshotToImage(Screenshot screenshot)
{
Image screenshotImage;
using (var memStream = new MemoryStream(screenshot.AsByteArray))
{
screenshotImage = Image.FromStream(memStream);
}
return screenshotImage;
}
It appears as though full-screen screenshots are not yet implemented in the ChromeDriver, due to some inaccuracies in its previous implementation.
Source: https://code.google.com/p/chromedriver/issues/detail?id=294
I have recently written a Selenium based application to test an Internet Explorer UI and found that:
Taking screenshots with selenium was not as quick as using .NET, and
Selenium is unable to take screenshots when dialog boxes are present. This was a major drawback, as I needed to identify unexpected dialogs during interaction with the pages.
Investigate using the Graphics.CopyFromScreen method in System.Drawing as an alternative solution until the feature is implemented in Chrome. Once you have tried .the Net approach however, I don't think you will look back =]
I stumbled accross the same problem and ChromeDriver2 just does not support it.
So I created a little script which scrolls thru the page, takes screenshots and stitches everything together.
You can find the script in my blog post here:
http://dev.flauschig.ch/wordpress/?p=341