I'm trying to test a swipe across an image gallery on a responsive website. On a mobile device it would be a finger swipe, using a desktop browser it functions the same with a mouse click/hold/drag. It's the desktop browser functionality I'm trying to replicate.
The code I have is:
Driver.Navigate().GoToUrl(Properties.Settings.Default.TestPage);
var mainImage = Driver.FindElement(By.CssSelector(".main-image"));
var imageAcross = (mainImage.Size.Width / 4 ) * 3;
var imageDown = mainImage.Size.Height/2;
var builder = new Actions(Driver);
builder.MoveToElement(mainImage, imageAcross, imageDown).ClickAndHold().Build().Perform();
TakeScreenshot(TestContext.CurrentContext.Test.Name);
builder.MoveByOffset(-imageDown, 0).Release().Build().Perform();
TakeScreenshot(TestContext.CurrentContext.Test.Name);
The TakeScreenshot function, as the name suggests, saves a screenshot. I take one before and after the move.
The first builder.MoveToElement places the cursor where I would expect and after the builder.MoveByOffset the cursor has moved as expected. However the swipe didn't change the gallery image. It's as if the ClickAndHold was ignored.
The CSS element .main-image is a div tag that has Javascript attached that changes an image within it on the swipe. The browser I'm using for testing is PhantomJS.
Is there something I am doing wrong? Or any suggestions on how to get the swipe to work?
Related
I am having a great deal of trouble dragging and dropping an element on a web page on an Android tablet.
The code I am using is this:
IWebElement dragFrom = driver.FindElement(By.XPath(dragElement));
IWebElement dragTo = driver.FindElement(By.XPath(dropElement));
Point fromPoint = dragFrom.Location;
Point toPoint = dragTo.Location;
((AndroidDriver<AndroidElement>)driver).Context = "NATIVE_APP";
new TouchAction((AndroidDriver<AndroidElement>)driver).
LongPress(fromPoint.X, fromPoint.Y).
Wait(3000).
MoveTo(toPoint.X, toPoint.Y).
Release().Perform();
((AndroidDriver<AndroidElement>)driver).Context = "CHROMIUM";
However when I use this the very top of the browser gets tapped, either the browser tab or maybe it pulls down slightly from the top slide menu on the device.
I've looked at the X and Y to and from points and they are not the top left of the screen (X 740, Y 50)
Originally I just used Actions but I get some crazy error message with it "OpenQA.Selenium.WebDriverException: 'An unknown server-side error occurred while processing the command. Original error: Could not proxy. Proxy error: Could not proxy command to remote server. Original error: 501 - "unimplemented command: "
Actions actions = new Actions(driver);
actions.DragAndDrop(
driver.FindElement(By.XPath(dragElement)),
driver.FindElement(By.XPath(dropElement)))
.Release()
.Build().Perform();
I also used the below code, but now an error tells me that Swipe does not exist
TouchAction action = new TouchAction((IPerformsTouchActions)driver);
action.Press(startX, startY).Wait(1000).MoveTo(endX, endY).Release().Perform();
((AndroidDriver<IWebElement>)driver).Swipe(startX, startY, endX, endY, 100);
Does anyone have a solid way of dragging and dropping an element on a webpage when using an Android device? Or know what I've done wrong in the above code?
It appears that when appium taps the screen, its not taking into account the size of the browser header, so when it taps Y 24 for example, its going by the total screen size, and not the web page screen size. So it taps 24 pixels down from the top of the phone screen, and not from the top of the web page.
Thanks in advance.
I have managed to get this working. After updating the Appium desktop programme to 1.19 I could get Actions working again.
So I used this
Actions actions = new Actions(driver);
actions.DragAndDrop(
driver.FindElement(By.XPath(dragElement)),
driver.FindElement(By.XPath(dropElement)))
.Release()
.Build().Perform();
Thread.Sleep(2000);
actions = null;
So I have searched and can't seem to find the right resources or tools for what I am wanting to do so I thought I'd ask for some help. Below is what I am wanting to do.
I am making a .Net core application that will simulate some key strokes and that is all working fine, but I want to also move the mouse based on where certain text appears on the screen, however, I will not have control over the application where the text appears, so I will need to somehow either be streaming the desktop view and using something to analyze it, or constantly taking a screen shot every 10 seconds or so and analyzing that screen shot if the text is there and then move the mouse to that position of text and simulate a left click.
I am not really confident in the screenshot approach as I am not sure how I'd get the mouse coordinates, so I have a feeling I will need to feed a desktop stream into something in order to get the image I am analyzing and then overlay my own transparent overlay on top of that in order to move the cursor to appropriate position.
I hope this makes sense and any libraries or whatever that is recommended to do this is appreciated. I know how to move the cursor on the screen and left click, just haven't found on google the right library for analyzing a desktop screen in real time and getting coordinates based on searching conditions.
Thank you everyone.
So I figured out what I needed to do. Trying to match the text with and OCR proved to be impossible as the text was layered into an image where they actively tried to prevent detection, this is because its a video game and they don't want automation I guess. So what I did was determine I will always know what the button is going to look like, so I took a screen shot of the button and now I take screenshots of the game every 10 seconds and then search that image for my button template image. If the image has a match, then I generate coordinates and send mouse there and click it. Below is the code I used and you will need the libraries from AForge to use this code.
Bitmap sourceImage = (Bitmap)Bitmap.FromFile(#"C:\testjpg.jpg");
Bitmap template = (Bitmap)Bitmap.FromFile(#"C:\buttonjpg.jpg");
// create template matching algorithm's instance
//Resize value
var resizePercent = 0.4;
//Resizing images to increase process time. Please note this will result in skewed coordinates.
//You must divide the coordinates by your resizePercent to get native coordinates taken from screen shots.
sourceImage = new ResizeBicubic((int)(sourceImage.Width * resizePercent), (int)(sourceImage.Height * resizePercent)).Apply(sourceImage);
template = new ResizeBicubic((int)(template.Width * resizePercent), (int)(template.Height * resizePercent)).Apply(template);
// (set similarity threshold to 0.951f = 95.1%)
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.951f);
// find all matchings with specified above similarity
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings
BitmapData data = sourceImage.LockBits(
new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{
Drawing.Rectangle(data, m.Rectangle, Color.White);
Console.WriteLine(m.Rectangle.Location.ToString());
var x = m.Rectangle.Location.X;
var y = m.Rectangle.Location.Y;
//Fixing the coordinates to reflex the origins of the original screenshot, not the resized one.
var xResized = (int)(x / resizePercent);
var yResized = (int)(y / resizePercent);
// do something else with matching
}
sourceImage.UnlockBits(data);
I did notice my coordinates were off usually around ~4.5%-8% from dead center(the button still gets clicked, but I want a perfect center click), so for me I am calculating a range of coordinates, which in this range are not very many to ensure the button I want to click is clicked and not missed and just having program click all the potential coordinates.
So I hope this helps someone!
Kind Regards,
Aaron
Depending on the application there are a number of approaches you might want to try.
For example, using Application Insights and UI Automation, you might be able to capture the text directly.
If you have the exe file, then you might be able to decompile it with something like dotPeek and then use Visual Studio's Live debugging feature to access the text.
You might be able to inspect the heap of the running process via a tool like HeapMemView.
Optical Character Recognition might be an option worth considering. In that case Tesseract is an open source OCR engine worth considering. This can be used in conjunction with screenshots to extract text.
I use this code as mentioned in document of play services for Unity3d.
BannerView bannerView = new BannerView(adUnitId, adSize, AdPosition.Bottom);
The code works, but here is the problem:
For Samsung phones I do not have any problem because, back-home buttons are not on screen. But I have another physical device for testing: LG
For Lg, I got a problem. Back, home etc buttons on screen, so when I use AdPosition.Bottom, It deploys it in bottom but make a space for buttons. So it is not appear in exact bottom.
Do you have suggestions?
I have a canvas and there i need to move the mouse from one point to another point to draw a rectangle. CapabilityType.HasNativeEvents is set to false. I'm using IE11 and .Net. The issue is while using selenium grid in continuous integration machine but works fine in local .
I tried this but it does not work. Any clue?
Actions builder = new Actions(_webDriver);
builder.MoveToElement(Canvas, 800, 250).Click().Build().Perform();
System.Threading.Thread.Sleep(2000);
builder.ClickAndHold(Canvas).MoveByOffset(500, -30).Build().Perform();
How can I write or draw controls to the Windows 7 preview area using C#? For an example of what I am talking about, open Windows Media Player in Windows 7 and play a song. While the song is playing, minimize Windows Media player, then hover your mouse over the Windows Media Player icon and you will see a pause, rewind and fast forward button just below the actual Media Player preview window. How can I duplicate this kind of behavior in C#?
You're looking for Windows 7 Thumbnail Toolbars:
Thumbnail toolbars provide a mini
"remote-control" opportunity for a
window from its thumbnail. For
example, to switch to the next song in
Windows Media Player you don't need to
use the clumsy Media Player desk band
or to switch to the Media Player
application. Instead, you can use the
thumbnail toolbar directly to perform
this task, without interrupting your
work flow by jumping to another
application.
Copied shamelessly from that MSDN article:
//In your window procedure:
switch (msg) {
case g_wmTBC://TaskbarButtonCreated
THUMBBUTTON buttons[2];
buttons[0].dwMask = THB_ICON|THB_TOOLTIP|THB_FLAGS;
buttons[0].iId = 0;
buttons[0].hIcon = GetIconForButton(0);
wcscpy(buttons[0].szTip, L"Tooltip 1");
buttons[0].dwFlags = THBF_ENABLED;
buttons[1].dwMask = THB_ICON|THB_TOOLTIP|THB_FLAGS;
buttons[1].iId = 1;
buttons[1].hIcon = GetIconForButton(1);
wcscpy(buttons[0].szTip, L"Tooltip 2");
buttons[1].dwFlags = THBF_ENABLED;
VERIFY(ptl->ThumbBarAddButtons(hWnd, 2,buttons));
break;
case WM_COMMAND:
if (HIWORD(wParam) == THBN_CLICKED)
{
if (LOWORD(wParam) == 0)
MessageBox(L"Button 0 clicked", ...);
if (LOWORD(wParam) == 1)
MessageBox(L"Button 1 clicked", ...);
}
break;
}
Since this had the C# tag I'm guessing you would like to do this in managed code. Take a look at the Windows API Code Pack which includes samples of live thumbnails, thumbnail buttons, clipped thumbnails, tabbed thumbnails etc. It is thumbnail buttons that you are looking for and two or three lines of code will take care of it.
BTW, the preview area is what you get in Windows explorer when you select say a .txt file and can see the content to the right. Most office files have previewers and you can write your own too.