I am trying to get puppeteer stealth working and found the "PuppeteerExtraSharp" package of NuGet. I however although no problems showing don't get the browser to launch and because I must use the async mehtod and don't get any errors whenever I run async I don't know how to fix the issue. I used the code that can be found on the official GitHub page of the publisher.
official link:
https://github.com/Overmiind/Puppeteer-sharp-extra
The code:
// Initialization plugin builder
var extra = new PuppeteerExtra();
// Use stealth plugin
extra.Use(new StealthPlugin());
// Launch the puppeteer browser with plugins
var browser = await extra.LaunchAsync(new LaunchOptions()
{
Headless = false
});
// Create a new page
var page = await browser.NewPageAsync();
await page.GoToAsync("http://google.com");
// Wait 2 second
await page.WaitForTimeoutAsync(2000);
// Take the screenshot
await page.ScreenshotAsync("extra.png");
I have also tried to define the path to my chrome and have tried the same for chromium and although it works when using the NuGet package "Puppeteersharp" is does not work with PuppeteerExtraSharp.
My end goal: is to run either playwright stealth or puppeteer stealth with C#
Related
I am using example code from Puppeteer-sharp but faced timeout error when calling browser.NewPageAsync(). Then I turned on Devtools and saw the page is crashed in Chromium.
var browserFetcher = new BrowserFetcher();
await browserFetcher.DownloadAsync();
var browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
Headless = true,
Devtools = true
});
// page is crashed in Chromium and timeout after 5 mins
var page = await browser.NewPageAsync();
await page.GoToAsync("http://www.google.com");
await page.DisposeAsync();
await browser.DisposeAsync();
*PuppeteerSharp version is 4.0.0
Should be due to PuppeteerSharp requires to run on Linux system in Azure (which can download and install Chromium by coding). If want to test on Windows, need to pre-install Chrome/Chromium and add ExecutablePath in LaunchOptions
// DEBUG: for running on local
ExecutablePath = "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"
Please share if you have other solutions.
Couple of other things that you can try out during your troubleshooting:
Make sure to dispose all the pages & browser being opened (as I suggested in comments).
Try adding the --full-memory-crash-report flag (& other flags related to Crash-handling) in your arguments to get more insight as in what's going wrong with your execution. You can literally take a mini-dump of your execution to understand the issue. The entire list of Chromium flags can be found here: Chromium Command Line Flags
Make sure nothing is going wrong with Garbage Collection. Since it's just a sample code, you'll have to make sure yourself that you're adding in all the necessary code required for a clean/optimised execution.
I have a task, that requires me of creating microservice, that uses puppeteersharp in order to make page screenshots. In order to do so, I use ASP.net core web api project template. In Startup.cs file I launch Puppeteersharp. Below is the code for that:
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
Browser puppeteerBrowser = null;
Task.Run(async () => puppeteerBrowser = await LaunchPuppeteerBrowserAsync());
services.AddSingleton(puppeteerBrowser);
}
public static async Task<Browser> LaunchPuppeteerBrowserAsync()
{
Console.WriteLine("Starting to launch CHROMIUM...");
// Uncomment to let puppeteer download chromium
//await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
var browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
// Comment to let puppeteer run downloaded chromium
ExecutablePath = "/usr/bin/chromium",
Args = new[]{ "--no-sandbox", "--disable-gpu-rasterization", "--disable-remote-extensions" },
Headless = true,
// Didn't help
//IgnoredDefaultArgs = new []{ "--enable-gpu-rasterization", "--enable-remote-extensions", "--load-extension=" }
});
Console.WriteLine("CHROMIUM launched successfully");
return browser;
}
Further, the microsevice is required to run in the container inside our server. I created dockerfile, that uses standard docker-image for .Net Core applications from dockerhub, created by Microsoft. In order to Illustrate my problem, I created test application and uploaded it to github repo, you can find it here:
ASP.net Core Web Api project
There is a dockerfile, which can be used to build image like so:
*Please note, that I'm using Windows 10, and for test purposes I installed Docker onto my machine.
So, as you might know, docker images have layered structure. In order to take advantage of that feature, I decided to add the download and installation of chromium browser as 1 of the steps in dockerfile. That way, when request comes to server for the first time since launch, puppeteersharp library will not have to download the browser first and then do the job it required to do, and will use the binary (or whatever it is called in linux, I mean the application launching file; in windows that would be .exe).
You can see, that I provide executable path to Puppeteersharp browser when launching new browser in the code example I provided earlier (ExecutablePath = "/usr/bin/chromium").
The container can be started the following way:
And finally, I can describe my problem. First, using CMD command: docker exec -it e95e6d9fca63 bash I tune into container's bash.
There I run this:
In order to "ps" command to work. I also install "less" package (apt-get install less).
Then I run ps aux | less to show running processes inside the container. I get the following result, which show the command line parameters for chromium process, how it was launched. I underlined the ones, which bother me:
/usr/lib/chromium/chromium --show-component-extension-options --enable-gpu-rasterization --no-default-browser-check --disable-pings --media-router=0 --enable-remote-extensions --load-extension= --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI,BlinkGenPropertyTrees --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --headless --hide-scrollbars --mute-audio about:blank --no-sandbox --disable-gpu-rasterization --disable-remote-extensions --remote-debugging-port=0 --user-data-dir=/tmp/kgkyt1d1.bqe
The latter --disable-gpu-rasterization --disable-remote-extensions arguments are the ones I set manually in Args property of LaunchOptions - check out the first code example I provided.
I also tried to use "IgnoredDefaultArgs" property of LaunchOptions, because according to documenation of the library, the options set there will be ignored. You can also see that happening in source code.
Launcher code - creates new chromium process, which has "PrepareChromiumArgs" method
The parameters put into "IgnoredDefaultArgs" array get deleted from the result array. But that didn't help, the --enable-gpu-rasterization --enable-remote-extensions --load-extension= are still there.
The strange thing is, that when I let the Puppeteersharp library to fetch the browser itself, and comment out "ExecutablePath" property from "LaunghOptions" and start it, there are no such redundant parameters. I have checked it.
My guess is that redundant arguments occur from the executable I downloaded when it gets started by the library. I mean in somewhat similar fashion that Windows allows you to add to file properties some extra command line arguments. But is this possible in Linux?
Can anyone please help?
I have a skill for the Amazon Echo. The lambda function is written in C#/.Net Core, and hosted on AWS. I now need to extend it with functionality that uses PuppeteerSharp. My first problem with this was that it downloads chromium and puts it in your application's directory by default. This gave me an error, as the filesystem is read only. I then tried the package HeadlessChromium.Puppeteer.Lambda.Dotnet, but my resulting skill was too large to upload as a lambda function, because it now had chromium embedded.
I'm now trying to override the default puppeteer downloadpath, using a BrowserFetchOption. This allows me to download chromium on AWS to the system temp path, but then Puppeteer can't find it. I need to add ExecutablePath to the LaunchOptions object when I try to launch puppeteer. I've tested this locally under windows and it works, I now just need to write some general code to find the chromium executable under AWS.
Is there a better way than searching for either chrome or chrome.exe under my download path (I need it to run under windows as well as AWS for the unit tests)? It feels like if Puppeteer provides this pair of options then there must be a better way to use them?
Is there a better way to use Puppeteer in a lambda function than what I'm attempting?
Here is the windows code that is currently working, with the chrome path hard coded:
string cpath = System.IO.Path.Combine(System.IO.Path.GetTempPath(), "chromium");
var bfopt = new BrowserFetcherOptions() { Path = cpath };
await new BrowserFetcher(bfopt).DownloadAsync(BrowserFetcher.DefaultRevision);
var browser = await Puppeteer.LaunchAsync(new LaunchOptions {
ExecutablePath=System.IO.Path.Combine(cpath, "Win64-674921\\chrome-win\\chrome.exe"),
Headless = true
});
I am trying to use phantomjs for one of our application to perform headless browser testing. I am stuck with below error, when trying to enter a value into the field:
OpenQA.Selenium.WebDriverException : Unexpected error. TypeError -
undefined is not a constructor (evaluating
'_getTagName(currWindow).toLowerCase()')
It works perfectly with Chrome, FF, IE and also with headless Chrome. I did some research, but not able to resolve the issue.
Understand that PhantomJs is faster than any other browser so added explicit wait and tried with Thread.Sleep, but still the same issue. However, click on button on the same page works properly, finding issue with SendKeys.
Below is the code ( phantomjs 2.2.1 ):
var driver = new PhantomJSDriver();
driver.Navigate().GoToUrl("https://portal-sandbox.afterpay.com");
driver.Manage().Window.Maximize();
driver.Manage().Cookies.DeleteAllCookies();
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(30));
wait.Until(ExpectedConditions.ElementIsVisible(By.ClassName("button")));
Thread.Sleep(1000);
//taking screenshot to debug
sc = driver.GetScreenshot();
sc.SaveAsFile("f://atest1.jpg");
var email = driver.FindElement(By.CssSelector("input[name=\"email\"]"));
email.SendKeys("abc#xyz.com"); //getting error on this line
driver.FindElementByClassName("button").Click();
sc = driver.GetScreenshot();
sc.SaveAsFile("f://atest2.jpg");
Tried following different solutions
1) google.com with entering values it works fine (just to check if sendKeys works)
2) updated PhantomJs.exe 1.9.8, but the same result.
3) Javascript executor to enter the value - no luck
4) Used Actions to build and perform - same result
So, not sure, if this issue is with phantomJs or with application not supporting entering values with phantomJs.
Any Help really appreciated.
i have issue in cross browser testing using codedui.
Using below code,
Process.Start("firefox", url);
BrowserWindow.CurrentBrowser = "firefox";
Browser = BrowserWindow.Launch(new System.Uri(url));
Keyboard.SendKeys("^{0}");
all code developed in IE . but now i have to execute code in firefox or chrome.I am going to execute the code in forefox.I am using this code here
Browser = BrowserWindow.Launch(new System.Uri(url));
in this line getting error like "An error occurred while connecting to Firefox".how to resolve this issue?I installed selenium components also. if i remove this line I am getting diffrent error like " Unable to find browser"...Please help.
Out of the box Visual Studio doesn't support cross browser CodedUI testing.
You're going to need to install Selenium components to allow for cross browser testing in Visual Studio.
Details on that can be found here:
http://blogs.msdn.com/b/visualstudioalm/archive/2012/10/30/introducing-cross-browser-testing-with-coded-ui-tests.aspx
Selenium components can be found here:
http://visualstudiogallery.msdn.microsoft.com/11cfc881-f8c9-4f96-b303-a2780156628d
Looks like CodedUI doesn't support playback on very many different browsers http://msdn.microsoft.com/en-us/library/dd380742(VS.100).aspx
There are some other tools out there http://watin.org/ is one, but I can't find anything myself that will really solve your problem.
Try
BrowserWindow.CurrentBrowser = "firefox";
BrowserWindow WebApp;
WebApp.CopyFrom(BrowserWindow.LaunchUrl(new System.Uri(url)));
I set up something similar to the following and it works well (I hand code everything, bypassing the UIMap).
public class WebApp : BrowserWindow
{
private string _url;
public WebApp(string url)
{
//define search properties using this keyword so the web application can be treated as a browser
_url = url;
BrowserWindow.CurrentBrowser = "Chrome";
this.CopyFrom(BrowserWindow.Launch(new Uri(url));
}
}
You can overload the constructor, of course, by adding a parameter for the browser to use or whether to start up the browser or not.
By setting up the web application as a BrowserWindow, you can have one open and ready and the playback engine should find it. I find this helps when working on tests (in IE).
Just a reminder, you do need the Selenium plug-in and that plug-in will only work for playback, not recording.
Cheers!
In our environment, we set up the Launch() method by doing the following:
public void LaunchBrowser(string uri)
{
BrowserWindow.CurrentBrowser = "firefox";
BrowserWindow myBrowser = BrowserWindow.Launch(new System.Uri(uri));
}
One thing to note is that if there is a firefox process already running in the background, the WebDriver will not launch a new instance, so be sure that all instances of firefox are shut down before the LaunchBrowser() is called. I've found that there are Java plugins that can keep it running in the background, so try to disable those that you don't need. Another good place to look, if you check your Task Manager and this is the case, is here.