Waiting for javascript-rendered page loading in CefSharp - c#

I started to learn CefSharp. It is a convinient tool for web scraping. But I have a problem.
I want to wait for a page to be loaded. But the page loading is completed by ajax. And I'm waiting for some html tags.
I wrote like this.
public ChromiumWebBrowser brser;
Thread main_thread;
public Form1()
{
brser = new ChromiumWebBrowser("some_url_here");
panelBrowser.Controls.Add(brser);
brser.Dock = DockStyle.Fill;
main_thread = new Thread(mainStart);
main_thread.Start();
}
private void mainStart()
{
brser.LoadingStateChanged += LoadingStateChanged;
brser.Load("other_url_here");
}
private async void LoadingStateChanged(object sender, LoadingStateChangedEventArgs e)
{
//... Omitted Code ...
Context.LogInfo("[LOG]\t----");
frame = brser.GetBrowser().GetFrame("iframe1");
if(frame == null)
{
Context.LogInfo("[LOG]\tiframe1 is null. return");
return;
}
int nretry = 100;
try
{
while (nretry > 0)
{
html = await Task.Run(frame.GetSourceAsync);
if (html.Contains("<ul class=\"some_class_to_be_loaded\">"))
{
break;
}
nretry--;
Thread.Sleep(100);
}
if (nretry == 0)
{
MessageBox.Show("Connection Error");
try
{
main_thread.Abort();
return;
}
catch (Exception ex)
{
return;
}
}
}
catch
{
return;
}
}
But it does not work. I debugged it. But when the thread sleeps, the page loading sleeps too. As a result, waiting 'loading complete event' failed. Please help me.

You can wrap your waiting part by delegate.
Like this.
Invoke(new Action(async delegate ()
{
//waiting code
int nretry = 100;
while (nretry > 0){
//...
Thread.Sleep(100);
}
}

Related

Async task didn't run - c#

btnSave Event: in this event i wana update datagridview but it's not working :(
private async void btnSave_Click(object sender, EventArgs e)
{
if (this.ValidateChildren(ValidationConstraints.Enabled))
{
Remember remember = new Remember();
remember.RememberTitle = txtTitle.Text;
remember.RememberDate = dateTimePickerRememberDate.Value.Date;
remember.RememberTime = dateTimePickerRememberTime.Value.TimeOfDay;
remember.RememberContent = txtDescription.Text;
remember.RememberTransaction = false;
remember.RememberExpire = false;
var res = await Rep_App.NewRemember(remember).ConfigureAwait(false);
if (res)
{
MessageBox.Show("Success");
//this line did not executed
await GetRemembers();
}
else
{
MessageBox.Show("...");
}
}
}
GetRemember method :
private async Task GetRemembers()
{
try
{
// Instantiate a new DBContext
ParkingManager.Context.DataBaseContext dbContext = new ParkingManager.Context.DataBaseContext(PublicVariables.ConnectionString);
// Call the LoadAsync method to asynchronously get the data for the given DbSet from the database.
await dbContext.Remembers.LoadAsync().ContinueWith(loadTask =>
{
// Bind data to control when loading complete
remembersBindingSource.DataSource = dbContext.Remembers.ToList();
}, System.Threading.Tasks.TaskScheduler.FromCurrentSynchronizationContext());
}
catch (InvalidOperationException)
{
return;
}
catch (Exception)
{
MessageBox.Show(MessagesStruct.Exception);
}
}
GetRemember() did not execute in Save Event but when the form loaded GetRemember() executed completely, what's wrong?
The problem is caused by ConfigureAwait(false) and ContinueWith. ConfigureAwait(false) will cause execution to continue on a background thread instead of getting back to the UI. LoadAsync isn't needed either, the data will be loaded by ToList or ToListAsync().
EF Core and async/await don't need so much code. All of this can be replaced with
var res = await Rep_App.NewRemember(remember);
if (res)
{
MessageBox.Show("Success");
using(var dbContext = new DataBaseContext(PublicVariables.ConnectionString))
{
remembersBindingSource.DataSource = await dbContext.Remembers.ToListAsync();
}
}
else
{
MessageBox.Show("...");
}

Program come out of function without executing await

I'm using Two await function in my program, and first one is working flawlessly but on encountering second await, my program comes out of function without executing that awaitable function and any line after that.
Not giving error and not even crashing.
Tried "wait()", "Running on main thread", "Task Run", "Threading Sleep"..
here a snippet of the code along
private async void Btn_Clicked(object sender, EventArgs e)
{
//this statement works perfectly fine
var status = await CrossPermissions.Current.RequestPermissionAsync<LocationPermission>();
//on debugging i found out it return out of function at this point without executing
var check = await CrossPermissions.Current.RequestPermissionAsync<CameraPermission>();
//None of the code is executed
Console.WriteLine("Granted");
Console.WriteLine("Granted");
}
You try to use the below method :
private async void Btn_Clicked(object sender, EventArgs e)
{
await GetPermissions();
}
public static async Task<bool> GetPermissions()
{
bool permissionsGranted = true;
var permissionsStartList = new List<Permission>()
{
Permission.Location,
Permission.Camera,
};
var permissionsNeededList = new List<Permission>();
try
{
foreach (var permission in permissionsStartList)
{
var status = await CrossPermissions.Current.CheckPermissionStatusAsync(permission);
if (status != Plugin.Permissions.Abstractions.PermissionStatus.Granted)
{
permissionsNeededList.Add(permission);
}
}
}
catch (Exception ex)
{
}
var results = await CrossPermissions.Current.RequestPermissionsAsync(permissionsNeededList.ToArray());
try
{
foreach (var permission in permissionsNeededList)
{
var status = Plugin.Permissions.Abstractions.PermissionStatus.Unknown;
//Best practice to always check that the key exists
if (results.ContainsKey(permission))
status = results[permission];
if (status == Plugin.Permissions.Abstractions.PermissionStatus.Granted || status == Plugin.Permissions.Abstractions.PermissionStatus.Unknown)
{
permissionsGranted = true;
}
else
{
permissionsGranted = false;
break;
}
}
}
catch (Exception ex)
{
}
return permissionsGranted;
}

Set a WPF Mahapps Progress Dialog

I'm trying to replace my ProgressBar to a Progress Dialog using Mahapps.
So I started writing this:
private void btnClick(object sender, RoutedEventArgs e)
{
ConfRelais();
}
public async void ConfRelais()
{
var controller = await this.ShowProgressAsync("hey", "hoy");
controller.Maximum = 128;
while (flag == 0)
{
string data = RelayBoard_Port.ReadTo("\r\n");
if (data == "ok") { controller.SetMessage("Done Process");
flag = 1; }
else { controller.SetProgress(Int32.Parse(data)); }
}
await controller.CloseAsync();
}
But the progress dialog only displays when it's over.. As I'm still a beginner in c# maybe I'm missing some importants points to setup that kind of function.
You should execute the loop on a background thread:
public async void ConfRelais()
{
var controller = await this.ShowProgressAsync("hey", "hoy");
controller.Maximum = 128;
await Task.Run(() =>
{
while (flag == 0)
{
string data = RelayBoard_Port.ReadTo("\r\n");
if (data == "ok")
{
controller.SetMessage("Done Process");
flag = 1;
}
else { controller.SetProgress(Int32.Parse(data)); }
}
});
await controller.CloseAsync();
}
A single thread cannot both update the UI and execute your loop simultaneously.
You also don't really need a flag. You could just break out of the loop when you receive "ok":
while (true)
{
string data = RelayBoard_Port.ReadTo("\r\n");
if (data == "ok")
{
controller.SetMessage("Done Process");
break;
}
else { controller.SetProgress(Int32.Parse(data)); }
}

try to run task n-times before throwing error [duplicate]

This question already has answers here:
Retry a task multiple times based on user input in case of an exception in task
(3 answers)
Closed 8 years ago.
I have method that is sending email using smtp server. Using Task.Factory I'm calling that method to not block UI:
Task.Factory.StartNew(() => SendMail("mail#example.com", "Test title", "TEST body"), CancellationToken.None, TaskCreationOptions.None, TaskScheduler.Default)
.ContinueWith(p =>
{
if (p.IsFaulted)
{
if (p.Exception != null)
{
MessageBox.Show(p.Exception.ToString());
}
return;
}
MessageBox.Show("ok");
}, TaskScheduler.FromCurrentSynchronizationContext());
Now I would like to modify my code to be able to try to call SendMail 10 times if something goes wrong. I've tried using do/while block, but I can't get this working:
private void button1_Click(object sender, EventArgs e)
{
bool success = false;
int i = 0;
int max = 10;
do
{
Task.Factory.StartNew(() => SendMail("mail#example.com", "Test", "TEST1"), CancellationToken.None, TaskCreationOptions.None, TaskScheduler.Default)
.ContinueWith(p =>
{
if (p.IsFaulted)
{
if (p.Exception != null)
{
MessageBox.Show(p.Exception.ToString());
}
return;
}
success = true;
MessageBox.Show("ok");
}, TaskScheduler.FromCurrentSynchronizationContext());
i++;
} while (!success && i < max);
if (!success)
{
MessageBox.Show("error");
}
else
{
MessageBox.Show("ok", "success", MessageBoxButtons.OK, MessageBoxIcon.Information);
}
}
private void SendMail(string address, string title, string body)
{
Thread.Sleep(10000);
MailClient.Instance.Send(address, title, body);
}
That I would like to do is to be able to call Specific method inside task, if I get exception then I would like to call it again and again, 10 times, if after those 10 times it won't be successful I want to show exception.
This is a bit off topic, but every time I see someone using threading for IO bound operations i get the chills :)
As sending mail is a network bound operation, you can use the awaitable SmtpClient.SendMailAsync added in .NET 4.5.
If I may take the implementation posted by JoelC and refactor it a bit:
private int _maxAttempts = 10;
private async Task TrySendMailAsync(int attemptNumber)
{
var smtpClient = new SmtpClient();
var mailMsg = new MailMessage("from#test.com", "to#test.com", "Test Subject", "Test Body");
while (!success && attempts <= maxAttempts)
{
try
{
await smtpClient.SendMailAsync(mailMsg)).ConfigureAwait(false);
success = true;
}
catch
{
if (attempts >= maxAttempts)
{
throw;
}
}
attempts++;
}
}
This will let you loop as requested, but let the main job which is the async IO work without the unnessacery execution of a thread pool thread.
Something like this might fix the issue:
private int _maxAttempts = 10;
private void TrySendMail(int attemptNumber)
Task.Factory.StartNew(() => SendMail("mail#example.com", "Test title", "TEST body"), CancellationToken.None, TaskCreationOptions.None, TaskScheduler.Default)
.ContinueWith(p =>
{
attemptNumber++;
if (p.IsFaulted)
{
if (p.Exception != null)
{
if (_attempts < _maxAttempts)
{
// Try again
TrySendMail(attemptNumber);
}
else
{
MessageBox.Show(p.Exception.ToString());
}
}
return;
}
success = true;
MessageBox.Show("ok");
}, TaskScheduler.FromCurrentSynchronizationContext());
}
Its not the prettiest and you want to watch that you don't call it recursively too many times and get a stack overflow! Ten times should be fine.
EDIT:
I changed the attempts count to an argument to be safer with threading, in case you are calling this send mail possibly on a thread many times.
EDIT2:
The implementation of the method #mazharenko mentioned above could look something like this:
private void TryAndRepeat(Action routine, int maxAttempts)
{
int attempts = 1 ;
bool success = false;
while (!success && attempts <= maxAttempts)
{
try
{
routine.Invoke();
success = true;
}
catch
{
if (attempts >= maxAttempts)
{
throw;
}
}
attempts++;
}
}

Get ReadyState from WebBrowser control without DoEvents

This has been awnsered many times here and at other sites and its working, but I would like ideas to other ways to:
get the ReadyState = Complete after using a navigate or post, without using DoEvents because of all of its cons.
I would also note that using the DocumentComplete event woud not help here as I wont be navigating on only one page, but one after another like this.
wb.navigate("www.microsoft.com")
//dont use DoEvents loop here
wb.Document.Body.SetAttribute(textbox1, "login")
//dont use DoEvents loop here
if (wb.documenttext.contais("text"))
//do something
The way it is today its working by using DoEvents. I would like to know if anyone have a proper way to wait the async call of the browser methods to only then proceed with the rest of the logic. Just for the sake of it.
Thanks in advance.
Below is a basic WinForms app code, illustrating how to wait for the DocumentCompleted event asynchronously, using async/await. It navigates to multiple pages, one after another. Everything is taking place on the main UI thread.
Instead of calling this.webBrowser.Navigate(url), it might be simulating a form button click, to trigger a POST-style navigation.
The webBrowser.IsBusy async loop logic is optional, its purpose is to account (non-deterministically) for the page's dynamic AJAX code which may take place after window.onload event.
using System;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace WebBrowserApp
{
public partial class MainForm : Form
{
WebBrowser webBrowser;
public MainForm()
{
InitializeComponent();
// create a WebBrowser
this.webBrowser = new WebBrowser();
this.webBrowser.Dock = DockStyle.Fill;
this.Controls.Add(this.webBrowser);
this.Load += MainForm_Load;
}
// Form Load event handler
async void MainForm_Load(object sender, EventArgs e)
{
// cancel the whole operation in 30 sec
var cts = new CancellationTokenSource(30000);
var urls = new String[] {
"http://www.example.com",
"http://www.gnu.org",
"http://www.debian.org" };
await NavigateInLoopAsync(urls, cts.Token);
}
// navigate to each URL in a loop
async Task NavigateInLoopAsync(string[] urls, CancellationToken ct)
{
foreach (var url in urls)
{
ct.ThrowIfCancellationRequested();
var html = await NavigateAsync(ct, () =>
this.webBrowser.Navigate(url));
Debug.Print("url: {0}, html: \n{1}", url, html);
}
}
// asynchronous navigation
async Task<string> NavigateAsync(CancellationToken ct, Action startNavigation)
{
var onloadTcs = new TaskCompletionSource<bool>();
EventHandler onloadEventHandler = null;
WebBrowserDocumentCompletedEventHandler documentCompletedHandler = delegate
{
// DocumentCompleted may be called several time for the same page,
// if the page has frames
if (onloadEventHandler != null)
return;
// so, observe DOM onload event to make sure the document is fully loaded
onloadEventHandler = (s, e) =>
onloadTcs.TrySetResult(true);
this.webBrowser.Document.Window.AttachEventHandler("onload", onloadEventHandler);
};
this.webBrowser.DocumentCompleted += documentCompletedHandler;
try
{
using (ct.Register(() => onloadTcs.TrySetCanceled(), useSynchronizationContext: true))
{
startNavigation();
// wait for DOM onload event, throw if cancelled
await onloadTcs.Task;
}
}
finally
{
this.webBrowser.DocumentCompleted -= documentCompletedHandler;
if (onloadEventHandler != null)
this.webBrowser.Document.Window.DetachEventHandler("onload", onloadEventHandler);
}
// the page has fully loaded by now
// optional: let the page run its dynamic AJAX code,
// we might add another timeout for this loop
do { await Task.Delay(500, ct); }
while (this.webBrowser.IsBusy);
// return the page's HTML content
return this.webBrowser.Document.GetElementsByTagName("html")[0].OuterHtml;
}
}
}
If you're looking to do something similar from a console app, here is an example of that.
The solution is simple:
// MAKE SURE ReadyState = Complete
while (WebBrowser1.ReadyState.ToString() != "Complete") {
Application.DoEvents();
}
// Move on to your sub-sequence code...
Dirty and quick.. I am a VBA guys, this logic has been working forever, just took me days and found none for C# but I just figured this out myself.
Following is my complete function, the objective is to obtain a segment of info from a webpage:
private int maxReloadAttempt = 3;
private int currentAttempt = 1;
private string GetCarrier(string webAddress)
{
WebBrowser WebBrowser_4MobileCarrier = new WebBrowser();
string innerHtml;
string strStartSearchFor = "subtitle block pull-left\">";
string strEndSearchFor = "<";
try
{
WebBrowser_4MobileCarrier.ScriptErrorsSuppressed = true;
WebBrowser_4MobileCarrier.Navigate(webAddress);
// MAKE SURE ReadyState = Complete
while (WebBrowser_4MobileCarrier.ReadyState.ToString() != "Complete") {
Application.DoEvents();
}
// LOAD HTML
innerHtml = WebBrowser_4MobileCarrier.Document.Body.InnerHtml;
// ATTEMPT (x3) TO EXTRACT CARRIER STRING
while (currentAttempt <= maxReloadAttempt) {
if (innerHtml.IndexOf(strStartSearchFor) >= 0)
{
currentAttempt = 1; // Reset attempt counter
return Sub_String(innerHtml, strStartSearchFor, strEndSearchFor, "0"); // Method: "Sub_String" is my custom function
}
else
{
currentAttempt += 1; // Increment attempt counter
GetCarrier(webAddress); // Recursive method call
} // End if
} // End while
} // End Try
catch //(Exception ex)
{
}
return "Unavailable";
}
Here is a "quick & dirty" solution. It's not 100% foolproof but it doesn't block UI thread and it should be satisfactory to prototype WebBrowser control Automation procedures:
private async void testButton_Click(object sender, EventArgs e)
{
await Task.Factory.StartNew(
() =>
{
stepTheWeb(() => wb.Navigate("www.yahoo.com"));
stepTheWeb(() => wb.Navigate("www.microsoft.com"));
stepTheWeb(() => wb.Navigate("asp.net"));
stepTheWeb(() => wb.Document.InvokeScript("eval", new[] { "$('p').css('background-color','yellow')" }));
bool testFlag = false;
stepTheWeb(() => testFlag = wb.DocumentText.Contains("Get Started"));
if (testFlag) { /* TODO */ }
// ...
}
);
}
private void stepTheWeb(Action task)
{
this.Invoke(new Action(task));
WebBrowserReadyState rs = WebBrowserReadyState.Interactive;
while (rs != WebBrowserReadyState.Complete)
{
this.Invoke(new Action(() => rs = wb.ReadyState));
System.Threading.Thread.Sleep(300);
}
}
Here is a bit more generic version of testButton_Click method:
private async void testButton_Click(object sender, EventArgs e)
{
var actions = new List<Action>()
{
() => wb.Navigate("www.yahoo.com"),
() => wb.Navigate("www.microsoft.com"),
() => wb.Navigate("asp.net"),
() => wb.Document.InvokeScript("eval", new[] { "$('p').css('background-color','yellow')" }),
() => {
bool testFlag = false;
testFlag = wb.DocumentText.Contains("Get Started");
if (testFlag) { /* TODO */ }
}
//...
};
await Task.Factory.StartNew(() => actions.ForEach((x)=> stepTheWeb (x)));
}
[Update]
I have adapted my "quick & dirty" sample by borrowing and sligthly refactoring #Noseratio's NavigateAsync method from this topic.
New code version would automate/execute asynchronously in UI thread context not only navigation operations but also Javascript/AJAX calls - any "lamdas"/one automation step task implementation methods.
All and every code reviews/comments are very welcome. Especially, from #Noseratio. Together, we will make this world better ;)
public enum ActionTypeEnumeration
{
Navigation = 1,
Javascript = 2,
UIThreadDependent = 3,
UNDEFINED = 99
}
public class ActionDescriptor
{
public Action Action { get; set; }
public ActionTypeEnumeration ActionType { get; set; }
}
/// <summary>
/// Executes a set of WebBrowser control's Automation actions
/// </summary>
/// <remarks>
/// Test form shoudl ahve the following controls:
/// webBrowser1 - WebBrowser,
/// testbutton - Button,
/// testCheckBox - CheckBox,
/// totalHtmlLengthTextBox - TextBox
/// </remarks>
private async void testButton_Click(object sender, EventArgs e)
{
try
{
var cts = new CancellationTokenSource(60000);
var actions = new List<ActionDescriptor>()
{
new ActionDescriptor() { Action = ()=> wb.Navigate("www.yahoo.com"), ActionType = ActionTypeEnumeration.Navigation} ,
new ActionDescriptor() { Action = () => wb.Navigate("www.microsoft.com"), ActionType = ActionTypeEnumeration.Navigation} ,
new ActionDescriptor() { Action = () => wb.Navigate("asp.net"), ActionType = ActionTypeEnumeration.Navigation} ,
new ActionDescriptor() { Action = () => wb.Document.InvokeScript("eval", new[] { "$('p').css('background-color','yellow')" }), ActionType = ActionTypeEnumeration.Javascript},
new ActionDescriptor() { Action =
() => {
testCheckBox.Checked = wb.DocumentText.Contains("Get Started");
},
ActionType = ActionTypeEnumeration.UIThreadDependent}
//...
};
foreach (var action in actions)
{
string html = await ExecuteWebBrowserAutomationAction(cts.Token, action.Action, action.ActionType);
// count HTML web page stats - just for fun
int totalLength = 0;
Int32.TryParse(totalHtmlLengthTextBox.Text, out totalLength);
totalLength += !string.IsNullOrWhiteSpace(html) ? html.Length : 0;
totalHtmlLengthTextBox.Text = totalLength.ToString();
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Error");
}
}
// asynchronous WebBroswer control Automation
async Task<string> ExecuteWebBrowserAutomationAction(
CancellationToken ct,
Action runWebBrowserAutomationAction,
ActionTypeEnumeration actionType = ActionTypeEnumeration.UNDEFINED)
{
var onloadTcs = new TaskCompletionSource<bool>();
EventHandler onloadEventHandler = null;
WebBrowserDocumentCompletedEventHandler documentCompletedHandler = delegate
{
// DocumentCompleted may be called several times for the same page,
// if the page has frames
if (onloadEventHandler != null)
return;
// so, observe DOM onload event to make sure the document is fully loaded
onloadEventHandler = (s, e) =>
onloadTcs.TrySetResult(true);
this.wb.Document.Window.AttachEventHandler("onload", onloadEventHandler);
};
this.wb.DocumentCompleted += documentCompletedHandler;
try
{
using (ct.Register(() => onloadTcs.TrySetCanceled(), useSynchronizationContext: true))
{
runWebBrowserAutomationAction();
if (actionType == ActionTypeEnumeration.Navigation)
{
// wait for DOM onload event, throw if cancelled
await onloadTcs.Task;
}
}
}
finally
{
this.wb.DocumentCompleted -= documentCompletedHandler;
if (onloadEventHandler != null)
this.wb.Document.Window.DetachEventHandler("onload", onloadEventHandler);
}
// the page has fully loaded by now
// optional: let the page run its dynamic AJAX code,
// we might add another timeout for this loop
do { await Task.Delay(500, ct); }
while (this.wb.IsBusy);
// return the page's HTML content
return this.wb.Document.GetElementsByTagName("html")[0].OuterHtml;
}

Categories