I am trying to make create a function to get the source code from a number of pages. After each page is grabbed, I want to update a label on my form indicating the progress (1 of 5, 2 of 5, etc.).
However, no matter what I try, the GUI completely freezes until the for loop has ended.
public List<List<string>> GetPages(string base_url, int num_pages)
{
var pages = new List<List<string>>();
var webGet = new HtmlWeb();
var task = Task.Factory.StartNew(() => {
for (int i = 0; i <= num_pages; i++)
{
UpdateMessage("Fetching page " + i + " of " + num_pages + ".");
var page = new List<string>();
var page_source = webGet.Load(url+i);
// (...)
page.Add(url+i);
page.Add(source);
pages.Add(page);
}
});
task.Wait();
return pages;
}
The call to this method looks like this:
List<List<string>> pages = site.GetPages(url, num_pages);
If I remove task.Wait(); the GUI unfreezes, the label updates properly, but the code continues without the needed multidimensional list.
I should say that I'm very new to C#. What the heck am I doing wrong?
Update
As per Darin, I have changed my method:
public async Task<List<List<string>>> GetPages(string url, int num_pages)
{
var pages = new List<List<string>>();
var webGet = new HtmlWeb();
for (int i = 0; i <= num_pages; i++)
{
UpdateMessage("Fetching page " + i + " of " + num_pages + ".");
var page = new List<string>();
var page_source = webGet.Load(url+i);
// (...)
page.Add(url+i);
page.Add(source);
pages.Add(page);
}
return pages;
}
And the call:
List<List<string>> pages = await site.GetPages(url, num_pages);
However, now I am getting this error:
The 'await' operator can only be used within an async method. Consider marking this method with the 'async' modifier and changing its return type to 'Task'.
But when I mark the method with async, the GUI still freezes.
Update 2
Woops! I seem to missed a piece of Darin's new method. I have now included await webGet.LoadAsync(url + i); in the method. I have also marked the method I am calling from as async.
Now, unfortunately, I'm getting this error:
'HtmlWeb' does not contain a definition for 'LoadAsync' and no extension method 'LoadAsync' accepting a first argument of type 'HtmlWeb' could be found (are you missing a using directive or an assembly reference?)
I've checked, I'm using .NET 4.5.2, and the HtmlAgilityPack in my References is the Net45 version. I have no idea what's going on now.
If I remove task.Wait(); the GUI unfreezes, the label updates
properly, but the code continues without the needed multidimensional
list.
That's normal. You should update your function so that it doesn't return the value but rather the task:
public Task<List<List<string>>> GetPages(string base_url, int num_pages)
{
var webGet = new HtmlWeb();
var task = Task.Factory.StartNew(() =>
{
var pages = new List<List<string>>();
for (int i = 0; i <= num_pages; i++)
{
UpdateMessage("Fetching page " + i + " of " + num_chapters + ".");
var page = new List<string>();
var page_source = webGet.Load(url+i);
// (...)
page.Add(url+i);
page.Add(source);
pages.Add(page);
}
return pages;
});
return task;
}
and then when calling this function you will use the ContinueWith on the result:
var task = GetPages(baseUrl, numPages);
task.ContinueWith(t =>
{
List<List<string>> chapters = t.Result;
// Do something with the results here
});
Obviously before accessing t.Result in the continuation you probably would like to first check the other properties to see if the task completed successfully or if some exception was thrown so that you can act accordingly.
Also if you are using .NET 4.5 you may consider taking advantage of the async/await constructs:
public async Task<List<List<string>>> GetPages(string base_url, int num_pages)
{
var webGet = new HtmlWeb();
var pages = new List<List<string>>();
for (int i = 0; i <= num_pages; i++)
{
UpdateMessage("Fetching page " + i + " of " + num_chapters + ".");
var page = new List<string>();
var page_source = await webGet.LoadAsync(url+i);
// (...)
page.Add(url+i);
page.Add(source);
pages.Add(page);
}
return pages;
}
and then:
List<List<string>> chapters = await GetPages(baseUrl, numPages);
// Do something with the results here.
Assuming WinForms, start by making the toplevel eventhandler async void .
You then have an async method that can await a Task<List<List<string>>> method. That method does not have to be async itself.
private async void Button1_Click(...)
{
var pages = await GetPages(...);
// update the UI here
}
public Task<List<List<string>>> GetPages(string url, int num_pages)
{
...
return task;
}
Related
I want to run a function over all rows, but instead of it running for each row at a time, I want it to run in parallel. The issue is that it seems to only run twice (while there are 217 products on the list) and it seems rows are picked randomly. I'm not sure why it's happening - the synchronous version works as intended.
The function makes an API call to a SOAP service that uses row data from WPF SfDataGrid. Maybe the API closes connection after so many requests? It's really hard to tell and I don't know how I could check it besides writing a ticket and waiting a week.
A part of event from which the function is called:
var tasks = new List<Task>();
using (var db = new SqliteConnection("Data Source=file:products.sqlite"))
{
using (var client = new ApiOrdersPortTypeClient(binding, address))
{
db.Open();
// run ScanProduct() on each row in a WPF SfDataGrid
foreach (var row in ProductList.View.Records)
{
tasks.Add(Task.Factory.StartNew(
() => ScanProduct(row, desiredDays, deliveryTime, client, db)));
}
Task.WaitAll(tasks.ToArray());
}
}
ScanProduct():
private async Task ScanProduct(RecordEntry row, int desiredDays, int deliveryTime, ApiOrdersPortTypeClient client, SqliteConnection db)
{
await db.OpenAsync();
var selectedItem = (Product)row.Data;
if (selectedItem.IsIgnored == true | selectedItem.IsInDelivery == true)
{
return;
}
else
{
var pagesCount = await ApiRequests
.GetOrdersPages(int.Parse(selectedItem.Id), desiredDays, client);
var listSold = await ApiRequests
.GetOrdersFromApi(int.Parse(selectedItem.Id), desiredDays, pagesCount);
foat x = 100.0; // cut irrelevant stuff
if (x > selectedItem.Stock)
{
using (var command =
new SqliteCommand($"UPDATE products_settings SET requires_purchase = 1,"
+ $" was_checked_with = {desiredDays},"
+ $"sold_amount = {listSold.Count},"
+ $"average_daily = '{averageSales.ToString()}',"
+ $"average_delivery = '{x}'"
+ $"WHERE id = {selectedItem.Id};", db))
{
await db.OpenAsync();
command.ExecuteReader();
}
}
else
{
using (var command2 =
new SqliteCommand($"UPDATE products_settings SET requires_purchase = 0,"
+ $"was_checked_with = {desiredDays},"
+ $"sold_amount = {listSold.Count},"
+ $"average_daily = '{averageSales.ToString()}',"
+ $"average_delivery = '{x}"
+ $"WHERE id = {selectedItem.Id};", db))
{
await db.OpenAsync();
command2.ExecuteReader();
}
}
}
}
Your issue is here:
tasks.Add(Task.Factory.StartNew(() => ScanProduct(row, desiredDays, deliveryTime, client, db)));
ScanProduct is an asynchronous function, and thus needs to be awaited.
Each call to Task.Factory.StartNew returns a Task, which will be awaited using Task.WaitAll, however, within each task, a call to ScanProduct will return another Task, which is not being awaited by Task.WaitAll.
The solution is to use an async lambda in combination with Task.Run, then in each case, the outer Task will not complete until ScanProduct has also completed.
tasks.Add(Task.Run(async () => await ScanProduct(row, desiredDays, deliveryTime, client, db)));
I wrote an async function for calling data from Facebook, it works, but the problem is I dun suppose it works. Can someone explain to me?
public class FacebookData
{
static string fb_api_version = ConfigurationManager.AppSettings["fb_ver"];
static string accessToken = ConfigurationManager.AppSettings["accessToken"];
static string fb_id = "";
private HttpClient _httpClient;
public FacebookData(string input_id)
{
fb_id = input_id;
_httpClient = new HttpClient
{
BaseAddress = new Uri("https://graph.facebook.com/" + fb_api_version + "/"),
Timeout = TimeSpan.FromSeconds(15)
};
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
public async Task<T> getData<T>()
{
var response = await _httpClient.GetAsync($"{fb_id}?access_token={accessToken}");
if (!response.IsSuccessStatusCode)
return default(T);
var result = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(result);
}
}
The calling class is typical, I make it await for the response.
But the problem is where I call it.
In main
static void Main(string[] args)
{
string[] data_Set = [//ids_group]
for (int i = 0; i < data_Set.length; ++i){
Console.WriteLine("Running Thread " + (i+1).ToString());
var dataSet = facebookRequestCmd(data_Set[i]);
writeToTXT(dataSet);
Console.WriteLine("Finished Thread " + (i + 1).ToString());
//do sth
}
}
In facebookRequestCmd
static Dictionary<string, string[]> facebookRequestCmd(string ids){
Dictionary<string, string[]> allData = new Dictionary<string, string[]>();
string[] ids_arr = ids.split(",")
for (var i = 0; i < ids.length; i++){
var facebook_client = new FacebookData(sqlData);
var response = facebook_client.getData<dynamic>();
Task.WaitAll(response);
//then use the result to do sth
}
}
In my understanding, each time I call getData, it already come back to main thread, as it is awaiting the data. So the Task doesn't really start a new thread.
Thus, async await works for waiting the http request, but the Threading should not work.
However,
Console.WriteLine("Running Thread " + (i+1).ToString());
jumps out simultaneously like I really make the Thread in the for loop in main function.
Why? And is that the way to use Multithreading with Async-await. As I want to make multiple calls at the same time.
Originally I use Parallel.ForEach to kick starting the calling, however, thats not asynchronous and will block the thread.
Ok, feel free to ignore all the changes I've made but I couldn't help but modify the way some of the variables read and the code looked. This is not a working application and I have obviously not tested it. This is just a cleaned up version of what you have with a suggested way of using Task. It's also mocked using just the code you've provided so it is what it is. #2 is, what I believe, the answer you needed.
In Main I removed the words 'thread' since that's not actually what's happening. It may be, but we don't know if the HttpClient is indeed starting a new thread or just holding / returning from the rest call. Using async / await does not always mean a Thread was started (although it's common to think of it that way).
I used .Result (not Wait() as I suggested in comments) to get the result of the task. This is ok since it's a console app but not ideal for a real world app that needs to operate without blocking. I also removed Task.WaitAll with this change.
I renamed functions to have verbage because, IMO, functions should be doing work and the naming should describe the work being done.
I renamed some variables because, IMO, variables should be PascalCase when their scope isn't in a method or private and camelCase when they are. The names should also, IMO, be what it is followed by the Type that makes sense.
I appended 'Async' to function names that return a running Task.
Changed FacebookClient to be singleton and allowing only one HttpClient to be used instead of many and allowing it to be disposed; plus more.
Added alternate version of the GetFacebookData function that calls the tasks and awaits them all simultaneously.
static void Main(string[] args)
{
string[] dataSet = new string[] { /* mocked */ }; // [ids_group]; <- no idea what this is so I mocked it.
for (int i = 0; i < dataSet.Length; i++)
{
Console.WriteLine("Main... " + (i + 1).ToString());
var result = GetFacebookData(dataSet[i]);
WriteToTxt(result);
Console.WriteLine("Complete... " + (i + 1).ToString());
//do sth
}
Console.Read();
}
private static Dictionary<string, string[]> GetFacebookData(string idsString)
{
var allDataDictionary = new Dictionary<string, string[]>();
var idsArray = idsString.Split(',');
foreach (var id in idsArray)
{
var response = FacebookClient.Instance.GetDataAsync<string[]>(id).Result;
allDataDictionary.Add(id, response);
}
return allDataDictionary;
}
public class FacebookClient
{
private readonly HttpClient httpClient;
private readonly string facebookApiVersion;
private readonly string accessToken;
public static FacebookClient Instance { get; } = new FacebookClient();
FacebookClient()
{
facebookApiVersion = ConfigurationManager.AppSettings["fb_ver"];
accessToken = ConfigurationManager.AppSettings["accessToken"];
httpClient = new HttpClient
{
BaseAddress = new Uri("https://graph.facebook.com/" + facebookApiVersion + "/"),
Timeout = TimeSpan.FromSeconds(15)
};
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
public async Task<T> GetDataAsync<T>(string facebookId)
{
var response = await httpClient.GetAsync($"{facebookId}?access_token={accessToken}");
if (!response.IsSuccessStatusCode) return default;
var result = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(result);
}
~FacebookClient() => httpClient.Dispose();
}
Here's a version that's starting all the tasks and then awaiting them all at the same time. I believe this might give you some issues on the HttpClient but we'll see.
private static Dictionary<string, string[]> GetFacebookData(string idsString)
{
var allDataDictionary = new Dictionary<string, string[]>();
var idsArray = idsString.Split(',');
var getDataTasks = new List<Task<string[]>>();
foreach (var id in idsArray)
{
getDataTasks.Add(FacebookClient.Instance.GetDataAsync<string[]>(id));
}
var tasksArray = getDataTasks.ToArray();
Task.WaitAll(tasksArray);
var resultsArray = tasksArray.Select(task => task.Result).ToArray();
for (var i = 0; i < idsArray.Length; i++)
{
allDataDictionary.Add(idsArray[i], resultsArray[i]);
}
return allDataDictionary;
}
I have a method collecting user data from MS Teams, while this code works as intended, with this method added it makes a previously working method crash.
public async void GetUsers()
{
string teamId = "TeamsID";
string tenantId = "TenantID";
var connector = new ConnectorClient(new Uri(Instance.Activity.ServiceUrl));
members = await connector.Conversations.GetTeamsConversationMembersAsync(teamId, tenantId);
Instance.EmailList.Clear();
foreach (var member in members)
{
Instance.EmailList.Add(member.Email);
}
}
I believe that the Line:
members = await connector.Conversations.GetTeamsConversationMembersAsync(teamId, tenantId);
While receiving the user information, makes the bot think that a user is inputing, causing my later methods to trigger without user input, and crashing because there is no input or because the input if the chunk of data that is the user data.
This is just my theory and I may be incorrect.
The following is the method that crashes:
async Task ReplyToQuestions(IDialogContext context, IAwaitable<IMessageActivity> argument)
{
var AnswerHolder = await argument;
Answers.Add(AnswerHolder.Text);
Answers[AnswerCounter] = Answers[AnswerCounter].First().ToString().ToUpper() + Answers[AnswerCounter].Substring(1);
var isQuit = AnswerHolder.Text;
var isQuit2 = Regex.Match(isQuit, #"\b(Quit|quit|QUIT)\b").Value;
Regex rgx = new Regex(#"\b(Quit|quit|QUIT)\b");
if (rgx.IsMatch(isQuit2)) // checks for the user trying to quit/restart bot
{
await context.PostAsync(string.Format("Exiting current process. Restarting."));
context.Done(isQuit); // returns to the start of dialog (StartAsync)
}
else
{
if (QuestionCounter < 5)
{
await context.PostAsync(string.Format($"Question {QuestionCounter + 1}: {Question[QuestionCounter]}"));
}
AnswerCounter += 1;
QuestionCounter += 1;
if (AnswerCounter < 5)
{
context.Wait(ReplyToQuestions);
}
else if (AnswerCounter == 5)
{
PostToChannel($"{Name}'s answers to the questions are as follows: \n\nQuestion1 Answer: {Answers[0]} \n\nQuestion2 Answer: {Answers[1]} \n\n" +
$"Question3 Answer: {Answers[2]} \n\nQuestion4 Answer: {Answers[3]} \n\nQuestion5 Answer: {Answers[4]} \n\n", context);
await context.PostAsync(string.Format("Your answers have been posted to the 'What's Up' channel."));
AnswerCounter = 0;
QuestionCounter = 1;
context.Done(Answers[4]);
}
else
{
await context.PostAsync($"{AnswerCounter}");
await context.PostAsync(string.Format("Oh no! it appears something has gone wrong. please try re-entering your answers"));
AnswerCounter = 0;
QuestionCounter = 1;
context.Wait(ReplyToQuestions);
}
}
}
And the code that calls it:
async Task Choice(IDialogContext context, IAwaitable<IMessageActivity> argument) // this method recives and validates a question
{
var Choice = await argument;
var isQuit = Choice.Text;
var isQuit2 = Regex.Match(isQuit, #"\b(Quit|quit|QUIT)\b").Value;
Regex rgx = new Regex(#"\b(Quit|quit|QUIT)\b");
var isEnter = Regex.Match(isQuit, #"\b(Enter|ENTER|enter)\b").Value;
Regex rgx2 = new Regex(#"\b(Enter|ENTER|enter)\b");
var isReply = Regex.Match(isQuit, #"\b(Reply|REPLY|reply)\b").Value;
Regex rgx3 = new Regex(#"\b(Reply|REPLY|reply)\b");
GetUsers();
if (rgx.IsMatch(isQuit2)) // if the user choses to quit
{
await context.PostAsync(string.Format("Exiting current process. Restarting."));
context.Done(isQuit); // restarts the program, calls the first method
}
else if (rgx2.IsMatch(isEnter)) // if the user choses to quit
{
await context.PostAsync(string.Format("Please enter your custom question."));
context.Wait(EnterQuestion);
}
else if (rgx3.IsMatch(isReply)) // if the user choses to quit
{
Answers.Clear();
await context.PostAsync(string.Format("Please answer the following questions:"));
await context.PostAsync(string.Format($"Question 1: {Question[0]}"));
context.Wait(ReplyToQuestions);
}
else
{
await context.PostAsync(string.Format("sorry this was not a choice, try again."));
}
}
Does anyone know a way I can fix this? As I have spent 2 full days on this without success.
I'm not sure what error you are seeing. But, the method being used to retrieve conversation members has been deprecated: https://msdn.microsoft.com/en-us/microsoft-teams/botapis#net-example The note on that page should state:
you should migrate your code to use GetConversationMembersAsync(conversationId)
var connector = new ConnectorClient(new Uri(activity.ServiceUrl));
var members = await connector.Conversations.GetConversationMembersAsync(activity.Conversation.Id);
So I have a block of code like
EventLog.WriteEntry("About to run the task");
// run the dequeued task
var task = PageRetrievals.Dequeue();
PageRetrieval retrieval = new PageRetrieval();
var continuation = task.ContinueWith(t => retrieval = t.Result);
task.Wait();
EventLog.WriteEntry("html = " + retrieval.Html.DocumentNode.OuterHtml);
where the WriteEntrys are just my sanity-check that this is working. But the 2nd isn't getting called and I'm trying to figure out why my code isn't reaching that point.
The above block of code is inside a method that is
MyTimer.Elapsed += new ElapsedEventHandler(MethodThatInvokesTheAboveCode);
and the type of task is like
PageRetrievals.Enqueue(new Task<PageRetrieval>(() =>
new PageRetrieval()
{
Html = PageOpener.GetBoardPage(pagenum),
Page = PageType.Board,
Number = pagenum
}
));
where PageOpener.GetBoardPage simply gets the HTML from a URL, like
private static HtmlDocument GetDocumentFromUrl(string url)
{
var webget = new HtmlWeb();
var doc = webget.Load(url);
return webget.StatusCode == System.Net.HttpStatusCode.OK ? doc : null;
}
public static HtmlDocument GetBoardPage(int pageNumber)
{
return GetDocumentFromUrl(string.Format(BoardPageUrlFormat, pageNumber));
}
Is there anything about this that looks obviously wrong?
I have made two functions in a WCF service and call them in silverlight using asynchronization. I call one method after the other, but before completion of the first method, silverlight executes the second method. I want the first method to completely finish executing before the second method call.
thanks for your reply i m pasting my code please suggest in code how i implement.
private GDOperations.GDDoneOperationsClient _gdDoneOperation;
private ImageOperationsClient proxy = null;
foreach (var file in _filesCollection)
{
clsImageTransactionEntity _clsImageEntity = new clsImageTransactionEntity();
_clsImageEntity.ImageByte = GetFileData(file.OpenRead());
_clsImageEntity.ImageExtension = file.Extension;
_clsImageEntity.ImageName = file.Name;
_clsImageEntity.ImageType = 2;
_clsImageEntity.ImagePath = "~/CMSImages/FinalImages/" + lblSelectedBarcode.Content.ToString() + "/" + file.Name;
_clsImageEntity.JabongBarcode = lblSelectedBarcode.Content.ToString();
GDOperations.clsImageTransactionEntity _clsImageGDEntity = new GDOperations.clsImageTransactionEntity();
_clsImageGDEntity.ImageExtension = file.Extension;
_clsImageGDEntity.ImageName = file.Name;
_clsImageGDEntity.ImageType = 2;
_clsImageGDEntity.ImagePath = "~/CMSImages/FinalImages/" + lblSelectedBarcode.Content.ToString() + "/" + file.Name;
_clsImageGDEntity.JabongBarcode = lblSelectedBarcode.Content.ToString();
_clsImageGDEntity.RoleId = roleID;
_clsImageGDEntity.TaskID = taskID;
_clsImageGDEntity.UserID = UserId;
_clsImageGDEntity.SystemIP = systemIP;
_clsGdAllotment.clsImageTransactionEntity.Add(_clsImageGDEntity);
----- first method calling-----
proxy.UploadFinalImageCompleted += (s, e) =>
{
if (e.Error == null)
{
}
};
proxy.UploadFinalImageAsync(_clsImageEntity);
countfile = countfile + 1;
pbUploadFiles.Value = countfile;
}
_clsGdAllotment.GdID = int.Parse(lblUserID.Content.ToString());
_clsGdAllotment.JabongBarcode = lblSelectedBarcode.Content.ToString();
_clsGdAllotment.TaskID = taskID;
--- after for loop completion calling second method -----
_gdDoneOperation.InsertGDDoneInformationCompleted += _gdDoneOperation_InsertGDDoneInformationCompleted;
_gdDoneOperation.InsertGDDoneInformationAsync(_clsGdAllotment);`
Please help its urgent.
If you're using Task-Based Async Pattern:
var task1 = CallFirstAsyncMethod();
task1.Wait(); // waiting task to finish
var task2 = CallAnotherAsyncMethod();
// or subscribe to the task continuation to call second
// method when first operation will finished
task1.ContinueWith(t =>
{
// add error handling
var task2 = CallAnotherAsyncMethod();
});
If you're using Classical Async Pattern (a.k.a. APM):
IAsyncResult ar1 = CallFirstAsyncMethod();
ar1.WaitHandle.Wait();
IAsyncResult ar2 = CallSecondAsyncMethod();
// or use the same technique asynchronously
CallFirstAsyncMethod(ar => // Suppose we should provide appropriate callback
{
// Call to appropriate EndInvoke method
IAsyncResult ar2 = CallSecondAsyncMethod();
}, state);
You can call the second in the callback of the first, no ?
Or is Visual Studio < 2012
You can use AutoResetEvent :
MyServiceClient clientService;
AutoResetEvent autoResetEvent = new AutoResetEvent(false);
public void Execute()
{
InvokeCompletedEventArgs data = null;
clientService.InvokeCompleted += (e, f) =>
{
data = f;
autoResetEvent.Set();
};
m_proxy.MyCallAync();
autoResetEvent.WaitOne(); // Wait the set of autoResetEvent
m_proxy.MySecondCallAync(data); // Data return by the first call
}
Antoine