I wrote an async function for calling data from Facebook, it works, but the problem is I dun suppose it works. Can someone explain to me?
public class FacebookData
{
static string fb_api_version = ConfigurationManager.AppSettings["fb_ver"];
static string accessToken = ConfigurationManager.AppSettings["accessToken"];
static string fb_id = "";
private HttpClient _httpClient;
public FacebookData(string input_id)
{
fb_id = input_id;
_httpClient = new HttpClient
{
BaseAddress = new Uri("https://graph.facebook.com/" + fb_api_version + "/"),
Timeout = TimeSpan.FromSeconds(15)
};
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
public async Task<T> getData<T>()
{
var response = await _httpClient.GetAsync($"{fb_id}?access_token={accessToken}");
if (!response.IsSuccessStatusCode)
return default(T);
var result = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(result);
}
}
The calling class is typical, I make it await for the response.
But the problem is where I call it.
In main
static void Main(string[] args)
{
string[] data_Set = [//ids_group]
for (int i = 0; i < data_Set.length; ++i){
Console.WriteLine("Running Thread " + (i+1).ToString());
var dataSet = facebookRequestCmd(data_Set[i]);
writeToTXT(dataSet);
Console.WriteLine("Finished Thread " + (i + 1).ToString());
//do sth
}
}
In facebookRequestCmd
static Dictionary<string, string[]> facebookRequestCmd(string ids){
Dictionary<string, string[]> allData = new Dictionary<string, string[]>();
string[] ids_arr = ids.split(",")
for (var i = 0; i < ids.length; i++){
var facebook_client = new FacebookData(sqlData);
var response = facebook_client.getData<dynamic>();
Task.WaitAll(response);
//then use the result to do sth
}
}
In my understanding, each time I call getData, it already come back to main thread, as it is awaiting the data. So the Task doesn't really start a new thread.
Thus, async await works for waiting the http request, but the Threading should not work.
However,
Console.WriteLine("Running Thread " + (i+1).ToString());
jumps out simultaneously like I really make the Thread in the for loop in main function.
Why? And is that the way to use Multithreading with Async-await. As I want to make multiple calls at the same time.
Originally I use Parallel.ForEach to kick starting the calling, however, thats not asynchronous and will block the thread.
Ok, feel free to ignore all the changes I've made but I couldn't help but modify the way some of the variables read and the code looked. This is not a working application and I have obviously not tested it. This is just a cleaned up version of what you have with a suggested way of using Task. It's also mocked using just the code you've provided so it is what it is. #2 is, what I believe, the answer you needed.
In Main I removed the words 'thread' since that's not actually what's happening. It may be, but we don't know if the HttpClient is indeed starting a new thread or just holding / returning from the rest call. Using async / await does not always mean a Thread was started (although it's common to think of it that way).
I used .Result (not Wait() as I suggested in comments) to get the result of the task. This is ok since it's a console app but not ideal for a real world app that needs to operate without blocking. I also removed Task.WaitAll with this change.
I renamed functions to have verbage because, IMO, functions should be doing work and the naming should describe the work being done.
I renamed some variables because, IMO, variables should be PascalCase when their scope isn't in a method or private and camelCase when they are. The names should also, IMO, be what it is followed by the Type that makes sense.
I appended 'Async' to function names that return a running Task.
Changed FacebookClient to be singleton and allowing only one HttpClient to be used instead of many and allowing it to be disposed; plus more.
Added alternate version of the GetFacebookData function that calls the tasks and awaits them all simultaneously.
static void Main(string[] args)
{
string[] dataSet = new string[] { /* mocked */ }; // [ids_group]; <- no idea what this is so I mocked it.
for (int i = 0; i < dataSet.Length; i++)
{
Console.WriteLine("Main... " + (i + 1).ToString());
var result = GetFacebookData(dataSet[i]);
WriteToTxt(result);
Console.WriteLine("Complete... " + (i + 1).ToString());
//do sth
}
Console.Read();
}
private static Dictionary<string, string[]> GetFacebookData(string idsString)
{
var allDataDictionary = new Dictionary<string, string[]>();
var idsArray = idsString.Split(',');
foreach (var id in idsArray)
{
var response = FacebookClient.Instance.GetDataAsync<string[]>(id).Result;
allDataDictionary.Add(id, response);
}
return allDataDictionary;
}
public class FacebookClient
{
private readonly HttpClient httpClient;
private readonly string facebookApiVersion;
private readonly string accessToken;
public static FacebookClient Instance { get; } = new FacebookClient();
FacebookClient()
{
facebookApiVersion = ConfigurationManager.AppSettings["fb_ver"];
accessToken = ConfigurationManager.AppSettings["accessToken"];
httpClient = new HttpClient
{
BaseAddress = new Uri("https://graph.facebook.com/" + facebookApiVersion + "/"),
Timeout = TimeSpan.FromSeconds(15)
};
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
public async Task<T> GetDataAsync<T>(string facebookId)
{
var response = await httpClient.GetAsync($"{facebookId}?access_token={accessToken}");
if (!response.IsSuccessStatusCode) return default;
var result = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(result);
}
~FacebookClient() => httpClient.Dispose();
}
Here's a version that's starting all the tasks and then awaiting them all at the same time. I believe this might give you some issues on the HttpClient but we'll see.
private static Dictionary<string, string[]> GetFacebookData(string idsString)
{
var allDataDictionary = new Dictionary<string, string[]>();
var idsArray = idsString.Split(',');
var getDataTasks = new List<Task<string[]>>();
foreach (var id in idsArray)
{
getDataTasks.Add(FacebookClient.Instance.GetDataAsync<string[]>(id));
}
var tasksArray = getDataTasks.ToArray();
Task.WaitAll(tasksArray);
var resultsArray = tasksArray.Select(task => task.Result).ToArray();
for (var i = 0; i < idsArray.Length; i++)
{
allDataDictionary.Add(idsArray[i], resultsArray[i]);
}
return allDataDictionary;
}
Related
I wrote a web crawler and I want to know if my approach is correct. The only issue I'm facing is that it stops after some hours of crawling. No exception, it just stops.
1 - the private members and the constructor:
private const int CONCURRENT_CONNECTIONS = 5;
private readonly HttpClient _client;
private readonly string[] _services = new string[2] {
"https://example.com/items?id=ID_HERE",
"https://another_example.com/items?id=ID_HERE"
}
private readonly List<SemaphoreSlim> _semaphores;
public Crawler() {
ServicePointManager.DefaultConnectionLimit = CONCURRENT_CONNECTIONS;
_client = new HttpClient();
_semaphores = new List<SemaphoreSlim>();
foreach (var _ in _services) {
_semaphores.Add(new SemaphoreSlim(CONCURRENT_CONNECTIONS));
}
}
Single HttpClient instance.
The _services is just a string array that contains the URL, they are not the same domain.
I'm using semaphores (one per domain) since I read that it's not a good idea to use the network queue (I don't remember how it calls).
2 - The Run method, which is the one I will call to start crawling.
public async Run(List<int> ids) {
const int BATCH_COUNT = 1000;
var svcIndex = 0;
var tasks = new List<Task<string>>(BATCH_COUNT);
foreach (var itemId in ids) {
tasks.Add(DownloadItem(svcIndex, _services[svcIndex].Replace("ID_HERE", $"{itemId}")));
if (++svcIndex >= _services.Length) {
svcIndex = 0;
}
if (tasks.Count >= BATCH_COUNT) {
var results = await Task.WhenAll(tasks);
await SaveDownloadedData(results);
tasks.Clear();
}
}
if (tasks.Count > 0) {
var results = await Task.WhenAll(tasks);
await SaveDownloadedData(results);
tasks.Clear();
}
}
DownloadItem is an async function that actually makes the GET request, note that I'm not awaiting it here.
If the number of tasks reaches the BATCH_COUNT, I will await all to complete and save the results to file.
3 - The DownloadItem function.
private async Task<string> DownloadItem(int serviceIndex, string link) {
var needReleaseSemaphore = true;
var result = string.Empty;
try {
await _semaphores[serviceIndex].WaitAsync();
var r = await _client.GetStringAsync(link);
_semaphores[serviceIndex].Release();
needReleaseSemaphore = false;
// DUE TO JSON SIZE, I NEED TO REMOVE A VALUE (IT'S USELESS FOR ME)
var obj = JObject.Parse(r);
if (obj.ContainsKey("blah")) {
obj.Remove("blah");
}
result = obj.ToString(Formatting.None);
} catch {
result = string.Empty;
// SINCE I GOT AN EXCEPTION, I WILL 'LOCK' THIS SERVICE FOR 1 MINUTE.
// IF I RELEASED THIS SEMAPHORE, I WILL LOCK IT AGAIN FIRST.
if (!needReleaseSemaphore) {
await _semaphores[serviceIndex].WaitAsync();
needReleaseSemaphore = true;
}
await Task.Delay(60_000);
} finally {
// RELEASE THE SEMAPHORE, IF NEEDED.
if (needReleaseSemaphore) {
_semaphores[serviceIndex].Release();
}
}
return result;
}
4- The function that saves the result.
private async Task SaveDownloadedData(List<string> myData) {
using var fs = new FileStream("./output.dat", FileMode.Append);
foreach (var res in myData) {
var blob = Encoding.UTF8.GetBytes(res);
await fs.WriteAsync(BitConverter.GetBytes((uint)blob.Length));
await fs.WriteAsync(blob);
}
await fs.DisposeAsync();
}
5- Finally, the Main function.
static async Task Main(string[] args) {
var crawler = new Crawler();
var items = LoadItemIds();
await crawler.Run(items);
}
After all this, is my approach correct? I need to make millions of requests, will take some weeks/months to gather all data I need (due to the connection limit).
After 12 - 14 hours, it just stops and I need to manually restart the app (memory usage is ok, my VPS has 1 GB and it never used more than 60%).
static void Main(string[] args)
{
token objtoken = new token();
var location = AddLocations();
OutPutResults outPutResultsApi = new OutPutResults();
GCPcall gCPcall = new GCPcall();
OutPutResults finaloutPutResultsApi = new OutPutResults();
var addressdt = new AddressDataDetails();
finaloutPutResultsApi.addressDatas = new List<AddressDataDetails>();
Console.WriteLine("Hello World!");
List<string> placeId = new List<string>();
var baseUrl = "https://maps.googleapis.com/maps/api/place/textsearch/json?";
var apiKey = "&key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx".Trim();
foreach (var itemlocations in location)
{
var searchtext = "query=" + itemlocations.Trim();
var finalUrl = baseUrl + searchtext + apiKey;
gCPcall.RecursiveApiCall(finalUrl, ref placeId, objtoken.NextToken);
}
var ids = gCPcall.myPalceid;
}
public List<string> RecursiveApiCall(string finalUrl, ref List<string> placeId, string nextToken = null)
{
try
{
var token = "&pagetoken=" + nextToken;
using (var client = new HttpClient())
{
var responseTask = client.GetAsync(finalUrl + token);
responseTask.Wait();
var result = responseTask.Result;
if (result.IsSuccessStatusCode)
{
var readTask = result.Content.ReadAsStringAsync();
readTask.Wait();
var students = readTask.Result;
Rootobject studentsmodel = JsonConvert.DeserializeObject<Rootobject>(students);
nextToken = studentsmodel.next_page_token;
foreach (var item in studentsmodel.results)
{
placeId.Add(item.place_id);
}
}
}
if (nextToken != null)
{
RecursiveApiCall(finalUrl, ref placeId, nextToken);
}
return placeId;
}
catch (Exception ex)
{
throw;
}
}
My recursive method has some issue. Here whenever I am debugging this code it work fine. It goes in recursive call twice.
As debugging result I am getting list place_id with 20 items in first call and next call 9 items total 29 items in place_id object which is correct in static main method.
But if I run without debugging mode I am getting only 20 place_id. next recursive iteration data is missing even if it has next valid token.
I don't have any clue why this is happening. Can someone tell me what is the issue with my code?
Here are my suggestions, which may or may not solve the problem:
// First of all, let's fix the signature : Go async _all the way_
//public List<string> RecursiveApiCall(string finalUrl, ref List<string> placeId, string nextToken = null)
// also reuse the HttpClient!
public async Task ApiCallAsync(HttpClient client, string finalUrl, List<string> placeId, string nextToken = null)
{
// Loop, don't recurse
while(!(nextToken is null)) // C# 9: while(nextToken is not null)
{
try
{
var token = "&pagetoken=" + nextToken;
// again async all the way
var result = await client.GetAsync(finalUrl+token);
if (result.IsSuccessStatusCode)
{
// async all the way!
var students = await result.Content.ReadAsStringAsync();
Rootobject studentsmodel = JsonConvert.DeserializeObject<Rootobject>(students);
nextToken = studentsmodel.next_page_token;
foreach (var item in studentsmodel.results)
{
// Will be reflected in main, so no need to return or `ref` keyword
placeId.Add(item.place_id);
}
}
// NO recursion needed!
// if (nextToken != null)
// {
// RecursiveApiCall(finalUrl, ref placeId, nextToken);
// }
}
catch (Exception ex)
{
// rethrow, only is somewhat useless
// I'd suggest using a logging framework and
// log.Error(ex, "Some useful message");
throw;
// OR remove try/catch here all together and wrap the call to this method
// in try / catch with logging.
}
}
Mind that you'll need to make your main :
async Task Main(string[] args)
and call this as
await ApiCallAsync(client, finalUrl, placeId, nextToken);
Also create an HttpClient in main and reuse that:
"HttpClient is intended to be instantiated once and re-used throughout the life of an application. Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads. This will result in SocketException errors." - Remarks
... which shouldn't do much harm here, but it's a "best practice" anyhow.
Now, as to why you get only 20 instead of expected 29 items, I cannot say if this resolves that issue. I'd highly recommend to introduce a Logging Framework and make log entries accordingly, so you may find the culprid.
I am trying to call HttpClient request inside for loop as follows. It needs to do multiple consecutive calls to third party rest api.
But it only gives me fist service call result while loop exit before getting result from rest of the service call.
private void Search()
{
try
{
var i = 1;
using (var httpClient = new HttpClient())
{
while (i < 5)
{
string url = "https://jsonplaceholder.typicode.com/posts/" + i;
var response = httpClient.GetAsync(url).Result;
string jsonResult = response.Content.ReadAsStringAsync().Result;
Console.WriteLine(jsonResult.ToString());
i++;
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
When I run with debug points the program gives me all the result. But when I run it without debug points it gives me only the first result.
I tried this with using async, await methods too. It also gives me same result.
As I feel Program needs to wait until the async call returns data.
Please help me to solve this.
EDIT - async way
private async Task<string> SearchNew()
{
try
{
var i = 1;
var res = string.Empty;
using (var httpClient = new HttpClient())
{
while (i < 5)
{
string url = "https://jsonplaceholder.typicode.com/posts/" + i;
var response = httpClient.GetAsync(url).Result;
string jsonResult = await response.Content.ReadAsStringAsync();
res = res + jsonResult + " --- ";
i++;
}
}
return res;
}
catch (Exception ex)
{
return ex.Message;
}
}
Both are giving same result.
There's a few things here that you should be doing. First, move the HttpClient creation outside of your method and make it static. You only need one of them and having multiple can be really bad for stability (see here):
private static HttpClient _client = new HttpClient();
Next, extract the calls to the HttpClient into a single method, something simple like this:
//Please choose a better name than this
private async Task<string> GetData(string url)
{
var response = await _client.GetAsync(url);
return await response.Content.ReadAsStringAsync();
}
And finally, you create a list of tasks and wait for them all to complete asynchronously using Task.WhenAll:
private async Task<string[]> SearchAsync()
{
var i = 1;
var tasks = new List<Task<string>>();
//Create the tasks
while (i < 5)
{
string url = "https://jsonplaceholder.typicode.com/posts/" + i;
tasks.Add(GetData(url));
i++;
}
//Wait for the tasks to complete and return
return await Task.WhenAll(tasks);
}
And to call this method:
var results = await SearchAsync();
foreach (var result in results)
{
Console.WriteLine(result);
}
I am trying to make lots of calls from my server to a REST API that is exposed by another server but the code is taking too long to run.
Below is the code which takes a lot of time. Right now I am using C# Async/Await along with HTTPClient, is there a better way I can do this?
static async Task RunAsync()
{
HttpClient client = new HttpClient();
client.BaseAddress = new Uri(URL);
foreach (var workbk in (workbooksforuserresultingMessage.Items[1] as workbookListType).workbook)
{
if (workbk.project.name == "Ascend")
{
tsResponse viewresultingMessage = null;
//Get View Data
HttpRequestMessage viewrequestMessage = new HttpRequestMessage(HttpMethod.Get, "sites/" + ((siteresultingMessage.Items[0] as siteType).id.ToString()) + "/workbooks/" + workbk.id + "/views");
// Add our custom headers
viewrequestMessage.Headers.Add("X-Tableau-Auth", (resultingMessage.Items[0] as credentialsType).token.ToString());
HttpResponseMessage viewrequestMessageresponse = await client.SendAsync(viewrequestMessage);
if (viewrequestMessageresponse.IsSuccessStatusCode)
{
var viewresponsecontent = await viewrequestMessageresponse.Content.ReadAsStringAsync();
XmlSerializer siteserializer = new XmlSerializer(typeof(tsResponse));
using (TextReader reader = new StringReader(viewresponsecontent))
{
viewresultingMessage = (tsResponse)siteserializer.Deserialize(reader);
}
}
}
}
foreach (var workbk in (workbooksforuserresultingMessage.Items[1] as workbookListType).workbook)
{
if (workbk.project.name == "Ascend")
{
foreach (var vu in workbk.views)
{
tsResponse viewImageresultingMessage = null;
//Get View Data
HttpRequestMessage viewImagerequestMessage = new HttpRequestMessage(HttpMethod.Get, "sites/" + ((siteresultingMessage.Items[0] as siteType).id.ToString()) + "/workbooks/" + workbk.id + "/views/" + vu.id + "/previewImage");
// Add our custom headers
viewImagerequestMessage.Headers.Add("X-Tableau-Auth", (resultingMessage.Items[0] as credentialsType).token.ToString());
HttpResponseMessage viewImagewrequestMessageresponse = await client.SendAsync(viewImagerequestMessage);
if (viewImagewrequestMessageresponse.IsSuccessStatusCode)
{
var viewImageresponsecontent = await viewImagewrequestMessageresponse.Content.ReadAsByteArrayAsync();
XmlSerializer siteserializer = new XmlSerializer(typeof(tsResponse));
using (var ms = new MemoryStream(viewImageresponsecontent))
{
Image returnImage = Image.FromStream(ms);
//return returnImage;
}
}
}
}
}
}
Those 'await's inside your loop will serialize the requests. Consider instead using .ContinueWith on the task to do some work with the result, keep track of the tasks in an array, then call Task.WhenAll once you're done setting up all the parallel work.
AS #TheESJ said the awaits make your code execute in a serial fashion. That is once execution reaches the first await, execution is effectively "paused" and will not resume until that Task is complete. You can avoid this by not awaiting the Task, just add it to a list of tasks to keep track of.
To facilitate this I think it will help if you introduced a couple of helper methods to execute the body of your loops and return a Task.
public Task<tsResponse> GetViewString(HttpRequestMessage viewrequestMessage)
{
tsResponse viewresultingMessage = null;
HttpResponseMessage viewrequestMessageresponse = await client.SendAsync(viewrequestMessage);
if (viewrequestMessageresponse.IsSuccessStatusCode)
{
var viewresponsecontent = await viewrequestMessageresponse.Content.ReadAsStringAsync();
XmlSerializer siteserializer = new XmlSerializer(typeof(tsResponse));
using (TextReader reader = new StringReader(viewresponsecontent))
{
viewresultingMessage = (tsResponse)siteserializer.Deserialize(reader);
}
}
return viewresultingMessage;
}
public Task<tsResponse> GetViewImage(HttpRequestMessage viewImagerequestMessage)
{
tsResponse viewImageresultingMessage = null;
//Get View Data
HttpResponseMessage viewImagewrequestMessageresponse = await client.SendAsync(viewImagerequestMessage);
if (viewImagewrequestMessageresponse.IsSuccessStatusCode)
{
var viewImageresponsecontent = await viewImagewrequestMessageresponse.Content.ReadAsByteArrayAsync();
XmlSerializer siteserializer = new XmlSerializer(typeof(tsResponse));
using (var ms = new MemoryStream(viewImageresponsecontent))
{
return Image.FromStream(ms);
}
}
}
Then in your main loop you just add the tasks to a list as the methods are kicked off, this will essentially execute the tasks in parallel. You may need to increase the number of concurrent connections your server can make if it causes a problem. Your main method then becomes.
static async Task RunAsync()
{
HttpClient client = new HttpClient();
client.BaseAddress = new Uri(URL);
var tasks = new List<Task<tsResponse>>();
foreach (var workbk in (workbooksforuserresultingMessage.Items[1] as workbookListType).workbook)
{
if (workbk.project.name == "Ascend")
{
tsResponse viewresultingMessage = null;
//Get View Data
HttpRequestMessage viewrequestMessage = new HttpRequestMessage(HttpMethod.Get, "sites/" + ((siteresultingMessage.Items[0] as siteType).id.ToString()) + "/workbooks/" + workbk.id + "/views");
// Add our custom headers
viewrequestMessage.Headers.Add("X-Tableau-Auth", (resultingMessage.Items[0] as credentialsType).token.ToString());
tasks.Add(GetViewString(viewrequestMessage);
}
}
foreach (var workbk in (workbooksforuserresultingMessage.Items[1] as workbookListType).workbook)
{
if (workbk.project.name == "Ascend")
{
foreach (var vu in workbk.views)
{
tsResponse viewImageresultingMessage = null;
//Get View Data
HttpRequestMessage viewImagerequestMessage = new HttpRequestMessage(HttpMethod.Get, "sites/" + ((siteresultingMessage.Items[0] as siteType).id.ToString()) + "/workbooks/" + workbk.id + "/views/" + vu.id + "/previewImage");
// Add our custom headers
viewImagerequestMessage.Headers.Add("X-Tableau-Auth", (resultingMessage.Items[0] as credentialsType).token.ToString());
tasks.Add(GetViewImage(viewImagerequestMessage);
}
}
}
// wait for all the tasks to complete (non blocking)
await Task.WhenAll(tasks);
}
I'm using the following code to post an image to a server.
var image= Image.FromFile(#"C:\Image.jpg");
Task<string> upload = Upload(image);
upload.Wait();
public static async Task<string> Upload(Image image)
{
var uriBuilder = new UriBuilder
{
Host = "somewhere.net",
Path = "/path/",
Port = 443,
Scheme = "https",
Query = "process=false"
};
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("locale", "en_US");
client.DefaultRequestHeaders.Add("country", "US");
var content = ConvertToHttpContent(image);
content.Headers.ContentType = MediaTypeHeaderValue.Parse("image/jpeg");
using (var mpcontent = new MultipartFormDataContent("--myFakeDividerText--")
{
{content, "fakeImage", "myFakeImageName.jpg"}
}
)
{
using (
var message = await client.PostAsync(uriBuilder.Uri, mpcontent))
{
var input = await message.Content.ReadAsStringAsync();
return "nothing for now";
}
}
}
}
I'd like to modify this code to run multiple threads. I've used "ThreadPool.QueueUserWorkItem" before and started to modify the code to leverage it.
private void UseThreadPool()
{
int minWorker, minIOC;
ThreadPool.GetMinThreads(out minWorker, out minIOC);
ThreadPool.SetMinThreads(1, minIOC);
int maxWorker, maxIOC;
ThreadPool.GetMaxThreads(out maxWorker, out maxIOC);
ThreadPool.SetMinThreads(4, maxIOC);
var events = new List<ManualResetEvent>();
foreach (var image in ImageCollection)
{
var resetEvent = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(
arg =>
{
var img = Image.FromFile(image.getPath());
Task<string> upload = Upload(img);
upload.Wait();
resetEvent.Set();
});
events.Add(resetEvent);
if (events.Count <= 0) continue;
foreach (ManualResetEvent e in events) e.WaitOne();
}
}
The problem is that only one thread executes at a time due to the call to "upload.Wait()". So I'm still executing each thread in sequence. It's not clear to me how I can use PostAsync with a thread-pool.
How can I post images to a server using multiple threads by tweaking the code above? Is HttpClient PostAsync the best way to do this?
I'd like to modify this code to run multiple threads.
Why? The thread pool should only be used for CPU-bound work (and I/O completions, of course).
You can do concurrency just fine with async:
var tasks = ImageCollection.Select(image =>
{
var img = Image.FromFile(image.getPath());
return Upload(img);
});
await Task.WhenAll(tasks);
Note that I removed your Wait. You should avoid using Wait or Result with async tasks; use await instead. Yes, this will cause async to grow through you code, and you should use async "all the way".