I am using the WebClient class in C#, 4.0. I need to hit a REST service with 30,000 different IDs, and grab the status result (200 or 404). Here is the method that makes the calls (eventCounter is a CountdownEvent object):
private void doWork()
{
initDB();
List<string> _lines = new List<string>();
//pull all UpcIds into a List
using (StreamReader _rdr = new StreamReader(#"C:\Users\kkohut\Dropbox\ROVI\Application Support\BestBuy\upc_test2.txt"))
{
string _line;
while ((_line = _rdr.ReadLine()) != null)
{
_lines.Add(_line);
}
}
numIds = _lines.Count();
for (int i = 0; i < numIds; i++)
{
string _upcId = _lines[i];
WebClient c = new WebClient();
c.DownloadDataCompleted += new DownloadDataCompletedEventHandler(c_DownloadDataCompleted);
c.DownloadDataAsync(new Uri(BASE_URL + _upcId), _upcId);
}
//this is not working correctly. Code execution hits this line and waits, without processing any of the
//the DownloadDataCompleted eventhandlers
eventCounter.Wait();
}
Here is the DownloadDataCompleted event handler
void c_DownloadDataCompleted(object sender, DownloadDataCompletedEventArgs e)
{
DataSet _ds = new DataSet();
string _upcId = e.UserState.ToString();
string _status = "404";
try
{
if (!e.Cancelled && e.Error == null)
{
string _result = System.Text.Encoding.UTF8.GetString(e.Result);
if (_result.IndexOf("<code>200</code>") > 0)
{
_status = "200";
}
}
}
catch (Exception ex)
{
_status = "404";
}
finally
{
updateDB(_upcId, _status);
eventCounter.Signal(1);
txtLog.Text += string.Format("{0}\t{1}\t{2}\r\n",ctr, _upcId, _status);
}
}
If I comment out the eventCounter.Wait() statement, the calls work, but I have no way of knowing when they complete. This is a winforms app, so as long as I keep the form running, all the calls complete. But if I uncomment the eventCounter.Wait() statement, no calls get processed. It appears that the Wait() statement is blocking the async calls to start with. Every example I have found uses this approach, but none of them are signaling the CountdownEvent in the completed event handler. Thoughts?
The WebClient Class implements the Event-based Asynchronous Pattern (EAP).
In this pattern, the XXXAsync Method captures the current SynchronizationContext (i.e. the UI thread in a WPF or WinForms application). When the operation completes, the event handler is executed in this context.
(See also: On which thread(s) does WebClient raise its events?)
Problem: If you invoke a blocking method on the UI thread, the event handler will not run before the blocking method returns.
Solution: Wait asynchronously for the CountdownEvent to complete, not synchronously.
You can use the ThreadPool.RegisterWaitForSingleObject Method to register a callback for the WaitHandle of the CountdownEvent.
Related
Here is my code:
public void ConnectToWorldServer()
{
if (socketReady)
{
return;
}
//Default host and port values;
string host = ClientWorldServer.ServerIP;
int port = ClientWorldServer.TCPPort;
//ClientLoginServer ClientLoginServer = new ClientLoginServer();
try
{
socket = new TcpClient(host, port);
stream = socket.GetStream();
socket.NoDelay = true;
writer = new StreamWriter(stream);
reader = new StreamReader(stream);
socketReady = true;
//Preserve the connection to worldserver thrue scenes
UnityThread.executeInUpdate(() =>
{
DontDestroyOnLoad(worldserverConnection);
});
// Start listening for connections.
while (true)
{
if (socketReady)
{
if (stream.DataAvailable)
{
string sdata = reader.ReadLine();
if (sdata != null)
{
Task<JsonData> jsonConvert = Task<JsonData>.Factory.StartNew(() => convertJson(sdata));
UnityThread.executeInUpdate(() =>
{
OnIncomingData(jsonConvert.Result);
});
}
}
}
}
}
catch (Exception e)
{
Debug.Log("Socket error : " + e.Message);
}
}
private JsonData convertJson(string data)
{
return JsonConvert.DeserializeObject<JsonData>(data);
}
What I am wondering now is does this part of the code:
UnityThread.executeInUpdate(() =>
{
OnIncomingData(jsonConvert.Result);
});
block until this task returns back a result:
Task<JsonData> jsonConvert = Task<JsonData>.Factory.StartNew(() => convertJson(sdata));
I am really not that familiar with Tasks. My goal is to run the json conversion and then execute OnIncomingData(jsonConvert.Result);.
I think my code is not doing that. Why?
When a thread invokes Task.Result it will block until the Task completes, either by returning a value, throwing an exception, or being canceled. From the documentation:
Accessing the property's get accessor blocks the calling thread until the asynchronous operation is complete; it is equivalent to calling the Wait method.
So, to be clear, calling Task<JsonData>.Factory.StartNew creates a Task (which represents some computation to be executed), and schedules it for execution (when it gets executed and on what thread is up to the default TaskScheduler, but StartNew should return immediately). Your call to UnityThread.executeInUpdate will then happen without waiting for the Task you created to complete. At the point where UnityThread calls the anonymous function you passed to executeInUpdate that thread will block until the Task completes. I'm not familiar with UnityThread.executeInUpdate so I cannot tell you whether it will block until that callback completes or not.
One thing to be aware of is that depending on how Unity works with threads, it is possible to create a deadlock by accessing the Result property. In some cases a Task will attempt to use a specific context to execute, and if you cause that context to block waiting for the Task to complete, it will never get a chance to run: https://blog.stephencleary.com/2012/07/dont-block-on-async-code.html
If you want to wait for the result then what is the point of using the Task. The right way of doing thing asynchronously is making your function async all the way.
public async void ConnectToWorldServer()
{
.....
.....
// Here await will put this method call on a queue to finish later and will return from this method.
Task<JsonData> jsonConvert = await Task<JsonData>.Factory.StartNew(() => convertJson(sdata));
// After the task is finished, it will resume to this method here to execute next statement.
UnityThread.executeInUpdate(() =>
{
OnIncomingData(jsonConvert.Result);
});
.....
.....
}
The Windows Forms GUI thread keeps stalling for a second now and then during async HttpClient SendAsync. Except for that, everything works fine (I get data). I want to do frequent and hopefully parallel requests to different servers to update the screen frequently. I am trying to launch multiple requests that post the responses for processing by the GUI thread later. The code below is a simplification of my code.
I've checked the time before and after SendAsync and see it is sometimes up to 2 seconds while the GUI window is frozen (can't be moved, scrolled, etc) and the polling timer is inactive (counter never incremented).
Using async Task DoWork did not help.
class Worker
{
HttpClient client = new HttpClient();
bool busy = false;
string data = "";
//public async Task DoWork()
public async void DoWork()
{
if ( busy ) return;
busy = true;
HttpRequestMessage request = new HttpRequestMessage(method, requestUrl);
HttpResponseMessage response = await client.SendAsync( request );
data = ... from response ...
busy = false;
}
}
int counter;
private void Update(object sender, EventArgs e)
{
++counter;
foreach ( Worker worker in workers )
worker.DoWork();
}
...
List<Worker> workers = ...
var poll = new System.Windows.Forms.Timer();
poll.Tick += Update;
poll.Interval = 250;
poll.Enabled = true;
...
If you are using Windows.Forms you may consider BackgroundWorker
SendAsync is sufficiently asynchronous (even if it uses threads internally).
Updating UI itself is really slow and the cause of UI freezing.
When I execute the following code:
public static async Task UploadFile(string serverPath, string pathToFile, string authToken)
{
serverPath = #"C:\_Series\S1\The 100 S01E03.mp4";
var client = new WebClient();
var uri = new Uri($"http://localhost:50424/api/File/Upload?serverPath={WebUtility.UrlEncode(serverPath)}");
client.UploadProgressChanged += UploadProgressChanged;
client.UploadFileCompleted += UploadCompletedCallback;
//client.UploadFileAsync(uri, "POST", pathToFile);
client.UploadFile(uri, "POST", pathToFile);
}
I get the exception:
System.Net.WebException: 'The remote server returned an error: (404)
Not Found.'
I'm not too worried about the 404, I'm busy tracing down why the WebClient can't find it, but my big concern is that if I call UploadFileAsync with the same uri, the method just executes as if nothing is wrong.
The only indication that something is wrong is that neither of the two event handlers is invoked. I strongly suspect that I don't get an exception because the async call is not async/await but event based, but then I would expect some kind of event or property that indicates an exception has occurred.
How is one supposed to use code that hides errors like this, especially network errors which are relatively more common, in production?
Why no error notification for UploadFileAsync with WebClient?
Citing WebClient.UploadFileAsync Method (Uri, String, String) Remarks
The file is sent asynchronously using thread resources that are automatically allocated from the thread pool. To receive notification when the file upload completes, add an event handler to the UploadFileCompleted event.
Emphasis mine.
You get no errors because it is being executed on another thread so as not to block the current thread. To see the error you can access it in the stated event handler via the UploadFileCompletedEventArgs.Exception.
I was curious as to why using WebClient and not HttpClient which is already primarily async, but then my assumption was because of the upload progress.
I would suggest wrapping the WebClient call with event handlers in a Task using a TaskCompletionSource to take advantage of TAP.
The following is similar to the examples provided here How to: Wrap EAP Patterns in a Task
public static async Task UploadFileAsync(string serverPath, string pathToFile, string authToken, IProgress<int> progress = null) {
serverPath = #"C:\_Series\S1\The 100 S01E03.mp4";
using (var client = new WebClient()) {
// Wrap Event-Based Asynchronous Pattern (EAP) operations
// as one task by using a TaskCompletionSource<TResult>.
var task = client.createUploadFileTask(progress);
var uri = new Uri($"http://localhost:50424/api/File/Upload?serverPath={WebUtility.UrlEncode(serverPath)}");
client.UploadFileAsync(uri, "POST", pathToFile);
//wait here while the file uploads
await task;
}
}
Where createUploadFileTask is a custom extension method used to wrap the Event-Based Asynchronous Pattern (EAP) operations of the WebClient as one task by using a TaskCompletionSource<TResult>.
private static Task createTask(this WebClient client, IProgress<int> progress = null) {
var tcs = new TaskCompletionSource<object>();
#region callbacks
// Specifiy the callback for UploadProgressChanged event
// so it can be tracked and relayed through `IProgress<T>`
// if one is provided
UploadProgressChangedEventHandler uploadProgressChanged = null;
if (progress != null) {
uploadProgressChanged = (sender, args) => progress.Report(args.ProgressPercentage);
client.UploadProgressChanged += uploadProgressChanged;
}
// Specify the callback for the UploadFileCompleted
// event that will be raised by this WebClient instance.
UploadFileCompletedEventHandler uploadCompletedCallback = null;
uploadCompletedCallback = (sender, args) => {
// unsubscribing from events after asynchronous
// events have completed
client.UploadFileCompleted -= uploadCompletedCallback;
if (progress != null)
client.UploadProgressChanged -= uploadProgressChanged;
if (args.Cancelled) {
tcs.TrySetCanceled();
return;
} else if (args.Error != null) {
// Pass through to the underlying Task
// any exceptions thrown by the WebClient
// during the asynchronous operation.
tcs.TrySetException(args.Error);
return;
} else
//since no result object is actually expected
//just set it to null to allow task to complete
tcs.TrySetResult(null);
};
client.UploadFileCompleted += uploadCompletedCallback;
#endregion
// Return the underlying Task. The client code
// waits on the task to complete, and handles exceptions
// in the try-catch block there.
return tcs.Task;
}
Going one step further and creating another extension method to wrap the upload file to make it await able...
public static Task PostFileAsync(this WebClient client, Uri address, string fileName, IProgress<int> progress = null) {
var task = client.createUploadFileTask(progress);
client.UploadFileAsync(address, "POST", fileName);//this method does not block the calling thread.
return task;
}
Allowed your UploadFile to be refactored to
public static async Task UploadFileAsync(string serverPath, string pathToFile, string authToken, IProgress<int> progress = null) {
using (var client = new WebClient()) {
var uri = new Uri($"http://localhost:50424/api/File/Upload?serverPath={WebUtility.UrlEncode(serverPath)}");
await client.PostFileAsync(uri, pathToFile, progress);
}
}
This now allow you to call the upload asynchronously and even keep track of the progress with your very own Progress Reporting (Optional)
For example if in an XAML based platform
public class UploadProgressViewModel : INotifyPropertyChanged, IProgress<int> {
public int Percentage {
get {
//...return value
}
set {
//...set value and notify change
}
}
public void Report(int value) {
Percentage = value;
}
}
Or using the out of the box Progress<T> Class
So now you should be able to upload the file without blocking the thread and still be able to await it, get progress notifications, and handle exceptions, provided you have a try/catch in place.
I have a situation that i export data to a file and what i have been asked to do is to provide a cancel button which on click will stop the export if it takes too much time to export.
I started exporting to the file in a thread. And i try to abort the thread on the button click. But it do not work.
I searched on Google and i found that abort() is not recommended. But what else should I choose to achieve it?
My current code is:
private void ExportButtonClick(object param)
{
IList<Ur1R2_Time_Points> data = ct.T_UR.ToList();
DataTable dtData = ExportHelper.ToDataTable(data);
thread = new Thread(new ThreadStart(()=>ExportHelper.DataTableToCsv(dtData, "ExportFile.csv")));
thread.SetApartmentState(ApartmentState.STA);
thread.IsBackground = true;
thread.Name = "PDF";
thread.Start();
}
private void StopButtonClick(object param)
{
if (thread.Name == "PDF")
{
thread.Interrupt();
thread.Abort();
}
}
Aborting a thread is a bad idea, especially when dealing with files. You won't have a chance to clean up half-written files or clean-up inconsistent state.
It won't harm the .NET Runtime bat it can hurt your own application eg if the worker method leaves global state, files or database records in an inconsistent state.
It's always preferable to use cooperative cancellation - the thread periodically checks a coordination construct like a ManualResetEvent or CancellationToken. You can't use a simple variable like a Boolean flag, as this can lead to race conditions, eg if two or more threads try to set it at the same time.
You can read about cancellation in .NET in the Cancellation in Managed Threads section of MSDN.
The CancellationToken/CancellationTokenSource classes were added in .NET 4 to make cancellation easier that passing around events.
In your case, you should modify your DataTableToCsv to accept a CancellationToken. That token is generated by a CancellationTokenSource class.
When you call CancellationTokenSource.Cancel the token's IsCancellationRequested property becomes true. Your DataTableToCsv method should check this flag periodically. If it's set, it should exit any loops, delete any inconsistent files etc.
Timeouts are directly supported with CancelAfter. Essentially, CancelAfter starts a timer that will fire Cancel when it expires.
Your code could look like this:
CancellationTokenSource _exportCts = null;
private void ExportButtonClick(object param)
{
IList<Ur1R2_Time_Points> data = ct.T_UR.ToList();
DataTable dtData = ExportHelper.ToDataTable(data);
_exportCts=new CancellationTokenSource();
var token=_exportCts.Token;
thread = new Thread(new ThreadStart(()=>
ExportHelper.DataTableToCsv(dtData, "ExportFile.csv",token)));
thread.SetApartmentState(ApartmentState.STA);
thread.IsBackground = true;
thread.Name = "PDF";
_exportCts.CancelAfter(10000);
thread.Start();
}
private void StopButtonClick(object param)
{
if (_exportCts!=null)
{
_exportCts.Cancel();
}
}
DataTableToCsv should contain code similar to this:
foreach(var row in myTable)
{
if (token.IsCancellationRequested)
{
break;
}
//else continue with processing
var line=String.Join(",", row.ItemArray);
writer.WriteLine(line);
}
You can clean up your code quite a bit by using tasks instead of raw threads:
private async void ExportButtonClick(object param)
{
IList<Ur1R2_Time_Points> data = ct.T_UR.ToList();
DataTable dtData = ExportHelper.ToDataTable(data);
_exportCts=new CancellationTokenSource();
var token=_exportCts.Token;
_exportCts.CancelAfter(10000);
await Task.Run(()=> ExportHelper.DataTableToCsv(dtData, "ExportFile.csv",token)));
MessageBox.Show("Finished");
}
You could also speed it up by using asynchronous operations, eg to read data from the database or write to text files without blocking or using threads. Windows IO (both file and network) is asynchronous at the driver level. Methods like File.WriteLineAsync don't use threads to write to a file.
Your Export button handler could become :
private void ExportButtonClick(object param)
{
IList<Ur1R2_Time_Points> data = ct.T_UR.ToList();
DataTable dtData = ExportHelper.ToDataTable(data);
_exportCts=new CancellationTokenSource();
var token=_exportCts.Token;
_exportCts.CancelAfter(10000);
await Task.Run(async ()=> ExportHelper.DataTableToCsv(dtData, "ExportFile.csv",token)));
MessageBox.Show("Finished");
}
and DataTableToCsv :
public async Task DataTableToCsv(DataTable table, string file,CancellationToken token)
{
...
foreach(var row in myTable)
{
if (token.IsCancellationRequested)
{
break;
}
//else continue with processing
var line=String.Join(",", row.ItemArray);
await writer.WriteLineAsync(line);
}
You can use a boolean flag. Use a volatile boolean for that.
In the helper do something like:
this.aborted = false;
while(!finished && !aborted) {
//process one row
}
Whenever you want to cancel the operation, you call a method to set aborted to true:
public void Abort() {
this.aborted = true;
}
Have a read here: https://msdn.microsoft.com/en-us/library/system.threading.threadabortexception(v=vs.110).aspx
When a call is made to the Abort method to destroy a thread, the common language runtime throws a ThreadAbortException. ThreadAbortException is a special exception that can be caught, but it will automatically be raised again at the end of the catch block. When this exception is raised, the runtime executes all the finally blocks before ending the thread. Because the thread can do an unbounded computation in the finally blocks or call Thread.ResetAbort to cancel the abort, there is no guarantee that the thread will ever end. If you want to wait until the aborted thread has ended, you can call the Thread.Join method. Join is a blocking call that does not return until the thread actually stops executing.
Since Thread.Abort() is executed by another thread, it can happen anytime and when it happens ThreadAbortException is thrown on target thread.
Inside ExportHelper.DataTableToCsv:
catch(ThreadAbortException e) {
Thread.ResetAbort();
}
On StopButtonClick
if (thread.Name == "PDF")
{
thread.Interrupt();
thread.Join();
}
To Stop a thread you have one option of Thread.Abort.However because this method thrown ThreadAbortException on the target thread when it executed by another thead.
Which is not recommended.
The second option to stop a thread is by using shared variable that both your target and your calling thread can access.
See the Example ::
public static class Program
{
public static void ThreadMethod(object o)
{
for (int i = 0; i < (int)o; i++)
{
Console.WriteLine("ThreadProc: { 0}", i);
Thread.Sleep(0);
}
}
public static void Main()
{
bool stopped = false;
Thread t = new Thread(new ThreadStart(() =>
{
while (!stopped)
{
Console.WriteLine("Running...");
Thread.Sleep(1000);
}
}));
t.Start();
Console.WriteLine("Press any key to exit");
Console.ReadKey();
stopped = true;
t.Join();
}
}
//Source :: Book --> Programming in c#
I have specific problem with WebClient in my Windows Phone app (using MVVM)
private string _lastCurrencyRatesJson;
private bool _lastCurrencyRatesJsonLoaded = false;
private void GetLastCoursesFromApiAsync()
{
var uri = new Uri(string.Format(OperationGetLastCourses, AppSettings.ApiEndpoint, AppSettings.ApiKey));
var client = new WebClient { Encoding = Encoding.UTF8 };
client.DownloadStringCompleted += client_DownloadStringCompleted;
client.DownloadStringAsync(uri);
}
void client_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
_lastCurrencyRatesJson = e.Result;
_lastCurrencyRatesJsonLoaded = true;
}
public List<CurrencyRate> GetLastCourses()
{
var worker = new Thread(GetLastCoursesFromApiAsync);
worker.Start();
while (!_lastCurrencyRatesJsonLoaded)
{
}
.....
The problem is that client_DownloadStringCompleted is never fired BUT when I change GetLastCourses this way:
public List<CurrencyRate> GetLastCourses()
{
var worker = new Thread(GetLastCoursesFromApiAsync);
worker.Start();
// whetever here, but any while...
client_DownloadStringCompleted is fired and data are obtained. It means, connectivity is ok.
I had very similar problems with DownloadStringTaskAsyn. Example:
private async Task<string> GetCoursesForDayFromApiAsJson(DateTime date)
{
var uri = new Uri(string.Format(OperationGetCoursesForDay, AppSettings.ApiEndpoint, AppSettings.ApiKey, date.ToString(DateFormat)));
var client = new WebClient { Encoding = Encoding.UTF8 };
return await client.DownloadStringTaskAsync(uri);
}
Again, at the line with await is application waiting for the data but the DownloadStringTaskAsync is never finished and my UI is still loading.
Any ideas what could be wrong?
SITUATION ONE DAY AGO
So, it looks that WP application is working just with one thread. It means, current thread have to be "finished" and then is DownloadStringTaskAsync finished and the code under the await executed. When I want to work with Task.Result I can not. Never.
When I create another Thread and I am trying to wait for thread completetion (using Join()), created Thread is never finsihed and the code after Join() is never executed.
There is any example on the Internet and I absolutely don't know, why exists some DownloadStringTaskAsync when it is not applicable.
You're blocking the UI thread by your while loop and at the same time, the DownloadStringCompleted event wants to execute on the UI loop. This causes a deadlock, so nothing happens. What you need to do is to let GetLastCourses() return (and whatever method calls that), so that the event handler can execute. This means that the code that handles the results should be in that event handler (not in GetLastCourses()).
With async-await, you didn't provide all of your code, but it's likely that you're encountering pretty much the same issue by calling Wait() or Result on the returned Task. If replace that with await, you code will work. Though that requires you to make all your code from GetCoursesForDayFromApiAsJson() up async.
I'd recommend to use the HttpClient class from Microsoft NuGet package and use the async/await downloading pattern rather than using event-based WebClient class:
Uri uri = new Uri(string.Format(OperationGetLastCourses, AppSettings.ApiEndpoint, AppSettings.ApiKey));
using (HttpClient client = new HttpClient())
{
string result = await client.GetStringAsync(uri);
}