I have a program that sends/receives POST requests/responses from an online API, similar to a market bot that buys and sells a commodity. It works great, however when i run it, it locks up the current thread and im unable to use anything else in the program. In the future i would also like to make buying and selling asynchronous, so they could happen at the same time. Here is the code that executes, and as you can see it loops continuously until conditions are met:
private void RunBot()
{
numToBuy = (int)nudNumToBuy.Value;
for (int i = 0; i < numToBuy; i++)
{
while (true)
{
if (AttachAndBuy())
{
break;
}
}
}
}
private bool AttachAndBuy()
{
string data = "<DATA HERE>";
string URL = "<URL HERE>";
Cookies.SetCookies(cookie, "PHPSESSID=" + SessionIDTextBox.Text.Replace("PHPSESSID=", ""));
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create(URL);
Request.ContentType = "application/json";
Request.Accept = "*/*";
Request.CookieContainer = Cookies;
Request.Host = "<HOST HERE>";
Request.Method = "POST";
Request.UserAgent = "<USER AGENT HERE>";
Request.Headers.Add("<HEADER>", "<VALUE>");
Request.KeepAlive = true;
byte[] CompressedRequest = SimpleZlib.CompressToBytes(Encoding.UTF8.GetBytes(data), 9);
Stream RequestStream = Request.GetRequestStream();
RequestStream.Write(CompressedRequest, 0, CompressedRequest.Length);
RequestStream.Flush();
Stream CompressedResponseStream = Request.GetResponse().GetResponseStream();
byte[] CompressedResponseData = ReadToEnd(CompressedResponseStream);
string DecompressedResponseData = SimpleZlib.Decompress(CompressedResponseData, null);
OffersResponse Return = Json.Deserialize<OffersResponse>(DecompressedResponseData);
int LowestCost = 1000000000;
Offer BestOffer = new Offer();
foreach (Offer CurrentOffer in Return.data.offers)
{
bool moneyOffer = false;
int Costs = CurrentOffer.requirementsCost;
string id = CurrentOffer._id;
foreach (Requirement CurrentRequirement in CurrentOffer.requirements)
{
if (CurrentRequirement._tpl == "<TEMPLATE ID HERE>")
{
moneyOffer = true;
}
}
if (moneyOffer == false)
{
continue;
}
if (Costs < LowestCost)
{
LowestCost = Costs;
BestOffer = CurrentOffer;
}
}
BestOfferID = BestOffer._id;
BestOfferCost = LowestCost;
string MoneyID = getStack(BestOfferCost);
while (true)
{
BuyRequestAttemptCounter++;
if (LowestCost > 140000)
{
AddLog("No Suitable Item! Skipping! Lowest Item Cost: " + LowestCost.ToString());
return false;
}
else
AddLog("Best Item Cost: " + LowestCost.ToString() + " | ID: " + BestOfferID);
int Result = buyOrder(MoneyID);
if (Result == 0)
{
//log info for averaging
numberPurchased++;
TotalCost += BestOfferCost;
averageCost = TotalCost / numberPurchased;
lblNumPurchased.Text = numberPurchased.ToString();
lblAverageCost.Text = averageCost.ToString();
lstPricesPurchased.Items.Add(LowestCost.ToString());
AddLog("====================================");
AddLog("Number Purchased: " + numberPurchased);
AddLog("Average Cost: " + averageCost);
AddLog("====================================");
System.Media.SystemSounds.Exclamation.Play();
return true;
}
else if (Result == 1)
return false;
else if (Result == 2)
continue;
else
return false;
}
}
private int buyOrder(string MoneyID)
{
string data = "<DATA HERE>";
string URL = "<URL HERE>";
Cookies.SetCookies(cookie, "PHPSESSID=" + SessionIDTextBox.Text.Replace("PHPSESSID=", ""));
HttpWebRequest Request = (HttpWebRequest)WebRequest.Create(URL);
Request.ContentType = "application/json";
Request.Accept = "*/*";
Request.CookieContainer = Cookies;
Request.Host = "<HOST HERE>";
Request.Method = "POST";
Request.UserAgent = "<USER AGENT HERE>";
Request.Headers.Add("<HEADER>", "<VALUE>");
Request.KeepAlive = true;
byte[] CompressedRequest = SimpleZlib.CompressToBytes(Encoding.UTF8.GetBytes(data), 9);
Stream RequestStream = Request.GetRequestStream();
RequestStream.Write(CompressedRequest, 0, CompressedRequest.Length);
RequestStream.Flush();
Stream CompressedResponseStream = Request.GetResponse().GetResponseStream();
byte[] CompressedResponseData = ReadToEnd(CompressedResponseStream);
string DecompressedResponseData = SimpleZlib.Decompress(CompressedResponseData, null);
ResponseRoot Return = Json.Deserialize<ResponseRoot>(DecompressedResponseData);
string returnErrorCode = DecompressedResponseData.ToString();
//AddLog(DecompressedResponseData);
if (Return.err == 0 && returnErrorCode.Contains("id"))
{
System.Windows.Forms.Clipboard.SetText(DecompressedResponseData);
//AddLog("Successful Purchase!");
return 0;
}
else if (returnErrorCode.Contains("1503"))
{
//AddLog("Failed with 1503!");
return 1;
}
else if (returnErrorCode.Contains("1512"))
{
// AddLog("Failed with 1512!");
return 2;
}
return 3;
}
As stated above, ideally i would like to run both the "attachandbuy" and "buyorder" functions at the same time, and then eventually i will add a sell function, also running concurrently. Is this possible? Thanks.
You can fix this by creating a list of tasks and executing them, this stops the main thread from being blocked while a call hasn't returned yet, e.g. -
var tasks = new List<Task>();
for (int i = 0; i < numToBuy; i++)
{
var task = new Task(() =>
{
AttachAndBuy()
});
tasks.Add(task);
task.Start();
}
Task.WaitAll(tasks.ToArray());
(Note: I've not actually tested this code, it's just a rough example)
Misunderstanding below -
You'll want to use some parallel programming for this - https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/task-based-asynchronous-programming?redirectedfrom=MSDN within your for loop e.g. -
private void RunBot()
{
numToBuy = (int)nudNumToBuy.Value;
for (int i = 0; i < numToBuy; i++)
{
while (true)
{
Parallel.Invoke(() => AttachAndBuy(), () => BuyOrder());
}
}
}
I have almost 200 text files in my laptop, I wrote the code in C# which reads these text files line by line and makes a directory per each line in FTP server.
This is my code:
static void Main()
{
for (int i = 0; i <= 200; i++)
{
var lines = File.ReadAllLines(#"D:\file_" + i.ToString().PadLeft(5, '0') + ".txt");
foreach (var line in lines)
{
try
{
WebRequest request = WebRequest.Create("ftp://myftp/dir/" + line);
request.Method = WebRequestMethods.Ftp.MakeDirectory;
request.Credentials = new NetworkCredential("user", "pass");
request.GetResponse();
}
catch (Exception ex)
{}
}
}
}
But this is very slow to create directories, are there other faster ways to do this? For example, get text file as an array and than create all of its directories.
Reading of the text file is really not the problem. The slow part is the FTP.
Use more threads to parallelize the processing:
List<Task> tasks = new List<Task>();
for (int i = 0; i <= 200; i++)
{
tasks.Add(new Task(() =>
{
var lines =
File.ReadAllLines(#"D:\file_" + i.ToString().PadLeft(5, '0') + ".txt");
foreach (var line in lines)
{
WebRequest request = WebRequest.Create("ftp://myftp/dir/" + line);
request.Method = WebRequestMethods.Ftp.MakeDirectory;
request.Credentials = new NetworkCredential("user", "pass");
request.GetResponse();
}
}
));
}
Task.WaitAll(tasks.ToArray());
Though note that you should also take care of disposing the WebResponse's.
The slow part of your programm are the requests you are sending one by one.
You can do some tricks to speed them up:
// allow more connections at a time
ServicePointManager.DefaultConnectionLimit = 30;
// don't wait the 100ms every request do
ServicePointManager.Expect100Continue = false;
Further you can send the request in multi-threading, so you don't have to wait for every request until it's finished. But be aware, that a lot of request can bring down a server. 200 shouldn't be a problem.
Here you have some code you could test:
static void Main()
{
// allow more connections at a time
ServicePointManager.DefaultConnectionLimit = 30;
// don't wait the 100ms every request do
ServicePointManager.Expect100Continue = false;
List<Task> tasks = new List<Task>();
for (int i = 0; i <= 200; i++)
{
var lines = File.ReadAllLines(#"D:\file_" + i.ToString().PadLeft(5, '0') + ".txt");
foreach (var line in lines)
{
tasks.Add(Task.Run(() =>
{
try
{
WebRequest request = WebRequest.Create("ftp://myftp/dir/" + line);
request.Method = WebRequestMethods.Ftp.MakeDirectory;
request.Credentials = new NetworkCredential("user", "pass");
request.GetResponse();
}
catch (Exception ex)
{ }
}
));
}
}
Task.WaitAll(tasks.ToArray());
}
I have the following code to download some files from a FTP Server :
EDIT : I've solved the problem using DotNet, a good FTP WPF Library !
public partial class MainWindow
{
DispatcherTimer dispatcherTimer = new System.Windows.Threading.DispatcherTimer();
private byte[] downloadedData;
string FTPAddress = "ftp://ftp.cluster007.ovh.net";
double currentBytes;
double oldBytes;
public MainWindow()
{
InitializeComponent();
// DispatcherTimer setup
dispatcherTimer.Tick += new EventHandler(dispatcherTimer_Tick);
dispatcherTimer.Interval = new TimeSpan(0, 0, 1);
}
public static void DoEvents()
{
Application.Current.Dispatcher.Invoke(DispatcherPriority.Background,
new Action(delegate { }));
}
private void dispatcherTimer_Tick(object sender, EventArgs e)
{
currentBytes = Dl_ProgressBar.Value;
Dl_Speed.Content = "Vitesse : " + ((currentBytes - oldBytes) / 1000000).ToString("0.00") + " Mo/s";
oldBytes = Dl_ProgressBar.Value;
// Forcing the CommandManager to raise the RequerySuggested event
CommandManager.InvalidateRequerySuggested();
}
private void Dl_Button_Click(object sender, RoutedEventArgs e)
{
downloadFile();
}
private void downloadFile()
{
downloadedData = new byte[0];
try
{
//Create FTP request
//Note: format is ftp://server.com/file.ext
FtpWebRequest request = FtpWebRequest.Create(FTPAddress + "/" + filename) as FtpWebRequest;
//Get the file size first (for progress bar)
request.Method = WebRequestMethods.Ftp.GetFileSize;
request.Credentials = new NetworkCredential(username, password);
request.UsePassive = true;
request.UseBinary = true;
request.KeepAlive = true; //don't close the connection
int dataLength = (int)request.GetResponse().ContentLength;
Dl_Status.Content = "Téléchargement en cours...";
DoEvents();
//Now get the actual data
request = FtpWebRequest.Create(FTPAddress + "/" + filename) as FtpWebRequest;
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential(username, password);
request.UsePassive = true;
request.UseBinary = true;
request.KeepAlive = false; //close the connection when done
//Set up progress bar
Dl_ProgressBar.Value = 0;
Dl_ProgressBar.Maximum = dataLength;
//Streams
FtpWebResponse response = request.GetResponse() as FtpWebResponse;
Stream reader = response.GetResponseStream();
//Download to memory
//Note: adjust the streams here to download directly to the hard drive
MemoryStream memStream = new MemoryStream();
byte[] buffer = new byte[1024]; //downloads in chuncks
dispatcherTimer.Start();
while (true)
{
DoEvents(); //prevent application from crashing
int bytesRead = reader.Read(buffer, 0, buffer.Length);
if (bytesRead == 0)
{
//Nothing was read, finished downloading
Dl_ProgressBar.Value = Dl_ProgressBar.Maximum;
Dl_Percent.Content = "Progression : 100%";
DoEvents();
break;
}
else
{
//Write the downloaded data
memStream.Write(buffer, 0, bytesRead);
//Update the progress bar
if (Dl_ProgressBar.Value + bytesRead <= Dl_ProgressBar.Maximum)
{
Dl_ProgressBar.Value += bytesRead;
Dl_Percent.Content = "Progression : " + ((Dl_ProgressBar.Value / 1000000000000000) * dataLength).ToString("0.00") + "%";
DoEvents();
}
}
}
//Convert the downloaded stream to a byte array
downloadedData = memStream.ToArray();
//Clean up
reader.Close();
memStream.Close();
response.Close();
Dl_Status.Content = "Téléchargement terminé";
DoEvents();
}
catch (Exception)
{
Dl_Status.Content = "Erreur de connexion au FTP";
}
}
}
My problem is that when I pass the mouse over the window, the download speed is dropping significantly...
It changes from 3.70Mb/s to 2.20Mb/s.
When I have the mouse out of the window, there's no problem, but when I'm over it, it slows down, particularly when I do some very short movements, the download speed go to 0.20Mb/s.
I've tried to use Threads and Dispatcher but it was the same.
To answer your specific question, WPF's Dispatcher uses a priority queue, and Input level events (like those originating from mouse movement) take priority over Background level events. Your DoEvents() method periodically drains the message queue of all events with Background priority or higher, so when you move the mouse over the window, the queue fills up with input events to process. This means that DoEvents takes longer to return, and more time elapses before you can resume processing the download.
That said, this is a terrible way to accomplish a download; you should never use this kind of DoEvents() hack in WPF; do some research on the async and await features of C# (or, if that is not an option, BackgroundWorker). You will find many examples on StackOverflow of how to perform asynchronous downloads without having to resort to this sort of Dispatcher trickery to keep the UI responsive.
I am having a hard time convert the below code which i have created in 4.0 to 4.5 using HttpClient.
According to my understand i guess if i create multiple web requests in the GUI thread itself without blocking the GUI if i got with asynchronous requeest.
how to convert the below code to Asynchronous using HttpClient in 4.5
// This is what called when button is clicked
Task t3 = new Task(SpawnTask);
t3.Start();
//if noofthreads are less 50 then GUI is woking fine.. if number increases then takes much time for repaint..
//where as other softwares are working without any problem even if the threads are more than 500!! in the same system
public void SpawnTask()
{
try
{
ParallelOptions po = new ParallelOptions();
po.CancellationToken = cts.Token;
po.MaxDegreeOfParallelism = noofthreads;
Parallel.ForEach(
urls,
po,
url => checkpl(url));
}
catch (Exception ex)
{
}
}
public void checkpl(string url)
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Timeout = 60*1000;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string stext = "";
using (BufferedStream buffer = new BufferedStream(response.GetResponseStream()))
{
using (StreamReader reader = new StreamReader(buffer))
{
stext = reader.ReadToEnd();
}
}
response.Close();
if (stext .IndexOf("domainname.com") != -1)
{
tfound = tfound + 1;
string lext = "Total Found : "+tfound.ToString();
label3.BeginInvoke(new InvokeDelegate(UpdateLabel), ltext);
slist.Add(url);
textBox2.BeginInvoke(new InvokeDelegate4(UpdateText), "Working Url " + url);
}
}
catch (Exception ex)
{
}
}
Since you are using .NET 4.5 you can use the new async and await keywords. Here is what it might look like.
private async void YourButton_Click(object sender, EventArgs args)
{
YourButton.Enabled = false;
try
{
var tasks = new List<Task>();
foreach (string url in Urls)
{
tasks.Add(CheckAsync(url));
}
await TaskEx.WhenAll(tasks);
}
finally
{
YourButton.Enabled = true;
}
}
private async Task CheckAsync(string url)
{
bool found = await UrlResponseContainsAsync(url, "domainname.com");
if (found)
{
slist.Add(url);
label3.Text = "Total Found: " + slist.Count.ToString();
textbox2.Text = "Working Url " + url;
}
}
private async Task<bool> UrlResponseContainsAsync(string url, string find)
{
var request = WebRequest.Create(url);
request.Timeout = 60 * 1000;
using (WebResponse response = await request.GetResponseAsync())
{
using (var buffer = new BufferedStream(response.GetResponseStream()))
using (var reader = new StreamReader(buffer))
{
string text = reader.ReadToEnd();
return text.Contains(find);
}
}
}
I've got WPF application I'm writing that posts files to one of social networks.
Upload itself working just fine, but I'd like to provide some indication of how far along I am with the uploading.
I tried a bunch of ways to do this:
1) HttpWebRequest.GetStream method:
using (
var FS = File.Open(
localFilePath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
long len = FS.Length;
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url);
request.Method = "POST";
request.ProtocolVersion = HttpVersion.Version11;
request.ContentType = "multipart/form-data; boundary=--AaB03x";
//predata and postdata is two byte[] arrays, that contains
//strings for MIME file upload (defined above and is not important)
request.ContentLength = predata.Length + FS.Length + postdata.Length;
request.AllowWriteStreamBuffering = false;
using (var reqStream = request.GetRequestStream())
{
reqStream.Write(predata, 0, predata.Length);
int bytesRead = 0;
int totalRead = 0;
do
{
bytesRead = FS.Read(fileData, 0, MaxContentSize);
totalRead += bytesRead;
reqStream.Write(fileData, 0, bytesRead);
reqStream.Flush(); //trying with and without this
//this part will show progress in percents
sop.prct = (int) ((100*totalRead)/len);
} while (bytesRead > 0);
reqStream.Write(postdata, 0, postdata.Length);
}
HttpWebResponse responce = (HttpWebResponse) request.GetResponse();
using (var respStream = responce.GetResponseStream())
{
//do things
}
}
2) WebClient way (much shorter):
void UploadFile (url, localFilePath)
{
...
WebClient client = new WebClient();
client.UploadProgressChanged += new UploadProgressChangedEventHandler(UploadPartDone);
client.UploadFileCompleted += new UploadFileCompletedEventHandler(UploadComplete);
client.UploadFileAsync(new Uri(url), localFilePath);
done.WaitOne();
//do things with responce, received from UploadComplete
JavaScriptSerializer jssSer = new JavaScriptSerializer();
return jssSer.Deserialize<UniversalJSONAnswer>(utf8.GetString(UploadFileResponce));
//so on...
...
}
void UploadComplete(object sender, UploadFileCompletedEventArgs e)
{
UploadFileResponce=e.Result;
done.Set();
}
void UploadPartDone(object sender, UploadProgressChangedEventArgs e)
{
//this part expected to show progress
sop.prct=(int)(100*e.BytesSent/e.TotalBytesToSend);
}
3) Even TcpClient way:
using (
var FS = File.Open(
localFilePath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
long len = FS.Length;
long totalRead = 0;
using (var client = new TcpClient(urli.Host, urli.Port))
{
using (var clearstream = client.GetStream())
{
using (var writer = new StreamWriter(clearstream))
using (var reader = new StreamReader(clearstream))
{
//set progress to 0
sop.prct = 0;
// Send request headers
writer.WriteLine("POST " + urli.AbsoluteUri + " HTTP/1.1");
writer.WriteLine("Content-Type: multipart/form-data; boundary=--AaB03x");
writer.WriteLine("Host: " + urli.Host);
writer.WriteLine("Content-Length: " + (predata.Length + len + postdata.Length).ToString());
writer.WriteLine();
//some data for MIME
writer.Write(utf8.GetString(predata));
writer.Flush();
int bytesRead;
do
{
bytesRead = FS.Read(fileData, 0, MaxContentSize);
totalRead += bytesRead;
writer.BaseStream.Write(fileData, 0, bytesRead);
writer.BaseStream.Flush();
sop.prct = (int) ((100*totalRead)/len);
} while (bytesRead > 0)
writer.Write(utf8.GetString(postdata));
writer.Flush();
//read line of response and do other thigs...
respStr = reader.ReadLine();
...
}
}
}
}
In all cases the file was successfully sent to the server.
But always progress looks like this: for a few seconds it runs from 0 to 100 and then waits until file actually uploading (about 5 minutes - file is 400MB).
So I think the data from a file is buffered somewhere and I'm tracking not uploading, but buffering data. And then must wait until it's uploaded.
My questions are:
1) Is there any way to track actual uploading data? That the method Stream.Write() or Flush() (which as I read somewhere, does not work for NetworkStream) did not return until it receives confirmation from the server that the TCP packets received.
2) Or can I deny buffering (AllowWriteStreamBUffering for HttpWebRequest doesn't work)?
3) And does it make sense to go further "down" and try with Sockets?
updated:
To avoid any doubts in the way of progress displaying on UI, I rewrote the code to log a file.
so, here is code:
using (var LogStream=File.Open("C:\\123.txt",FileMode.Create,FileAccess.Write,FileShare.Read))
using (var LogWriter=new StreamWriter(LogStream))
using (var FS = File.Open(localFilePath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
long len = FS.Length;
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url);
request.Timeout = 7200000; //2 hour timeout
request.Method = "POST";
request.ProtocolVersion = HttpVersion.Version11;
request.ContentType = "multipart/form-data; boundary=--AaB03x";
//predata and postdata is two byte[] arrays, that contains
//strings for MIME file upload (defined above and is not important)
request.ContentLength = predata.Length + FS.Length + postdata.Length;
request.AllowWriteStreamBuffering = false;
LogWriter.WriteLine(DateTime.Now.ToString("o") + " Start write into request stream. ");
using (var reqStream = request.GetRequestStream())
{
reqStream.Write(predata, 0, predata.Length);
int bytesRead = 0;
int totalRead = 0;
do
{
bytesRead = FS.Read(fileData, 0, MaxContentSize);
totalRead += bytesRead;
reqStream.Write(fileData, 0, bytesRead);
reqStream.Flush(); //trying with and without this
//sop.prct = (int) ((100*totalRead)/len); //this part will show progress in percents
LogWriter.WriteLine(DateTime.Now.ToString("o") + " totalRead= " + totalRead.ToString() + " / " + len.ToString());
} while (bytesRead > 0);
reqStream.Write(postdata, 0, postdata.Length);
}
LogWriter.WriteLine(DateTime.Now.ToString("o") + " All sent!!! Waiting for responce... ");
LogWriter.Flush();
HttpWebResponse responce = (HttpWebResponse) request.GetResponse();
LogWriter.WriteLine(DateTime.Now.ToString("o") + " Responce received! ");
using (var respStream = responce.GetResponseStream())
{
if (respStream == null) return null;
using (var streamReader = new StreamReader(respStream))
{
string resp = streamReader.ReadToEnd();
JavaScriptSerializer jssSer = new JavaScriptSerializer();
return jssSer.Deserialize<UniversalJSONAnswer>(resp);
}
}
}
and here is result (I cut the middle):
2011-11-19T22:00:54.5964408+04:00 Start write into request stream.
2011-11-19T22:00:54.6404433+04:00 totalRead= 1048576 / 410746880
2011-11-19T22:00:54.6424434+04:00 totalRead= 2097152 / 410746880
2011-11-19T22:00:54.6434435+04:00 totalRead= 3145728 / 410746880
2011-11-19T22:00:54.6454436+04:00 totalRead= 4194304 / 410746880
2011-11-19T22:00:54.6464437+04:00 totalRead= 5242880 / 410746880
2011-11-19T22:00:54.6494438+04:00 totalRead= 6291456 / 410746880
.......
2011-11-19T22:00:55.3434835+04:00 totalRead= 408944640 / 410746880
2011-11-19T22:00:55.3434835+04:00 totalRead= 409993216 / 410746880
2011-11-19T22:00:55.3464837+04:00 totalRead= 410746880 / 410746880
2011-11-19T22:00:55.3464837+04:00 totalRead= 410746880 / 410746880
2011-11-19T22:00:55.3464837+04:00 All sent!!! Waiting for responce...
2011-11-19T22:07:23.0616597+04:00 Responce received!
as you can see program thinks that it uploaded ~400MB for about 2 seconds. And after 7 minutes file actually uploads and I receive responce.
updated again:
Seems to this is happening under WIndows 7 (not shure about x64 or x86).
When I run my code uder XP everything works perfectly and progress is shown absolute correctly
it's more than year since this question was posted, but I think my post can be usefull for someone.
I had the same problem with showing progress and it behaved exactly like you described. So i decided to use HttpClient which shows upload progress correctly. Then I've encountered interesting bug - when I had Fiddler launched HttpClient started to show its upload progress in unexpected way like in WebClient/HttpWebRequest above so I thinked maybe that was a problem of why WebClient showed upload progres not correctly (I think I had it launched). So I tried with WebClient again (without fiddler-like apps launched) and all works as it should, upload progress has correct values. I have tested in on several PC with win7 and XP and in all cases progress was showing correctly.
So, I think that such program like Fiddler (probably not only a fiddler) has some affect on how WebClient and other .net classes shows upload progress.
this discussion approves it:
HttpWebRequest doesn't work except when fiddler is running
You could use the WebClient's UploadFile to upload file rather than using writing file as a file stream. In order to track the percentage of the data received and uploaded you can use UploadFileAsyn and subscribe to its events.
In the code bellow I've used UploadFileAsyn to the upload files synchronously, but it need not to be synchronous as far as you don't dispose the instance of the uploader.
class FileUploader : IDisposable
{
private readonly WebClient _client;
private readonly Uri _address;
private readonly string _filePath;
private bool _uploadCompleted;
private bool _uploadStarted;
private bool _status;
public FileUploader(string address, string filePath)
{
_client = new WebClient();
_address = new Uri(address);
_filePath = filePath;
_client.UploadProgressChanged += FileUploadProgressChanged;
_client.UploadFileCompleted += FileUploadFileCompleted;
}
private void FileUploadFileCompleted(object sender, UploadFileCompletedEventArgs e)
{
_status = (e.Cancelled || e.Error == null) ? false : true;
_uploadCompleted = true;
}
private void FileUploadProgressChanged(object sender, UploadProgressChangedEventArgs e)
{
if(e.ProgressPercentage % 10 == 0)
{
//This writes the pecentage data uploaded and downloaded
Console.WriteLine("Send: {0}, Received: {1}", e.BytesSent, e.BytesReceived);
//You can have a delegate or a call back to update your UI about the percentage uploaded
//If you don't have the condition (i.e e.ProgressPercentage % 10 == 0 )for the pecentage of the process
//the callback will slow you upload process down
}
}
public bool Upload()
{
if (!_uploadStarted)
{
_uploadStarted = true;
_client.UploadFileAsync(_address, _filePath);
}
while (!_uploadCompleted)
{
Thread.Sleep(1000);
}
return _status;
}
public void Dispose()
{
_client.Dispose();
}
}
Client Code:
using (FileUploader uploader = new FileUploader("http://www.google.com", #"C:\test.txt"))
{
uploader.Upload();
}
You can register a custom callback (may be a delegate) on the FileUploadProgressChanged event handler to update your WPF UI.
The upload progress changed event get called more often if your callback for the event does any IO then that'll slowdown the download progress. It's best to have infrequent update e.g. the following code update only evey 10% up.
private int _percentageDownloaded;
private void FileUploadProgressChanged(object sender, UploadProgressChangedEventArgs e)
{
if (e.ProgressPercentage % 10 == 0 && e.ProgressPercentage > _percentageDownloaded)
{
_percentageDownloaded = e.ProgressPercentage;
//Any callback instead of printline
Console.WriteLine("Send: {0} Received: {1}", e.BytesSent, e.BytesReceived);
}
}
my suggestion is to use new HTTPClient class (available in .NET 4.5). It supports progress.
This article helped me a lot with this:
http://www.strathweb.com/2012/06/drag-and-drop-files-to-wpf-application-and-asynchronously-upload-to-asp-net-web-api/
My code for upload file:
private void HttpSendProgress(object sender, HttpProgressEventArgs e)
{
HttpRequestMessage request = sender as HttpRequestMessage;
Console.WriteLine(e.BytesTransferred);
}
private void Window_Loaded_1(object sender, RoutedEventArgs e)
{
ProgressMessageHandler progress = new ProgressMessageHandler();
progress.HttpSendProgress += new EventHandler<HttpProgressEventArgs>(HttpSendProgress);
HttpRequestMessage message = new HttpRequestMessage();
StreamContent streamContent = new StreamContent(new FileStream("e:\\somefile.zip", FileMode.Open));
message.Method = HttpMethod.Put;
message.Content = streamContent;
message.RequestUri = new Uri("{Here your link}");
var client = HttpClientFactory.Create(progress);
client.SendAsync(message).ContinueWith(task =>
{
if (task.Result.IsSuccessStatusCode)
{
}
});
}
This one has been bugging me for at least one day. I have started with using WebClient.UploadFileAsync, next tried the ProgressMessageHandler for HttpClient then rolled my own HttpContent for the HttpClient API. None of those approaches worked (for me).
It appears HttpWebRequest, which sits at the bottom of most (all?) .NET Http abstraction like WebClient and HttpClient, buffers the request and response stream by default, which I confirmed by looking at it in ILSpy.
As others have noted, you can make your request use chunked encoding one way or another which will effectively disable buffering the request stream, but still this is not going to fix the progress reporting.
I found that it was necessary to flush the request stream after each block that I send in order to accurately reflect sending progress, or else your data will simply be buffered one step further down the pipeline (probably somewhere in NetworkStream or OS, didn't check). The sample code below works for me and also does a minimalistic job at translating back from a HttpWebResponse to HttpResponseMessage (which you may not need, YMMV).
public async Task<HttpResponseMessage> UploadFileAsync( string uploadUrl, string absoluteFilePath, Action<int> progressPercentCallback )
{
var length = new FileInfo( absoluteFilePath ).Length;
var request = new HttpWebRequest( new Uri(uploadUrl) ) {
Method = "PUT",
AllowWriteStreamBuffering = false,
AllowReadStreamBuffering = false,
ContentLength = length
};
const int chunkSize = 4096;
var buffer = new byte[chunkSize];
using (var req = await request.GetRequestStreamAsync())
using (var readStream = File.OpenRead(absoluteFilePath))
{
progressPercentCallback(0);
int read = 0;
for (int i = 0; i < length; i += read)
{
read = await readStream.ReadAsync( buffer, 0, chunkSize );
await req.WriteAsync( buffer, 0, read );
await req.FlushAsync(); // flushing is required or else we jump to 100% very fast
progressPercentCallback((int)(100.0 * i / length));
}
progressPercentCallback(100);
}
var response = (HttpWebResponse)await request.GetResponseAsync();
var result = new HttpResponseMessage( response.StatusCode );
result.Content = new StreamContent( response.GetResponseStream() );
return result;
}
At fast guess, you are running this code on UI thread. You need to run upload stuff on new thread.
At that point you have 2 options. 1) You run timer on UI thread and update UI. 2) You update UI using Invoke(because you can't access UI from another thread) calls to update UI.
In the first example I think your progress bar is showing how fast you write into the stream from the file on disk - not the actual upload progress (which is why it all happens to 100% really quickly then the upload chugs on*).
I might be wrong ^^ and have no WPF experience but I have uploaded massive files from Silverlight to WCF and the model used there is (as you do) to break up the file into blocks. Send each block. When you get a response from the server ("block 26 received ok"), update the progress bar as really, you can't (or should not) update the progress bar unless you /know/ that block x made it - and a good way to know that is if the server says it got it.
*I wish I could upload 400Mb in 5 mins. Would take me all day...
I had the same problem. I spent a lot of time and solved the problem as follows:
Antivirus AVAST. When I turn it off my program works perfectly...