We Keep Coding

C# HTTP request with proxy [duplicate]

I am using a web client class in my source code for downloading a string using http.
This was working fine. However, the clients in the company are all connected now to a proxy server. And the problem started from this.
When I have tested my application I don't think it can pass through the proxy server, as the exception that keeps getting thrown is "no response from xxx.xxx.xxx.xxx which is the proxy server IP address.
However, I can still navigate to the web site URL and it displays the string correctly in the browser when connecting through a proxy server, but not when I use my web client.
Is there something in the web client that I have to configure to allow me to access the url from behind a proxy server?
using (WebClient wc = new WebClient())
{
string strURL = "http://xxxxxxxxxxxxxxxxxxxxxxxx";
//Download only when the webclient is not busy.
if (!wc.IsBusy)
{
string rtn_msg = string.Empty;
try
{
rtn_msg = wc.DownloadString(new Uri(strURL));
return rtn_msg;
}
catch (WebException ex)
{
Console.Write(ex.Message);
return false;
}
catch (Exception ex)
{
Console.Write(ex.Message);
return false;
}
}
else
{
System.Windows.Forms.MessageBox.Show("Busy please try again");
return false;
}
}

My solution:
WebClient client = new WebClient();
WebProxy wp = new WebProxy(" proxy server url here");
client.Proxy = wp;
string str = client.DownloadString("http://www.google.com");

If you need to authenticate to the proxy, you need to set UseDefaultCredentials to false, and set the proxy Credentials.
WebProxy proxy = new WebProxy();
proxy.Address = new Uri("mywebproxyserver.com");
proxy.Credentials = new NetworkCredential("usernameHere", "pa****rdHere"); //These can be replaced by user input
proxy.UseDefaultCredentials = false;
proxy.BypassProxyOnLocal = false; //still use the proxy for local addresses
WebClient client = new WebClient();
client.Proxy = proxy;
string doc = client.DownloadString("http://www.google.com/");
If all you need is a simple proxy, you skip most of the lines above though. All you need is:
WebProxy proxy = new WebProxy("mywebproxyserver.com");

The answer proposed by Jonathan is proper, but requires that you specify the proxy credentials and url in the code. Usually, it is better to allow usage of the credentials as setup in the system by default (Users typically configure LAN Settings anyway in case they use a proxy)...
The below answer has been provided by Davide in earlier answer, but that requires modifying the app.config files. This solution is probably more useful since it does the same thing IN CODE.
In order to let the application use the default proxy settings as used in the user's system, one can use the following code:
IWebProxy wp = WebRequest.DefaultWebProxy;
wp.Credentials = CredentialCache.DefaultCredentials;
wc.Proxy = wp;
This will allow the application code to use the proxy (with logged-in credentials and default proxy url settings)... No headaches! :)
Hope this helps future viewers of this page to solve their problem!

I've encountered the same issue but using a webclient for downloading a file from the internet with a Winform application the solution was adding in the app.config:
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>
The same solution will work for an asp.net app inserting the same rows in web.config.
Hope it will help.

You need to configure the proxy in the WebClient object.
See the WebClient.Proxy property:
http://msdn.microsoft.com/en-us/library/system.net.webclient.proxy(VS.80).aspx

byte[] data;
using (WebClient client = new WebClient())
{
ICredentials cred;
cred = new NetworkCredential("xmen#test.com", "mybestpassword");
client.Proxy = new WebProxy("192.168.0.1",8000);
client.Credentials = cred;
string myurl="http://mytestsite.com/source.jpg";
data = client.DownloadData(myUrl);
}
File.WriteAllBytes(#"c:\images\target.jpg", data);

All previous answers have some merit, but the actual answer only needs ONE line:
wc.Proxy = new WebProxy("127.0.0.1", 8888);
where wc is the WebClient object, and 8888 is the port number of the proxy server located on the same machine.

ftp-ssl fails after "Expecting TLS Negotiation"

The following code for simple secure-ftp fails on my Windows 7 computer but succeeds on a Server 2012 computer on a different network: If I remove the EnableSsl = true it works fine.
public static string TheContent(string url, string username, string password)
{
ServicePointManager.ServerCertificateValidationCallback = OnValidateCertificate;//returns true.
string result = "";
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.Timeout = 7000;
request.EnableSsl = true;
request.UsePassive = true;
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.Credentials = new NetworkCredential(username, password);
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
result = reader.ReadToEnd();
return result;
}
I've tried many things. Some of them:
Disabling the firewall. (Though I'm using passive mode)
Setting port forwarding on my router.
Using "active".
Using FileZilla (fails the same way)
Connecting to the modem by cable
Much longer timeout
And more.
I always get an exception at (FtpWebResponse)request.GetResponse()
"Unable to read data from the transport connection: A connection
attempt failed because the connected party did not properly respond
after a period of time, or established connection failed because
connected host has failed to respond."
When using Microsoft Network Monitor 3.4 I see that the error seems to be:
FTP:Response to Port , '234 AUTH command ok. Expecting
TLS Negotiation.'
FTP:Response to Port , '451 The parameter is incorrect.
'
Any ideas? Can it be that the non-server Windows aren't capable for this by default and something has to be enabled first?

Why does my FTP upload fail intermittently?

I have a bit of C# code (part of a scheduled SSIS job) that uploads a file an FTP. The file is already present on the FTP site, so the job is just overwriting it with a newer version.
Most of the time it works great, but every few weeks it will begin to fail. The most recent time this has happened, I found that the initial error is "427 Aborted, file write error". Other times I have received "451, Local error in processing". After the initial failure, every subsequent time the job tries to run it returns "550 File Unavailable". When I open the site using Filezilla after getting this error, I find that my target file is there but empty. If I delete the empty file using Filezilla, the job will begin working again.
I have altered my code to first delete the file in question, then upload the new one in hopes that will serve as a workaround. I have been unable to reproduce this error, so time will tell if the workaround helps.
That said, I would like to figure out the root cause. I've been able to find very little info on the 427 error. Anyone have any idea how to avoid it or an idea for a better workaround?
Note: the FTP server is a proprietary thing, and I don't have much control over it other than username, password, & port.
My code is below (edited for brevity)...
private void CreateFile(FtpWebRequest request, FtpWebResponse response, string ipAddress, Int16 port, string folderName, string username, string password)
{
string filename = "myfile.csv";
request = (FtpWebRequest)FtpWebRequest.Create(string.Format("ftp://{0}:{1}/{2}/{3}", ipAddress, port, folderName, filename));
request.KeepAlive = false;
request.Credentials = new System.Net.NetworkCredential(username, password);
request.Proxy = null;
request.UsePassive = false;
request.Method = WebRequestMethods.Ftp.DeleteFile;
request.GetResponse();
request = (FtpWebRequest)FtpWebRequest.Create(string.Format("ftp://{0}:{1}/{2}/{3}", ipAddress, port, folderName, filename));
request.KeepAlive = false;
request.Credentials = new System.Net.NetworkCredential(username, password);
request.Proxy = null;
request.UsePassive = false;
request.Method = WebRequestMethods.Ftp.UploadFile;
using (StreamWriter sw = new StreamWriter(request.GetRequestStream()))
{
DataTable myList = new DataTable();
System.Data.OleDb.OleDbDataAdapter da = new System.Data.OleDb.OleDbDataAdapter();
da.Fill(myList, Dts.Variables["MyList"].Value);
foreach (DataRow row in myList.Rows)
{
sw.WriteLine(row["col1"]);
}
sw.Flush();
}
}

Delete remote files?

I have files that I want to delete. Connection can be from file sharing, http, and ftp.
Example of files to delete:
//mytest//delete//filename.bin
ftp://mytest/delete/filename.bin
http://mytest/delete/filename.bin
Here's what I did:
Uri target = new Uri(#"ftp://mytest/delete/filename.bin");
FileInfo fi = new FileInfo(target.AbsoluteUri);
fi.Delete();
The error I get is:
The given paths format is not supported
Is there a single code that can delete in all these file types?
I have created a simple code for this task(based on thread response).
This is the input:
Uri target = new Uri(#"ftp://tabletijam/FileServer/upload.bin");
Uri target = new Uri(#"http://tabletijam/FileServer/upload.bin");
Uri target = new Uri(#"\\tabletijam\FileServer\upload.bin");
This is the code:
bool DeleteFileOnServer(Uri serverUri)
{
if (serverUri.Scheme == Uri.UriSchemeFtp)
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(serverUri);
request.Method = WebRequestMethods.Ftp.DeleteFile;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
lblStatus.Content = response.StatusDescription;
response.Close();
return true;
}
else if (serverUri.Scheme == Uri.UriSchemeFile)
{
System.IO.File.Delete(serverUri.LocalPath);
return true;
}
else if (serverUri.Scheme == Uri.UriSchemeHttp || serverUri.Scheme == Uri.UriSchemeHttps)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(serverUri);
request.Method = WebRequestMethods.Http.DeleteFile;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
lblStatus.Content = response.StatusDescription;
response.Close();
return true;
}
else
{
lblStatus.Content = "Unknown uri scheme.";
return false;
}
}
Ftp and File deleted successfully. WebRequestMethods.Http does not contain DeleteFile.
So my question is, how do I delete file from this URI?
http://tabletijam/FileServer/upload.bin

Because FileInfo only works with local files. For each connection you will need a special implementation.
For FTP: (example from MSDN)
public static bool DeleteFileOnServer(Uri serverUri)
{
// The serverUri parameter should use the ftp:// scheme.
// It contains the name of the server file that is to be deleted.
// Example: ftp://contoso.com/someFile.txt.
//
if (serverUri.Scheme != Uri.UriSchemeFtp)
{
return false;
}
// Get the object used to communicate with the server.
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(serverUri);
request.Method = WebRequestMethods.Ftp.DeleteFile;
FtpWebResponse response = (FtpWebResponse) request.GetResponse();
Console.WriteLine("Delete status: {0}",response.StatusDescription);
response.Close();
return true;
}

Using the \\server... notation, you can delete file (that you have access) to on remote servers.
Using FTP, you should the FtpWebRequest.
For HTTP, you could issue a DELETE request, using HttpWebRequest.
For both FTP and HTTP, you might need to supply a username and password. Also normally HTTP servers are not configured to delete files when receiving a DELETE request by default.

For a whole lot of reasons, no, there is not a single unified way you can delete files via each of these protocols.
You could abstract this away into some implementation of your own, however, using an implementation specific to each of the protocols you want to support...

how do i delete file from this uri?
request.Method = "DELETE";
Also, there's a different header supported by WebDAV for controlling the delete...

No, this is not possible. FTP and HTTP are protocols that you are required to stick to. Even though you may be able to delete files when viewing FTP folders in the Explorer doesn't mean it works from C#, too, because the Explorer uses an integrated FTP client. Deleting files via HTTP like that isn't possible at all.

C# webclient and proxy server

I am using a web client class in my source code for downloading a string using http.
This was working fine. However, the clients in the company are all connected now to a proxy server. And the problem started from this.
When I have tested my application I don't think it can pass through the proxy server, as the exception that keeps getting thrown is "no response from xxx.xxx.xxx.xxx which is the proxy server IP address.
However, I can still navigate to the web site URL and it displays the string correctly in the browser when connecting through a proxy server, but not when I use my web client.
Is there something in the web client that I have to configure to allow me to access the url from behind a proxy server?
using (WebClient wc = new WebClient())
{
string strURL = "http://xxxxxxxxxxxxxxxxxxxxxxxx";
//Download only when the webclient is not busy.
if (!wc.IsBusy)
{
string rtn_msg = string.Empty;
try
{
rtn_msg = wc.DownloadString(new Uri(strURL));
return rtn_msg;
}
catch (WebException ex)
{
Console.Write(ex.Message);
return false;
}
catch (Exception ex)
{
Console.Write(ex.Message);
return false;
}
}
else
{
System.Windows.Forms.MessageBox.Show("Busy please try again");
return false;
}
}

My solution:
WebClient client = new WebClient();
WebProxy wp = new WebProxy(" proxy server url here");
client.Proxy = wp;
string str = client.DownloadString("http://www.google.com");

If you need to authenticate to the proxy, you need to set UseDefaultCredentials to false, and set the proxy Credentials.
WebProxy proxy = new WebProxy();
proxy.Address = new Uri("mywebproxyserver.com");
proxy.Credentials = new NetworkCredential("usernameHere", "pa****rdHere"); //These can be replaced by user input
proxy.UseDefaultCredentials = false;
proxy.BypassProxyOnLocal = false; //still use the proxy for local addresses
WebClient client = new WebClient();
client.Proxy = proxy;
string doc = client.DownloadString("http://www.google.com/");
If all you need is a simple proxy, you skip most of the lines above though. All you need is:
WebProxy proxy = new WebProxy("mywebproxyserver.com");

The answer proposed by Jonathan is proper, but requires that you specify the proxy credentials and url in the code. Usually, it is better to allow usage of the credentials as setup in the system by default (Users typically configure LAN Settings anyway in case they use a proxy)...
The below answer has been provided by Davide in earlier answer, but that requires modifying the app.config files. This solution is probably more useful since it does the same thing IN CODE.
In order to let the application use the default proxy settings as used in the user's system, one can use the following code:
IWebProxy wp = WebRequest.DefaultWebProxy;
wp.Credentials = CredentialCache.DefaultCredentials;
wc.Proxy = wp;
This will allow the application code to use the proxy (with logged-in credentials and default proxy url settings)... No headaches! :)
Hope this helps future viewers of this page to solve their problem!

I've encountered the same issue but using a webclient for downloading a file from the internet with a Winform application the solution was adding in the app.config:
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>
The same solution will work for an asp.net app inserting the same rows in web.config.
Hope it will help.

You need to configure the proxy in the WebClient object.
See the WebClient.Proxy property:
http://msdn.microsoft.com/en-us/library/system.net.webclient.proxy(VS.80).aspx

byte[] data;
using (WebClient client = new WebClient())
{
ICredentials cred;
cred = new NetworkCredential("xmen#test.com", "mybestpassword");
client.Proxy = new WebProxy("192.168.0.1",8000);
client.Credentials = cred;
string myurl="http://mytestsite.com/source.jpg";
data = client.DownloadData(myUrl);
}
File.WriteAllBytes(#"c:\images\target.jpg", data);

All previous answers have some merit, but the actual answer only needs ONE line:
wc.Proxy = new WebProxy("127.0.0.1", 8888);
where wc is the WebClient object, and 8888 is the port number of the proxy server located on the same machine.