Hi how do I get the source of an html page through a proxy. When I use the code below I get an error saying "Proxy Authentication Required." and I have to go through a proxy.
Dim client As New WebClient()
Dim htmlCode As String = client.DownloadString("http://www.stackoverflow.com")
Then use a proxy that does not need authentication
see here for more info
http://msdn.microsoft.com/en-us/library/system.net.webclient.proxy.aspx
string source = GetPageSource("http://www.stackoverflow.com");
private string GetPageSource(string url)
{
string htmlSource = string.Empty;
try
{
System.Net.WebProxy myProxy = new System.Net.WebProxy("Proxy IP", 8080);
using (System.Net.WebClient client = new System.Net.WebClient())
{
client.Proxy = myProxy;
client.Proxy.Credentials = new System.Net.NetworkCredential("username", "password");
htmlSource = client.DownloadString(url);
}
}
catch (WebException ex)
{
// log any exceptions
}
return htmlSource;
}
Related
I am trying to save my images using weblclient Object.
Here it is my code..
try
{
WebClient client = new WebClient();
NetworkCredential nc = new NetworkCredential("****", "*****");
Uri addy = new Uri(#"\\104.199.191.5\Images\");
client.Credentials = nc;
byte[] arrReturn = client.UploadFile(addy, "Post", #"E:\free-html-admin-templates.JPG");
}
catch (Exception ex)
{
}
Right now I am getting error Network path not found..
Where am I wrong ??
This is the code I currently have
using (WebClient client = new WebClient()) {
WebProxy proxy = new WebProxy();
proxy.Address = new Uri(96.44.147.138:6060);
proxy.Credentials = new NetworkCredential(proxyUsername.Text, proxyPassword.Text);
proxy.UseDefaultCredentials = false;
proxy.BypassProxyOnLocal = false;
Console.WriteLine(client.DownloadString("http://bot.whatismyipaddress.com/"));
}
The proxy needs credentials.
I get an error on line proxy.Address = new Uri(96.44.147.138:6060);
saying
"The URI scheme is not valid."
Not sure what kind of value it's expecting
The Uri should consist of scheme host and optiona port. So you should use
proxy.Address = new Uri("http://96.44.147.138:6060");
Must be like;
using (var client = new WebClient())
{
var proxy = new WebProxy();
proxy.Address = new Uri("http://96.44.147.138:6060");
proxy.Credentials = new NetworkCredential(proxyUsername.Text, proxyPassword.Text);
proxy.UseDefaultCredentials = false;
proxy.BypassProxyOnLocal = false;
Console.WriteLine(client.DownloadString("http://bot.whatismyipaddress.com/"));
}
Example edit: Setting a global HTTP proxy in C# and .NET client classes
I have some code that's throwing a 407 unauthorized exception.
I am trying to download a file and below is my example code. I've tried with netcredentials and webproxy but in vain.
WebClient webClient = new WebClient();
NetworkCredential netCred=new NetworkCredential();
netCred.UserName="<<userid>>";
netCred.Password="<<password>>";
netCred.Domain="<<windowsdomainname>>";
webClient.Credentials = netCred;
WebProxy wp = new WebProxy();
wp.Credentials = netCred;
wp.Address = new Uri(#"http://proxy-xx.xxxx.co.uk:8080/proxy.pac");
webClient.Proxy = wp;
webClient.DownloadFile("http://www.win-rar.com/postdownload.html?&L=0", #"c:\winrar.exe");
I was getting 407 before finding your question. Modified your source to the following, which works for me:
try
{
var netCred = new NetworkCredential { UserName = "ununun", Password = #"pwpwpw", Domain = #"domain" };
var webProxy = new WebProxy { Credentials = netCred };
var webClient = new WebClient { Proxy = webProxy };
webClient.DownloadFile(url, saveFileName);
}
catch (Exception ex)
{
Console.WriteLine("Exception:\n{0}\n{1}", ex.Message, ex.StackTrace);
return;
}
This is what I do yesterday.
WebClient client = new WebClient();
ICredentials cred;
cred = new NetworkCredential(user, password);
client.Proxy = new WebProxy("xxxx-proxy", true, null, cred);
Hope it help !
I am trying to get the location of client machine using ip address. Client can access the internet only if
he/she provide the proxy authenication.
Let us say client need to access the 'www.google.com' on the browser then immediately Authenication Required
prompt window open and then client enter his/her username and password. But it is possible the few users does
not required the provide the authenication in order to access internet.
This segment of code does not helped me...
string url = "http://freegeoip.net/xml/";
WebClient wc = new WebClient();
WebProxy proxyObj = new WebProxy("http://freegeoip.net/xml/");
proxyObj.Credentials = CredentialCache.DefaultCredentials;
Uri uri = new Uri(url);
MemoryStream ms = new MemoryStream(wc.DownloadData(uri));
XmlTextReader rdr = new XmlTextReader(url);
XmlDocument doc = new XmlDocument();
ms.Position = 0;
doc.Load(ms);
ms.Dispose();
In the above code if i add network credential instance with username, password and domain then it's work perfectly
Instead of providing the default net credential in code itself, I need to get the username and password from the users(client
machine).
My question is how to prompt the Authentication Required Window and get the username and password to load the download from url
I would be glad if someone throw light on this issue...
Edit: Somehow basic authentication window prompt and now i can get the username and password which can use for credential
try
{
var reg = HttpContext.Current.Request;
if (!String.IsNullOrEmpty(reg.Headers["Authorization"]))
{
var cred = System.Text.ASCIIEncoding.ASCII.GetString(Convert.FromBase64String(Request.Headers["Authorization"].Substring(6))).Split(':');
var user = new { Name = cred[0], Pass = cred[1] };
string url = "http://freegeoip.net/xml/";
WebClient wc = new WebClient();
WebProxy wProxy = new WebProxy();
ICredentials crd;
crd = new NetworkCredential("'" + cred[0] + "'", "'" + cred[1] + "'");
wProxy = new WebProxy("myproxy", true, null, crd);
wc.Proxy = wProxy;
Uri uri = new Uri(url);
string content = wc.DownloadString(uri);
}
else
{
try
{
//var reg = HttpContext.Current.Request;
if (String.IsNullOrEmpty(reg.Headers["Authorization"]))
{
var res = HttpContext.Current.Response;
res.StatusCode = 401;
res.AddHeader("WWW-Authenticate", "Basic realm = \"freegeoip\"");
//res.End();
}
}
catch (Exception ex)
{
}
}
}
catch(Exception ex)
{
}
But Still It throwing the "Unable to connect to the remote server"
This is a follow-up to this question: How to load XML into a DataTable?
I want to read an XML file on the internet into a DataTable. The XML file is here: http://rates.fxcm.com/RatesXML
If I do:
public DataTable GetCurrentFxPrices(string url)
{
WebProxy wp = new WebProxy("http://mywebproxy:8080", true);
wp.Credentials = CredentialCache.DefaultCredentials;
WebClient wc = new WebClient();
wc.Proxy = wp;
MemoryStream ms = new MemoryStream(wc.DownloadData(url));
DataSet ds = new DataSet("fxPrices");
ds.ReadXml(ms);
DataTable dt = ds.Tables["Rate"];
return dt;
}
It works fine. I'm struggling with how to use the default proxy set in Internet Explorer. I don't want to hard-code the proxy. I also want the code to work if no proxy is specified in Internet Explorer.
You can use Console.WriteLine(System.Net.WebProxy.GetDefaultProxy().Address.AbsoluteUri); ...
Add the following setting to your app.config/web.config to use the system default proxy automatically:
<system.net>
<defaultProxy useDefaultCredentials="true"/>
</system.net>
#region Function to get x-rate via proxy
public string fncProxyGetRate(string countryCode)// use 'GBP' for British Pounds
{
string rtnTxt = "";
try
{
string url = "http://rss.timegenie.com/forex.xml";
string proxyUrl = "http://xxx.xxx.x.x:8080/";
string myXratePath = "/forex/data/code[text()='" + countryCode + "']";
WebProxy wp = new WebProxy(proxyUrl, true);
wp.Credentials = CredentialCache.DefaultCredentials;
WebClient wc = new WebClient();
wc.Proxy = wp;
MemoryStream ms = new MemoryStream(wc.DownloadData(url));
XmlTextReader rdr = new XmlTextReader(ms);
XmlDocument doc = new XmlDocument();
doc.Load(rdr);
rtnTxt = doc.SelectSingleNode(myXratePath).ParentNode.SelectSingleNode("rate").InnerXml;
}
catch (Exception ex)
{
rtnTxt = ex.Message;
}
return rtnTxt;
}
#endregion