This is a follow-up to this question: How to load XML into a DataTable?
I want to read an XML file on the internet into a DataTable. The XML file is here: http://rates.fxcm.com/RatesXML
If I do:
public DataTable GetCurrentFxPrices(string url)
{
WebProxy wp = new WebProxy("http://mywebproxy:8080", true);
wp.Credentials = CredentialCache.DefaultCredentials;
WebClient wc = new WebClient();
wc.Proxy = wp;
MemoryStream ms = new MemoryStream(wc.DownloadData(url));
DataSet ds = new DataSet("fxPrices");
ds.ReadXml(ms);
DataTable dt = ds.Tables["Rate"];
return dt;
}
It works fine. I'm struggling with how to use the default proxy set in Internet Explorer. I don't want to hard-code the proxy. I also want the code to work if no proxy is specified in Internet Explorer.
You can use Console.WriteLine(System.Net.WebProxy.GetDefaultProxy().Address.AbsoluteUri); ...
Add the following setting to your app.config/web.config to use the system default proxy automatically:
<system.net>
<defaultProxy useDefaultCredentials="true"/>
</system.net>
#region Function to get x-rate via proxy
public string fncProxyGetRate(string countryCode)// use 'GBP' for British Pounds
{
string rtnTxt = "";
try
{
string url = "http://rss.timegenie.com/forex.xml";
string proxyUrl = "http://xxx.xxx.x.x:8080/";
string myXratePath = "/forex/data/code[text()='" + countryCode + "']";
WebProxy wp = new WebProxy(proxyUrl, true);
wp.Credentials = CredentialCache.DefaultCredentials;
WebClient wc = new WebClient();
wc.Proxy = wp;
MemoryStream ms = new MemoryStream(wc.DownloadData(url));
XmlTextReader rdr = new XmlTextReader(ms);
XmlDocument doc = new XmlDocument();
doc.Load(rdr);
rtnTxt = doc.SelectSingleNode(myXratePath).ParentNode.SelectSingleNode("rate").InnerXml;
}
catch (Exception ex)
{
rtnTxt = ex.Message;
}
return rtnTxt;
}
#endregion
Related
I need to use a proxy with HtmlAgilityPack.
I give a link to my app RefURL. After that I want the app get url from a proxy address. For instance "101.109.44.157:8080"
I searched and found out this:
WebClient wc = new WebClient();
wc.Proxy = new WebProxy(host,port);
var page = wc.DownloadString(url);
and used it like this.
RefURL = new Uri(refLink.Text);
WebClient wc = new WebClient();
wc.Proxy = new WebProxy("101.109.44.157:8080");
var page = wc.DownloadString(RefURL);
RefURL.ToString();
HtmlWeb web = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = web.Load(RefURL.ToString());
but it does not work!
The proxy IP is not responding but also you're not passing web proxy in this code line:
HtmlAgilityPack.HtmlDocument doc = web.Load(RefURL.ToString());
Should be:
HtmlAgilityPack.HtmlDocument doc = web.Load(RefURL.ToString(),"GET", webProxy);
First step is finding "fresh proxy IP" list, for example:
https://geonode.com/free-proxy-list/
https://free-proxy-list.net/uk-proxy.html
https://hidemy.name/en/proxy-list/
http://free-proxy.cz
http://nntime.com
Most of these addresses would work for few hours. Check out how to set proxy IP in a browser. If the proxy is anonymous, this page should be unable to detect your location and IP.
Once you have a proxy IP and port that works, you can create webProxy object or simply pass IP and port.
string RefURL = "https://www.whatismyip.com/";
string myProxyIP = "119.81.197.124"; //check this is still available
int myPort = 3128;
string userId = string.Empty; //leave it blank
string password = string.Empty;
try
{
HtmlWeb web = new HtmlWeb();
var doc = web.Load(RefURL.ToString(), myProxyIP, myPort, userId, password);
Console.WriteLine(doc.DocumentNode.InnerHtml);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
I am trying to download a file from sharepoint, I have this code and throw error code 500.
static void DownloadFile(){
string serverFilePath = "Here goes my URL, that open the file from any tab";
var password = new SecureString();
foreach (char c in Configuration.password) {
password.AppendChar(c);
}
// theese are the credentials and work fine because I tested in another method
var o365credentials = new SharePointOnlineCredentials(Configuration.userName, password);
var url = string.Format("{0}/{1}", Configuration.siteUrl, serverFilePath);
// My destination folder
string destPath = #"C:\publisher";
var request = System.Net.HttpWebRequest.Create(url);
request.Credentials = o365credentials;
using (var sReader = new StreamReader(request.GetResponse().GetResponseStream())) {
using (var sWriter = new StreamWriter(destPath)) {
sWriter.Write(sReader.ReadToEnd());
}
}
}
you can achieve this task using WebRequest in order to download files from sharepoint site:
public void DownloadFile(string serverFilePath, string destPath)
{
var url = string.Format("{0}/{1}", ServerURL, serverFilePath);
Directory.CreateDirectory(Path.GetDirectoryName(destPath)); // this method creates your directory
var request = System.Net.HttpWebRequest.Create(url);
request.Credentials = System.Net.CredentialCache.DefaultCredentials;
using (var sReader = new StreamReader(request.GetResponse().GetResponseStream()))
{
using (var sWriter = new StreamWriter(destPath))
{
sWriter.Write(sReader.ReadToEnd());
}
}
}
if you wish to use the Client-object-model you read that:
How to get a file using SharePoint Client Object Model with only an absolute url at hand?
Edit: fixed the spelling of CreateDirectory call
I am trying to get a list of files on FTP folder.
The code was working when I ran it locally, but on deploying it I started receiving html instead of file name
ArrayList fName = new ArrayList();
try
{
StringBuilder result = new StringBuilder();
//create the directory
FtpWebRequest requestDir =
(FtpWebRequest) FtpWebRequest.Create(new Uri(directory));
requestDir.Method = WebRequestMethods.Ftp.ListDirectory;
requestDir.Credentials = new NetworkCredential(FTP_USER_NAME, FTP_PASSWORD);
requestDir.UsePassive = true;
requestDir.UseBinary = true;
requestDir.KeepAlive = false;
requestDir.Proxy = null;
FtpWebResponse response = (FtpWebResponse)requestDir.GetResponse();
Stream ftpStream = response.GetResponseStream();
StreamReader reader = new StreamReader(ftpStream, Encoding.ASCII);
while (!reader.EndOfStream)
{
fName.Add(reader.ReadLine().ToString());
}
response.Close();
ftpStream.Close();
reader.Close();
You can try with GetFileName
Uri uri = new Uri(hreflink);
string filename = Path.GetFileName(uri.LocalPath);
I was able to access the file names in a list format(not as html) by using the ip address of the ftp server and not the name
i.e.
ftp://xxx.x.x.xxx/folder_name/
instead of
ftp://abc.some_company.com/folder_name/
I will edit this answer with more details.
Hi how do I get the source of an html page through a proxy. When I use the code below I get an error saying "Proxy Authentication Required." and I have to go through a proxy.
Dim client As New WebClient()
Dim htmlCode As String = client.DownloadString("http://www.stackoverflow.com")
Then use a proxy that does not need authentication
see here for more info
http://msdn.microsoft.com/en-us/library/system.net.webclient.proxy.aspx
string source = GetPageSource("http://www.stackoverflow.com");
private string GetPageSource(string url)
{
string htmlSource = string.Empty;
try
{
System.Net.WebProxy myProxy = new System.Net.WebProxy("Proxy IP", 8080);
using (System.Net.WebClient client = new System.Net.WebClient())
{
client.Proxy = myProxy;
client.Proxy.Credentials = new System.Net.NetworkCredential("username", "password");
htmlSource = client.DownloadString(url);
}
}
catch (WebException ex)
{
// log any exceptions
}
return htmlSource;
}
I am trying to get the location of client machine using ip address. Client can access the internet only if
he/she provide the proxy authenication.
Let us say client need to access the 'www.google.com' on the browser then immediately Authenication Required
prompt window open and then client enter his/her username and password. But it is possible the few users does
not required the provide the authenication in order to access internet.
This segment of code does not helped me...
string url = "http://freegeoip.net/xml/";
WebClient wc = new WebClient();
WebProxy proxyObj = new WebProxy("http://freegeoip.net/xml/");
proxyObj.Credentials = CredentialCache.DefaultCredentials;
Uri uri = new Uri(url);
MemoryStream ms = new MemoryStream(wc.DownloadData(uri));
XmlTextReader rdr = new XmlTextReader(url);
XmlDocument doc = new XmlDocument();
ms.Position = 0;
doc.Load(ms);
ms.Dispose();
In the above code if i add network credential instance with username, password and domain then it's work perfectly
Instead of providing the default net credential in code itself, I need to get the username and password from the users(client
machine).
My question is how to prompt the Authentication Required Window and get the username and password to load the download from url
I would be glad if someone throw light on this issue...
Edit: Somehow basic authentication window prompt and now i can get the username and password which can use for credential
try
{
var reg = HttpContext.Current.Request;
if (!String.IsNullOrEmpty(reg.Headers["Authorization"]))
{
var cred = System.Text.ASCIIEncoding.ASCII.GetString(Convert.FromBase64String(Request.Headers["Authorization"].Substring(6))).Split(':');
var user = new { Name = cred[0], Pass = cred[1] };
string url = "http://freegeoip.net/xml/";
WebClient wc = new WebClient();
WebProxy wProxy = new WebProxy();
ICredentials crd;
crd = new NetworkCredential("'" + cred[0] + "'", "'" + cred[1] + "'");
wProxy = new WebProxy("myproxy", true, null, crd);
wc.Proxy = wProxy;
Uri uri = new Uri(url);
string content = wc.DownloadString(uri);
}
else
{
try
{
//var reg = HttpContext.Current.Request;
if (String.IsNullOrEmpty(reg.Headers["Authorization"]))
{
var res = HttpContext.Current.Response;
res.StatusCode = 401;
res.AddHeader("WWW-Authenticate", "Basic realm = \"freegeoip\"");
//res.End();
}
}
catch (Exception ex)
{
}
}
}
catch(Exception ex)
{
}
But Still It throwing the "Unable to connect to the remote server"