I want to access HTML content from wikipedia .But it is showing access denied.
How can i access Wiki. Please give some suggestion
Use HttpWebRequest
Try the following:
string Text = "http://www.wikipedia.org/";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Text);
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)";
HttpWebResponse respons;
respons = (HttpWebResponse)request.GetResponse();
Encoding enc = Encoding.GetEncoding(respons.CharacterSet);
StreamReader reader = new StreamReader(respons.GetResponseStream(), enc);
string sr = reader.ReadToEnd();
Related
I would like to download an image from this url http://squirlytraffic.com/surficon.php?ts=1491591235
I tried this code but I don't see the image when I open it.
using (WebClient client = new WebClient())
{
client.DownloadFile("http://squirlytraffic.com/surficon.php?ts=1491591235", #"D:\image.jpg");
}
You need to set your credentials using the the WebClient Credentials property. You can do this by assigning it an instance of NetworkCredential. See below:
using (WebClient client = new WebClient()){
client.Credentials = new NetworkCredential("user-name", "password");
client.DownloadFile("url", #"file-location");
}
EDIT
If you don't want to hard code a username and password, you can set the UseDefaultCredentials property of the WebClient to true. This will use the credentials of the currently logged in user. From the documentation.
The Credentials property contains the authentication credentials used to access a resource on a host. In most client-side scenarios, you should use the DefaultCredentials, which are the credentials of the currently logged on user. To do this, set the UseDefaultCredentials property to true instead of setting this property.
Which would mean you could amend the above code to:
using (WebClient client = new WebClient()){
client.UseDefaultCredentials = true;
client.DownloadFile("url", #"file-location");
}
Try this way, those parameters are passed when login
StringBuilder postData = new StringBuilder();
postData.Append("login=" + HttpUtility.UrlEncode("username") + "&");
postData.Append("password=" + HttpUtility.UrlEncode("password") + "&");
postData.Append("Submit=" + HttpUtility.UrlEncode("Login"));
ASCIIEncoding ascii = new ASCIIEncoding();
byte[] postBytes = ascii.GetBytes(postData.ToString());
CookieContainer cc = new CookieContainer();
HttpWebRequest webReq = (HttpWebRequest)WebRequest.Create("http://squirlytraffic.com/members.php");
webReq.Method = "POST";
webReq.ContentType = "application/x-www-form-urlencoded";
webReq.ContentLength = postBytes.Length;
webReq.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
webReq.CookieContainer = cc;
Stream postStream = webReq.GetRequestStream();
postStream.Write(postBytes, 0, postBytes.Length);
postStream.Flush();
postStream.Close();
HttpWebResponse res = (HttpWebResponse)webReq.GetResponse();
HttpWebRequest ImgReq = (HttpWebRequest)WebRequest.Create("http://squirlytraffic.com/surficon.php?ts=1491591235");
ImgReq.Method = "GET";
ImgReq.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
ImgReq.CookieContainer = cc;
HttpWebResponse ImgRes = (HttpWebResponse)ImgReq.GetResponse();
Stream Img = ImgRes.GetResponseStream();
I want to read atom xml, and used following code
string str1 = "http://moss:133333/_vti_bin/ExcelRest.aspx/Document Library/OrdersExcel.xlsx/Model/Tables('Table1')?$format=atom";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(str1);
req.UseDefaultCredentials = true;
req.PreAuthenticate = true;
req.Credentials = CredentialCache.DefaultCredentials;
req.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0)";
WebResponse response = req.GetResponse();
Encoding enc = System.Text.Encoding.GetEncoding(1252);
StreamReader loResponseStream = new StreamReader(response.GetResponseStream(), enc);
string Response = loResponseStream.ReadToEnd();
The last line in above code basically reads whole stream reader into the string Response.
Now I do not know how do I read the above string's atom xml into the data table.
DataTable.ReadXml reads XML schema and data into the DataTable using the specified TextReader.
var reader = new System.IO.StreamReader(xmlStream);
var newTable = new DataTable();
newTable.ReadXml(reader);
I am trying to access a website through C# using the WebRequest and the WebResponse object,
I logged on to the site and preserved the cookie to further browse it,
The problem is that the website is arabic and somehow I got a formatted message from the website indicating that my browser does not support arabic.
Perhaps I can add something to the request object to ensure the website that arabic is supported.
This is the code I used, please let me know how to update it:
string formUrl = "http://www.kuwaitlook.com/Ar/Residential.asp";
string formParams = string.Format("Mega={0}", searchTarget);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1);Accept-Language:ar";
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
req.Headers.Add("Cookie", cookieHeader);
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream()) {
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
StreamReader streamReader = new StreamReader(resp.GetResponseStream());
using (StreamWriter writer = new StreamWriter("text.xml")) {
string line;
while ((line = streamReader.ReadLine()) != null) {
writer.WriteLine(line);
}
}
Like Mikael suggested try this one:
HttpWebRequest request=(HttpWebRequest)WebRequest.Create("http://www.yourdomain.com");
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1);Accept-Language:ar"
Here is how you do it in vb.net:
Dim SW As StreamWriter
Dim ar As System.Text.UTF8Encoding = New System.Text.UTF8Encoding
Request.ContentLength = ar.GetByteCount(your_string) ' Here
SW = New StreamWriter(Request.GetRequestStream(), ar) ' And Here
SW.Write(your_string)
After authorisation on www.vkontakte.ru through ie8 me spans on page: www.vkontakte.ru/MyPage. But I cannot receive www.vkontakte.ru/MyPage through a code
HttpWebRequest authRequest = (HttpWebRequest)WebRequest.Create(new Uri("http://vkontakte.ru/login.php", UriKind.Absolute));
authRequest.CookieContainer = new CookieContainer();
authRequest.AllowAutoRedirect = false;
string param = string.Format("email={0}&pass={1}&expire=1", HttpUtility.UrlEncode("---"), HttpUtility.UrlEncode("---"));
authRequest.Method = "POST";
authRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)";
authRequest.ContentType = "application/x-www-form-urlencoded";
authRequest.ContentLength = param.Length;
authRequest.GetRequestStream().Write(Encoding.GetEncoding(1251).GetBytes(param), 0, param.Length);
HttpWebResponse authResponse = (HttpWebResponse)authRequest.GetResponse();
listBox1.Items.Add(authRequest.Address);
Returns http://vkontakte.ru/ instead of www.vkontakte.ru/MyPage =(
HttpContext.Current.Request.Url.AbsoluteUri - can help?
help me!
You forgot to close the request stream.
You should write the following:
using (Stream requestStream = authRequest.GetRequestStream())
using (StreamWriter writer = new StreamWriter(requestStream, Encoding.GetEncoding(1251))
writer.Write(param);
Also, you should run Fiddler check what the request and response look like.
I'm trying to interface with the Google Reader (undocumented/unofficial) API using information from this page. My first step is to get a SID and token, which works fine, but I can't seem to POST anything without getting a 401 error.
Here is the code I'm using to get my SID and token:
static string getSid()
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://www.google.com/accounts/ClientLogin?service=reader&Email=username&Passwd=password");
req.Method = "GET";
string sid;
HttpWebResponse response = (HttpWebResponse)req.GetResponse();
using (var stream = response.GetResponseStream())
{
StreamReader r = new StreamReader(stream);
string resp = r.ReadToEnd();
int indexSid = resp.IndexOf("SID=") + 4;
int indexLsid = resp.IndexOf("LSID=");
sid = resp.Substring(indexSid, indexLsid - 5);
return sid;
}
}
And to generate a cookie and get the token:
static Cookie getCookie(string sid)
{
Cookie cookie = new Cookie("SID", sid, "/", ".google.com");
return cookie;
}
static string getToken(string sid, Cookie cookie)
{
string token;
string url = "http://www.google.com/reader/api/0/token";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.CookieContainer = new CookieContainer();
req.CookieContainer.Add(cookie);
HttpWebResponse response = (HttpWebResponse)req.GetResponse();
using (var stream = response.GetResponseStream())
{
StreamReader r = new StreamReader(stream);
token = r.ReadToEnd();
return token;
}
}
Now if I try to do a POST (in this example, insert a new tag) using the following, I get the 401 error.
static void insertTag(string tag, Cookie cookie)
{
string result = "";
string data = Uri.EscapeDataString("a="+tag+"&T=" + Program.token);
byte[] buffer = Encoding.GetEncoding(1252).GetBytes(data);
HttpWebRequest WebReq = (HttpWebRequest)WebRequest.Create
("http://www.google.com/reader/api/0/edit-tag?client=-");
WebReq.Method = "POST";
WebReq.Credentials = new NetworkCredential("username", "password");
WebReq.ContentType = "application/x-www-form-urlencoded";
WebReq.ContentLength = buffer.Length;
Stream PostData = WebReq.GetRequestStream();
PostData.Write(buffer, 0, buffer.Length);
PostData.Close();
HttpWebResponse WebResp = (HttpWebResponse)WebReq.GetResponse();
Stream Answer = WebResp.GetResponseStream();
StreamReader _Answer = new StreamReader(Answer);
result = _Answer.ReadToEnd();
if (result.Length < 0)
result = "";
}
The error occurs on the line Stream Answer = WebResp.GetResponseStream();
You need to double check that you have a user agent set. I have run into this same problem before when i didnt have it set.
For example:
WebClient client = new WebClient();
client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
Or: MSDN Link
myHttpWebRequest=(HttpWebRequest)WebRequest.Create("http://www.contoso.com");
myHttpWebRequest.UserAgent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
Turns out I was using the wrong URL to access the Google Reader APIs thanks to some outdated documentation! The correct URL for adding labels in Google Reader (as of August 2009) is
http://www.google.com/reader/api/0/subscription/edit?client=scroll
with POST arguments
a=user/-/label/[your label]&s=feed/[feed url]&ac=edit&T=[token]