I just got the following problem:
I need to download data which is behind a login page. When I however make my get request, the server is providing bad data - the content is there, but no content length in header set, its an empty field. I looked it up with Fiddler, and its the same when I try to download the file with browser, but browser completes the download ok, while C# drops with exception when getting the response object from my request.
The header looks like this :
HTTP/1.1 200 OK
Date: Sat, 06 Dec 2014 11:55:06 GMT
Server: Apache
Content-Disposition: attachment; filename=;
Pragma: no-cache
Expires: 0
Content-Length:
X-Powered-By: PleskLin
Connection: close
Content-Type: application/octet-stream
Hersteller;"Hersteller Art-Nr";"Lieferant Art-Nr";Ma�stab;Bezeichnung;EAN;"EK (netto)";UVP;USt;Verkaufseinheit;Hinweis;"Letzte Pro...
My code looks like this
public string ReadPage(string path, string method = "GET"){
var result = "";
var request = (HttpWebRequest)WebRequest.Create(path);
request.Method = method;
request.Host = "somehost.de";
request.CachePolicy = new System.Net.Cache.RequestCachePolicy(System.Net.Cache.RequestCacheLevel.NoCacheNoStore);
request.Referer = #"http://somehost.de/login.php?redir=list.php%3Ftype%3Dmm";
request.AllowAutoRedirect = true;
request.Headers.Add("Cookie", LoginCookie);
try
{
var response = request.GetResponse();
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
result = sr.ReadToEnd();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
//throw;
}
return result;
}
The exception appears in the var response = request.GetResponse(); line. Any idea how to fix this problem? I just want it to carry on and let me read out the data.
Forgot the exception - its a WebException with a message
The server committed a protocol violation. Section=ResponseHeader Detail='Content-Length' header value is invalid
I'm really loving the answer of Matthew: .Net HttpWebRequest.GetResponse() raises exception when http status code 400 (bad request) is returned
litte bit changed
public static class WebRequestExtensions
{
public static WebResponse GetWEResponse(this WebRequest request)
{
if (request == null)
{
throw new ArgumentNullException("request");
}
try
{
return request.GetResponse();
}
catch (WebException e)
{
return e.Response;
}
}
}
You can use this now by using request.GetWEResponse
public string ReadPage(string path, string method = "GET") {
var result = "";
var request = (HttpWebRequest)WebRequest.Create(path);
request.Method = method;
request.Host = "somehost.de";
request.CachePolicy = new System.Net.Cache.RequestCachePolicy(System.Net.Cache.RequestCacheLevel.NoCacheNoStore);
request.Referer = #"http://somehost.de/login.php?redir=list.php%3Ftype%3Dmm";
request.AllowAutoRedirect = true;
request.Headers.Add("Cookie", LoginCookie);
try
{
var response = request.GetWEResponse();
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
result = sr.ReadToEnd();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
//throw;
}
return result;
}
or better
public string ReadPage(string path, string method = "GET")
{
var request = (HttpWebRequest)WebRequest.Create(path);
request.Method = method;
request.Host = "somehost.de";
request.CachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);
request.Referer = #"http://somehost.de/login.php?redir=list.php%3Ftype%3Dmm";
request.AllowAutoRedirect = true;
request.Headers.Add("Cookie", LoginCookie);
using (var response = (HttpWebResponse)request.GetWEResponse())
using (var reader = new StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
Related
Trying to call a remote server using the following code I get always the unauthorized exception from the server:
HttpWebRequest webRequest;
var myURI = "https://myURI";
webRequest = (HttpWebRequest)WebRequest.Create(myURI);
webRequest.Method = "GET";
string myCredentials = Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes("myUsername" + ":" + "myPassword"));
webRequest.Headers.Add("Authorization", "Basic " + myCredentials);
HttpWebResponse webResponse;
webResponse = (HttpWebResponse)webRequest.GetResponse();
string response = string.Empty;
using (StreamReader sr = new StreamReader(webResponse.GetResponseStream()))
{
response = sr.ReadToEnd();
sr.Close();
}
System.Net.WebException: 'The remote server returned an error: (401) Unauthorized.'
WWW-Authenticate: Basic realm="geoserver"
Cache-Control: proxy-revalidate
Connection: Keep-Alive
Set-Cookie: BCSI-CS-**********=1; Path=/
Proxy-Support: Session-based-authentication
Content-Type: text/plain; charset=UTF-8
Content-Length: 23
It is a litte bit weird because, if I try (almost) the same thing via postman, it works and i get the correct response from the remote server (without 401).
I've also tried other ways like the follwing code, but nothing helps:
CredentialCache cc = new CredentialCache();
cc.Add(
new Uri("https://mywebserver/webpage"),
"Basic", //also tried NTLM
new NetworkCredential("user", "password"));
webRequest.Credentials = cc;
If I try to call an other server with the same code it works.
As you can see in the Exception, has a Session-based-authentication. The problem is: There is no documentation for the reverse proxy and I cannot find an endpoint in order to get a clean cookie. As workaround i've tried the fllowing code and it works now.
It's maybe not the best way to set the Cookie as HttpHeader.
static void Main(string[] args)
{
HttpWebRequest webRequest = CreateWebRequest();
string cookie = null;
HttpWebResponse webResponse;
try
{
webResponse = (HttpWebResponse) webRequest.GetResponse();
}
catch (WebException ex)
{
var headers = (WebHeaderCollection)ex.Response.GetType().GetProperty("Headers").GetValue(ex.Response, null);
cookie = headers.Get("Set-Cookie");
webRequest = CreateWebRequest();
webRequest.Headers.Add("Cookie", cookie);
webResponse = (HttpWebResponse) webRequest.GetResponse();
}
string response = string.Empty;
using (StreamReader sr = new StreamReader(webResponse.GetResponseStream()))
{
response = sr.ReadToEnd();
sr.Close();
}
}
private static HttpWebRequest CreateWebRequest()
{
HttpWebRequest webRequest;
var myURI = "https://myURI";
webRequest = (HttpWebRequest)WebRequest.Create(myURI);
webRequest.Method = "GET";
string myCredentials = Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes("myUsername" + ":" + "myPassword"));
webRequest.Headers.Add("Authorization", "Basic " + myCredentials);
return webRequest;
}
I'm trying to convert this curl command to c#
curl -i 'http://developer.echonest.com/api/v4/artist/profile?api_key=[API KEY]&name=weezer'
This returns a response header with content like so:
HTTP/1.1 200 OK
Content-Length: 135
X-RateLimit-Limit: 120
X-RateLimit-Remaining: 62
X-RateLimit-Used: 58
I tried this code but when I run it, it gets an exception: "HTTP Error 405 Method not allowed".
string baseurl = "http://developer.echonest.com/api/v4/artist/profile?api_key=[API KEY]&name=weezer";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(baseurl);
request.Method = "POST";
request.Accept = "application/json";
request.UserAgent = "curl/7.37.0";
request.ContentType = "application/x-www-form-urlencoded";
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
string data = "browser=Win7x64-C1|Chrome32|1024x768&url=http://www.google.com";
streamWriter.Write(data);
}
var response = request.GetResponse();
string text;
using (var sr = new StreamReader(response.GetResponseStream()))
{
text = sr.ReadToEnd();
Console.WriteLine(text);
}
Any help will be appreciated.
As well as the changes Daniel suggested, I would remove the UserAgent. After that, it looks pretty much like the C# I'm using successfully. Here's a somewhat shortened version of that:
string baseurl = "http://developer.echonest.com/api/v4/artist/profile?api_key=[API KEY]&name=weezer";
var req = (HttpWebRequest) WebRequest.Create(baseurl);
req.Method = WebRequestMethods.Http.Get;
req.Accept = "application/json";
using (var response = (HttpWebResponse) req.GetResponse())
{
if (response.StatusCode == HttpStatusCode.OK)
{
var stream = response.GetResponseStream();
if (stream != null)
{
using (var sr = new StreamReader(stream))
{
responseString = sr.ReadToEnd();
}
var remaining = GetRateInfo(response.Headers, "X-RateLimit-Remaining");
var used = GetRateInfo(response.Headers, "X-RateLimit-Used");
var limit = GetRateInfo(response.Headers, "X-RateLimit-Limit");
Trace.WriteLine($"Excedeed EchoNest Limits: remaining {remaining} - used {used} - limit {limit}");
}
}
else
{
// Error handling
}
// And the header parsing code:
private static int GetRateInfo(WebHeaderCollection headers, string type)
{
var s = headers.Get(type);
if (s == null)
return -1;
int info;
return int.TryParse(s, out info) ? info : -1;
}
I am working with GoogleApi. I want to get accesstoken as response using Google api. when I am sending httpwebrequest for getting access token then
When I used :- request.Method = "POST"
Exception:- HTTP method POST is not supported by this URL
When I used :- request.Method = "GET"
Exception:- System.Net.ProtocolViolationException: Cannot send a content-body with this verb-type
The actual request might look like:
POST /o/oauth2/token HTTP/1.1
Host: accounts.google.com
Content-Type: application/x-www-form-urlencoded
code=4/P7q7W91a-oMsCeLvIaQm6bTrgtp7&
client_id=8819981768.apps.googleusercontent.com&
client_secret={client_secret}&
redirect_uri=https://oauth2-login-demo.appspot.com/code&
grant_type=authorization_code
A successful response is returned as a JSON array, similar to the following:
{
"access_token":"1/fFAGRNJru1FTz70BzhT3Zg",
"expires_in":3920,
"token_type":"Bearer"
}
My Code is :-
var request = (HttpWebRequest)WebRequest.Create(#"https://accounts.google.com");
request.Method = "POST";
request.ContentType = "application/json";
//request.KeepAlive = false;
// request.Headers[HttpRequestHeader.Authorization] = "";
//request.ContentLength = 0;
using (StreamWriter streamWriter = new StreamWriter(request.GetRequestStream()))
{
string json = "{\"code\":\"4/M1IIC8htCuvYORuVJK16oadDb3Gd.cigIKgaPjvUYXE-sT2ZLcbSrckCLgwI\"," + "\"client_id\":\"841994137170.apps.googleusercontent.com\"," + "\"client_secret\":\"MXjKvevD_cKp5eQWZ1RFXfdo\"," + "\"redirect_uri\":\"http://gmailcheck.com/response.aspx\"," + "\"grant_type\":\"authorization_code\"}";
streamWriter.Write(json);
// streamWriter.Flush();
//streamWriter.Close();
}
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
StreamReader responsereader = new StreamReader(response.GetResponseStream());
var responsedata = responsereader.ReadToEnd();
//Session["responseinfo"] = responsereader;
}
}
catch (WebException ex)
{
using (WebResponse response = ex.Response)
{
var httpResponse = (HttpWebResponse)response;
using (Stream data = response.GetResponseStream())
{
var sr = new StreamReader(data);
throw new Exception(sr.ReadToEnd());
}
}
}
This is the problem:
var request = (HttpWebRequest)WebRequest.Create(#"https://accounts.google.com");
That's not the URL you showed originally. That's just the domain root. You need:
var request = (HttpWebRequest)WebRequest.Create("https://accounts.google.com/o/oauth2/token");
I've removed the # as your string doesn't include any line breaks or backslashes, so there's no benefit in using a verbatim string literal.
(Additionally, I'd expect this to be covered in the Google Client APIs - is it not?)
I am trying to post a form using HttpWebRequest but get an internal server error. I have tried setting useragent in the request but no success.
Here is the url which has got the form.
https://portal.llg.de/tracking/xml/sender.asp?cust=anonym_
and this is my sample code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
namespace postdata
{
public class RequestManager
{
public static string LastResponse { protected set; get; }
public static void Main(string[] args)
{
System.Net.ServicePointManager.Expect100Continue = false;
string content = "XML=<VORGANG><KUNDE>6400320</KUNDE><LIEFADR>..... Tokio, Japan</LIEFADR><POS NUMMER='1'><ARTNR>9000032</ARTNR><MENGE>10</MENGE><BESTAND>25</BESTAND></POS><POS NUMMER='2'><ARTNR>9161161</ARTNR><MENGE>100</MENGE><BESTAND>33</BESTAND><TERMIN>20050401</TERMIN><PREIS>80.00</PREIS></POS><VERSAND>80.00</VERSAND></VORGANG>";
string url = "https://portal.llg.de/tracking/xml/receiver.asp?cust=_anonym_";
var response = SendPOSTRequest(url, content, true);
var responsecontent = GetResponseContent(response);
}
public static string GetResponseContent(HttpWebResponse response)
{
if (response == null)
{
throw new ArgumentNullException("response");
}
Stream dataStream = null;
StreamReader reader = null;
string responseFromServer = null;
try
{
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
reader = new StreamReader(dataStream);
// Read the content.
responseFromServer = reader.ReadToEnd();
// Cleanup the streams and the response.
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
if (reader != null)
{
reader.Close();
}
if (dataStream != null)
{
dataStream.Close();
}
response.Close();
}
LastResponse = responseFromServer;
return responseFromServer;
}
public static HttpWebResponse SendPOSTRequest(string uri, string content, bool allowAutoRedirect)
{
HttpWebRequest request = GeneratePOSTRequest(uri, content, allowAutoRedirect);
return GetResponse(request);
}
public static HttpWebRequest GeneratePOSTRequest(string uri, string content, bool allowAutoRedirect)
{
return GenerateRequest(uri, content, "POST", null, null, allowAutoRedirect);
}
internal static HttpWebRequest GenerateRequest(string uri, string content, string method, string login, string password, bool allowAutoRedirect)
{
if (uri == null)
{
throw new ArgumentNullException("uri");
}
// Create a request using a URL that can receive a post.
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
// Set the Method property of the request to POST.
request.Method = method;
// Set cookie container to maintain cookies
// request.CookieContainer = cookies;
request.AllowAutoRedirect = false;
// If login is empty use defaul credentials
if (string.IsNullOrEmpty(login))
{
// request.Credentials = CredentialCache.DefaultNetworkCredentials;
request.Credentials = CredentialCache.DefaultCredentials;
}
else
{
request.Credentials = new NetworkCredential(login, password);
}
if (method == "POST")
{
// Convert POST data to a byte array.
byte[] byteArray = Encoding.UTF8.GetBytes(content);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
Stream dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
}
return request;
}
internal static HttpWebResponse GetResponse(HttpWebRequest request)
{
if (request == null)
{
throw new ArgumentNullException("request");
}
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
Console.WriteLine("Web exception occurred. Status code: {0}", ex.Status);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return response;
}
}
}
I debugged with fiddler both posts.
Something is weird in the form post. The xml conatains "+" between the tags.
If you use this string the post works.
content = "XML=<VORGANG><KUNDE>6400320</KUNDE><LIEFADR>.....+Tokio,+Japan</LIEFADR><POS+NUMMER=\"1\">++<ARTNR>9000032</ARTNR>++<MENGE>10</MENGE>++<BESTAND>25</BESTAND></POS><POS+NUMMER=\"2\">++<ARTNR>9161161</ARTNR>++<MENGE>100</MENGE>++<BESTAND>33</BESTAND>++<TERMIN>20050401</TERMIN>++<PREIS>80.00</PREIS></POS><VERSAND>80.00</VERSAND></VORGANG>";
Fiddler Debug:
POST https://portal.llg.de/tracking/xml/receiver.asp?cust=_anonym_ HTTP/1.1
Host: portal.llg.de
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
DNT: 1
Referer: https://portal.llg.de/tracking/xml/sender.asp?cust=anonym_
Cookie: ASPSESSIONIDSQSACSTQ=CIGFABLBPHPMLBDOPJIKPHJB
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 588
XML=%3CVORGANG%3E%0D%0A%3CKUNDE%3E6400320%3C%2FKUNDE%3E%0D%0A%3CLIEFADR%3E.....+Tokio%2C+Japan%3C%2FLIEFADR%3E%0D%0A%3CPOS+NUMMER%3D%221%22%3E%0D%0A++%3CARTNR%3E9000032%3C%2FARTNR%3E%0D%0A++%3CMENGE%3E10%3C%2FMENGE%3E%0D%0A++%3CBESTAND%3E25%3C%2FBESTAND%3E%0D%0A%3C%2FPOS%3E%0D%0A%3CPOS+NUMMER%3D%222%22%3E%0D%0A++%3CARTNR%3E9161161%3C%2FARTNR%3E%0D%0A++%3CMENGE%3E100%3C%2FMENGE%3E%0D%0A++%3CBESTAND%3E33%3C%2FBESTAND%3E%0D%0A++%3CTERMIN%3E20050401%3C%2FTERMIN%3E%0D%0A++%3CPREIS%3E80.00%3C%2FPREIS%3E%0D%0A%3C%2FPOS%3E%0D%0A%3CVERSAND%3E80.00%3C%2FVERSAND%3E%0D%0A%3C%2FVORGANG%3E%0D%0A
if you use url decode to decode the xml="..:"
you get this:
XML=<VORGANG>
<KUNDE>6400320</KUNDE>
<LIEFADR>.....+Tokio,+Japan</LIEFADR>
<POS+NUMMER="1">
++<ARTNR>9000032</ARTNR>
++<MENGE>10</MENGE>
++<BESTAND>25</BESTAND>
</POS>
<POS+NUMMER="2">
++<ARTNR>9161161</ARTNR>
++<MENGE>100</MENGE>
++<BESTAND>33</BESTAND>
++<TERMIN>20050401</TERMIN>
++<PREIS>80.00</PREIS>
</POS>
<VERSAND>80.00</VERSAND>
</VORGANG>
I'm having a bit of trouble calling a method from a generic handler we have. I've tried using two separate techniques to call a simple 'HelloWorld()' method but I get two different errors:
The first technique is as follows:
WebClient wc = new WebClient();
NameValueCollection formData = new NameValueCollection();
formData["method"] = "HelloWorld";
byte[] data;
try
{
data = wc.UploadValues(_domain, formData);
}
catch (WebException ex)
{
Label1.Text = ex.Message;
return;
}
string response = Encoding.UTF8.GetString(data);
Label1.Text = response;
wc.Dispose();
and I get the following error:
{"id":null,"error":{"name":"Found String where Object was expected."}}
and the second technique I've tried is:
var httpWebRequest = (HttpWebRequest)WebRequest.Create(_domain);
httpWebRequest.ContentType = "text/json";
httpWebRequest.Method = "POST";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
string json = "{\"method\":\"helloWorld\"}"; //," +
//"\"password\":\"bla\"}";
streamWriter.Write(json);
streamWriter.Flush();
streamWriter.Close();
try
{
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
}
}
catch (WebException wex)
{
Label2.Text = wex.Message;
}
catch (Exception ex)
{
Label2.Text = ex.Message;
}
}
and with this, I get the following error:
The remote server returned an error: (500) Internal Server Error.
When I test the call from the ".ashx?test" page the method runs and the details at the bottom of the screen are:
Pragma: no-cache
Date: Tue, 23 Jul 2013 13:46:19 GMT
Server: ASP.NET Development Server/11.0.0.0
X-AspNet-Version: 2.0.50727
Content-Type: application/json; charset=utf-8
Cache-Control: no-cache
Connection: Close
Content-Length: 32
Expires: -1
Any ideas as to why this wouldn't be working?
Thanks!
An ASHX handler is not a web service. You don't call methods within the ASXH handler. You just call the handler, and it delivers data directly, be it a text or binary data - that's up to you.