GetResponse keeps throwing page not found exception - c#

Can you please help me out here? I am trying to get content of any web page. But GetResponse keeps throwing exception page not found. I appreciate your help. Following is my code.
try
{
System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.smallchiptechnologies.com/");
request.Method = "POST";
request.ContentType = "text/plain";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
}

First, it looks like it should be GET response, not POST (since you just trying to get data from server and not posting any form data or something similar) so change request.Method = "POST"; to request.Method = "GET";
Second, you're not reading anything from response stream. Add something like this to your code to get page content:
string text;
using (var reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
text = reader.ReadToEnd();
}

Have you discarded potential proxy issues? For example, if you are running that code behind a corporate webproxy, you'll need to modify slightly your code in order to support the connection thru that proxy.
Something like this...
webrequest.Proxy = new WebProxy(ProxyServer,ProxyPort);
More details here: https://msdn.microsoft.com/en-us/library/czdt10d3(v=vs.110).aspx

Related

read onedrive api errors

I am current using the following to try and create an upload session with the onedrive api
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.Headers["Authorization"] = "Bearer " + nODUserTokenObj.Access_Token;
request.ContentType = "application/json";
string data =
#"{
""#microsoft.graph.conflictBehavior"": ""replace""
}";
byte[] postBytes = Encoding.ASCII.GetBytes(data);
request.ContentLength = postBytes.Length;
request.Accept = "application/json";
request.GetRequestStream().Write(postBytes, 0, postBytes.Length);
using (HttpWebResponse httpWebResponse = (HttpWebResponse)request.GetResponse())
{
using(StreamReader sr = new StreamReader(httpWebResponse.GetResponseStream()))
{
string theData = sr.ReadToEnd();
JObject test = JObject.Parse(theData);
}
}
Currently I am receiving an exception stating
The remote server returned an error: (409) Conflict.
However looking at the Microsoft documentation I should have a json object returned to me with more information.
What am I doing wrong that I am getting an exception thrown when it hits (HttpWebResponse)request.GetResponse() as opposed to receiving a json object with more error details?
I came across two solutions:
I initially resolved this by sending the request through Fiddler and reading the response there. This was fine for my situation as I was developing and just trying to figure out how the onedrive api functioned.
A solution going forward would be to follow something like this post suggests (How to get error information when HttpWebRequest.GetResponse() fails)
To summarize the post:
catch (WebException ex)
{
using (var stream = ex.Response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
Console.WriteLine(reader.ReadToEnd());
}
}
catch (Exception ex)
{
// Something more serious happened
// like for example you don't have network access
// we cannot talk about a server exception here as
// the server probably was never reached
}

c# App skip after reading JSON response

I been trying to use an online API that returns an a json.
I am using winform application at the moment.
So far i tried
WebClient cHttp = new WebClient();
string htmlCode = cHttp.DownloadString(path); <--------
///-----------And then this
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(path);
request.Method = WebRequestMethods.Http.Get;
request.Accept = "application/json";
HttpWebResponse response = (HttpWebResponse)request.GetResponse(); <-----
Where i point the arrow to, the program doesn't crash it just hit that line and then skip all the code below it. Then my form open without running my entire code. What am i doing wrong?
Thank you
Use try-catch block, and you'll see an error:
try
{
WebClient cHttp = new WebClient();
string htmlCode = cHttp.DownloadString(path);
}
catch(Exception e)
{
Debug.WriteLine(e);
}
It jumps out of your method. You must catch the exception and return null object. or find the error.

HttpWebRequest.GetResponse() keeps getting timed out

i wrote a simple C# function to retrieve trade history from MtGox with following API call:
https://data.mtgox.com/api/1/BTCUSD/trades?since=<trade_id>
documented here: https://en.bitcoin.it/wiki/MtGox/API/HTTP/v1#Multi_currency_trades
here's the function:
string GetTradesOnline(Int64 tid)
{
Thread.Sleep(30000);
// communicate
string url = "https://data.mtgox.com/api/1/BTCUSD/trades?since=" + tid.ToString();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string json = reader.ReadToEnd();
reader.Close();
reader.Dispose();
response.Close();
return json;
}
i'm starting at tid=0 (trade id) to get the data (from the very beginning). for each request, i receive a response containing 1000 trade details. i always send the trade id from the previous response for the next request. it works fine for exactly 4 requests & responses. but after that, the following line throws a "System.Net.WebException", saying that "The operation has timed out":
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
here are the facts:
catching the exception and retying keeps causing the same exception
the default HttpWebRequest .Timeout and .ReadWriteTimeout are already high enough (over a minute)
changing HttpWebRequest.KeepAlive to false didn't solve anything either
it seems to always work in the browser even while the function is failing
it has no problems retrieveing the response from https://www.google.com
the amount of successful responses before the exceptions varies from day to day (but browser always works)
starting at the trade id that failed last time causes the exception immediately
calling this function from the main thread instead still caused the exception
running it on a different machine didn't work
running it from a different IP didn't work
increasing Thread.Sleep inbetween requests does not help
any ideas of what could be wrong?
I had the very same issue.
For me the fix was as simple as wrapping the HttpWebResponse code in using block.
using (HttpWebResponse response = (HttpWebResponse) request.GetResponse())
{
// Do your processings here....
}
Details: This issue usually happens when several requests are made to the same host, and WebResponse is not disposed properly. That is where using block will properly dispose the WebResponse object properly and thus solving the issue.
There are two kind of timeouts. Client timeout and server timeout. Have you tried doing something like this:
request.Timeout = Timeout.Infinite;
request.KeepAlive = true;
Try something like this...
I just had similar troubles calling a REST Service on a LINUX Server thru ssl. After trying many different configuration scenarios I found out that I had to send a UserAgent in the http head.
Here is my final method for calling the REST API.
private static string RunWebRequest(string url, string json)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
// Header
request.ContentType = "application/json";
request.Method = "POST";
request.AllowAutoRedirect = false;
request.KeepAlive = false;
request.Timeout = 30000;
request.ReadWriteTimeout = 30000;
request.UserAgent = "test.net";
request.Accept = "application/json";
request.ProtocolVersion = HttpVersion.Version11;
request.Headers.Add("Accept-Language","de_DE");
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
byte[] bytes = Encoding.UTF8.GetBytes(json);
request.ContentLength = bytes.Length;
using (var writer = request.GetRequestStream())
{
writer.Write(bytes, 0, bytes.Length);
writer.Flush();
writer.Close();
}
var httpResponse = (HttpWebResponse)request.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var jsonReturn = streamReader.ReadToEnd();
return jsonReturn;
}
}
This is not a solution, but just an alternative:
These days i almost only use WebClient instead of HttpWebRequest. Especially WebClient.UploadString for POST and PUT and WebClient.DownloadString. These simply take and return strings. This way i don't have to deal with streams objects, except when i get a WebException. i can also set the content type with WebClient.Headers["Content-type"] if necessary. The using statement also makes life easier by calling Dispose for me.
Rarely for performance, i set System.Net.ServicePointManager.DefaultConnectionLimit high and instead use HttpClient with it's Async methods for simultaneous calls.
This is how i would do it now
string GetTradesOnline(Int64 tid)
{
using (var wc = new WebClient())
{
return wc.DownloadString("https://data.mtgox.com/api/1/BTCUSD/trades?since=" + tid.ToString());
}
}
2 more POST examples
// POST
string SubmitData(string data)
{
string response;
using (var wc = new WebClient())
{
wc.Headers["Content-type"] = "text/plain";
response = wc.UploadString("https://data.mtgox.com/api/1/BTCUSD/trades", "POST", data);
}
return response;
}
// POST: easily url encode multiple parameters
string SubmitForm(string project, string subject, string sender, string message)
{
// url encoded query
NameValueCollection query = HttpUtility.ParseQueryString(string.Empty);
query.Add("project", project);
query.Add("subject", subject);
// url encoded data
NameValueCollection data = HttpUtility.ParseQueryString(string.Empty);
data.Add("sender", sender);
data.Add("message", message);
string response;
using (var wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
response = wc.UploadString( "https://data.mtgox.com/api/1/BTCUSD/trades?"+query.ToString()
, WebRequestMethods.Http.Post
, data.ToString()
);
}
return response;
}
Error handling
try
{
Console.WriteLine(GetTradesOnline(0));
string data = File.ReadAllText(#"C:\mydata.txt");
Console.WriteLine(SubmitData(data));
Console.WriteLine(SubmitForm("The Big Project", "Progress", "John Smith", "almost done"));
}
catch (WebException ex)
{
string msg;
if (ex.Response != null)
{
// read response HTTP body
using (var sr = new StreamReader(ex.Response.GetResponseStream())) msg = sr.ReadToEnd();
}
else
{
msg = ex.Message;
}
Log(msg);
}
For what it's worth, I was experiencing the same issues with timeouts every time I used it, even though calls went through to the server I was calling. The problem in my case was that I had Expect set to application/json, when that wasn't what the server was returning.

HttpWebRequest method HEAD returns body

My web request with the method "Head" keeps returning the body of my webpage (on localhost).
Here is how it is basically created:
HttpWebRequest webrequest = WebRequest.Create(url.ToString()) as HttpWebRequest;
webrequest.Method = "HEAD";
WebResponse response = webrequest.GetResponse();
As I put a breakpoint in my aspx.cs page, I step into the OnInit() method and also the Page_Load() method where I believe I'm not supposed to step in with a Head method request (am I wrong?).
In my Page_Load() I execute some code that I do not want to be executed when I call with the Head method, but later when I call with the Get method (once I got the headers).
Am I missing something? (not too familiar with Http requests and responses yet... :/)
Thank you for your help!
Try this sample code approach ....
for (int i = 0; i < ParsedLinks.Count; i++)
{
Thread.Sleep(500);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(ParsedLinks[i]);
req.Method = "HEAD";
req.Credentials = CredentialCache.DefaultCredentials;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
bool b_Result = int.TryParse(resp.Headers.Get("Content-Length"), out i_ContentLength);
int i_Size = (int)(i_ContentLength / 1024);
req.Abort();
resp.Close();
}
hope it helps
http://forums.asp.net/t/1412824.aspx/1
The difference is, if you try to read the response you get nothing in case of HEAD. Where as you can see the response content in case of GET
var response = webrequest.GetResponse().GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(response, encode);
Console.WriteLine(readStream.ReadToEnd().Length) // you should see 0
readStream.Close();

500 internal server error at GetResponse()

I have a heavy traffic aspx page calling a web service upon every user`s request as follows.
string uri = "Path.asmx";
string soap = "soap xml string";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Headers.Add("SOAPAction", "\"http://xxxxxx"");
request.ContentType = "text/xml;charset=\"utf-8\"";
request.Accept = "text/xml";
request.Method = "POST";
using (Stream stm = request.GetRequestStream())
{
using (StreamWriter stmw = new StreamWriter(stm))
{
stmw.Write(soap);
}
}
WebResponse response = request.GetResponse();
response.close();
Everything is working fine but sometimes I am getting the following error.
The remote server returned an error: (500) Internal Server Error.
at System.Net.HttpWebRequest.GetResponse()
Does anybody have any idea about this error or can anybody tell me if I am doing wrong.
For me this error occurred because I had 2 web API actions that had the exact same signatures and both had the same verbs, HttpPost, what I did was change one of the verbs (the one used for updating) to PUT and the error was removed. The following in my catch statement helped in getting to the root of the problem:
catch (WebException webex)
{
WebResponse errResp = webex.Response;
using (Stream respStream = errResp.GetResponseStream())
{
StreamReader reader = new StreamReader(respStream);
string text = reader.ReadToEnd();
}
}
From that error, I would say that your code is fine, at least the part that calls the webservice. The error seems to be in the actual web service.
To get the error from the web server, add a try catch and catch a WebException. A WebException has a property called Response which is a HttpResponse. you can then log anything that is returned, and upload you code. Check back later in the logs and see what is actually being returned.
Finally I get rid of internal server error message with the following code. Not sure if there is another way to achieve it.
string uri = "Path.asmx";
string soap = "soap xml string";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Headers.Add("SOAPAction", "\"http://xxxxxx"");
request.ContentType = "text/xml;charset=\"utf-8\"";
request.Accept = "text/xml";
request.Method = "POST";
using (Stream stm = request.GetRequestStream())
{
using (StreamWriter stmw = new StreamWriter(stm))
{
stmw.Write(soap);
}
}
using (WebResponse webResponse = request.GetResponse())
{
}
Have you tried to specify UserAgent for your request? For example:
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
looking at your error message first of all I would suggest you to recompile your whole application, make sure all the required dlls are there in bin folder when you recompile it.
In my case my request object inherited from base object. Without knowingly I added a property with int? in my request object and my base object also has same property ( same name ) with int datatype. I noticed this and deleted the property which I added in request object and after that it worked fine.
For me the error was misleading. I discovered the true error by testing the errant web service with SoapUI.
In my case I just remove the SoapAction instruction from the HttpWebRequest object. So, I don't define .Headers.Add("SOAPAction","someurl") in HttpWebRequest definitions and my code works fine.
ResultXML is an XDocument.
ResultString is a string.
try
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Url);
//req.Headers.Add("SOAPAction", "http://tempuri.org/IWebService/GetMessage");
req.ProtocolVersion = HttpVersion.Version11;
req.ContentType = "text/xml;charset=\"utf-8\"";
req.Accept = "text/xml";
req.KeepAlive = true;
req.Method = "POST";
using (Stream stm = req.GetRequestStream())
{
using (StreamWriter stmw = new StreamWriter(stm))
stmw.Write(soapStr);
}
using (StreamReader responseReader = new StreamReader(req.GetResponse().GetResponseStream()))
{
string result = responseReader.ReadToEnd();
ResultXML = XDocument.Parse(result);
ResultString = result;
}
}

Categories