I'm trying to get a JSON response from a service in my C# code. I generate the URL, and pass it to the webClient.DownloadString method, where it comes back with a 400 error. I can copy the URL into a browser and get a correct response.
What might I be missing?
Here's the code:
using (var webClient = new WebClient())
{
try
{
var requestUrl = GetRequestUrl(email);
// Error here
var jsonResponse = webClient.DownloadString(requestUrl);
//...
}
catch (Exception exception)
{
// ...
}
}
Here's a sample of the URL as it's generated from GetRequestUrl(email) which returns a string. (Certain values redacted)
http://api.someservice.org/api.php?usr=xxxxyyyy&pwd=zzz2345mmm22&check=none#nowhere.com
And here's what I get with an actual URL from a browser.
{"authentication_status":1,"limit_status":0,"limit_desc":"Not Limited","verify_status":1,"verify_status_desc":"MX record about nowhere.com exists.Connection succeeded to nowhere-com.mail.protection.outlook.com SMTP.220 BN3NAM01FT045.mail.protection.outlook.com Microsoft ESMTP MAIL Service ready at Mon, 29 Feb 2016 14:52:44 +0000\n> HELO verify-email.org250 BN3NAM01FT045.mail.protection.outlook.com Hello [verify-email.org]\n> MAIL FROM: <check#verify-email.org>=250 2.1.0 Sender OK\n> RCPT TO: <none#nowhere.com>=250 2.1.5 Recipient OK\n"}
UPDATE: Here's what GetRequestUrl does:
protected string GetRequestUrl(string email)
{
if (string.IsNullOrEmpty(email))
throw new ArgumentNullException("email");
var requestUrl = string.Format("http://api.someservice.org/api.php?usr={0}&pwd={1}&check={2}", ApiUsername, ApiPassword, Uri.EscapeUriString(email));
return requestUrl;
}
UPDATE: Here is the exception message, InnerException is NULL.
The remote server returned an error: (400) Bad Request.
UPDATE: Caught the WebException, and got the full ResponseStream. I don't think the call is actually going out, Fiddler doesn't catch a call to the address. Here's the exception.Response:
<h2>Bad Request - Invalid Hostname</h2>
<p>HTTP Error 400. The request hostname is invalid.</p>
Related
Hey guys,
I have a problem with my code. Since about a week my code is not working anymore without any changes. I am pretty sure, that my could should work. All I get is Error 404: forbidden.
Below is a snippet of my Code. I also read about adding a header of the webclient, which did not help. Any other suggestions? I am sorry if my syntax is not that good, it is my first post on stackoverflow.
Thanks in advance!
string epicId = "ManuelNotManni";
WebClient webClient = new WebClient();
Uri uri = new Uri("https://api.tracker.gg/api/v2/rocket-league/standard/profile/epic/");
string result = String.Empty;
try
{
string website = $"{uri.ToString()}{epicId}?";
result = webClient.DownloadString(website);
}
catch (Exception ex)
{
Console.WriteLine($"Error:\n{ex}");
Console.ReadLine();
}
finally
{
webClient.Dispose();
}
This is the exact error:
System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.GetResponse()
at System.Net.WebClient.GetWebResponse(WebRequest request)
at System.Net.WebClient.DownloadBits(WebRequest request, Stream writeStream)
at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
at System.Net.WebClient.DownloadString(Uri address)
at System.Net.WebClient.DownloadString(String address)
at TestProject.Program.Main(String[] args) in > C:\Users\Manue\source\repos\TestProject\Program.cs:line 17
You're right. Your code should work fine.
Issue is that URL you're requesting which is actually:
https://api.tracker.gg/api/v2/rocket-league/standard/profile/epic/ManuelNotManni?
This returns a 403 status code in any case - no matter if you use a browser, your code or for example postman.
I suggest to have a look at the response body while using postman.
It shows this
<html class="no-js" lang="en-US">
<!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta name="captcha-bypass" id="captcha-bypass" />
Tracker.gg wants API users to register their apps with them before they're given access to the API.
What you need to do is to first head to their Getting Started page. Here you will have to create an app, which should give you an authentication key.
When you have done this, you want to change your code slightly to add the Authentication Header. Like so for example:
var webClient = new WebClient();
webclient.Headers.Add("TRN-Api-Key", "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX")
As a sidenote, WebClient has been deprecated and it's recommended to use HttpClient from now on. Here's your code with HttpClient instead:
var epicId = "ManuelNotManni";
var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Add("TRN-Api-Key", "YOUR API KEY GOES HERE");
// Simplifying Uri creation:
var uri = new Uri($"https://api.tracker.gg/api/v2/rocket-league/standard/profile/epic/{epicId}");
var result = string.Empty; // C# prefers lowercase string
try
{
var response = await httpClient.GetAsync(uri);
if (response.IsSuccessStatusCode)
{
result = await response.Content.ReadAsStringAsync();
}
else
{
Console.WriteLine($"Unable to retrieve data for {epicId}.");
Console.WriteLine($"Statuscode: {response.StatusCode}");
Console.WriteLine($"Reason: {response.ReasonPhrase}");
}
}
catch (Exception ex)
{
Console.WriteLine($"Error:\n{ex}");
Console.ReadLine();
}
finally
{
httpClient.Dispose();
}
This happens when we violate the Firewall rule set by Cloudflare, you can visit this blog for more details.
https://community.cloudflare.com/t/community-tip-fixing-error-1020-access-denied/66439
I have a restful web service. If someone calls a particular method with the wrong key, I return this:
OutgoingWebResponseContext response = WebOperationContext.Current.OutgoingResponse;
response.StatusCode = System.Net.HttpStatusCode.Unauthorized;
response.StatusDescription = "Invalid key provided.";
If I call it using Firefox with Firebug running, send an invalid key and handle the error in my calling code like:
string serviceResponse = "";
try
{
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
string statusCode = response.StatusCode.ToString();
string statusDescription = response.StatusDescription.ToString();
}
}
catch(Exception ex)
{
serviceResponse = ex.Message;
}
the Response headers show:
HTTP:1.1 200 OK
Having handled the error, I can show a message on the screen. However, as soon as the 'using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)' line is hit, an error is thrown and the response object seems to be discarded.
If I don't put the HttpWebResponse code in a try catch block, a 401 unauthorized message shows in the browser but Firebug now shows:
HTTP:1.2 500 Internal Server Error
... which is weird, as I am sending a 401 back.
What is the point, in a Restful web service, of setting the response StatusCode and StatusDescription as, in a browser calling the web service, as soon as a 401 or 500 StatusCode is encountered, an error is thrown and the Response object discarded.
If a 500 or 401 is returned, how are you supposed to read the StatusDescription?
I am using the Google Translate API and am trying to capture the data returned when I get an error. (FYI: I know the API Key is wrong, I am just testing this).
The issue is that the browser, as you can see by clicking the link, displays the error info, but C# throws a WebException and I can't seem to get the response data.
Here is my code:
string url = "https://www.googleapis.com/language/translate/v2?key=INSERT-YOUR-KEY&source=en&target=de&q=Hello%20world";
WebClient clnt = new WebClient();
//Get string response
try
{
strResponse = clnt.DownloadString(url);
System.Diagnostics.Debug.Print(strResponse);
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
return null;
}
How do I get the JSON error returned even when the response is a (400) Bad Request (or any other error resonse for that matter)? Do I need to use different classes other than a WebClient?
This may help you
catch ( WebException exception )
{
string responseText;
using(var reader = new StreamReader(exception.Response.GetResponseStream()))
{
responseText = reader.ReadToEnd();
}
}
That will get you the json text, that you can then convert from JSON using whichever method you prefer.
Retrieved from: Get WebClient errors as string
I would catch the specific exception you are receiving - it will have pertinent data concerning the failure.
According to MSDN, WebException.Response will contain the response received from the server.
Once you are able to retrieve the JSON data from this response object, then you'll need to deserialize it yourself if you want.
I am using a simple .NET service (asmx) that works fine when invoking via the test form (POST). When invoking via a HttpWebRequest object, I get a WebException "System.Net.WebException: The remote server returned an error: (500) Internal Server Error." Digging deeper, reading the WebException.Response.GetResponseStream() I get the message: "Missing parameter: serviceType." but I've clearly included this parameter.
I'm at a loss here, and its worse that I don't have access to debug the service itself.
Here is the code being used to make the request:
string postData = String.Format("serviceType={0}&SaleID={1}&Zip={2}", request.service, request.saleId, request.postalCode);
byte[] data = (new ASCIIEncoding()).GetBytes(postData);
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.Timeout = 60000;
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.ContentLength = data.Length;
using (Stream newStream = httpWebRequest.GetRequestStream())
{
newStream.Write(data, 0, data.Length);
}
try
{
using (response = (HttpWebResponse)httpWebRequest.GetResponse())
{
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception("There was an error with the shipping freight service.");
string responseData;
using (StreamReader responseStream = new StreamReader(httpWebRequest.GetResponse().GetResponseStream(), System.Text.Encoding.GetEncoding("iso-8859-1")))
{
responseData = responseStream.ReadToEnd();
responseStream.Close();
}
if (string.IsNullOrEmpty(responseData))
throw new Exception("There was an error with the shipping freight service. Request went through but response is empty.");
XmlDocument providerResponse = new XmlDocument();
providerResponse.LoadXml(responseData);
return providerResponse;
}
}
catch (WebException webExp)
{
string exMessage = webExp.Message;
if (webExp.Response != null)
{
using (StreamReader responseReader = new StreamReader(webExp.Response.GetResponseStream()))
{
exMessage = responseReader.ReadToEnd();
}
}
throw new Exception(exMessage);
}
Anyone have an idea what could be happening?
Thanks.
UPDATE
Stepping through the debugger, I see the parameters are correct. I also see the parameters are correct in fiddler.
Examining fiddler, I get 2 requests each time this code executes. The first request is a post that sends the parameters. It gets a 301 response code with a "Document Moved Object Moved This document may be found here" message. The second request is a GET to the same URL with no body. It gets a 500 server error with "Missing parameter: serviceType." message.
It seems like you found your problem when you looked at the requests in Fiddler. Taking an excerpt from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html:
10.3.2 301 Moved Permanently
The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible.
.....
Note: When automatically redirecting a POST request after
receiving a 301 status code, some existing HTTP/1.0 user agents
will erroneously change it into a GET request.
Here's a couple options that you can take:
Hard-code your program to use the new Url that you see in the 301 response in Fiddler
Adjust your code to retrieve the 301 response, parse out the new Url from the response, and build a new response with the new Url.
The latter option would be ideal if you're dealing with user-based input on the Url (like a web browser), since you don't know where the user is going to want your program to go.
I'm querying Wikipedia using the following code, but I always get an error (403 forbidden). When I type the exact same url in my browser, however, it works. I've been using the same code before to query other web apis, so I am not sure what's causing the trouble.
private static string query(string text)
{
text = text.Replace(" ", "%20");
string url = "http://en.wikipedia.org/w/api.php?action=opensearch&search=" + text + "&format=json&callback=spellcheck";
WebClient client = new WebClient();
client.Headers.Add("User-Agent", "whatever"); // <-- this line was missing
try
{
string response = client.DownloadString(url);
return response;
}
catch(Exception e)
{
Console.WriteLine(e.Message);
return null;
}
}
Try setting the user agent header to something that matches your browser. If this doesn't work, fire up Fiddler, take a peek at your browser headers and copy them to your web request.
http://msdn.microsoft.com/en-us/library/system.net.webclient.headers.aspx
EDIT
The advice I gave was generic. Please observe the policies of the website you are downloading from, as spoofing a browser user-agent may contravene policy or be considered malicious by default:
http://meta.wikimedia.org/wiki/User-Agent_policy :
Do not copy a browser's user agent for your bot, as bot-like behavior with a browser's user agent will be assumed malicious.