We Keep Coding

Difficulty receiving an HTTP Response from API -- Bad Request Error

I'm attempting to pass username/password from an application to the API to receive a token authorization key. When I attempt to do so, I receive a 400 Bad Request error and I cannot figure out why. Below is the method in question:
public User UserAuthentication(string username, string password)
{
string endpoint = baseURL + "/TOKEN";
// Could be POST maybe
string method = "POST";
Credential jsonObj = new Credential
{
grant_type = "password",
username = username,
password = password
};
string jsonStr = JsonConvert.SerializeObject(jsonObj);
WebClient wc = new WebClient();
//x - www - form - urlencoded
wc.Headers[HttpRequestHeader.ContentType] = "application/x - www - form - urlencoded";
wc.Headers.Add("Access-Control-Allow-Headers", "content-type");
wc.Headers.Add("Access-Control-Allow-Origin", "*");
wc.Headers[HttpRequestHeader.Authorization] = "Bearer <token>";
wc.Headers.Add("Access-Control-Allow-Methods", "POST, PUT, GET, DELETE, OPTIONS");
string header = wc.Headers.ToString();
try
{
string response = wc.UploadString(endpoint, method, jsonStr);
return JsonConvert.DeserializeObject<User>(response);
}
catch(Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
I've messed around altering just about everything in this method in search of a fix.
What I've done:
/TOKEN was /values & /api/values
POST method was GET -- With this, I received a "Cannot send a content-body with this verb-type." error.
ContentType was changed to "application/json"
Access-Control-Allow-Origin had the baseURL
Checked the format of header & body:
Header:
{Content-Type: application/x - www - form - urlencoded
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Origin: *
Authorization: Bearer <token>
Access-Control-Allow-Methods: POST, PUT, GET, DELETE, OPTIONS}
Body:
{"grant_type":"password",
"username":"test#gmail.com",
"password":"password123"}
I obviously have something wrong in my request, I've just run out of ideas to try. I'm not entirely sure if UploadString() is the correct method to be using in this situation, but I couldn't find another method in the WebClient class that would be better. Any help to try and push me in the right direction would be very much appreciated.

So what I think you are trying to do is a form-urlencoded post to a "token" endpoint with a username/password grant. These are typically done like so:
using (var request = new HttpRequestMessage(HttpMethod.Post, new Uri("https://example.com/token"))
{
Content = new FormUrlEncodedContent(new Dictionary<string, string>
{
{ "grant_type", "password" },
{ "username", "username#site.com" },
{ "password", "password12345" }
})
})
{
using (var resp = await _client.SendAsync(request))
{
resp.EnsureSuccessStatusCode();
//await resp.Content.ReadAsAsync<BearerToken>();
// for testing purposes, try this:
var returnData = await resp.Content.ReadAsStringAsync();
Console.WriteLine(returnData);
}
}
You should define this outside all scopes where you need to do Http requests:
private static readonly HttpClient _client = new HttpClient();
So, first off, try to stick with HttpClient. Other patterns such as WebClient are considered legacy.
Next, CORS headers are typically returned from the server when an OPTIONS call is sent to the server. You aren't doing that here, and you should never have to worry about that kind of stuff inside a C# program running from your computer. So you can drop the access-control header stuff.
Form-urlencoded data is not JSON data. It's a different way to format data. If you want to send JSON data, you should use the content-type application/json
Finally, you are trying to add an Authorization header. But that doesn't make much sense as you are trying to authenticate yourself to become authorized. If you send the right username/password, you will receive a bearer token that you can use in an Authorization header for future requests to said service.
Oh and I forgot to add: Whenever you see an error in the [400,499] range (in this case "400 - bad request") it means that you sent something wrong and the server doesn't understand what you are trying to do. For example: a 401 means you sent invalid or missing authorization information. A 400 means your data was probably malformed.
But I like your question... I can see what you were doing and you tried all kinds of different things.
Download a program called Fiddler if you want to see how HTTP works. It's a great tool to debug your HTTP calls.

What is the role of my machine's Application pool in Windows service development when consuming third party rest services [duplicate]

I am working on Windows Service in visual studio 2017. In the rest api's call, getting exceptions while debugging code. Sometimes first 2 3 calls working after that getting exceptions.
System.Net.WebException: 'The remote server returned an error: (503)
Server Unavailable.'
The remote server returned an error: (429)
Unable to connect to the remote server
When calling same api's from Postman, getting response successfully.
This is my code
private void timer1_Tick(object sender, ElapsedEventArgs e)
{
WriteToFile("timer1_Tick method called..");
try
{
string jsonString = "";
string jsonstring2 = "";
string prodfetchurl = HOST;
var req = WebRequest.Create(prodfetchurl) as HttpWebRequest;
req.Method = "GET";
InitializeRequest(req);
req.Accept = MIME_TYPE;
//System.Threading.Thread.Sleep(5000);
var response = (HttpWebResponse)req.GetResponse();
WriteToFile("First service called...");
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
jsonString = responseReader.ReadToEnd();
}
var deserialsseobj = JsonConvert.DeserializeObject<ProductList>(jsonString).Products.Where(i => i.Failed > 0).ToList();
foreach (var a in deserialsseobj)
{
var pid = a.ID;
string url = FailedDevicesUrl + pid.Value + "/failed";
var req2 = WebRequest.Create(url) as HttpWebRequest;
req2.Method = "GET";
InitializeRequest(req2);
req2.Timeout = 300000;
req2.Accept = MIME_TYPE;
var response1 = (HttpWebResponse)req2.GetResponse();
Stream responsestream2 = response1.GetResponseStream();
WriteToFile("Second service called...");
if (response1.StatusCode == HttpStatusCode.OK)
{
StreamReader responsereader1 = new StreamReader(responsestream2);
jsonstring2 = responsereader1.ReadToEnd();
}
var output = JsonConvert.DeserializeObject<List<FailedDeviceList>>(jsonstring2); // Will get List of the Failed devices
List<int> deviceids = new List<int>();
Reprocessdata reproc = new Reprocessdata();
Reprocessdata.DeviceId rprod = new Reprocessdata.DeviceId();
reproc.ForceFlag = true;
reproc.ProductID = pid.Value;
foreach (var dd in output)
{
rprod.ID = dd.DeviceId;
reproc.DeviceIds.Add(rprod);
}
// Reprocess the Product in Devices
var req3 = WebRequest.Create(ReprocessUrl) as HttpWebRequest;
req3.Method = "POST";
InitializeRequest(req3);
req3.Accept = MIME_TYPE;
req3.Timeout = 300000;
req3.ContentType = "application/json";
using (StreamWriter writer = new StreamWriter(req3.GetRequestStream()))
{
string json = new JavaScriptSerializer().Serialize(reproc);
writer.Write(json);
writer.Close();
}
System.Threading.Thread.Sleep(5000);
var response5 = (HttpWebResponse)req3.GetResponse();
WriteToFile("Third service called...");
if (response5.StatusCode == HttpStatusCode.OK)
{
string result;
using (StreamReader rdr = new StreamReader(response5.GetResponseStream()))
{
result = rdr.ReadToEnd();
}
}
}
response.Close();
}
catch (Exception ex)
{
WriteToFile("Simple Service Error on: {0} " + ex.Message + ex.StackTrace);
}
}
Methods used in above code
protected override void OnStart(string[] args)
{
base.OnStart(args);
timer1 = new System.Timers.Timer();
timer1.Interval = 60000; //every 1 min
timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Tick);
timer1.Enabled = true;
WriteToFile("Service has started..");
}
public void InitializeRequest(HttpWebRequest request)
{
request.Headers.Add("aw-tenant-code", API_TENANT_CODE);
request.Credentials = new NetworkCredential(USER_NAME, PASSWORD);
request.KeepAlive = false;
request.AddRange(1024);
}
When I contacted service provide they said everything fine from there side. Is this my code is buggy or windows service not reliable? How can I fix this issue?
Note: All APIS are working fine from Angular application using Visual Studio Code. It means my code is not working.
Edit1: Three below services I am using from this document of VMware.
private const string HOST = "https:host/api/mdm/products/search?";
private const string FailedDevicesUrl = "https:host/api/mdm/products/";
private const string ReprocessUrl = "https:host/api/mdm/products/reprocessProduct";

Response http code 429 indicates that you sending too many requests on target web service.
This means service you trying to send requests has a policies that blocks some requests by request-per-time limit.
Also I admit that external service can be manually configured to throw 403 code in specific cases that you can't know about. If that, this information can be explained in external service documentation... or not :)
What you can do with this?
Fit in limitations
You can make detailed research what limits target webservice has and set up your code to fit in this limitations. For example if service has limitation for receiving only one request per 10 minutes - you must set up your timer to send one request each 10 or more minutes. If documentation not provide such information - you can test it manually by finding some patterns with external service responses.
Use proxy
Every limitation policy based on information about requests senders. Usually this information consists of IP address of sender only. This means if you send 2 requests from two different IP addresses - limitation policy will perceive that like 2 different computers sending these requests. So you can find/buy/rent some proxy IP addresses and send requests through there on target web server.
How to connect through proxy in C# using WebRequest you can see in this answer.
Negotiate with external service provider
If you have possibility to communicate with external service developers or help center, you can ask their to reduce limitations for your IP address (if it static) or provide some mechanisms to avoid limitation policy for you. If for some reason they cannot provide this opportunity, at least you can ask detailed information about limitations.
Repetition mechanism
Some times 503 error code that is outer exception you received may be caused by service unavailable. It means that server can be under maintenance or temporary overloaded. So you can write repetition mechanism to make continious sending requests to server until it'll be accessible.
Polly library may help you with repetition mechanism creation

The inner error of that 503 is:
The remote server returned an error: (429)
HTTP 429 indicates too many requests. Maybe your upstream server can’t process all requests sent.
This can happen when you reached rate limiting / throttling value if you’re calling a third party API.
UPDATE
As per page 28 in the API docs, you could configure throttling when creating a new API. Check if the throttling is too small or maybe turn off the throttling and see if that could fix the error?

Web API2 proxy doesn't redirect the requests

I have a proxy controller in order to redirect the Ajax requests and pass the same cookies from the current domain to the Web API endpoint, but it doesn't work as what I expected. e.g. the cookies in "https://www.example.com", the Web API URL "https://api.example.com/xyz/abc/". What I am trying to do is sending an Ajax request to
"https://www.example.com/api/proxy/something"
and hoping it to be redirected to
"https://api.example.com/xyz/abc/something" with the same settings (especially the cookies).
Here is the API controller in the web site:
public class ProxyController : ApiController
{
private string _baseUri = "https://api.example.com/xyz/abc/";
[AcceptVerbs(Http.Get, Http.Head, Http.MkCol, Http.Post, Http.Put)]
public async Task<HttpResponseMessage> Proxy()
{
using (HttpClient http = new HttpClient())
{
string proxyURL = this.Request.RequestUri.AbsolutePath;
int indexOfProxy = proxyURL.IndexOf("proxy/") + 6;
_baseUri = _baseUri + proxyURL.Substring(indexOfProxy, proxyURL.Length - indexOfProxy);
this.Request.RequestUri = new Uri(_baseUri);
//For some reason Ajax request sets Content in the Get requests, because
//of that it was denied complaining about "cannot send a content-body"
if (this.Request.Method == HttpMethod.Get)
{
this.Request.Content = null;
}
return await http.SendAsync(this.Request);
}
}
}
It doesn't redirect the requests. In the response, the Requested URL is the same as the original request. The Host in the request header is "www.example.com" instead of "api.example.com', I am going nuts with this issue for the last few days.

HttpClient will work fine. We have done this many times. However, cookies are tricky as they are tied to the domain. Here is some code of a proxy without cookies that should get you started.
[AcceptVerbs(Http.Get, Http.Head, Http.MkCol, Http.Post, Http.Put)]
public async Task<HttpResponseMessage> Proxy()
{
var request = this.Request;
var proxyUri = this.GetProxyUri(request.RequestUri);
request.RequestUri = proxyUri;
request.Headers.Host = proxyUri.Host;
if (request.Method == HttpMethod.Get)
{
request.Content = null;
}
//todo: Clone all cookies with the domain set to the domain of the proxyUri. Remove the old cookies and add the clones.
using (var client = new HttpClient())
{
//default is 60 seconds or so
client.Timeout = TimeSpan.FromMinutes(5);
return await client.SendAsync(request, HttpCompletionOption.ResponseContentRead);
}
}
private string _baseUri = "https://api.example.com/xyz/abc/";
private Uri GetProxyUri(Uri originalUri)
{
var proxyUri = originalUri.AbsolutePath;
var indexOfProxy = proxyUri.IndexOf("proxy/") + 6;
var finalUri = _baseUri + proxyUri.Substring(indexOfProxy, proxyUri.Length - indexOfProxy);
return new Uri(finalUri);
}
You may not be able to get cookies to work properly because of the domain switching. Your client and server will be limited to it's domain. If you own the proxy destination, you may want to change it to allow other mechanisms besides cookies. Can you use headers, or querystring, etc? The domain just may be a killer.

I had exactly this kind of problem like two days ago.
Haven't figured out exactly what was causing it, I'm going to investigate today and let you know.
But until then, you can try using RestSharp, it fixed my problem.
I think it is a problem with the BaseAddress property of the HttpClient, it somehow gets initialized with the initial address and makes the request to that address instead of the proxy one.
I will investigate and let you know.

.NET service responds 500 internal error and "missing parameter" to HttpWebRequest POSTS but test form works fine

I am using a simple .NET service (asmx) that works fine when invoking via the test form (POST). When invoking via a HttpWebRequest object, I get a WebException "System.Net.WebException: The remote server returned an error: (500) Internal Server Error." Digging deeper, reading the WebException.Response.GetResponseStream() I get the message: "Missing parameter: serviceType." but I've clearly included this parameter.
I'm at a loss here, and its worse that I don't have access to debug the service itself.
Here is the code being used to make the request:
string postData = String.Format("serviceType={0}&SaleID={1}&Zip={2}", request.service, request.saleId, request.postalCode);
byte[] data = (new ASCIIEncoding()).GetBytes(postData);
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.Timeout = 60000;
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.ContentLength = data.Length;
using (Stream newStream = httpWebRequest.GetRequestStream())
{
newStream.Write(data, 0, data.Length);
}
try
{
using (response = (HttpWebResponse)httpWebRequest.GetResponse())
{
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception("There was an error with the shipping freight service.");
string responseData;
using (StreamReader responseStream = new StreamReader(httpWebRequest.GetResponse().GetResponseStream(), System.Text.Encoding.GetEncoding("iso-8859-1")))
{
responseData = responseStream.ReadToEnd();
responseStream.Close();
}
if (string.IsNullOrEmpty(responseData))
throw new Exception("There was an error with the shipping freight service. Request went through but response is empty.");
XmlDocument providerResponse = new XmlDocument();
providerResponse.LoadXml(responseData);
return providerResponse;
}
}
catch (WebException webExp)
{
string exMessage = webExp.Message;
if (webExp.Response != null)
{
using (StreamReader responseReader = new StreamReader(webExp.Response.GetResponseStream()))
{
exMessage = responseReader.ReadToEnd();
}
}
throw new Exception(exMessage);
}
Anyone have an idea what could be happening?
Thanks.
UPDATE
Stepping through the debugger, I see the parameters are correct. I also see the parameters are correct in fiddler.
Examining fiddler, I get 2 requests each time this code executes. The first request is a post that sends the parameters. It gets a 301 response code with a "Document Moved Object Moved This document may be found here" message. The second request is a GET to the same URL with no body. It gets a 500 server error with "Missing parameter: serviceType." message.

It seems like you found your problem when you looked at the requests in Fiddler. Taking an excerpt from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html:
10.3.2 301 Moved Permanently
The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible.
.....
Note: When automatically redirecting a POST request after
receiving a 301 status code, some existing HTTP/1.0 user agents
will erroneously change it into a GET request.
Here's a couple options that you can take:
Hard-code your program to use the new Url that you see in the 301 response in Fiddler
Adjust your code to retrieve the 301 response, parse out the new Url from the response, and build a new response with the new Url.
The latter option would be ideal if you're dealing with user-based input on the Url (like a web browser), since you don't know where the user is going to want your program to go.

Using uTorrent Web API via .NET

I'm trying to get list of torrents from uTorrent using Web API. Getting required token goes O.K.:
WebClient client = new WebClient() { Credentials = new NetworkCredential(UserName, pass) };
StreamReader Reader = new StreamReader(client.OpenRead("http://localhost:" + port + "/gui/token.html"));
string token = Reader.ReadToEnd();
token = token.Split('>')[2].Split('<')[0];
// token is now something like 3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA
But when I try to use it to get list of torrents:
Reader = new StreamReader(client.OpenRead("http://localhost:" + port + "/gui/?list=1&token=" + token));
all I get is "Error 400 Bad request".
I've tried to get token manually. In browser page "http://localhost:30303/gui/?list=1&token=3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA" opens as it should, but in C# with the same link without any variables I still get error 400.
The interesting part is that if switch off token authentication WebClient load page perfectly with and without
"&token=3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA"
but token auth enabled by default, so my and any app should use it.
And yes, WebRequest/HttpWebRequest didn't help also.
P.S. sorry for my English, I was never able to make it work right

you have to save the cookie from the request
Classes.CookieAwareWebClient client = new Classes.CookieAwareWebClient() { Credentials = new NetworkCredential("shehab", "shehab") };
StreamReader Reader = new StreamReader(client.OpenRead("http://localhost:" + "8080" + "/gui/token.html"));
string token = HtmlRemoval.StripTagsRegexCompiled(Reader.ReadToEnd());
MessageBox.Show(token);
Reader = new StreamReader(client.OpenRead("http://localhost:" + "8080" + "/gui/?list=1&token=" + token));
MessageBox.Show(Reader.ReadToEnd());
and for the cookie aware class go to the following link(Using CookieContainer with WebClient class) as web client doesn't support cookies.

You should save cookies from request
WebRequest request = WebRequest.Create("http://localhost:" + port + "/gui/token.html");
CookieContainer cookies = new CookieContainer();
(request as HttpWebRequest).CookieContainer = cookies;
And then use it in every other request to uTorrent when using the same token:
request = WebRequest.Create("http://localhost:" + port + "/gui/?list=1&token=" + token);
(request as HttpWebRequest).CookieContainer = cookies;

I have a simple 3-step suggestion:
When you use your browser with the token, use Fiddler2 to analyze the HTTP traffic between the server and browser.
Open up your C# app and use Fiddler2 to analyze the HTTP traffic between the server and your app.
Compare the HTTP requests and responses for the browser with the requests and responses for the C# app. If you see a significant difference, there is a good chance that could be the problem.