I am currently working on a local http server written in C#. At this point I am not yet processing post data, but only distinguish between get and post request. In both cases, of course, a 200 should be answered. While testing the server, I noticed that if I send an empty post request, it is answered by the server with a 200 and an html page just like a get request without any problems. However, if there are images attached to the post request, as in my example, the connection to the server fails immediately.
I handle a client connection as follows. I know it's not ideal to store the received bytes in a string, but for testing purposes I haven't seen any problems with it.
private void HandleClient(TcpClient client)
{
Byte[] bytes;
String requestData = "";
NetworkStream ns = client.GetStream();
if (client.ReceiveBufferSize > 0)
{
bytes = new byte[client.ReceiveBufferSize];
ns.Read(bytes, 0, client.ReceiveBufferSize);
requestData = Encoding.ASCII.GetString(bytes);
}
// Get Request out of message
Request request = Request.GetRequest(requestData);
// Create Response
Response response = Response.From(request);
response.Post(client.GetStream());
}
And here is the method I use to determine what type of request it is.
public static Request GetRequest(String request)
{
//return if request is null
if(String.IsNullOrEmpty(request))
{
return null;
}
//Split Request to get tokens - split by spaces
String[] tokens = request.Split(' ');
String type = tokens[0];
String url = tokens[1];
String host = tokens[4];
return new Request(type, url, host);
}
Surely it must be possible to read only the headers from a get as well as post request and then still give a 200 response. Is there a rule of behavior for an http server on how it should handle post-request data?
The answer to my question was quite simple in the end. The input stream of a request must be read completely before the server can respond to the request. In my case it was so, that I only read the header of the request to know if it is a Post or Get request, therefore the server could not respond to the request in case of an attached image, because the input stream was not read completely.
Related
I'm attempting to pass username/password from an application to the API to receive a token authorization key. When I attempt to do so, I receive a 400 Bad Request error and I cannot figure out why. Below is the method in question:
public User UserAuthentication(string username, string password)
{
string endpoint = baseURL + "/TOKEN";
// Could be POST maybe
string method = "POST";
Credential jsonObj = new Credential
{
grant_type = "password",
username = username,
password = password
};
string jsonStr = JsonConvert.SerializeObject(jsonObj);
WebClient wc = new WebClient();
//x - www - form - urlencoded
wc.Headers[HttpRequestHeader.ContentType] = "application/x - www - form - urlencoded";
wc.Headers.Add("Access-Control-Allow-Headers", "content-type");
wc.Headers.Add("Access-Control-Allow-Origin", "*");
wc.Headers[HttpRequestHeader.Authorization] = "Bearer <token>";
wc.Headers.Add("Access-Control-Allow-Methods", "POST, PUT, GET, DELETE, OPTIONS");
string header = wc.Headers.ToString();
try
{
string response = wc.UploadString(endpoint, method, jsonStr);
return JsonConvert.DeserializeObject<User>(response);
}
catch(Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
I've messed around altering just about everything in this method in search of a fix.
What I've done:
/TOKEN was /values & /api/values
POST method was GET -- With this, I received a "Cannot send a content-body with this verb-type." error.
ContentType was changed to "application/json"
Access-Control-Allow-Origin had the baseURL
Checked the format of header & body:
Header:
{Content-Type: application/x - www - form - urlencoded
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Origin: *
Authorization: Bearer <token>
Access-Control-Allow-Methods: POST, PUT, GET, DELETE, OPTIONS}
Body:
{"grant_type":"password",
"username":"test#gmail.com",
"password":"password123"}
I obviously have something wrong in my request, I've just run out of ideas to try. I'm not entirely sure if UploadString() is the correct method to be using in this situation, but I couldn't find another method in the WebClient class that would be better. Any help to try and push me in the right direction would be very much appreciated.
So what I think you are trying to do is a form-urlencoded post to a "token" endpoint with a username/password grant. These are typically done like so:
using (var request = new HttpRequestMessage(HttpMethod.Post, new Uri("https://example.com/token"))
{
Content = new FormUrlEncodedContent(new Dictionary<string, string>
{
{ "grant_type", "password" },
{ "username", "username#site.com" },
{ "password", "password12345" }
})
})
{
using (var resp = await _client.SendAsync(request))
{
resp.EnsureSuccessStatusCode();
//await resp.Content.ReadAsAsync<BearerToken>();
// for testing purposes, try this:
var returnData = await resp.Content.ReadAsStringAsync();
Console.WriteLine(returnData);
}
}
You should define this outside all scopes where you need to do Http requests:
private static readonly HttpClient _client = new HttpClient();
So, first off, try to stick with HttpClient. Other patterns such as WebClient are considered legacy.
Next, CORS headers are typically returned from the server when an OPTIONS call is sent to the server. You aren't doing that here, and you should never have to worry about that kind of stuff inside a C# program running from your computer. So you can drop the access-control header stuff.
Form-urlencoded data is not JSON data. It's a different way to format data. If you want to send JSON data, you should use the content-type application/json
Finally, you are trying to add an Authorization header. But that doesn't make much sense as you are trying to authenticate yourself to become authorized. If you send the right username/password, you will receive a bearer token that you can use in an Authorization header for future requests to said service.
Oh and I forgot to add: Whenever you see an error in the [400,499] range (in this case "400 - bad request") it means that you sent something wrong and the server doesn't understand what you are trying to do. For example: a 401 means you sent invalid or missing authorization information. A 400 means your data was probably malformed.
But I like your question... I can see what you were doing and you tried all kinds of different things.
Download a program called Fiddler if you want to see how HTTP works. It's a great tool to debug your HTTP calls.
I'm building a Xamarin app. I'm still on a very very noobish level, and I'm coming from Nativescript, and something (not much) of Native Android.
I have an Express server that performs long-time operations. During that time the Xamarin client waits with a spinner.
On the server I already calculate the percentage progress of the job, and I'd like to send it to the client each time it changes, in order to swap that spinner with a progress.
Still, on the server, the task was already achieved with a
response.write('10'); where the number 10 stands for "10%" of the Job done.
Now the tuff part. How can I read that 10 from the stream? Right now it works as a JSON response, because it waits for the whole response to come.
Xamarin client HTTP GET:
// Gets weather data from the passed URL.
async Task<JsonValue> DownloadSong(string url)
{
// Create an HTTP web request using the URL:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(url));
request.ContentType = "application/json";
request.Method = "GET";
// Send the request to the server and wait for the response:
using (WebResponse response = await request.GetResponseAsync())
{
// Get a stream representation of the HTTP web response:
using (System.IO.Stream stream = response.GetResponseStream())
{
// Use this stream to build a JSON document object:
JsonValue jsonDoc = await Task.Run(() => JsonValue.Load(stream));
// Return the JSON document:
return jsonDoc;
}
}
}
The server writes on the response each time the progress of the job changes, sending a plain string containing the percentage value. At the end of the job, it will write a final string, which will be a Base64 (very long) string. And the response will be then closed.
Can anyone indicate me how to change that script in order to read each data chunk the server sends?
First you need to define some protocol. For simplicity we can say that server sends:
(optional) current progress as 3-digit string ("010" - means 10%)
(required) final progress as "100"
(required) json data
So, valid response is, for example, "010020050090100{..json here..}".
Then you can read response in 3-byte chunks, until you find "100". Then you read json. Sample code:
using (System.IO.Stream stream = response.GetResponseStream()) {
while (true) {
// 3-byte buffer
byte[] buffer = new byte[3];
int offset = 0;
// this block of code reliably reads 3 bytes from response stream
while (offset < buffer.Length) {
int read = await stream.ReadAsync(buffer, offset, buffer.Length - offset);
if (read == 0)
throw new System.IO.EndOfStreamException();
offset += read;
}
// convert to text with UTF-8 (for example) encoding
// need to use encoding in which server sends
var progressText = Encoding.UTF8.GetString(buffer);
// report progress somehow
Console.WriteLine(progressText);
if (progressText == "100") // done, json will follow
break;
}
// if JsonValue has async api (like LoadAsync) - use that instead of
// Task.Run. Otherwise, in UI application, Task.Run is fine
JsonValue jsonDoc = await Task.Run(() => JsonValue.Load(stream));
return jsonDOc;
}
I am playing around with an app using HttpWebRequest to dialog with a web server.
I followed standard instructions I found on the web to build my request function that I tried to make as generic as possible (I try to get a unique method regardless of the method: PUT, POST, DELETE, REPORT, ...)
When I submit a "REPORT" request, I get two access logs on my server:
1) I get response 401 after following line is launched in debugger
reqStream.Write(Encoding.UTF8.GetBytes(body), 0, body.Length);
2) I get response 207 (multi-get, which is what I expect) after passing the line calling Request.GetResponse();
Actually, it seems to be the Request.GetRequestStream() line that is querying the server the first time, but the request is only committed once passing the reqStream.Write(...) line...
Same for PUT and DELETE, the Request.GetRequestStream() again generates a 401 access log on my server whereas the Request.GetResponse(); returns code 204.
I don't understand why for a unique request I have two server access logs, especially one that seems to be doing nothing as it always returns code 401... Could anybody explain what is going on? Is it a flaw in my code or a bad design due to my attempt to get a generic code for multiple methods?
Here is my full code:
public static HttpWebResponse getHttpWebRequest(string url, string usrname, string pwd, string method, string contentType,
string[] headers, string body) {
// Variables.
HttpWebRequest Request;
HttpWebResponse Response;
//
string strSrcURI = url.Trim();
string strBody = body.Trim();
try {
// Create the HttpWebRequest object.
Request = (HttpWebRequest)HttpWebRequest.Create(strSrcURI);
// Add the network credentials to the request.
Request.Credentials = new NetworkCredential(usrname.Trim(), pwd);
// Specify the method.
Request.Method = method.Trim();
// request headers
foreach (string s in headers) {
Request.Headers.Add(s);
}
// Set the content type header.
Request.ContentType = contentType.Trim();
// set the body of the request...
Request.ContentLength = body.Length;
using (Stream reqStream = Request.GetRequestStream()) {
// Write the string to the destination as a text file.
reqStream.Write(Encoding.UTF8.GetBytes(body), 0, body.Length);
reqStream.Close();
}
// Send the method request and get the response from the server.
Response = (HttpWebResponse)Request.GetResponse();
// return the response to be handled by calling method...
return Response;
}
catch (Exception e) {
throw new Exception("Web API error: " + e.Message, e);
}
I am using a simple .NET service (asmx) that works fine when invoking via the test form (POST). When invoking via a HttpWebRequest object, I get a WebException "System.Net.WebException: The remote server returned an error: (500) Internal Server Error." Digging deeper, reading the WebException.Response.GetResponseStream() I get the message: "Missing parameter: serviceType." but I've clearly included this parameter.
I'm at a loss here, and its worse that I don't have access to debug the service itself.
Here is the code being used to make the request:
string postData = String.Format("serviceType={0}&SaleID={1}&Zip={2}", request.service, request.saleId, request.postalCode);
byte[] data = (new ASCIIEncoding()).GetBytes(postData);
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.Timeout = 60000;
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.ContentLength = data.Length;
using (Stream newStream = httpWebRequest.GetRequestStream())
{
newStream.Write(data, 0, data.Length);
}
try
{
using (response = (HttpWebResponse)httpWebRequest.GetResponse())
{
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception("There was an error with the shipping freight service.");
string responseData;
using (StreamReader responseStream = new StreamReader(httpWebRequest.GetResponse().GetResponseStream(), System.Text.Encoding.GetEncoding("iso-8859-1")))
{
responseData = responseStream.ReadToEnd();
responseStream.Close();
}
if (string.IsNullOrEmpty(responseData))
throw new Exception("There was an error with the shipping freight service. Request went through but response is empty.");
XmlDocument providerResponse = new XmlDocument();
providerResponse.LoadXml(responseData);
return providerResponse;
}
}
catch (WebException webExp)
{
string exMessage = webExp.Message;
if (webExp.Response != null)
{
using (StreamReader responseReader = new StreamReader(webExp.Response.GetResponseStream()))
{
exMessage = responseReader.ReadToEnd();
}
}
throw new Exception(exMessage);
}
Anyone have an idea what could be happening?
Thanks.
UPDATE
Stepping through the debugger, I see the parameters are correct. I also see the parameters are correct in fiddler.
Examining fiddler, I get 2 requests each time this code executes. The first request is a post that sends the parameters. It gets a 301 response code with a "Document Moved Object Moved This document may be found here" message. The second request is a GET to the same URL with no body. It gets a 500 server error with "Missing parameter: serviceType." message.
It seems like you found your problem when you looked at the requests in Fiddler. Taking an excerpt from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html:
10.3.2 301 Moved Permanently
The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible.
.....
Note: When automatically redirecting a POST request after
receiving a 301 status code, some existing HTTP/1.0 user agents
will erroneously change it into a GET request.
Here's a couple options that you can take:
Hard-code your program to use the new Url that you see in the 301 response in Fiddler
Adjust your code to retrieve the 301 response, parse out the new Url from the response, and build a new response with the new Url.
The latter option would be ideal if you're dealing with user-based input on the Url (like a web browser), since you don't know where the user is going to want your program to go.
I know there are a lot of questions about sending HTTP POST requests with C#, but I'm looking for a method that uses WebClient rather than HttpWebRequest. Is this possible? It'd be nice because the WebClient class is so easy to use.
I know I can set the Headers property to have certain headers set, but I don't know if it's possible to actually do a POST from WebClient.
You can use WebClient.UploadData() which uses HTTP POST, i.e.:
using (WebClient wc = new WebClient())
{
byte[] result = wc.UploadData("http://stackoverflow.com", new byte[] { });
}
The payload data that you specify will be transmitted as the POST body of your request.
Alternatively there is WebClient.UploadValues() to upload a name-value collection also via HTTP POST.
You could use Upload method with HTTP 1.0 POST
string postData = Console.ReadLine();
using (System.Net.WebClient wc = new System.Net.WebClient())
{
wc.Headers.Add("Content-Type","application/x-www-form-urlencoded");
// Upload the input string using the HTTP 1.0 POST method.
byte[] byteArray = System.Text.Encoding.ASCII.GetBytes(postData);
byte[] byteResult= wc.UploadData("http://targetwebiste","POST",byteArray);
// Decode and display the result.
Console.WriteLine("\nResult received was {0}",
Encoding.ASCII.GetString(byteResult));
}