Greetings!
I've been fooling around with C# (again) and now i've stucked with simple HTTP web server implementation. Honestly, i don't want to get along with HTTP specification - i just need to write a very tiny (read as simple) HTTP web server. And i've encouraged this problem: client sends request to the server and then server parses it, runs some actions, builds response and sends it back to client. That seems to be obvious (for me at least).
Here's what i've got so far:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
TcpListener listener = new TcpListener(IPAddress.Any, 80);
listener.Start();
Socket sock = listener.AcceptSocket();
try
{
Stream s = new NetworkStream(sock);
s.ReadTimeout = 300;
StreamReader reader = new StreamReader(s);
StreamWriter writer = new StreamWriter(s);
writer.AutoFlush = true;
Console.WriteLine("Client stream read:\r\n");
string str = "none";
while (sock.Connected && !reader.EndOfStream && str.Length > 0) // here's where i'm stuck
{
str = reader.ReadLine();
Console.WriteLine("{0} ({1})", str, str.Length);
}
Console.WriteLine("Sending response...\r\n");
{
string response = "<h1>404: Page Not Found</h1>";
writer.WriteLine("HTTP / 1.1 404 Not Found");
writer.WriteLine("Content-Type: text/html; charset=utf-8");
writer.WriteLine("Content-Length: {0}", response.Length);
writer.WriteLine("\r\n{0}", response);
}
Console.WriteLine("Client: over\r\n");
s.Close();
sock.Close();
}
catch (Exception e)
{
Console.WriteLine("Exception: {0}\r\nTrace: \r\n{1}", e.Message, e.StackTrace);
}
Console.ReadKey();
}
}
}
But i've met an "underwater stone": i'm reading request via input stream, so input data flow will be terminated when client close page in his browser (let's talk about the most obvious actions, excluding curl, w3 and other "geek-stuff").
So, the question is: how to determine request' end? E.g. when should i stop reading request data and start sending response?
Why not use HttpListener? You can have a simple HTTP server in 5 lines of code.
This Wikipedia article explains pretty succinctly about the request message format.
The request line and headers must all end with <CR><LF> (that is, a carriage return followed by a line feed). The empty line must consist of only <CR><LF> and no other whitespace. In the HTTP/1.1 protocol, all headers except Host are optional.
Basically, watch for a blank line after the headers and/or a potential message body.
According to the HTTP specification, you can use certain headers to determine if there is a message body:
The presence of a message-body in a request is signaled by the inclusion of a Content-Length or Transfer-Encoding header field in the request's message-headers.
Another option is this one:
http://webserver.codeplex.com/
even if you don't want to use it, you can stole ideas because it implements the complete request lifecycle.
Related
using System.Net;
using System.Text;
namespace HttpServer
{
class Program
{
// Main method
static void Main()
{
using var listener = new HttpListener();
listener.Prefixes.Add("http://localhost:10060/");
listener.Start();
Console.WriteLine("Listening on port 10060...");
// Request handler
while (true)
{
HttpListenerContext context = listener.GetContext();
HttpListenerRequest req = context.Request;
Console.WriteLine($"Received request for {req.Url}");
// TODO: Login stuff
Uri? url = req.Url;
if (url.ToString() == "http://localhost:10060/login")
{
using HttpListenerResponse resp = context.Response;
resp.Headers.Set("Content-Type", "text/plain");
string data = "Hello there!";
byte[] buffer = Encoding.UTF8.GetBytes(data);
resp.ContentLength64 = buffer.Length;
using Stream ros = resp.OutputStream;
ros.Write(buffer, 0, buffer.Length);
}
}
}
}
}
The above code is a simple http server that listens for requests. The above code works when going to localhost:10060 in the browser. However, I want to access this server via the machine's IP address on the same network through a different device. When doing so, it results in a bad request due to the invalid hostname. Is there a way to fix this?
try using the ip address instead of localhost. or try *:10060
In another machine try to modify firewall rule to allow port 80.
Windows defender firewall with advance security on local Computer.
Inbound rules
New Rule
Select the port, Next, TCP, specific local port, type 80
Allow the connection and save the rule.
Let us know if this worked.
I have Netduino Plus and I need it to send Http requests to my server. I'm not a guru in C#, I've never tried it before, so I copy/paste code from internet and try to make it works. But even after several hours I can't get it work.
using System;
using System.IO;
using System.Net;
using System.Text;
using SecretLabs.NETMF.Hardware;
using SecretLabs.NETMF.Hardware.NetduinoPlus;
namespace NetduinoPlusApplication5
{
public class Program
{
static void Main()
{
var request = WebRequest.Create("http://example.com?variable=1");
request.Method = "GET";
var result = request.GetResponse();
}
}
}
What am I doing wrong?
You are executing a GET request so I think you want to get the response body from the server. In this case you have to use :
Stream respStream = resp.GetResponseStream();
instead of simple GetResponse(). In this way, you can read on the stream the response body.
Paolo.
I'm using HttpWebRequest to connect to my in-house built HTTP server. My problem is that it is a lot slower than connecting to the server via for instance PostMan (https://chrome.google.com/webstore/detail/postman-rest-client/fdmmgilgnpjigdojojpjoooidkmcomcm?hl=en), which is probably using the built-in functions in Chrome to request data.
The server is built using this example on MSDN (http://msdn.microsoft.com/en-us/library/dxkwh6zw.aspx) and uses a buffer size of 64. The request is a HTTP request with some data in the body.
When connecting via PostMan, the request is split into a bunch of chunks and BeginRecieve() is called multiple times, each time receiving 64B and taking about 2 milliseconds. Except the last one, which receives less than 64B.
But when connecting with my client using HttpWebRequest, the first BeginRecieve() callback receives 64B and takes about 1 ms, the following receives only 47B and takes almost 200 ms, and finally the third receives about 58B and takes 2ms.
What is up with the second BeginRecieve? I note that the connection is established as soon as I start to write data to the HttpWebRequest input stream, but the data reception does not start until I call GetResponse().
Here is my HttpWebRequest code:
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = verb;
request.Timeout = timeout;
request.Proxy = null;
request.KeepAlive = false;
request.Headers.Add("Content-Encoding", "UTF-8");
System.Net.ServicePointManager.Expect100Continue = false;
request.ServicePoint.Expect100Continue = false;
if ((verb == "POST" || verb == "PUT") && !String.IsNullOrEmpty(data))
{
var dataBytes = Encoding.UTF8.GetBytes(data);
try
{
var dataStream = request.GetRequestStream();
dataStream.Write(dataBytes, 0, dataBytes.Length);
dataStream.Close();
}
catch (Exception ex)
{
throw;
}
}
WebResponse response = null;
try
{
response = request.GetResponse();
}
catch (Exception ex)
{
throw;
}
var responseReader = new StreamReader(rStream, Encoding.UTF8);
var responseStr = responseReader.ReadToEnd();
responseReader.Close();
response.Close();
What am I doing wrong? Why is it behaving so much differently than a HTTP request from a web browser? This is effectively adding 200ms of lag to my application.
This looks like a typical case of the Nagle algorithm clashing with TCP Delayed Acknowledgement. In your case you are sending a small Http Request (~170 bytes according to your numbers). This is likely less than the MSS (Maximum Segment Size) meaning that the Nagle Algorithm will kick in. The server is probably delaying the ACK resulting in a delay of up to 500 ms. See links for details.
You can disable Nagle via ServicePointManager.UseNagleAlgorithm = false (before issuing the first request), see MSDN.
Also see Nagle’s Algorithm is Not Friendly towards Small Requests for a detailed discussion including a Wireshark analysis.
Note: In your answer you are running into the same situation when you do write-write-read. When you switch to write-read you overcome this problem. However I do not believe you can instruct the HttpWebRequest (or HttpClient for that matter) to send small requests as a single TCP write operation. That would probably be a good optimization in some cases. Althought it may lead to some additional array copying, affecting performance negatively.
200ms is the typical latency of the Nagle algorithm. This gives rise to the suspicion that the server or the client is using Nagling. You say you are using a sample from MSDN as the server... Well there you go. Use a proper server or disable Nagling.
Assuming that the built-in HttpWebRequest class has an unnecessary 200ms latency is very unlikely. Look elsewhere. Look at your code to find the problem.
It seems like HttpWebRequest is just really slow.
Funny thing: I implemented my own HTTP client using Sockets, and I found a clue to why HttpWebRequest is so slow. If I encoded my ASCII headers into its own byte array and sent them on the stream, followed by the byte array encoded from my data, my Sockets-based HTTP client behaved exactly like HttpWebRequest: first it fills one buffer with data (part of the header), then it uses another buffer partially (the rest of the header), waits 200 ms and then sends the rest of the data.
The code:
TcpClient client = new TcpClient(server, port);
NetworkStream stream = client.GetStream();
// Send this out
stream.Write(headerData, 0, headerData.Length);
stream.Write(bodyData, 0, bodyData.Length);
stream.Flush();
The solution was of course to append the two byte arrays before sending them out on the stream. My application is now behaving as espected.
The code with a single stream write:
TcpClient client = new TcpClient(server, port);
NetworkStream stream = client.GetStream();
var totalData = new byte[headerBytes.Length + bodyData.Length];
Array.Copy(headerBytes,totalData,headerBytes.Length);
Array.Copy(bodyData,0,totalData,headerBytes.Length,bodyData.Length);
// Send this out
stream.Write(totalData, 0, totalData.Length);
stream.Flush();
And HttpWebRequest seems to send the header before I write to the request stream, so it might be implemented somewhat like my first code sample. Does this make sense at all?
Hope this is helpful for anyone with the same problem!
Try this: you need to dispose of your IDisposables:
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = verb;
request.Timeout = timeout;
request.Proxy = null;
request.KeepAlive = false;
request.Headers.Add("Content-Encoding", "UTF-8");
System.Net.ServicePointManager.Expect100Continue = false;
request.ServicePoint.Expect100Continue = false;
if ((verb == "POST" || verb == "PUT") && !String.IsNullOrEmpty(data))
{
var dataBytes = Encoding.UTF8.GetBytes(data);
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(dataBytes, 0, dataBytes.Length);
}
}
string responseStr;
using (var response = request.GetResponse())
{
using (var responseReader = new StreamReader(rStream, Encoding.UTF8))
{
responseStr = responseReader.ReadToEnd();
}
}
I am using a simple .NET service (asmx) that works fine when invoking via the test form (POST). When invoking via a HttpWebRequest object, I get a WebException "System.Net.WebException: The remote server returned an error: (500) Internal Server Error." Digging deeper, reading the WebException.Response.GetResponseStream() I get the message: "Missing parameter: serviceType." but I've clearly included this parameter.
I'm at a loss here, and its worse that I don't have access to debug the service itself.
Here is the code being used to make the request:
string postData = String.Format("serviceType={0}&SaleID={1}&Zip={2}", request.service, request.saleId, request.postalCode);
byte[] data = (new ASCIIEncoding()).GetBytes(postData);
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.Timeout = 60000;
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.ContentLength = data.Length;
using (Stream newStream = httpWebRequest.GetRequestStream())
{
newStream.Write(data, 0, data.Length);
}
try
{
using (response = (HttpWebResponse)httpWebRequest.GetResponse())
{
if (response.StatusCode != HttpStatusCode.OK)
throw new Exception("There was an error with the shipping freight service.");
string responseData;
using (StreamReader responseStream = new StreamReader(httpWebRequest.GetResponse().GetResponseStream(), System.Text.Encoding.GetEncoding("iso-8859-1")))
{
responseData = responseStream.ReadToEnd();
responseStream.Close();
}
if (string.IsNullOrEmpty(responseData))
throw new Exception("There was an error with the shipping freight service. Request went through but response is empty.");
XmlDocument providerResponse = new XmlDocument();
providerResponse.LoadXml(responseData);
return providerResponse;
}
}
catch (WebException webExp)
{
string exMessage = webExp.Message;
if (webExp.Response != null)
{
using (StreamReader responseReader = new StreamReader(webExp.Response.GetResponseStream()))
{
exMessage = responseReader.ReadToEnd();
}
}
throw new Exception(exMessage);
}
Anyone have an idea what could be happening?
Thanks.
UPDATE
Stepping through the debugger, I see the parameters are correct. I also see the parameters are correct in fiddler.
Examining fiddler, I get 2 requests each time this code executes. The first request is a post that sends the parameters. It gets a 301 response code with a "Document Moved Object Moved This document may be found here" message. The second request is a GET to the same URL with no body. It gets a 500 server error with "Missing parameter: serviceType." message.
It seems like you found your problem when you looked at the requests in Fiddler. Taking an excerpt from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html:
10.3.2 301 Moved Permanently
The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible.
.....
Note: When automatically redirecting a POST request after
receiving a 301 status code, some existing HTTP/1.0 user agents
will erroneously change it into a GET request.
Here's a couple options that you can take:
Hard-code your program to use the new Url that you see in the 301 response in Fiddler
Adjust your code to retrieve the 301 response, parse out the new Url from the response, and build a new response with the new Url.
The latter option would be ideal if you're dealing with user-based input on the Url (like a web browser), since you don't know where the user is going to want your program to go.
I'm trying to test the behavior of an "expect: 100-continue" request being handled in an IHttpModule.
What I want to do is create a request in a client with the header expect: 100-continue and send it to the server. The server will immediately return a 500. Theoretically, the payload (file) should never be sent if the server returns a 500 instead of a 100.
I'm not seeing the expected behavior. This is what I'm doing...
Here is the server code (http module):
using System;
using System.Web;
namespace WebSite
{
public class Expect100ContinueModule : IHttpModule
{
private HttpApplication httpApplication;
public void Init(HttpApplication context)
{
httpApplication = context;
context.BeginRequest += ContextBeginRequest;
}
private void ContextBeginRequest(object sender, EventArgs e)
{
var request = httpApplication.Context.Request;
var response = httpApplication.Context.Response;
if(!request.Url.AbsolutePath.StartsWith("/Upload"))
{
return;
}
response.StatusCode = 500;
response.End();
}
public void Dispose()
{
}
}
}
I'm running this in IIS 7 with the integrated pipeline.
Here is the client code:
using System.IO;
using System.Net;
namespace ConsoleClient
{
class Program
{
static void Main(string[] args)
{
var request = (HttpWebRequest)WebRequest.Create("http://localhost:83/Upload/");
request.ServicePoint.Expect100Continue = true;
request.Method = "POST";
request.ContentType = "application/octet-stream";
var buffer = File.ReadAllBytes("Test - Copy.txt");
var text = File.ReadAllText("Test - Copy.txt");
using (var requestStream = request.GetRequestStream())
{
requestStream.Write(buffer, 0, buffer.Length);
requestStream.Flush();
}
var response = (HttpWebResponse)request.GetResponse();
var x = "";
}
}
}
From what I'm seeing, the request has the file content even when the 500 is returned.
One issue I ran into is, Fiddler automatically handles expect: 100-continue by buffering the request, returning 100, and continuing with the full request.
I then tried WireShark. To get that to work, I had to capture the traffic with RawCap and read the output with WireShark. From what I can tell, this still shows the full payload on the request.
Now I have a few questions.
Is the server code actually returning the 500 first, or has a 100 already been returned before I get the BegineRequest event?
Is there a better way to write the client? I don't understand why you would write your full payload to the request stream before the request is made. The server module doesn't get the BeginRequest event until the clients Request.GetResponse() method is called.
Is there a good way to test the actual traffic on localhost?
[Update]
I couldn't find a way to test locally so I used a neighbor employees desktop to test with also. I was able to test with WireShark this way.
From what I can tell, there is no way to have the HttpWebClient do a PUT request to IIS without having the full file sent, even if an IHttpModule returns an error immediately before the file is completely uploaded.
I don't know if the issue is in the client (HttpWebClient) or the server (IIS). I don't know if using raw sockets and implementing the HTTP protocol by hand would make a difference.
If anyone has any more insight into this, please let me know.