I am wondering when I read a response stream and a timeout occurs, can I retry the read? Or do I have to make a new request? The server that I am downloading from does not support resuming, so I need to protect against timeouts.
No, you can't retry. You need to reissue the request
How do you know that the server doesn't accept resumes? Is the Accept-Ranges header indeed lacking in the response? If not, you could resume the request using the Range header.
If the server indeed doesn't support it, then best what you can try is to inform the server that you want to keep the connection alive using Connection: keep-alive header in combination with the Keep-Alive header wherein you can specify the timeout.
Hope this helps.
Related
I'm using raw TCP sockets. I can send 200/404/302 without a problem.
If I serve 413 like a normal request it works just fine
"HTTP/1.1 413\r\nContent-Type: text/html; charset=utf-8;\r\nContent-Length: 7\r\n\r\nToo big"
However in the http 1/1 rfc it says I may close the connection before processing the request https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.14
Request Entity Too Large
The server is refusing to process a request because the request entity is larger than the server is willing or able to process. The server MAY close the connection to prevent the client from continuing the request.
However when I serve the results and don't process the entire request both firefox and chrome will show a connection error. When I send the results THEN READ THE REQUEST it will work. The browsers seem to not process what I send them until I read the entire stream which defeats the point.
Or maybe I'm sending it wrong. How am I suppose to send the 413? I send the string above, flush the socket, sleep for 100 milliseconds, close the connection, sleep for another 100 milliseconds then restart.
A request and response exchange isn't considered complete until both the request data is fully received and the response fully sent. So the only way to send a valid 413 response is to read (and presumably discard) the full request body, as well as sending the 413 as soon as possible as it appears you are doing.
If you send the 413 immediately, it's up to the client to detect that early or not. I think most clients will just continue uploading the request though.
Alternatively, you may close the connection which is how you can stop the client from sending the whole message. But then, the HTTP request and response is not complete, and clients may show a connection error rather than the 413.
So: if you want to stop clients uploading data then close the connection. If you want clients to get a nice valid 413 response then consume the whole request.
I need to consume a third-party WebSocket API in .NET Core and C#; the WebSocket server is implemented using socket.io (using protocol version 0.9), and I am having a hard time understanding how socket.io works... besides that the API requires SSL.
I found out that the HTTP handshake must be initiated via a certain path, which is...
socket.io/1/?t=...
...whereby the value of the parameter t is a Unix-timestamp (in seconds). The service replies with a session-key, timeout information, and a list of supported transport protocols. Due to simplicity, this first request is made via HttpClient and does not involve any additional headers.
Next, another HTTP request is required, which should result in an HTTP 101 Switching Protocol response. I specified the following headers in accordance to the previous request...
Connection: Upgrade
Upgrade: websocket
Sec-WebSocket-Key: ...
Sec-WebSocket-Version: 13
...whereby the value of the Key-header is a Base64-encoded GUID-value that the server will use to calculate the Sec-WebSocket-Accept header value. I also precalculate the expected Sec-WebSocket-Accept header value, for validation...
I tried to make that request using HttpClient as well, but that does not seem to work... I actually don´t understand why, because I expect an HTTP response. I also tried to make the request using TcpClient by sending a manually prepared GET request over a SslStream, which accepts the remote certificate as expected. Sending data seems to work, but there´s no response data... the Read-method returns zero.
What do I miss here? Do I need to setup a listener for the WebSocket connection as well, and if yes how? I don´t want to implement a feature complete socket.io client, I´d just like to keep it as simple as possible to catch some events...
The best way of debugging these issues is to use a sniffer like wireshark or fiddler. Often connect using an IE and compare IE results with my application and modify my app so it works like the IE. Using WebClient instead of HttpClient will also work better because the WebClient does more automatically than the HttpClient.
A web connection uses the header of the client and the headers in the server webpage to negotiate a connection mode. Adding additional headers to you client will change the connection mode. Cookies are also used to select the connection mode. Cookies are the results of previous connection to the same server which shortens the negotiations and stores info from previous connection so less data has to be downloaded from server. The server remembers the cookies. Cookies have a timeout and is kept until timeout expires. The IE history in your client has a list of IP addresses and Net automatically sends the cookies associated with the server IP.
If a bad connection is made to the server the cookies is also bad so the only was of connection is to remove the cookie. Usually I go into the IE and delete cookies manually in the IE history.
To check if a response is good the server returns a status. A completed response contains a status 200 DONE. You can get status which are errors. You can also get a 100 Continue which means you need to send another request to get the rest of the webpage.
Http has 1.0 (stream mode) and 1.1 (chunk mode). Net library doesn't work with chunk. Chunk requires client to send message to get next chunk and I have not found a way in Net to send the next chunk message. So if a server responds with a 1.1 then you have to add to your client headers to use 1.0 only.
Http uses TCP as the transport layer. So in a sniffer you will see TCP and HTTP. Usually you can filter sniffer just to return Http and look at header for debugging. Occasionally TCP disconnects and then you have to look at TCP to find why the disconnect occurs.
I understand that the keep-alive message aims to prevent the underlying TCP connection from closing so that multiple requests can be sent over the same TCP connection.
However it is unclear how this affects a request that has timed out:
If multiple requests are handled over the same TCP connection, when the client times-out, how will ASP.NET know? I'm not sure how to ask this question correctly, but I suspect that HttpWorkerRequest.IsClientConnected is false if the underlying TCP connection is closed? How would I determine if the client has timed out? Surely the response can no longer be sent?
Since the client can send keep-alive messages, what effect, if any does it have on the request? I expect that it will have no effect and mechanisms such as a timeout will work as usual.
Thank you for your time and assistance.
HTTP keep-alive (which should not be confused with TCP keep alive) asks the server to not close the connection after the HTTP response is fully sent. Thus HTTP keep-alive is only relevant for the time after the HTTP response is sent and before the next HTTP request will be received.
... when the client times-out, how will ASP.NET know?
The server is trying to read data. If the client closes the connection the server will get notified (like by reading 0 bytes) and will close the connection too.
... Surely the response can no longer be sent?
To repeat myself, HTTP keep-alive is only relevant after the full response was sent. So this question is unrelated to HTTP keep-alive.
Since the client can send keep-alive messages, what effect, if any does it have on the request?
No effect on the current request. But the client must be aware that the server might close the connection anytime after the HTTP response was sent and before the server gets the new request. Thus it might happen that the server closes the connection exactly in the moment when the client is trying to send a new request. In this case idempotent requests (GET) must be retried by the client within a new connection, while other requests should fail.
I'm using HttpClient to send an asynchronous POST request to a remote web server. That remote web server responds with the Connection header set to keep-alive. Eventually, it will close the connection.
What I can't figure out how to do is keep receiving data until the connection is set to close.
For example, consider the following code:
HttpClient client = new HttpClient();
string requestUri = ...; //My GET uri
string content = ...; //url-encoded contents of the request
HttpResponseMessage response = await client.PostAsync(requestUri, new StringContent(content));
In the code above, response will have the Connection header set to keep-alive. I couldn't find any members on HttpResponseMessage that seem to give me any means of continuing to receive information after the first keep-alive is received.
I can't figure out how to continue receiving responses until the Connection closes.
I was able to find all sorts of resources that dealt with sending keep-alive requests. That's not my issue. My issue is in receiving/handling a keep-alive response from the server.
Please let me know if there is anymore information that I could provide to improve this question.
Connection: keep-alive purpose is not to enable server to send multiple responses for the same single request - this would fundamentally violate how HTTP works.
Connection: keep-alive is server's way of signaling to the client that client can submit next request using the same, already established, TCP connection, thus help avoid the costs of establishing new connection (and possibly TLS handshake as well).
Reuse of the connection is (typically) handled by underlying library, and is abstracted away such that user (as in developer) of the library need not worry about handling.
All you should be doing is keep making as many client.Post/Get/etc requests, and the HttpClient library will take care of managing the underlying TCP connection and its reuse.
There is one area in HTTP that may look counter to the above and thus confusing, but is not. The case is when server returns HTTP Status 100 Continue. In this case you will see another response come from the server for still the same original request (and by "see" I mean not from HttpClient, but rather on the wire if you were to snoop with a tool like WireShark or a proxy). What is happening in this case is if a client is making a large request, the server simply signals to the client that it is continuing to accept and read request, and for the client to continue sending that original request. Once full request is received, the server will process and respond with a final response code and message. And again, HttpClient will abstract away this interim 100 Continue response, so as developer you need not worry about it (in most if not all cases).
If all you want is receive the response for a given request you don't need to deal with keep-alive at all. Keep-alive is purely a performance optimization. The framework transparently manages connections.
All you do is send a request and receive a response. There is exactly one response per request.
One side will eventually close the connection which is not of any concern to you. You don't notice any effect of that.
Can anyone guide me in acquiring the POST content-length of a website by just using sockets. Thanks and Kudos!
(I'm avoiding using httpwebrequest for some reason)
If it's a proxy application you don't need to be parsing headers at all. You just need to mirror the data from one side to the other, as bytes. The only thing you need to parse is for example the initial HTTP CONNECTION request, or whatever your initial handshake with the client is that causes you to set up the upstream connection. The rest of it is just byte copying and EOS and error propagation.
In the Http protocol the header is seperated from the content by a double crlf.
So you could either parse the header and get the Content-Length header or you can figure out the length of the content (since you know where the header ends and content starts).
HTTP/1.1 message length rules are described in section 4.4 of the RFC 2616.