We Keep Coding

WebSocket connection issues using c#

The response from the server is not a valid HTTP response. This problem occurs when the .NET Framework detects that the server response does not comply with HTTP 1.1 RFC. This problem may occur when the response contains incorrect headers or incorrect header delimiters.RFC 2616 defines HTTP 1.1 and the valid format for the response from the server. For more information, see RFC 2616 - Hypertext Transfer Protocol -- HTTP/1.1 at Internet Engineering Task Force (IETF) website.
Get a network trace of the transaction and examine the headers in the response.
If your application requires the server response without parsing (this could be a security issue), set useUnsafeHeaderParsing to true in the configuration file. See Element

It seems like you just forgentting to clear the buffer before you receive new message.
2 quick options for you:
-add '\0' in the end of each of your messages on your server side. This will prevent gliches like that from happening.
-use Array.Clear(buffer,0,buffer.length) to clear buffer before receiving on your client side.

setting Transfer-Encoding header not working with no error message

I am sending a page to the client in chunks, by calling Response.Flush() in the middle of my page. This is so the browser will get the first part of the html, and can start downloading resources while my server continues to process the rest of the request.
Because of certain 3rd party services between my IIS server and my client (CDN, Firewall, Load Balancing, etc.) I need to set the header Transfer-Encoding: Chunked so they will know that the response will return in chunks.
I try setting the header by calling : Response.Headers.Add("Transfer-Encoding", "chunked");
For some reason when i do this, I get a blank page back after waiting quite a long time, even when contacting my IIS server directly, without going through all the 3rd parties. When attaching to process to debug, I don't see any errors.
Removing the 'Transfer-Encoding' header works, but I need this header for some of the 3rd parties I'm using.
Anyone know how I can set this header in my web application ??
Btw - I also tried setting this header in 'Response Headers' section in IIS directly, and the response is still empty when doing this.

According to the description on Wikipedia, Chunked Transfer Encoding requires the response body to be encoded in a specific way, one of main points of the described format being :
Each chunk starts with the number of octets of the data it embeds
expressed as a hexadecimal number in ASCII followed by optional
parameters (chunk extension) and a terminating CRLF sequence, followed
by the chunk data. The chunk is terminated by CRLF. If chunk
extensions are provided, the chunk size is terminated by a semicolon
followed with the extension name and an optional equal sign and value.
As far as I know, calling Response.Flush() does not generate this specific markup. It just empties any buffered response content to the client.
You may have a look at "When does the server use chunked transfer encoding?" in this answer :
https://stackoverflow.com/a/2711405/1236044
It seems to imply that, with the correct settings, IIS should automatically switch to Chunked Transfer Encoding when needed

The server will be using chunked transfer encoding if you disable buffering:
Set context.Response.BufferOutput to false
According to this question
You might further need to set Server.ScriptTimeout (in seconds) to avoid your script being interrupted.

Packet fragmentation in the WCF stack using BasicHttpBinding, http header and soap envelope always split by a packet boundary?

I have a problem whereby I have lots of very small web service calls to a Java endpoint (hosted on Oracle GlassFish 3.1.X). I added this service as a Service Reference (using the remote wsdl file) and use a BasicHttpBinding to reach it.
Since the service is located half the world away plus going across the internet, we frequently experience some packet loss when reaching the destination. We are looking at any way possible to reduce the impact of these occurrences. We have been using Wireshark to give us detailed knowledge of what is going across the wire to our destination, and back again. I was curious to see that for every single request we generate, we are sending 2 packets. The packet boundary is always between the HTTP header and the <s:Envelope> tag. To me this is a big overhead, particularly in my environment where I want to minimise the amount of packets sent (to reduce overall packetloss).
In most cases (99% of our calls), the HTTP header packet is 210 bytes followed by a SOAP envelope packet of 291 bytes (excluding the 54 bytes of TCP/IP overhead for each packet). Totalling these gives 501 bytes - just over a third of our Max Segment Size of 1460 bytes. Why isn't WCF sending this HTTP POST request as a single packet of 501 bytes (555 bytes if you include the 54 bytes of TCP/IP overhead)?
Does anyone know why it does this? It almost seems as if the HttpWebRequest object is calling .Flush() on the stream after writing it's headers but I'm not sure why it would do this?
I've tried different combinations on these:
ServicePointManager.UseNagleAlgorithm = false;
ServicePointManager.Expect100Continue = false;
With no effect.
EDIT
Wrong: I've investigated a bit further and when a HttpWebRequest.GetRequestStream() is called, it does write the headers to the stream immediately. At some stage before you write to the Stream that is given back to you, the network would Flush these (I guess? Unless a deliberate flush is happening somewhere). When you finally start writing to the stream, it has already sent the header packet. Not sure how to prevent this, it seems a very hard assertion inside the HttpWebRequest code that called GetRequestStream() will write the headers. For my small requests, I want nothing to be written until I have closed the stream but that goes against the streaming nature of it.

And the answer is - Can't be done with WebHttpRequest (and hence BasicHttpBinding)
http://us.generation-nt.com/answer/too-packets-httpwebrequest-help-23298102.html

Extract HTTP header information using Packet.Net

I would like to extract the HTTP header information using Packet.Net. I am using SharpPcap to capture the packet and need to access the User-Agent field in the TCP packet. If I understand correctly Packet.Net is used to analyze the packet captured. Help would be appreciated on this regard. I have tried to display the TCP packet with the following code but I get bytes displayed. I am using C# as development language.
private static void device_OnPacketArrival(object sender,CaptureEventArgs packet){
Packet p =Packet.ParsePacket(packet.Device.LinkType,packet.Packet.Data);
System.Text.ASCIIEncoding ASCII = new System.Text.ASCIIEncoding();
String StringMessage = ASCII.GetString(p.Bytes);
Console.WriteLine(StringMessage);
}

Packet.Net doesn't currently have http decoding support. Because http messages can be split across multiple packets, it seems like a good approach would be to first add support to allow the following of tcp connections, then add http session detection and parsing on top of the tcp data stream. Trying to parse http data on a per-packet basis might work for the headers of the data or some http messages but isn't a robust solution as it would prevent being able to get the full content of the http message that might be several kilobytes in size.
(I have a commercial library that builds upon SharpPcap/Packet.Net that adds tcp session following and http session following and decode. Post your email here if you want me to email you with more details.)

Operate on HTTP Request as it is received in IIS in C#

I have a need to override the receipt of a raw HTTP request as it is being communicated to an IIS server. I want to know if this is possible.
We have a client who sends huge Web Service calls (tens of Mb) and we want to start acting on portions of those calls as they are being received (in order to get a faster total time of execution for the Web Service call).
Currently, using normal web service methods, our application code is handed the Web Service call after it is totally received.
I realize this isn't the ideal way of handling Web Services, and we're not building our business on this, but we do have an need that we're trying to fill for a limited range of customers.
I have created a handler that implements IHttpHandler, but it appears that at this point in the process pipeline, the Request has been fully received by IIS (which doesn't get us any benefit over our current model). That is, I can read the InputStream directly, but the full request has already been transferred over the wire before I have access to this stream.
I think the answer is that I have to code an ISAPI filter to get this far down, but I don't have the skills to do this in C/C++. Does anyone know if there's another way I can do this without the ISAPI filter route?
An acceptable answer could be, "You have to do this as and ISAPI filter, to do it in C#, check this doc".

You can use a custom HttpModule to hook almost any part of the IIS pipeline. They work in both IIS 6 (under ASP.NET) and are the primary extension mechanism in IIS 7.

There are plenty of examples of building ISAPI filters, but none in C#. I am sure it is possible, but not practical and not without lots dirty tricks.
Your C# investment will hold up well in C++, let me know if you need help. By the way, I recommend you invest in the my standard trio - try to keep up a healthy knowledge of C#, C++ and Java.
I also recommend you consider Apache modules, they may offer more overall flexibility. This is what I would do:
host these web services off of IIS - you never know when IIS will bite you by resetting the application.
Use WCF services, host these from Windows services, use redirection to route the service to the WCF service.
Consider writting a raw sockets application. This one would implement the minimal WS:* protocol to your service and act as a proxy for the real service. When the proxy detects that the inbound message is exceeding a threshold it would begin analyzing the message to extract out what it could process right away.
The result would be standard WCF (through proxy) for smaller messages and non-standard processing for everything else.
Let me know if I can help you build it - this is the kind of thing I like to do...
Oh - and I recall now that WCF is completely configurable. You will be able to provide your own handlers for a variaty of layers and resolve everything from within managed code after all.

Short Version
The solution is to use HttpRequest.GetBufferlessInputStream.
Long Version
The issue is that if you attempt to use:
Request.InputStream
or Request.Form
or Request.Files
you must wait until the whole request has been received before it returns a Stream object. In contrast, the GetBufferlessInputStream method returns the Stream object immediately. You can use the method to begin processing the entity body before the complete contents of the body have been received.
This method can be useful if the request is uploading a large file and you want to begin accessing the file contents before the upload is finished. However, you should only use this method for scenarios where you want to take over all processing of the entity body. This means that you cannot use this method from an .aspx page, because by the time an .aspx page runs, the entity body has already been read.
The only downside, and it is a huge downside, is that you are now reading the Request.InputStream. This means you have to handle the MIME multiparts, and the base64 encoding, yourself.
Example (untested) code
UploadFile.ashx
public class Default : IHttpHandler
{
public void ProcessRequest (HttpContext context)
{
var request = context.Request;
var stm = request.GetBufferlessInputStream(true); //true --> disable web.config limits on request size
if (!stm.CanRead)
throw new Exception("Request input stream is not readable");
//Setup the buffer we'll be shuffling stream data into
int bufferLength = 16 * 8040; //use a multiple of 8040 bytes, because SQL Server uses pages of 8040 bytes. And because i'm saving it into SQL Server.
byte[] buffer = new Byte[bufferLength];
int bytesRead;
bytesRead = stm.Read(buffer, 0, buffer.Length);
while (bytesRead > 0)
{
SavePiece(buffer, bytesRead); //whatever you want to do with it
bytesRead = stm.Read(buffer, 0, buffer.Length);
}
}
private void SavePiece(byte[] buffer, int bufferLength)
{
//It's all going to be multipart mime encoded nonsense.
//Good luck!
}
public bool IsReusable { get { return false;}
}
Bonus Reading
How can I decode a multipart HTTP response?
Are there any multipart/form-data parser in C# - (NO ASP)
https://stackoverflow.com/a/21689347/12597 (example usage of the StreamContent class in .NET)
MSDN: StreamContent Class