We Keep Coding

NetworkStream.read taking 2 seconds to read data from stream (TCP C#)

I have a TCP request response model in C# where I am communicating with a server. Once the server has written data to the stream, I am reading that data. But stream.read is taking 2 seconds to read the data. I need to send an explicit acknowledgement to the server, within 2 seconds but am unable to do so because of the time taken to read the data.
Below is my code to read data:
byte[] resp = new byte[100000];
var memoryStream = new MemoryStream();
int bytes;
String timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("Before reading data: ");
Console.WriteLine(timeStamp);
do
{
bytes = stream.Read(resp, 0, resp.Length);
memoryStream.Write(resp, 0, bytes);
}
while (bytes > 0);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After reading data: ");
Console.WriteLine(timeStamp);
GenerateAcknowledgemnt(stream);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After sending ack: ");
Console.WriteLine(timeStamp);
Below are the timestamps read, in the format yyyyMMddHHmmssff:
Before reading data:
2022050615490817
After reading data:
2022050615491019
After sending ack:
2022050615491020
I have highlighted the seconds bold.
How do I reduce the time that stream.read is taking to read? I have tried to wrap the network stream in a BufferedStream as well, but it didn't help.

At the moment, you are performing a read loop that keeps going until Read returns a non-positive number; in TCP, this means you are waiting until the other end hangs up (or at least hangs up their outbound socket) until you get out of that loop. I suspect what is happening is that the other end is giving up on you, closing their connection, and only then do you get out of the loop.
Basically: you can't loop like that; instead, what you need to do is to carefully read until either EOF (bytes <= 0) or until you have at least one complete frame that you can respond to, and in the latter case: respond then. This usually means a loop more like (pseudo-code):
while (TryReadSomeMoreData()) // performs a read into the buffer, positive result
{
// note: may have more than one frame per successful 'read'
while (TryParseOneFrame(out frame))
{
ProcessFrame(frame); // includes sending responses
// (and discard anything that you've now processed from the back-buffer)
}
}
(parsing a frame here means: following whatever rules apply about isolating a single message from the stream - this may mean looking for a sentinel value such as CR/LF/NUL, or may mean checking if you have enough bytes to read a header that includes a length, and then checking that you have however-many bytes the header indicates as the payload)
This is a little awkward if you're using MemoryStream as the backlog, as the discard step is not convenient; the "pipelines" API is more specifically designed for this, but: either way can work.
Secondly: you may prefer async IO, although sync IO is probably fine for a simple client application with only one connection (but not for servers, which may have many many connections).

C#, clear HttpListenerContext.Response.OutputStream

I have a web-service using HttpListener.
I have noticed this thing:
HttpListenerContext context = listener.GetContext();
...
context.Response.StatusCode = 200;
context.Response.OutputStream.Write(buffer, 0, bufferSize);
context.Response.StatusCode = 500;
context.Response.OutputStream.Close();
A client in this case receives a status code 200, so if i have wrote some data to the output network stream i can't change the status code, as, i suppose, it is already written to the response stream.
What i want: after i have started writing a response to the output stream, in some case i want to "abort and reset" the response, clear the output stream (so the client won't receive any data in HTTP response body), and change the status code.
I have no idea how to clear the output stream and change the status code. These two lines below won't help, they throw exceptions.
context.Response.OutputStream.SetLength(0);
context.Response.OutputStream.Position = 0;
I suppose, what the program writes buffer data into network device after i call context.Response.OutputStream.Close(), until this the data is stored in RAM and we can reset it, can't we?
EDIT: It seems what writing into the context.Response.OutputStream takes too much of time sometimes, in some case. From 100 to 1000 ms... That's why i would just interrupt writing, if it's possible.

You either could use a MemoryStream to cache the answer, and if you are sure it is complete, set the status to 200 and return it (e.g. with Stream.CopyTo).
You can't "clear" the OutputStream, since it isn't stored (for long), instead it is sent right away to the client, so you can't edit it anymore.
Apart from that, HTTP does not offer a way to gracefully say "DATADATADATA... oh forget that, this was wrong, use the Status Code 500 instead.". You only can try to kill the TCP connection (TCP RST instead of TCP FIN) and hope that the client will handle failing to continue reading on the connection in a suitable way, after it probably already started to process the data you've already sent.
Try context.Response.Abort() before closing, this won't allow you to set a status code, but will at least communicate that something went wrong.

Send / receive file over TCP [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
TcpClient send data and receive data over network
Loop until TcpClient response fully read
I am trying to send a file from a server to a client over TCP.
Server-side code, sending file:
NetworkStream netStream = client.GetStream();
FileStream fs = new FileStream("usb.exe",FileMode.Open, FileAccess.Read);
byte[] data = new byte[fs.Length];
fs.Read(data,0, data.Length);
fs.Flush();
fs.Close();
netStream.Write(data, 0, data.Length);
netStream.Flush();
Client-side code, receive file:
FileStream str = new FileStream("usb.exe", FileMode.Create, FileAccess.Write);
byte[] data = new byte[1024];
while ((dataCitit = netStream.Read(data,0, data.Length)) > 0)
{
Thread.Sleep(25);
Application.DoEvents();
str.Write(data, 0, dataCitit);
totalbytes += dataCitit;
}
str.Close();
Can someone point where I am getting it wrong ?
The file has 1036 kb, and it sends only 1032 kb and then gets stuck it won't get out the while loop on the client-side.
Also if I close the server and open it really quick it sends the last bytes and the files sends completely. (this file opens perfectly)
I think it`s a problem on the server side not sending all the bytes but why and where...

Well this is a problem in your server-side code to start with:
fs.Read(data,0, data.Length);
You're ignoring the value returned by Read. Never do that. With FileStream you're possibly okay, but I personally wouldn't trust it anyway. If you're using .NET 4, you don't need to do this anyway - just use Stream.CopyTo.
On the client-side code, your biggest initial problem is that you're doing all of this on the UI thread. That's a terrible idea - the UI will freeze if there's a network glitch, as the Read call is blocking.
Again, just use Stream.CopyTo, but do it in a background thread.
Additionally, in all of these cases, use a using statement for the streams, so that you close them cleanly whatever happens.
That's all just general hygiene. Now, as for why you're hanging...
... you're not closing the network stream on the server side. Therefore you never reach the end of the stream on the client side. If you only need to use the connection for a single file, then the answer is simple: just close the connection on the server side.
If, however, you need to use the same connection for multiple files, then you need more protocol - you need to some way of indicating the end of the data. There are three common ways of doing that:
Write the length of the data before the data itself, then on the reading side, first read the length, then read that many bytes, failing if the stream finishes before you've done so. This requires that you know how much data you're going to write before you start writing.
Use an "end of data" marker which you can detect on the reading side; this is a pain in general, as it requires escaping the marker if it appears in the text itself.
A variation on the first approach, where you write a length-prefixed chunk at a time, then a zero-length chunk to indicate "end of data". This is pretty flexible, but obviously a bit more work than the first approach if the first approach actually works for you.

how to write the stream to memory stream?

public void doprocess(TcpClient client)
{
MemoryStream ms = new MemoryStream();
Stream clStream = client.GetStream();
byte[] buffer_1 = new byte[8192];
int count = clStream.Read(buffer_1, 0, buffer_1.Length);
while (count > 0)
{
ms.Write(buffer_1, 0, count);
//the below line doesn't gives response and code hangs here
count = clStream.Read(buffer_1, 0, buffer_1.Length);
}
}
Is there any other way to write one stream to another? I want to use this Stream twice, which is why I need to write it to the MemoryStream.

In .NET 4 the copying part is really easy:
MemoryStream ms = new MemoryStream();
client.GetStream().CopyTo(ms);
If you're not using .NET 4, then code similar to what you've already got is basically the same thing.
However, note that this (and any other attempt) will only work if the network stream has been closed - otherwise the Read call will block waiting for more data. I suspect that's what's going wrong for you - and it's a fundamental problem.
If your network stream isn't closed, then it's unclear how you'd really want it to behave - should the two readers of the "split" stream basically read any buffered data, but then block until there's new data otherwise? The buffered data could be removed when it's been read from both streams, of course. If that is what you're after, I don't think there's anything in .NET to help you particularly - and it's probably pretty tricky.
If you could give us more context of what you're trying to do (and what the TcpClient is connecting to) that would really help.

The reason why your code hangs on clStream.Read is because you are tying to read 8192 bytes from the socket but on the other side no-one is writing that many bytes. So the client just sits there and waits for the other side to send the required number of bytes. Depending on the protocol you are trying to implement over TCP there must be some indincation from the server how much data it intends to send so that the client knows in advance and only tries to read that many bytes.
For example in the HTTP protocol the server sends in the headers the Content-Length header to indicate to the clients how much data is going to be sent in the body.

The operation has timed out with WebClient.DownloadFile and correct url's

I am batch uploading products to a database.
I am download the image urls to the site to be used for the products.
The code I written works fine for the first 25 iterations (always that number for some reason), but then throws me a System.Net.WebException "The operation has timed out".
if (!File.Exists(localFilename))
{
using (WebClient Client = new WebClient())
{
Client.DownloadFile(remoteFilename, localFilename);
}
}
I checked the remote url it was requesting and it is a valid image url that returns an image.
Also, when I step through it with the debugger, I don't get the timeout error.
HELP! ;)

If I were in your shoes, here's a few possibilities I'd investigate:
if you're running this code from multiple threads, you may be bumping up against the System.Net.ServicePointManager.DefaultConnectionLimit property. Try increasing it to 50-100 when you start up your app. note that I don't think this is your problem, but trying this is easier than the other stuff below. :-)
another possibility is that you're swamping the server. This is usually hard to do with a single-threaded client, but is possible since multiple other clients may be hitting the server also. But because the problem always happens at #25, this seems unlikely since you'd expect to see more variation.
you may be running into a problem with keepalive HTTP connections backing up between your client and the server. this also seems unlikely.
the hard cutoff of 25 makes me think that this may be a proxy or firewall limit, either on your end or the server's, where >25 connections made from one client IP to one server (or proxy) will get throttled.
My money is on the latter one, since the fact that it always breaks at a nice round number of requests, and that stepping in the debugger (aka slower!) doesn't trigger the problem.
To test all this, I'd start with the easy thing: stick in a delay (Thread.Sleep) before each HTTP call, and see if the problem goes away. If it does, reduce the delay until the problem comes back. If it doesn't, increase the delay up to a large number (e.g. 10 seconds) until the problem goes away. If it doesn't go away with a 10 second delay, that's truly a mystery and I'd need more info to diagnose.
If it does go away with a delay, then you need to figure out why-- and whether the limit is permanent (e.g. server's firewall which you can't change) or something you can change. To get more info, you'll want to time the requests (e.g. check DateTime.Now before and after each call) to see if you see a pattern. If the timings are all consistent and suddenly get huge, that suggests a network/firewall/proxy throttling. If the timings gradually increase, that suggests a server you're gradually overloading and lengthening its request queue.
In addition to timing the requests, I'd set the timeout of your webclient calls to be longer, so you can figure out if the timeout is infinite or just a bit longer than the default. To do this, you'll need an alternative to the WebClient class, since it doesn't support a timeout. This thread on MSDN Forums has a reasonable alternative code sample.
An alternative to adding timing in your code is to use Fiddler:
download fiddler and start it up.
set your webclient code's Proxy property to point to the fiddler proxy (localhost:8888)
run your app and look at fiddler.

it seems that WebClient is not closing the Response object it uses when done which will cause, in your case, many responses to be opened at the same time and with a limit of 25 connections on the remote server, you got the 'Timeout exception'. When you debug, early opened reponses get closed due to their inner timeout, etc...
(I inpected WebClient that with Reflector, I can't find an instruction for closing the response).
I propse that you use HttpWebRequest & HttpWebResponse so that you can clean objects after each download:
HttpWebRequest request;
HttpWebResponse response = null;
try
{
FileStream fs;
Stream s;
byte[] read;
int count;
read = new byte[256];
request = (HttpWebRequest)WebRequest.Create(remoteFilename);
request.Timeout = 30000;
request.AllowWriteStreamBuffering = false;
response = (HttpWebResponse)request.GetResponse();
s = response.GetResponseStream();
fs = new FileStream(localFilename, FileMode.Create);
while((count = s.Read(read, 0, read.Length))> 0)
{
fs.Write(read, 0, count);
count = s.Read(read, 0, read.Length);
}
fs.Close();
s.Close();
}
catch (System.Net.WebException)
{
//....
}finally
{
//Close Response
if (response != null)
response.Close();
}

Here's a slightly simplified version of manji's answer:
private static void DownloadFile(Uri remoteUri, string localPath)
{
var request = (HttpWebRequest)WebRequest.Create(remoteUri);
request.Timeout = 30000;
request.AllowWriteStreamBuffering = false;
using (var response = (HttpWebResponse)request.GetResponse())
using (var s = response.GetResponseStream())
using (var fs = new FileStream(localPath, FileMode.Create))
{
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = s.Read(buffer, 0, buffer.Length)) > 0)
{
fs.Write(buffer, 0, bytesRead);
bytesRead = s.Read(buffer, 0, buffer.Length);
}
}
}

I have the same problem and I solve it adding this lines to the configuration file app.config:
<system.net>
<connectionManagement>
<add address="*" maxconnection="100" />
</connectionManagement>
</system.net>