Is there any limit on the size of data that can be received by TCP client.
With TCP socket communication, server is sending more data but the client is only getting 4K and stopping.
I'm guessing that you're doing exactly 1 Send and exactly 1 Receive.
You need to do multiple reads, there is no guarantee that a single read from the socket will contain everything.
The Receive method will read as much data as is available, up to the size of the buffer. But it will return when it has some data so your program can use it.
You may consider splitting your read/writes over multiple calls. I've definitely had some problems with TcpClient in the past. To fix that we use a wrapped stream class with the following read/write methods:
public override int Read(byte[] buffer, int offset, int count)
{
int totalBytesRead = 0;
int chunkBytesRead = 0;
do
{
chunkBytesRead = _stream.Read(buffer, offset + totalBytesRead, Math.Min(__frameSize, count - totalBytesRead));
totalBytesRead += chunkBytesRead;
} while (totalBytesRead < count && chunkBytesRead > 0);
return totalBytesRead;
}
public override void Write(byte[] buffer, int offset, int count)
{
int bytesSent = 0;
do
{
int chunkSize = Math.Min(__frameSize, count - bytesSent);
_stream.Write(buffer, offset + bytesSent, chunkSize);
bytesSent += chunkSize;
} while (bytesSent < count);
}
//_stream is the wrapped stream
//__frameSize is a constant, we use 4096 since its easy to allocate.
No, it should be fine. I suspect that your code to read from the client is flawed, but it's hard to say without you actually showing it.
No limit, TCP socket is a stream.
There's no limit for data with TCP in theory BUT since we're limited by physical resources (i.e memory), implementors such as Microsoft Winsock utilize something called "tcp window size".
That means that when you send something with the Winsock's send() function for example (and didn't set any options on the socket handler) the data will be first copied to the socket's temporary buffer. Only when the receiving side has acknowledged that he got that data, Winsock will use this memory again.
So, you might flood this buffer by sending faster than it frees up and then - error!
Related
I've an UART device which I'm writing to it a command (via System.IO.Ports.SerialPort) and then immediately the device will respond.
So basically my approach is:
->Write to SerialPort->await Task.Delay->Read from the Port.
//The port is open all the time.
public async byte[] WriteAndRead(byte[] message){
port.Write(command, 0, command.Length);
await Task.Delay(timeout);
var msglen = port.BytesToRead;
if (msglen > 0)
{
byte[] message = new byte[msglen];
int readbytes = 0;
while (port.Read(message, readbytes, msglen - readbytes) <= 0)
;
return message;
}
This works fine on my computer. But if I try it on another computer for example, the bytesToRead property is sometimes mismatched. There are empty bytes in it or the answer is not completed. (E.g. I get two bytes, if I expect one byte: 0xBB, 0x00 or 0x00, 0xBB)
I've also looked into the SerialPort.DataReceived Event, but it fires too often and is (as far as I understand) not really useful for this write and read approach. (As I expect the answer immediately from the device).
Is there a better approach to a write-and-read?
Read carefully the Remarks in https://msdn.microsoft.com/en-us/library/ms143549(v=vs.110).aspx
You should not rely on the BytesToRead value to indicate message length.
You should know, how much data you expect to read to decompose the message.
Also, as #itsme85 noticed, you are not updating the readbytes, and therefore you are always writing received bytes to beginning of your array. Proper code with updating the readbytes should look like this:
int r;
while ((r = port.Read(message, readbytes, msglen - readbytes)) <= 0){
readbytes += r;
}
However, during the time you will read data, more data can come and your "message" might be incomplete.
Rethink, what you want to achieve.
On receiving Multipart data from the browser (which has a size of greater than ~2KB), I start receiving null '\0' bytes after the first few chunks which are relevant when I use:
_Stream.Read(ByteArray, Offset, ContentLength);
But, if I divide the ContentLength into small buffers (around 2KB each) AND add a delay of 1ms after each call to Read(), then it works fine:
for(int i = 0; i < x; i++)
{
_Stream.Read(ByteArray, Offset * i, BufferSize);
System.Threading.Thread.Sleep(1);
}
But, adding delay is quite slow. How to prevent reading null bytes. How can I know how many bytes have been written by the browser.
Thanks
The 0x00 bytes were not actually received, they were never written to.
Stream.Read() returns the number of bytes actually read, which is in your case often less than BufferSize. Small amounts of data typically arrive in a single message, in which case the problem does not occur.
The delay might "work" in your test scenario because by then the network layer has buffered more than BufferSize data. It will probably fail in a production environment.
So you'll need to change your code into something like:
int remaining = ContentLength;
int offset = 0;
while (remaining > 0)
{
int bytes = _Stream.Read(ByteArray, offset, remaining);
if (bytes == 0)
{
throw new ApplicationException("Server disconnected before the expected amount of data was received");
}
offset += bytes;
remaining -= bytes;
}
I need to transfer some large data buffer via UDP(User Datagram protocol ). This buffer is divided into 1452 bytes datagrams. The first two bytes is the datagram number (++dtgrNr % 65536). The number is for detection of lost datagrams. I'd like to control the bitrate as follows:
e.g., for 1 MBps the buffer length (bufferSize) is 100 kB, the timer function SendBuffer is called every 100ms.
for 2 MBps the buffer length is 200 kB and so on.
SendBuffer() // simplified version
{
int current_idx = 0;
while(current_idx < bufferSize)
{
GetCounterBytes(counter, ref b1, ref b2); // obtaining two bytes of datagram counter
datagram[0] = b1;
datagram[1] = b2;
Array.Copy(buffer, current_idx, datagram, 2, datagramSize);
try
{
int sentLen = udpClient.Send(datagram, datagram.Length);
if(sentLen <= 0)
Console.WriteLine("error");
}
catch .........
current_idx += datagramSize;
}
The problem is with sending datagrams. It seems that some of these are not send.
Send method is called inside the loop and it may cause some timeout problem?
But sentLen value is always > 0 and no catch block is called.
Could you help me with the problem?
Any suggestions?
Best regards
I am writing an app that will require to make hundreds of socket connections over tcp to read/write data.
I have come across this code snippet here and I'm wondering how I can make this more robust.
This is currently how I am calling the code:
foreach (var ip in listofIps)
{
IPEndPoint remoteEP = new IPEndPoint(IPAddress.Parse(ip), 4001);
Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
client.Connect(remoteEP);
await ReadAsync(client);
}
Is there anything wrong with the above, and how can it be optimized such that it runs concurrently?
In the code snippet, the buffer size is set to 1000. Just as a simple illustration, if I were to attempt to print out only the bytes received, and not the remaining 0x00s, I have to do something like this:
while (true)
{
await s.ReceiveAsync(awaitable);
int bytesRead = args.BytesTransferred;
if (bytesRead <= 0) break;
var hex = new StringBuilder(bytesRead * 2);
var msg = new byte[bytesRead];
for (int i = 0; i < bytesRead; i++)
msg[i] = args.Buffer[i];
foreach (byte b in msg)
hex.AppendFormat("{0:x2} ", b);
AppendLog(string.Format("RX: {0}", hex));
}
Is there a more efficient way of doing this? Previously, I would iterate the whole buffer and print out the data, but that will give me a whole bunch of trailing 0x00s as my protocol is anywhere between 60 to 70 bytes long.
I am writing an app that will require to make hundreds of socket connections over tcp to read/write data.
You don't need "high-performance sockets" for that. The code is far simpler with regular-performance sockets.
For starters, don't use the custom awaitables from the link you posted. They are perfectly fine for some people (and completely "robust"), but you don't need them and your code will be simpler without them.
Is there anything wrong with the above, and can it be further optimized?
Yes. You shouldn't mix blocking (Connect) and asynchronous (ReadAsync) code. I would recommend something like this:
foreach (var ip in listofIps)
{
IPEndPoint remoteEP = new IPEndPoint(IPAddress.Parse(ip), 4001);
Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
await client.ConnectTaskAsync(remoteEP);
...
}
Where ConnectTaskAsync is a standard TAP-over-APM wrapper:
public static Task ConnectTaskAsync(this Socket socket, EndPoint endpoint)
{
return TaskFactory.FromAsync(socket.BeginConnect, socket.EndConnect, endpoint, null);
}
As Marc Gravell pointed out, this code (and your original code) is connecting the sockets one at a time. You could use Task.WhenAll to connect them all simultaneously.
2) Is there a more efficient way of doing this?
First, you should define a TAP-over-APM ReceiveTaskAsync wrapper similar to the above. When dealing with binary data, I also like to have an extension method on byte arrays for dumping:
public string DumpHex(this ArraySegment<byte> data)
{
return string.Join(" ", data.Select(b => b.ToString("X2")));
}
Then you can have code like this:
while (true)
{
int bytesRead = await socket.ReceiveTaskAsync(buffer);
if (bytesRead == 0) break;
var data = new ArraySegment<byte>(buffer, 0, bytesRead);
AppendLog("RX: " + data.HexDump());
...
}
If you do a lot of binary manipulation, you may find my ArraySegments library helpful.
3) Where and how should I include the logic to check if my whole data has arrived within a single read
Oh, it's more complex than that. :) Sockets are a stream abstraction, not a message abstraction. So if you want to define "messages" in your protocol, you need to include a length prefix or delimiter byte so you can detect the message boundaries. Then you need to write code that will parse out your messages, keeping in mind that blocks of data read from the socket may contain only a partial message (so you have to buffer it), a complete message, multiple complete messages, and may also end with a partial message (again, buffering). And you have to also consider your existing buffer when receiving the new block.
I have a TCP/IP .NET Sockets FAQ on my blog that addresses this specifically and has some example code using my personal default preference for message framing (4-byte little-endian length prefixing).
4) How should I include a writeasync method such that I can send data through the socket in the middle of reads.
That one's surprisingly tricky:
public static Task<int> SendTaskAsync(this Socket socket, byte[] buffer, int offset, int size, SocketFlags flags)
{
return Task<int>.Factory.FromAsync(socket.BeginSend, socket.EndSend, buffer, offset, size, flags, null);
}
public static Task WriteAsync(this Socket socket, byte[] buffer)
{
int bytesSent = 0;
while (bytesSent != buffer.Length)
{
bytesSent += await socket.SendTaskAsync(data, bytesSent, buffer.Length - bytesSent, SocketFlags.None);
}
}
socket.Available does not work. Why?
Instead of size I need to use socket.Available. But it does not work. So I need a method that returns the number of bytes in socket.
void Receive(Socket socket, byte[] buffer, int offset, int size)
{
//int sockAvail = socket.Available; Here socket.Available always asigns 0 to sockAvail
int received = 0;
do
{
try
{
received += socket.Receive(buffer, offset + received, size - received, SocketFlags.None);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
} while (received < size);
MessageBox.Show(socket.Available.ToString());
}
Pass MSG_PEEK to recv().
Socket.Available has to be called before the receive() into the buffer, otherwise it's always 0.
socket.Available is for blocking sockets. With asynchronous reads, you just read what you can and then go back into a select to be told when it's ok to read more.
It does work. It just doesn't meet your expectations. Your expectations are wrong. Adjust them.