This is my first question posted on this forum, and I'm a beginner in c# world , so this is kind of exciting for me, but I'm facing some issues with sending a large amount of data through sockets so this is more details about my problem:
I'm sending a binary image of 5 Mo through a TCP socket, at the receiving part I'm saving the result(data received ) and getting only 1.5 Mo ==> data has been lost (I compared the original and the resulting file and it showed me the missed parts)
this is the code I use:
private void senduimage_Click(object sender, EventArgs e)
{
if (!user.clientSocket_NewSocket.Connected)
{
Socket clientSocket_NewSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
user.clientSocket_NewSocket = clientSocket_NewSocket;
System.IAsyncResult _NewSocket = user.clientSocket_NewSocket.BeginConnect(ip_address, NewSocket.Transceiver_TCP_Port, null, null);
bool successNewSocket = _NewSocket.AsyncWaitHandle.WaitOne(2000, true);
}
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(Uimage_Data);
user.clientSocket_NewSocket.Send(outStream);
}
In forums they say to divide data into chunks, is this a solution, if so how can I do this, I've tried but it didn't work!
There are lots of different solutions but chunking is usually a good solution, you can either do this blindly where you keep filling your temporary buffer and then putting it into some stateful buffer until you hit some arbitrary token or the buffer is not completely full, or you can adhere to some sort of contract per tcp message (a message being the overall data to recieve).
If you were to look at doing some sort of contract then do something like the first N bytes of a message is the descriptor, which you could make as big or as small as you want, but your temp buffer will ONLY read this size up front from the stream.
A typical header could be something like:
public struct StreamHeader // 5 bytes
{
public byte MessageType {get;set;} // 1 byte
public int MessageSize {get;set;} // 4 bytes
}
So you would read that in then if its small enough allocate the full message size to the temp buffer and read it all in, or if you deem it too big chunk it into sections and keep reading until the total bytes you have received match the MessageSize portion of your header structure.
Probably you haven't read the documentation on socket usage in C#. (http://msdn.microsoft.com/en-us/library/ms145160.aspx)
The internal buffer can not store all the data you provided to send methode. A possible solution to your problem can be is like you said to divide your data into chunks.
int totalBytesToSend = outstream.length; int bytesSend = 0;
while(bytesSend < totalBytesToSend )
bytesSend+= user.clientSocket_NewSocket.Send(outStream, bytesSend, totalBytesToSend - bytesSend,...);
I suspect that one of your problems is that you are not calling EndConnect. From the MSDN documentation:
The asynchronous BeginConnect operation must be completed by calling the EndConnect method.
Also, the wait:-
bool successNewSocket = _NewSocket.AsyncWaitHandle.WaitOne(2000, true);
is probably always false as there is nothing setting the event to the signaled state. Usually, you would specify a callback function to the BeginConnect function and in the callback you'd call EndConnect and set the state of the event to signaled. See the example code on this MSDN page.
UPDATE
I think I see another problem:-
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(Uimage_Data);
I don't know what type Uimage_Data but I really don't think you want to convert it to ASCII. A zero in the data may signal an end of data byte (or maybe a 26 or someother ASCII code). The point is, the encoding process is likely to be changing the data.
Can you provide the type for the Uimage_Data object?
Most likely the problem is that you are closing the client-side socket before all the data has been transmitted to the server, and it is therefore getting discarded.
By default when you close a socket, all untransmitted data (sitting in the operating system buffers) is discarded. There are a few solutions:
[1] Set SO_LINGER (see http://developerweb.net/viewtopic.php?id=2982)
[2] Get the server to send an acknowledgement to the client, and don't close the client-side socket until you receive it.
[3] Wait until the output buffer is empty on the client side before closing the socket (test using getsocketopt SO_SND_BUF - I'm not sure of the syntax on c#).
Also you really should be testing the return value of Send(). Although in theory it should block until it sends all the data, I would want to actually verify that and at least print an error message if there is a mismatch.
Related
I have a strange question that involves NetworkStreams in C# .Net Core 2.2.101
My setup is as follows:
I have a list of meters
Each meter has a list of registers (a register saves a value, for example: the voltage or current)
Meters are connected to a GSM modem via RS485 (irrelevant for the question)
I send commands to the modem to read specific registers for specific meters
For each register of each meter, I send a command using stream.Write(bytesToSend, 0, bytesToSend.Length); to ask the meter to send me the data that is saved in a specific register in the meter. Directly after sending, I read the response using stream.Read(buffer, 0, buffer.Length);. I also set a read timeout of 5 seconds, which will block and wait for 5 seconds before moving on to the next register if no response has been received before the timeout.
The problem:
What is happening is that when I ask the meter for data, sometimes it will take too long and the timeout will be reached, after which I will move on to ask the next register for data, but then sometimes the first register will reply with data after I have moved on to the next register (meaning that the NetworkStream now has the data from the previous register). Since I have already moved on in my for-loop, my program thinks that the data I am reading from the stream is for the current register, when in fact it is for the previous register from the previous iteration. This messes up the data that goes into the database, because I am saving the wrong value for the wrong register.
My question is: Is there a clever way to ignore any incoming data from a previous iteration in my for-loop? Unfortunately there is no information in the data received that could be used to identify for which register the data is for.
Here is a snippet of what my write and read requests look like:
stream.ReadTimeout = 5000;
stream.WriteTimeout = 2000;
foreach (Meter meter in metersToRead)
{
foreach (Register register in meter.Registers)
{
// Write the request to the meter
stream.Write(bytesToSend, 0, bytesToSend.Length);
// Read response from meter
requestedReadingDataCount = stream.Read(buffer, 0, buffer.Length);
// Extract the value from the response buffer and save the value to the database
...
}
}
I want to try and clone the stream and use the cloned stream for communication regarding the current register iteration, so that when a response comes in after I have closed the cloned stream and moved on to the next register, the response will fail since the stream has been closed. However, I am not sure if you can clone a C# NetworkStream? Anybody know?
My last resort will be to make a call to the database after I read the data for each register, to check if the data I received is reasonable for that register, but I fear that this might slow the program down with all the database calls and I would have to build up some rules that will determine whether a value is reasonable for the current register.
Any ideas would be greatly appreciated. If you have any questions, please let me know and I will try my best to explain it further.
Edit
Here is an updated code snippet, as well as an image that will better explain the issue that I am having.
private async Task ReadMeterRegisters(List<MeterWithRegisters> metersWithRegisters, NetworkStream stream)
{
stream.ReadTimeout = 5000; /* Read timeout set to 5 seconds */
stream.WriteTimeout = 2000; /* Write timeout set to 2 seconds */
foreach (Meter meter in metersToRead)
{
foreach (Register register in meter.Registers)
{
// Instantiate a new buffer to hold the response
byte[] readingResponseDataBuffer = new byte[32];
// Variable to hold number of bytes received
int numBytesReceived = 0;
try
{
// Write the request to the meter
stream.Write(bytesToSend, 0, bytesToSend.Length);
// Read response from meter
numBytesReceived = stream.Read(buffer, 0, buffer.Length);
}
catch (IOException) /* catch read/write timeouts */
{
// No response from meter, move on to next register of current meter
continue;
}
// Extract the value from the response buffer and save the value to the database
...
}
}
}
It sounds like the issue here is that in a timeout scenario, the read operation is still completing at some point in the future but writing to the old buffer. If that is the case, perhaps the simplest option is to not reuse the read buffer in the event of a timeout (meaning: assign a new byte[] to buffer), and consider that network-stream burned (since you now can't know what the internal state is).
An alternative approach would be to not read until you know there is data; you can't do that from NetworkStream, but on Socket you can check .Available to see whether there is data to be read; that way, you won't be performing ambiguous reads. You can also perform a zero-length read on a socket (at least on most OS-es); if you pass a zero-length buffer, it will block until either the timeout or until data becomes available, but without consuming any data (the idea being that you follow a zero-length read with a non-zero-length read if you find that data has become available).
In the more general case: you might find you get better throughput here if you switch to asynchronous IO rather than synchronous IO; you could even use the array-pool for the buffers. For dealing with large volumes of connections, async IO is almost always the way to go.
I am trying to receive multiple messages send by Python TCP client to a C# server.
I do receive the data, but all at once and I don't want that to happen.
I tried to set the server buffer size to the size of the byte[] which I am sending, but it didn't work (source).
Any ideas how can I fix this issue?
Client sending code:
import socket
def send_to_remote(sock, data):
data_bytes = bytearray()
data_bytes.extend(map(ord, data)) # appending the array with string converted to bytes
bytes_length = len(data_bytes) # length of the array that I am sending
sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, bytes_length) # setting the buffer size to the array length
print(bytes_length)
sock.send(data_bytes) # sending the data
Server receiving code:
public static string ReadData(TcpClient connection)
{
StringBuilder receivedDataStr = new StringBuilder();
NetworkStream connectionStream = connection.GetStream();
if (connectionStream.CanRead)
{
byte[] connectionBuffer = new byte[connection.ReceiveBufferSize];
Console.WriteLine(">> Reading from NODE");
int bytesRead = 0;
bytesRead = connectionStream.Read(connectionBuffer, 0, 1024);
receivedDataStr.Append(Encoding.ASCII.GetString(connectionBuffer, 0, bytesRead));
Console.WriteLine(">> " + connection.ReceiveBufferSize);
return receivedDataStr.ToString();
}
else
{
Console.WriteLine(">> Can't read from network stream");
return "none-read";
}
}
EDIT:
What I do is:
send_to_remote(socekt, "msg1")
send_to_remote(socekt, "msg1")
And then:
string msg1 = ReadData(client);
string msg2 = ReadData(client);
I receive buffer array of 106200.
The result is:
"msg1 (then new line) msg2"
"" (string msg2 is waiting for data)
Thank you in advance!!
TCP is a stream protocol, not a packet protocol. There is absolutely no guarantee that you're going to get data in the same exact chunks that it was sent - all that is guaranteed is that you'll get the bytes in the correct order (or a failure, or a timeout, etc). You can get each byte separately, or 14 messages at once. Or anything in between, including half of a message in one receive, and the other half in the next. In the case of unicode (etc), this also means that a single character can be split over multiple receives.
Any multi-message protocol based on TCP needs to include some kind of framing, i.e. some way for you to know where each separate message starts and ends. Since you're using a text-based protocol, a common option is to separate logical messages with a CR, LF, or CR+LF - and then it is your job to buffer data until you have an entire message - by looking for the line ending. For completeness: in the case of binary protocols, it is common to length-prefix payloads with some kind of header that indicates the amount of data in the payload, then you just need to parse the header, determine the length, and buffer until you have that much data.
In simple applications with a text protocol and no challenging scalability concerns, it may be possible to use StreamReader (on top of a NetworkStream) to do that part for you, then just use ReadLine or ReadLineAsync. Note, however, that this would be a bad idea in "real" servers (especially ReadLine) - you don't want a malicious or just buggy client locking a thread forever just because it sent a few characters without the line ending, and then didn't send anything else ... ever. Of course, in a serious server you wouldn't be using a thread per client anyway.
I'm creating a client/server socket app. And I'm not being able to solve this problem, probably due to lack of knowledge on the matter.
The client must send an answer in order to proceed communication:
while(comunicate)
{
if (chkCorrectAnswer.Checked)
answer = encoder.GetBytes('\x02' + "82SP|" + '\x03');
else
answer = encoder.GetBytes("bla");
ServerStream.Write(answer, 0, answer.Length);
//or ??
//tcp.Client.Send(answer);
}
And the server recieves it:
while(comunicate)
{
var validanswer= encoder.GetBytes("myvalidanswer");
answer = new byte[validanswer.Length];
stream.Read(answer, 0, validanswer.Length);
//or ??
//tcp.Client.Receive(answer);
if (answer.SequenceEqual(validanswer))
// continue communication
}
Each code is in different app, looped "comunication thread".
The answer seems being sent correctly but the server doesn't seem to be recieveing it porperly. Sometimes it recieves blablab or lablabl and variations with 7 chars. I thoung the recieving would fill the buffer only with the incoming data and somehow it is filling the buffer with repeated data.
Two questions here:
What should I use, stream.write/read or client.send/recieve?
How to ensure this answer verification?
0x02 and 0x03 is called start of text (STX) and end of text (ETX) and are separators used to identify where your messages starts and ends. There is really no need to use both, it was a common practice when doing serial communication.
You need to continue building a message until ETX is received.
something like (easiest solution but not very efficient if you have lots of clients)
string buffer = "";
var readBuffer = new byte[1];
int readBytes = 0;
while ((readBytes = stream.Read(readBuffer, 0, 1)) == 1 && readBuffer[0] != 3)
{
buffer += readBuffer[0];
}
you can of course read larger chunks. But then you need to check if more than one message was arrived and process the buffer accordingly. That's how I would do it though.
I thoung the recieving would fill the buffer only with the incoming data and somehow it is filling the buffer with repeated data.
Well, you are repeatedly sending the data in a loop, so this is to be expected.
If you want to read only a certain number of bytes off the stream you need to also send the size of the logical packet ahead of it so that the receiving end can first read the size (say as a fixed int value) and then the actual response.
When you do the read you'll get everything you wrote including amything you've written previously.
Implement a length header or some kind of seperator so you know what's what --
length + message
or
message + seperator
Then parse it when you do the read.
I'm currently in the process of developing a C# Socket server that can accept multiple connections from multiple client computers. The objective of the server is to allow clients to "subscribe" and "un-subscribe" from server events.
So far I've taken a jolly good look over here: http://msdn.microsoft.com/en-us/library/5w7b7x5f(v=VS.100).aspx and http://msdn.microsoft.com/en-us/library/fx6588te.aspx for ideas.
All the messages I send are encrypted, so I take the string message that I wish to send, convert it into a byte[] array and then encrypt the data before pre-pending the message length to the data and sending it out over the connection.
One thing that strikes me as an issue is this: on the receiving end it seems possible that Socket.EndReceive() (or the associated callback) could return when only half of the message has been received. Is there an easy way to ensure each message is received "complete" and only one message at a time?
EDIT: For example, I take it .NET / Windows sockets does not "wrap" the messages to ensure that a single message sent with Socket.Send() is received in one Socket.Receive() call? Or does it?
My implementation so far:
private void StartListening()
{
IPHostEntry ipHostInfo = Dns.GetHostEntry(Dns.GetHostName());
IPEndPoint localEP = new IPEndPoint(ipHostInfo.AddressList[0], Constants.PortNumber);
Socket listener = new Socket(localEP.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
listener.Bind(localEP);
listener.Listen(10);
while (true)
{
// Reset the event.
this.listenAllDone.Reset();
// Begin waiting for a connection
listener.BeginAccept(new AsyncCallback(this.AcceptCallback), listener);
// Wait for the event.
this.listenAllDone.WaitOne();
}
}
private void AcceptCallback(IAsyncResult ar)
{
// Get the socket that handles the client request.
Socket listener = (Socket) ar.AsyncState;
Socket handler = listener.EndAccept(ar);
// Signal the main thread to continue.
this.listenAllDone.Set();
// Accept the incoming connection and save a reference to the new Socket in the client data.
CClient client = new CClient();
client.Socket = handler;
lock (this.clientList)
{
this.clientList.Add(client);
}
while (true)
{
this.readAllDone.Reset();
// Begin waiting on data from the client.
handler.BeginReceive(client.DataBuffer, 0, client.DataBuffer.Length, 0, new AsyncCallback(this.ReadCallback), client);
this.readAllDone.WaitOne();
}
}
private void ReadCallback(IAsyncResult asyn)
{
CClient theClient = (CClient)asyn.AsyncState;
// End the receive and get the number of bytes read.
int iRx = theClient.Socket.EndReceive(asyn);
if (iRx != 0)
{
// Data was read from the socket.
// So save the data
byte[] recievedMsg = new byte[iRx];
Array.Copy(theClient.DataBuffer, recievedMsg, iRx);
this.readAllDone.Set();
// Decode the message recieved and act accordingly.
theClient.DecodeAndProcessMessage(recievedMsg);
// Go back to waiting for data.
this.WaitForData(theClient);
}
}
Yes, it is possible you'll have only part of message per one receiving, also it can be even worse during transfer only part of message will be sent. Usually you can see that during bad network conditions or under heavy network load.
To be clear on network level TCP guaranteed to transfer your data in specified order but it not guaranteed that portions of data will be same as you sent. There are many reasons for that software (take a look to Nagle's algorithm for example), hardware (different routers in trace), OS implementation, so in general you should never assume what part of data already transferred or received.
Sorry for long introduction, below some advices:
Try to use relatevely "new" API for high-performance socket server, here samples Networking Samples for .NET v4.0
Do not assume you always send full packet. Socket.EndSend() returns number of bytes actually scheduled to send, it can be even 1-2 bytes under heavy network load. So you have to implement resend rest part of buffer when it required.
There is warning on MSDN:
There is no guarantee that the data
you send will appear on the network
immediately. To increase network
efficiency, the underlying system may
delay transmission until a significant
amount of outgoing data is collected.
A successful completion of the
BeginSend method means that the
underlying system has had room to
buffer your data for a network send.
Do not assume you always receive full packet. Join received data in some kind of buffer and analyze it when it have enough data.
Usually, for binary protocols, I add field to indicate how much data incoming, field with message type (or you can use fixed length per message type (generally not good, e.g. versioning problem)), version field (where applicable) and add CRC-field to end of message.
It not really required to read, a bit old and applies directly to Winsock but maybe worth to study: Winsock Programmer's FAQ
Take a look to ProtocolBuffers, it worth to learn: http://code.google.com/p/protobuf-csharp-port/, http://code.google.com/p/protobuf-net/
Hope it helps.
P.S. Sadly sample on MSDN you refer in question effectively ruin async paradigm as stated in other answers.
Your code is very wrong. Doing loops like that defeats the purpose of asynchronous programming. Async IO is used to not block the thread but let them continue doing other work. By looping like that, you are blocking the thread.
void StartListening()
{
_listener.BeginAccept(OnAccept, null);
}
void OnAccept(IAsyncResult res)
{
var clientSocket = listener.EndAccept(res);
//begin accepting again
_listener.BeginAccept(OnAccept, null);
clientSocket.BeginReceive(xxxxxx, OnRead, clientSocket);
}
void OnReceive(IAsyncResult res)
{
var socket = (Socket)res.Asyncstate;
var bytesRead = socket.EndReceive(res);
socket.BeginReceive(xxxxx, OnReceive, socket);
//handle buffer here.
}
Note that I've removed all error handling to make the code cleaner. That code do not block any thread and is therefore much more effecient. I would break the code up in two classes: the server handling code and the client handling code. It makes it easier to maintain and extend.
Next thing to understand is that TCP is a stream protocol. It do not guarentee that a message arrives in one Receive. Therefore you must know either how large a message is or when it ends.
The first solution is to prefix each message with an header which you parse first and then continue reading until you get the complete body/message.
The second solution is to put some control character at the end of each message and continue reading until the control character is read. Keep in mind that you should encode that character if it can exist in the actual message.
You need to send fixed length messages or include in the header the length of the message. Try to have something that allows you to clearly identify the start of a packet.
I found 2 very useful links:
http://vadmyst.blogspot.com/2008/03/part-2-how-to-transfer-fixed-sized-data.html
C# Async TCP sockets: Handling buffer size and huge transfers
When reading data in chunks of say, 1024, how do I continue to read from a socket that receives a message bigger than 1024 bytes until there is no data left? Should I just use BeginReceive to read a packet's length prefix only, and then once that is retrieved, use Receive() (in the async thread) to read the rest of the packet? Or is there another way?
edit:
I thought Jon Skeet's link had the solution, but there is a bit of a speedbump with that code. The code I used is:
public class StateObject
{
public Socket workSocket = null;
public const int BUFFER_SIZE = 1024;
public byte[] buffer = new byte[BUFFER_SIZE];
public StringBuilder sb = new StringBuilder();
}
public static void Read_Callback(IAsyncResult ar)
{
StateObject so = (StateObject) ar.AsyncState;
Socket s = so.workSocket;
int read = s.EndReceive(ar);
if (read > 0)
{
so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read));
if (read == StateObject.BUFFER_SIZE)
{
s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0,
new AyncCallback(Async_Send_Receive.Read_Callback), so);
return;
}
}
if (so.sb.Length > 0)
{
//All of the data has been read, so displays it to the console
string strContent;
strContent = so.sb.ToString();
Console.WriteLine(String.Format("Read {0} byte from socket" +
"data = {1} ", strContent.Length, strContent));
}
s.Close();
}
Now this corrected works fine most of the time, but it fails when the packet's size is a multiple of the buffer. The reason for this is if the buffer gets filled on a read it is assumed there is more data; but the same problem happens as before. A 2 byte buffer, for exmaple, gets filled twice on a 4 byte packet, and assumes there is more data. It then blocks because there is nothing left to read. The problem is that the receive function doesn't know when the end of the packet is.
This got me thinking to two possible solutions: I could either have an end-of-packet delimiter or I could read the packet header to find the length and then receive exactly that amount (as I originally suggested).
There's problems with each of these, though. I don't like the idea of using a delimiter, as a user could somehow work that into a packet in an input string from the app and screw it up. It also just seems kinda sloppy to me.
The length header sounds ok, but I'm planning on using protocol buffers - I don't know the format of the data. Is there a length header? How many bytes is it? Would this be something I implement myself? Etc..
What should I do?
No - call BeginReceive again from the callback handler, until EndReceive returns 0. Basically, you should keep on receiving asynchronously, assuming you want the fullest benefit of asynchronous IO.
If you look at the MSDN page for Socket.BeginReceive you'll see an example of this. (Admittedly it's not as easy to follow as it might be.)
Dang. I'm hesitant to even reply to this given the dignitaries that have already weighed in, but here goes. Be gentle, O Great Ones!
Without having the benefit of reading Marc's blog (it's blocked here due the corporate internet policy), I'm going to offer "another way."
The trick, in my mind, is to separate the receipt of the data from the processing of that data.
I use a StateObject class defined like this. It differs from the MSDN StateObject implementation in that it does not include the StringBuilder object, the BUFFER_SIZE constant is private, and it includes a constructor for convenience.
public class StateObject
{
private const int BUFFER_SIZE = 65535;
public byte[] Buffer = new byte[BUFFER_SIZE];
public readonly Socket WorkSocket = null;
public StateObject(Socket workSocket)
{
WorkSocket = workSocket;
}
}
I also have a Packet class that is simply a wrapper around a buffer and a timestamp.
public class Packet
{
public readonly byte[] Buffer;
public readonly DateTime Timestamp;
public Packet(DateTime timestamp, byte[] buffer, int size)
{
Timestamp = timestamp;
Buffer = new byte[size];
System.Buffer.BlockCopy(buffer, 0, Buffer, 0, size);
}
}
My ReceiveCallback() function looks like this.
public static ManualResetEvent PacketReceived = new ManualResetEvent(false);
public static List<Packet> PacketList = new List<Packet>();
public static object SyncRoot = new object();
public static void ReceiveCallback(IAsyncResult ar)
{
try {
StateObject so = (StateObject)ar.AsyncState;
int read = so.WorkSocket.EndReceive(ar);
if (read > 0) {
Packet packet = new Packet(DateTime.Now, so.Buffer, read);
lock (SyncRoot) {
PacketList.Add(packet);
}
PacketReceived.Set();
}
so.WorkSocket.BeginReceive(so.Buffer, 0, so.Buffer.Length, 0, ReceiveCallback, so);
} catch (ObjectDisposedException) {
// Handle the socket being closed with an async receive pending
} catch (Exception e) {
// Handle all other exceptions
}
}
Notice that this implementation does absolutely no processing of the received data, nor does it have any expections as to how many bytes are supposed to have been received. It simply receives whatever data happens to be on the socket (up to 65535 bytes) and stores that data in the packet list, and then it immediately queues up another asynchronous receive.
Since processing no longer occurs in the thread that handles each asynchronous receive, the data will obviously be processed by a different thread, which is why the Add() operation is synchronized via the lock statement. In addition, the processing thread (whether it is the main thread or some other dedicated thread) needs to know when there is data to process. To do this, I usually use a ManualResetEvent, which is what I've shown above.
Here is how the processing works.
static void Main(string[] args)
{
Thread t = new Thread(
delegate() {
List<Packet> packets;
while (true) {
PacketReceived.WaitOne();
PacketReceived.Reset();
lock (SyncRoot) {
packets = PacketList;
PacketList = new List<Packet>();
}
foreach (Packet packet in packets) {
// Process the packet
}
}
}
);
t.IsBackground = true;
t.Name = "Data Processing Thread";
t.Start();
}
That's the basic infrastructure I use for all of my socket communication. It provides a nice separation between the receipt of the data and the processing of that data.
As to the other question you had, it is important to remember with this approach that each Packet instance does not necessarily represent a complete message within the context of your application. A Packet instance might contain a partial message, a single message, or multiple messages, and your messages might span multiple Packet instances. I've addressed how to know when you've received a full message in the related question you posted here.
You would read the length prefix first. Once you have that, you would just keep reading the bytes in blocks (and you can do this async, as you surmised) until you have exhausted the number of bytes you know are coming in off the wire.
Note that at some point, when reading the last block you won't want to read the full 1024 bytes, depending on what the length-prefix says the total is, and how many bytes you have read.
Also I troubled same problem.
When I tested several times, I found that sometimes multiple BeginReceive - EndReceive makes packet loss. (This loop was ended improperly)
In my case, I used two solution.
First, I defined the enough packet size to make only 1 time BeginReceive() ~ EndReceive();
Second, When I receive large size of data, I used NetworkStream.Read() instead of BeginReceive() - EndReceive().
Asynchronous socket is not easy to use, and it need a lot of understanding about socket.
For info (general Begin/End usage), you might want to see this blog post; this approach is working OK for me, and saving much pain...
There seems to be a lot of confusion surrounding this. The examples on MSDN's site for async socket communication using TCP are misleading and not well explained. The EndReceive call will indeed block if the message size is an exact multiple of the receive buffer. This will cause you to never get your message and the application to hang.
Just to clear things up - You MUST provide your own delimiter for data if you are using TCP. Read the following (this is from a VERY reliable source).
The Need For Application Data
Delimiting
The other impact of TCP treating
incoming data as a stream is that data
received by an application using TCP
is unstructured. For transmission, a
stream of data goes into TCP on one
device, and on reception, a stream of
data goes back to the application on
the receiving device. Even though the
stream is broken into segments for
transmission by TCP, these segments
are TCP-level details that are hidden
from the application. So, when a
device wants to send multiple pieces
of data, TCP provides no mechanism for
indicating where the “dividing line”
is between the pieces, since TCP
doesn't examine the meaning of the
data at all. The application must
provide a means for doing this.
Consider for example an application
that is sending database records. It
needs to transmit record #579 from the
Employees database table, followed by
record #581 and record #611. It sends
these records to TCP, which treats
them all collectively as a stream of
bytes. TCP will package these bytes
into segments, but in a manner the
application cannot predict. It is
possible that each will end up in a
different segment, but more likely
they will all be in one segment, or
part of each will end up in different
segments, depending on their length.
The records themselves must have some
sort of explicit markers so the
receiving device can tell where one
record ends and the next starts.
Source: http://www.tcpipguide.com/free/t_TCPDataHandlingandProcessingStreamsSegmentsandSequ-3.htm
Most examples I see online for using EndReceive are wrong or misleading. It usually causes no problems in the examples because only one predefined message is sent and then the connection is closed.
This a very old topic, but I got here looking for something else and found this:
Now this corrected works fine most of the time, but it fails when the packet's size is a multiple of the buffer. The reason for this is if the buffer gets filled on a read it is assumed there is more data; but the same problem happens as before. A 2 byte buffer, for exmaple, gets filled twice on a 4 byte packet, and assumes there is more data. It then blocks because there is nothing left to read. The problem is that the receive function doesn't know when the end of the packet is.
I had this same problem, and since none of the replies seems to solve this, the way I did it was using Socket.Available
public static void Read_Callback(IAsyncResult ar)
{
StateObject so = (StateObject) ar.AsyncState;
Socket s = so.workSocket;
int read = s.EndReceive(ar);
if (read > 0)
{
so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read));
if (s.Available == 0)
{
// All data received, process it as you wish
}
}
// Listen for more data
s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0,
new AyncCallback(Async_Send_Receive.Read_Callback), so);
}
Hope this helps others, SO have helped me many times, thank you all!