For context, I am creating a networking library and have internally implemented object pooling - instances of byte[] get reused in order to avoid creating constant garbage.
For users of my library, I have exposed a public ref struct Packet which allows the user to perform write operations to the pooled byte[]. Here is the code:
public ref struct Packet
{
private readonly byte[] _buffer;
public Packet()
{
// Here we take a byte-array from the pool.
_buffer = Pool.Get();
}
// Some write methods here...
public void Return()
{
// Here byte-array returns to the pool so it can be reused later.
Pool.Return(_buffer);
}
}
While all of this is great and it works, there are some problems that I have with this approach:
I am leaking implementation details: user shouldn't really know or care how packets get their byte[], they only want to get the packet, write some data to it and send it.
I am forcing my users to call Return method when they are done using the packet.
If user forgets to call Return, byte[] will not get returned and leak occurs.
To somewhat remedy said problems, here is what I tried. Whenever user sends the packet, that send method will call Return method for the user.
public class Client
{
public void Send(Packet packet)
{
// Performs socket operations here...
// Returns byte-array to the pool so users don't have to.
packet.Return();
}
}
While this works most of the time, what if user creates a packet, but does not send it? Again, leak will occur. There seems to be no way to actually hide that library is internally using object pooling and there must exist public Return method so user can manually return byte[] in case they don't actually send the packet.
So, what I want to achieve is:
User doesn't even have access to Return method, it is completely hidden from the user.
byte[] instances get reused (pooled) and are always returned to the pool, even if user does not send the packet or does anything with it.
Any suggestions?
I would consider implementing IDisposable. Nevertheless, it could be beneficial to only get from the Pool in case a Send request is invoked:
public void Send(Action<Packet> modifyPacket)
{
// Performs socket operations here...
var packet = new Packet();
try
{
modifyPacket(packet);
// send...
}
finally
{
packet.Return();
}
}
Thus the payload would be out of the Pool for the shortest amount of time. As I do not know the methods on Packet I do not know in how far you want to hide it. But I hope you get the idea.
Related
I created a static ip and log class.
the ip class find out the users ip address and log class log it into a text file.
every thing work just fine but i wonder what happens if so many requests came at a same time?
i mean the both classes are static and base on static classes it causes problem.
how can i managed them?
here is my ip class:
public static class IP
{
public static string IP()
{
System.Web.HttpContext context = System.Web.HttpContext.Current;
string ipAddress = context.Request.ServerVariables["HTTP_X_FORWARDED_FOR"];
if (!string.IsNullOrEmpty(ipAddress))
{
string[] addresses = ipAddress.Split(',');
if (addresses.Length != 0)
{
return addresses[0];
}
}
return context.Request.ServerVariables["REMOTE_ADDR"];
}
}
}
and here part of my log class which write into the text file:
private static void WriteLine(string message)
{
string filePath = FilePath();
CreateFile(filePath);
try
{
using (StreamWriter log = File.AppendText(filePath))
log.WriteLine(message);
}
catch (Exception)
{
//If can not access to file do nothing
//throw;
}
}
You aren't going to run into contention problems due to your classes being static. Your IP.IP() method class is pure (i.e. it does not change the state of anything) and contains no locks, so there is no chance of there being any contention there.
You do potentially have problems in WriteLine due to the fact that you are probably writing your log file on the same thread as you are doing your work. That means the file write is acting as a lock since only one write can occur at any one time.
What you want is to log to a queue and then to write that queue on a separate thread; that is a classic producer-consumer pattern.
Alternatively you could avoid reinventing the wheel and use an existing logging framework that will handle these things for you like log4net
Streamwriter has a default of 4kb buffer which can be modified if needed as defined by:
public StreamWriter(
Stream stream,
Encoding encoding,
int bufferSize
)
More than likely, your computer (including disk access) is most likely a lot faster than your internet access.
It will work fine, because you don't have any public variable which will be kept in memory and changing on every time class is accessed.
So as the method ends, the scope of your variables will be finished.
However if they are in memory, they will not be effected by how many users use it at the same time, and there will be no mess.
Multithread programming is a new concept for me. I’ve done a bunch of reading and even with many examples, I just can’t seem to figure it out. I'm new to C# and programming.
I have a winform project with lots of custom controls I’ve imported and will utilize many tcpclients. I’m trying to get each control to be hosted on it’s own separate thread. Right now, I’m trying to get 1 control to behave appropriately with it’s own thread.
I'll show you what I have and then follow up with some questions regarding guidance.
string asyncServerHolder; // gets the server name from a text_changed event
int asyncPortHolder; // gets the port # from a text_changed event
TcpClient wifiClient = new TcpClient();
private void btnStart_Click(object sender, EventArgs e)
{
... // variable initialization, etc.
... // XML setup, http POST setup.
send(postString + XMLString); // Content to send.
}
private void send(string msg)
{
AsyncCallback callBack = new AsyncCallback(ContentDownload);
wifiClient.BeginConnect(asyncServerHolder, asyncPortHolder, callBack, wifiClient);
wifiClient.Client.Send(System.Text.Encoding.ASCII.GetBytes(msg));
}
private void ContentDownload(IAsyncResult result)
{
if (wifiClient.Connected)
{
string response4 = "Connected!!"; //debug msg
byte[] buff = new byte[1024];
int i = wifiClient.Client.Receive(buff);
do
{
response1 = System.Text.Encoding.UTF8.GetString(buff, 0, i);
} while (response1.Length == 0);
response2 = response1.Substring(9, 3); // pick out status code to be displayed after
wifiClient.Client.Dispose();
wifiClient.Close();
}
}
If you're knowledgeable about this, I bet you see lots of problems above. As it stands right now, I always get an exception one my first iteration of running this sequence:
"A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied"
Why is this? I have confirmed that my asyncServerHolder and my asyncPortHolder are correct. My second iteration of attempting allowed me to see response4 = "Connected!!" but I get a null response on response1.
Eventually I'd like to substitute in my user controls which I have in a List. I'd just like to gracefully connect, send my msg, receive my response and then allow my form to notify me from that particular control which plays host to that tcp client. My next step would be link up many controls.
Some questions:
1) Do I need more TCP clients? Should they be in a list and be the # of controls I have enabled at that time of btnStart_Click?
2) My controls are on my GUI, does that mean I need to invoke if I'm interacting with them?
3) I see many examples using static methods with this context. Why is this?
Thanks in advance. All criticism is welcome, feel free to be harsh!
BeginConnect returns immediately. Probably, no connection has been established yet when Send runs. Make sure that you use the connection only after having connected.
if (wifiClient.Connected) and what if !Connected? You just do nothing. That's not a valid error recovery strategy. Remove this if entirely.
In your read loop you destroy the previously read contents on each iteration. In fact, you can't split up an UTF8 encoded string at all and decode the parts separately. Read all bytes into some buffer and only when you have received everything, decode the bytes to a string.
wifiClient.Client.Dispose();
wifiClient.Close();
Superstitious dispose pattern. wifiClient.Dispose(); is the canonical way to release everything.
I didn't quite understand what "controls" you are talking about. A socket is not a control. UI controls are single-threaded. Only access them on the UI thread.
Do I need more TCP clients?
You need one for each connection.
Probably, you should use await for all blocking operations. There are wrapper libraries that make the socket APIs usable with await.
This is to a degree a "basics of TCP" question, yet at the same time I have yet to find a convincing answer elsewhere and believe i have a ok/good understanding of the basics of TCP. I am not sure if the combination of questions (or the one questions and while i'm at it the request for confirmation of a couple of points) is against the rules. Hope not.
I am trying to write a C# implementation of a TCP client, that communicates with an existing app containing a TCP server (I don't have access to its code, so no WCF). How do I connect to it, send and receive as needed as new info comes in or out, and ultimately disconnect. Using the following MSDN code as an example where they list "Send" and "Receive" asynchronous methods (or just TcpClient), and ignoring the connect and disconnect as trivial, how can I best go about continuously checking for new packets received and at the same time send when needed?
I initially used TCPClient and GetStream(), and the msdn code still seems to require the loop and sleep described in a bit (counter intuitively), where I run the receive method in a loop in a separate thread with a sleep(10) milliseconds, and Send in the main (or third) thread as needed. This allows me to send fine, and the receive method effectively polls at regular intervals to find new packets. The received packets are then added to a queue.
Is this really the best solution? Shouldn't there be a DataAvailable event equivalent (or something i'm missing in the msdn code) that allows us to receive when, and only when, there is new data available?
As an afterthought I noticed that the socket could be cut from the other side without the client becoming aware till the next botched send. To clarify then, the client is obliged to send regular keepalives (and receive isn't sufficient, only send) to determine if the socket is still alive. And the frequency of the keepalive determines how soon I will know that link is down. Is that correct? I tried Poll, socket.connected etc only to discover why each just doesn't help.
Lastly, to confirm (i believe not but good to make sure), in the above scenario of sending on demand and receiving if tcpclient.DataAvailable every ten seconds, can there be data loss if sending and receiving at the same time? If at the same time I am receiving I try and send will one fail, overwrite the other or any other such unwanted behaviour?
There's nothing wrong necessarily with grouping questions together, but it does make answering the question more challenging... :)
The MSDN article you linked shows how to do a one-and-done TCP communication, that is, one send and one receive. You'll also notice it uses the Socket class directly where most people, including myself, will suggest using the TcpClient class instead. You can always get the underlying Socket via the Client property should you need to configure a certain socket for example (e.g., SetSocketOption()).
The other aspect about the example to note is that while it uses threads to execute the AsyncCallback delegates for both BeginSend() and BeginReceive(), it is essentially a single-threaded example because of how the ManualResetEvent objects are used. For repeated exchange between a client and server, this is not what you want.
Alright, so you want to use TcpClient. Connecting to the server (e.g., TcpListener) should be straightforward - use Connect() if you want a blocking operation or BeginConnect() if you want a non-blocking operation. Once the connection is establish, use the GetStream() method to get the NetworkStream object to use for reading and writing. Use the Read()/Write() operations for blocking I/O and the BeginRead()/BeginWrite() operations for non-blocking I/O. Note that the BeginRead() and BeginWrite() use the same AsyncCallback mechanism employed by the BeginReceive() and BeginSend() methods of the Socket class.
One of the key things to note at this point is this little blurb in the MSDN documentation for NetworkStream:
Read and write operations can be performed simultaneously on an
instance of the NetworkStream class without the need for
synchronization. As long as there is one unique thread for the write
operations and one unique thread for the read operations, there will
be no cross-interference between read and write threads and no
synchronization is required.
In short, because you plan to read and write from the same TcpClient instance, you'll need two threads for doing this. Using separate threads will ensure that no data is lost while receiving data at the same time someone is trying to send. The way I've approached this in my projects is to create a top-level object, say Client, that wraps the TcpClient and its underlying NetworkStream. This class also creates and manages two Thread objects, passing the NetworkStream object to each during construction. The first thread is the Sender thread. Anyone wanting to send data does so via a public SendData() method on the Client, which routes the data to the Sender for transmission. The second thread is the Receiver thread. This thread publishes all received data to interested parties via a public event exposed by the Client. It looks something like this:
Client.cs
public sealed partial class Client : IDisposable
{
// Called by producers to send data over the socket.
public void SendData(byte[] data)
{
_sender.SendData(data);
}
// Consumers register to receive data.
public event EventHandler<DataReceivedEventArgs> DataReceived;
public Client()
{
_client = new TcpClient(...);
_stream = _client.GetStream();
_receiver = new Receiver(_stream);
_sender = new Sender(_stream);
_receiver.DataReceived += OnDataReceived;
}
private void OnDataReceived(object sender, DataReceivedEventArgs e)
{
var handler = DataReceived;
if (handler != null) DataReceived(this, e); // re-raise event
}
private TcpClient _client;
private NetworkStream _stream;
private Receiver _receiver;
private Sender _sender;
}
Client.Receiver.cs
private sealed partial class Client
{
private sealed class Receiver
{
internal event EventHandler<DataReceivedEventArgs> DataReceived;
internal Receiver(NetworkStream stream)
{
_stream = stream;
_thread = new Thread(Run);
_thread.Start();
}
private void Run()
{
// main thread loop for receiving data...
}
private NetworkStream _stream;
private Thread _thread;
}
}
Client.Sender.cs
private sealed partial class Client
{
private sealed class Sender
{
internal void SendData(byte[] data)
{
// transition the data to the thread and send it...
}
internal Sender(NetworkStream stream)
{
_stream = stream;
_thread = new Thread(Run);
_thread.Start();
}
private void Run()
{
// main thread loop for sending data...
}
private NetworkStream _stream;
private Thread _thread;
}
}
Notice that these are three separate .cs files but define different aspects of the same Client class. I use the Visual Studio trick described here to nest the respective Receiver and Sender files under the Client file. In a nutshell, that's the way I do it.
Regarding the NetworkStream.DataAvailable/Thread.Sleep() question. I would agree that an event would be nice, but you can effectively achieve this by using the Read() method in combination with an infinite ReadTimeout. This will have no adverse impact on the rest of your application (e.g., UI) since it's running in its own thread. However, this complicates shutting down the thread (e.g., when the application closes), so you'd probably want to use something more reasonable, say 10 milliseconds. But then you're back to polling, which is what we're trying to avoid in the first place. Here's how I do it, with comments for explanation:
private sealed class Receiver
{
private void Run()
{
try
{
// ShutdownEvent is a ManualResetEvent signaled by
// Client when its time to close the socket.
while (!ShutdownEvent.WaitOne(0))
{
try
{
// We could use the ReadTimeout property and let Read()
// block. However, if no data is received prior to the
// timeout period expiring, an IOException occurs.
// While this can be handled, it leads to problems when
// debugging if we are wanting to break when exceptions
// are thrown (unless we explicitly ignore IOException,
// which I always forget to do).
if (!_stream.DataAvailable)
{
// Give up the remaining time slice.
Thread.Sleep(1);
}
else if (_stream.Read(_data, 0, _data.Length) > 0)
{
// Raise the DataReceived event w/ data...
}
else
{
// The connection has closed gracefully, so stop the
// thread.
ShutdownEvent.Set();
}
}
catch (IOException ex)
{
// Handle the exception...
}
}
}
catch (Exception ex)
{
// Handle the exception...
}
finally
{
_stream.Close();
}
}
}
As far as 'keepalives' are concerned, there is unfortunately not a way around the problem of knowing when the other side has exited the connection silently except to try sending some data. In my case, since I control both the sending and receiving sides, I've added a tiny KeepAlive message (8 bytes) to my protocol. This is sent every five seconds from both sides of the TCP connection unless other data is already being sent.
I think I've addressed all the facets that you touched on. I hope you find this helpful.
I have a problem with the SocketAsyncEventArgs class..the problem is when I try to send 8K of data
for example over internet, the socket sometimes only sends 1K or 2K, I know that this is normal for
TCP socket and that one send doesn't guarantee one receive.
now for this to work for me I modified my code to resend the remaining data, for example
when I the SocketAsyncEventArgs.SendAsync completes, in the callback i check whether it sent all the 8K or not if it's not, I call the SocketAsyncEventArgs.SendAsync again with the remaining data untill
I send it all.
Now, when I looked at some SocketAsyncEventArgs code.. I saw that most people don't do so!
and they just clean up when the send complete without checking if it sent all the data or not!
also when I looked at Microsoft's example, they were saying the ONE call to SocketAsyncEventArgs.SendAsync guarantees that all the data will be sent.
I mean I tested it myself and NO all data will not be sent in one call to SocketAsyncEventArgs.SendAsync.
What I'm doing wrong ?
thanks in advance.
Edit:
Here is the code which doesn't send all data(exactly like microsoft's)
the SendAsyncComplete will be called when the socket sends for example 1Kb of data
not all 8K!
public virtual void Send(byte[] packet, int offset, int length)
{
if (_tcpSock != null && _tcpSock.Connected)
{
var args = SocketHelpers.AcquireSocketArg();
if (args != null)
{
args.Completed += SendAsyncComplete;
args.SetBuffer(packet, offset, length);
args.UserToken = this;
var willRaiseEvent = _tcpSock.SendAsync(args);
if (!willRaiseEvent)
{
ProcessSend(args);
}
unchecked
{
_bytesSent += (uint)length;
}
Interlocked.Add(ref _totalBytesSent, length);
}
else
{
log.Error("Client {0}'s SocketArgs are null", this);
}
}
}
private static void ProcessSend(SocketAsyncEventArgs args)
{
args.Completed -= SendAsyncComplete;
SocketHelpers.ReleaseSocketArg(args);
}
private static void SendAsyncComplete(object sender, SocketAsyncEventArgs args)
{
ProcessSend(args);
}
There are many things I would change there. As a preamble, read this.
First of all, whenever you sent any data on the socket, you must process that event: either you stop the whole send process, or you issue another socket send operation to send the remaining data.
So it makes sense to have 3 methods, like this:
// This is the Send() to be used by your class' clients
1) public void Send(byte[] buffer);
This method will take care to apply any data formatting that you need, create (retrieve) a SocketAsyncEventArgs object, set the token to hold your buffer, and call the next method bellow:
2) private void Send(SocketAsyncEventArgs e);
This one actually calls
Socket.SendAsync(SocketAsyncEventArgs e)
and copies the contents from the token (the buffer, remember?) to the SAEA object. Now that is why because method number (2) might be called several times to send the remaining data that couldn't be sent in one operation by the socket. So here you copy the remaining data from the token to the SAEA buffer.
3) private void ProcessSent(SocketAsyncEventArgs e);
This last method will examine the data that has been sent by the socket. If all the data has been sent, the SAEA object will be released. If not, method (2) will be called again for the rest of the data. In order to keep track of sent data, you use SAEA.BytesTransferred. You should add this value to a value stored in the custom token I advise you to create (so do not use "this" as a token).
This is where you also check for SocketError on the SAEA parameter.
This last method will be called in two places:
in the 2nd method, like this:
// Attempt to send data in an asynchronous fashion
bool isAsync = this.Socket.SendAsync(e);
// Something went wrong and we didn't send the data async
if (!isAsync)
this.ProcessSent(e);
This bit is important any many people missed it even when using the more traditional Begin/EndXXX pattern (in that case, via IAsyncResult). If you don't place it, once in a while (quite rare), a StackOverflow exception will pop out of nowhere and will keep you puzzled for long time.
in the Completed event handler:
private void Completed(object sender, SocketAsyncEventArgs e)
{
// What type of operation did just completed?
switch (e.LastOperation)
{
case SocketAsyncOperation.Send:
{
ProcessSent(e);
break;
}
}
}
The tricky thing is to use one SocketAsyncEventArgs object per 1st Send(byte[]) operation, and release it in 3rd operation, if all data has been sent.
For that, you must create a custom token (class or immutable struct) to place in SocketAsyncEventArgs.UserToken inside the 1st method, and then keep track of how much data you have transferred on each Socket.SendAsync() operation.
When you are reading the article provided in the beginning, notice how the writer reuses the same SAEA object when at the end of Send() operation proceeds with a Receive() operation, if all the data has been sent. That is because his protocol is: each one of the parties (server and client) talk to each other in turns.
Now, should multiple calls to the 1st Send() method occur at the same time, there is no rule in which order they will be handled by the OS. If that is likely to happen and message order is important, and since any call to Send(byte[]) from an "outside entity" result in a Socket.SendAsync(), I suggest that the 1st method actually writes down the received bytes in an internal buffer. As long as this buffer is not empty, you keep sending this data internally. Think of it like a producer-consumer scenario, where the "outside entity" is the producer and the internal send op is the consumer. I prefer an optimistic concurrency scenario here.
The documentation on the matter is rather shallow, with the exception of this article, and even in this case, when you start implementing your own application, some things turn out to be different. This SocketAsyncEventArgs model is arguably a bit counter-intuitive.
Let me know if you need more help, I had my times of struggle with this a while ago when I developed my own library.
Edit:
If I were you, I would move
unchecked
{
_bytesSent += (uint)length;
}
Interlocked.Add(ref _totalBytesSent, length);
to your ProcessSend(SAEA), and use args.BytesTransferred instead of "length".
When reading data in chunks of say, 1024, how do I continue to read from a socket that receives a message bigger than 1024 bytes until there is no data left? Should I just use BeginReceive to read a packet's length prefix only, and then once that is retrieved, use Receive() (in the async thread) to read the rest of the packet? Or is there another way?
edit:
I thought Jon Skeet's link had the solution, but there is a bit of a speedbump with that code. The code I used is:
public class StateObject
{
public Socket workSocket = null;
public const int BUFFER_SIZE = 1024;
public byte[] buffer = new byte[BUFFER_SIZE];
public StringBuilder sb = new StringBuilder();
}
public static void Read_Callback(IAsyncResult ar)
{
StateObject so = (StateObject) ar.AsyncState;
Socket s = so.workSocket;
int read = s.EndReceive(ar);
if (read > 0)
{
so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read));
if (read == StateObject.BUFFER_SIZE)
{
s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0,
new AyncCallback(Async_Send_Receive.Read_Callback), so);
return;
}
}
if (so.sb.Length > 0)
{
//All of the data has been read, so displays it to the console
string strContent;
strContent = so.sb.ToString();
Console.WriteLine(String.Format("Read {0} byte from socket" +
"data = {1} ", strContent.Length, strContent));
}
s.Close();
}
Now this corrected works fine most of the time, but it fails when the packet's size is a multiple of the buffer. The reason for this is if the buffer gets filled on a read it is assumed there is more data; but the same problem happens as before. A 2 byte buffer, for exmaple, gets filled twice on a 4 byte packet, and assumes there is more data. It then blocks because there is nothing left to read. The problem is that the receive function doesn't know when the end of the packet is.
This got me thinking to two possible solutions: I could either have an end-of-packet delimiter or I could read the packet header to find the length and then receive exactly that amount (as I originally suggested).
There's problems with each of these, though. I don't like the idea of using a delimiter, as a user could somehow work that into a packet in an input string from the app and screw it up. It also just seems kinda sloppy to me.
The length header sounds ok, but I'm planning on using protocol buffers - I don't know the format of the data. Is there a length header? How many bytes is it? Would this be something I implement myself? Etc..
What should I do?
No - call BeginReceive again from the callback handler, until EndReceive returns 0. Basically, you should keep on receiving asynchronously, assuming you want the fullest benefit of asynchronous IO.
If you look at the MSDN page for Socket.BeginReceive you'll see an example of this. (Admittedly it's not as easy to follow as it might be.)
Dang. I'm hesitant to even reply to this given the dignitaries that have already weighed in, but here goes. Be gentle, O Great Ones!
Without having the benefit of reading Marc's blog (it's blocked here due the corporate internet policy), I'm going to offer "another way."
The trick, in my mind, is to separate the receipt of the data from the processing of that data.
I use a StateObject class defined like this. It differs from the MSDN StateObject implementation in that it does not include the StringBuilder object, the BUFFER_SIZE constant is private, and it includes a constructor for convenience.
public class StateObject
{
private const int BUFFER_SIZE = 65535;
public byte[] Buffer = new byte[BUFFER_SIZE];
public readonly Socket WorkSocket = null;
public StateObject(Socket workSocket)
{
WorkSocket = workSocket;
}
}
I also have a Packet class that is simply a wrapper around a buffer and a timestamp.
public class Packet
{
public readonly byte[] Buffer;
public readonly DateTime Timestamp;
public Packet(DateTime timestamp, byte[] buffer, int size)
{
Timestamp = timestamp;
Buffer = new byte[size];
System.Buffer.BlockCopy(buffer, 0, Buffer, 0, size);
}
}
My ReceiveCallback() function looks like this.
public static ManualResetEvent PacketReceived = new ManualResetEvent(false);
public static List<Packet> PacketList = new List<Packet>();
public static object SyncRoot = new object();
public static void ReceiveCallback(IAsyncResult ar)
{
try {
StateObject so = (StateObject)ar.AsyncState;
int read = so.WorkSocket.EndReceive(ar);
if (read > 0) {
Packet packet = new Packet(DateTime.Now, so.Buffer, read);
lock (SyncRoot) {
PacketList.Add(packet);
}
PacketReceived.Set();
}
so.WorkSocket.BeginReceive(so.Buffer, 0, so.Buffer.Length, 0, ReceiveCallback, so);
} catch (ObjectDisposedException) {
// Handle the socket being closed with an async receive pending
} catch (Exception e) {
// Handle all other exceptions
}
}
Notice that this implementation does absolutely no processing of the received data, nor does it have any expections as to how many bytes are supposed to have been received. It simply receives whatever data happens to be on the socket (up to 65535 bytes) and stores that data in the packet list, and then it immediately queues up another asynchronous receive.
Since processing no longer occurs in the thread that handles each asynchronous receive, the data will obviously be processed by a different thread, which is why the Add() operation is synchronized via the lock statement. In addition, the processing thread (whether it is the main thread or some other dedicated thread) needs to know when there is data to process. To do this, I usually use a ManualResetEvent, which is what I've shown above.
Here is how the processing works.
static void Main(string[] args)
{
Thread t = new Thread(
delegate() {
List<Packet> packets;
while (true) {
PacketReceived.WaitOne();
PacketReceived.Reset();
lock (SyncRoot) {
packets = PacketList;
PacketList = new List<Packet>();
}
foreach (Packet packet in packets) {
// Process the packet
}
}
}
);
t.IsBackground = true;
t.Name = "Data Processing Thread";
t.Start();
}
That's the basic infrastructure I use for all of my socket communication. It provides a nice separation between the receipt of the data and the processing of that data.
As to the other question you had, it is important to remember with this approach that each Packet instance does not necessarily represent a complete message within the context of your application. A Packet instance might contain a partial message, a single message, or multiple messages, and your messages might span multiple Packet instances. I've addressed how to know when you've received a full message in the related question you posted here.
You would read the length prefix first. Once you have that, you would just keep reading the bytes in blocks (and you can do this async, as you surmised) until you have exhausted the number of bytes you know are coming in off the wire.
Note that at some point, when reading the last block you won't want to read the full 1024 bytes, depending on what the length-prefix says the total is, and how many bytes you have read.
Also I troubled same problem.
When I tested several times, I found that sometimes multiple BeginReceive - EndReceive makes packet loss. (This loop was ended improperly)
In my case, I used two solution.
First, I defined the enough packet size to make only 1 time BeginReceive() ~ EndReceive();
Second, When I receive large size of data, I used NetworkStream.Read() instead of BeginReceive() - EndReceive().
Asynchronous socket is not easy to use, and it need a lot of understanding about socket.
For info (general Begin/End usage), you might want to see this blog post; this approach is working OK for me, and saving much pain...
There seems to be a lot of confusion surrounding this. The examples on MSDN's site for async socket communication using TCP are misleading and not well explained. The EndReceive call will indeed block if the message size is an exact multiple of the receive buffer. This will cause you to never get your message and the application to hang.
Just to clear things up - You MUST provide your own delimiter for data if you are using TCP. Read the following (this is from a VERY reliable source).
The Need For Application Data
Delimiting
The other impact of TCP treating
incoming data as a stream is that data
received by an application using TCP
is unstructured. For transmission, a
stream of data goes into TCP on one
device, and on reception, a stream of
data goes back to the application on
the receiving device. Even though the
stream is broken into segments for
transmission by TCP, these segments
are TCP-level details that are hidden
from the application. So, when a
device wants to send multiple pieces
of data, TCP provides no mechanism for
indicating where the “dividing line”
is between the pieces, since TCP
doesn't examine the meaning of the
data at all. The application must
provide a means for doing this.
Consider for example an application
that is sending database records. It
needs to transmit record #579 from the
Employees database table, followed by
record #581 and record #611. It sends
these records to TCP, which treats
them all collectively as a stream of
bytes. TCP will package these bytes
into segments, but in a manner the
application cannot predict. It is
possible that each will end up in a
different segment, but more likely
they will all be in one segment, or
part of each will end up in different
segments, depending on their length.
The records themselves must have some
sort of explicit markers so the
receiving device can tell where one
record ends and the next starts.
Source: http://www.tcpipguide.com/free/t_TCPDataHandlingandProcessingStreamsSegmentsandSequ-3.htm
Most examples I see online for using EndReceive are wrong or misleading. It usually causes no problems in the examples because only one predefined message is sent and then the connection is closed.
This a very old topic, but I got here looking for something else and found this:
Now this corrected works fine most of the time, but it fails when the packet's size is a multiple of the buffer. The reason for this is if the buffer gets filled on a read it is assumed there is more data; but the same problem happens as before. A 2 byte buffer, for exmaple, gets filled twice on a 4 byte packet, and assumes there is more data. It then blocks because there is nothing left to read. The problem is that the receive function doesn't know when the end of the packet is.
I had this same problem, and since none of the replies seems to solve this, the way I did it was using Socket.Available
public static void Read_Callback(IAsyncResult ar)
{
StateObject so = (StateObject) ar.AsyncState;
Socket s = so.workSocket;
int read = s.EndReceive(ar);
if (read > 0)
{
so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read));
if (s.Available == 0)
{
// All data received, process it as you wish
}
}
// Listen for more data
s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0,
new AyncCallback(Async_Send_Receive.Read_Callback), so);
}
Hope this helps others, SO have helped me many times, thank you all!