We Keep Coding

Handling multiple TcpClient connections without using threads

I've got a C# program with lots (let's say around a thousand) opened TcpClient objects. I want to enter a state which will wait for something to happen for any of those connections.
I would rather not launch a thread for each connection.
Something like...
while (keepRunning)
{
// Wait for any one connection to receive something.
TcpClient active = WaitAnyTcpClient(collectionOfOpenTcpClients);
// One selected connection has incomming traffic. Deal with it.
// (If other connections have traffic during this function, the OS
// will have to buffer the data until the loop goes round again.)
DealWithConnection(active);
}
Additional info:
The TcpClient objects come from a TcpListener.
The target environment will be MS .NET or Mono-on-Linux.
The protocol calls for long periods of idleness while the connection is open.

What you're trying to do is called an Async Pattern in Microsoft terminology. The overall idea is to change all I/O blocking operations to non-blocking. If this is done, the application usually needs as many system threads as there are CPU cores at the machine.
Take a look at Task Parallel Library in .Net 4:
http://msdn.microsoft.com/en-us/library/dd460717%28VS.100%29.aspx
It's a pretty mature wrapper over the plain old Begin/Callback/Context .Net paradigm.
Update:
Think about what to you will do with the data after you read from the connection. In real life you probably have to reply to the client or save the data to a file. In this case you will need some C# infrastructure to contain/manage your logic and still stay within a single thread. TPL provides it to you for free. Its only drawback is that it was introduced in .Net 4, so probably it's not in Mono yet.
Another thing to consider is connection's lifetime. How often your connections are opened/closed and how long do they live? This is important because accepting and disconnecting a TCP connection requires packet exchange with the client (which is asynchronous by nature, and moreover - a malicious client may not return ACK(-nowledged) packets at all). If you think this aspect is significant for your app, you may want to research how to handle this properly in .Net. In WinAPI the corresponding functions are AcceptEx and DisconnectEx. Probably they are wrapped in .Net with Begin/End methods - in this case you're good to go. Otherwise you'll probably have to create a wrapper over these WinAPI calls.

C#: SSL with SocketAsyncEventArgs?

I'm developing a socket server using C# .NET. I'm using the async model provided by the SocketAsyncEventArgs class, because it must be a high performance server to support many connections in short periods of time. Next, I want to secure the communication between
clients and server, and I think I could use SSL.
Is there any way of using SSL with the SocketAsyncEventArgs model? I know .NET has the SslStream class for SSL securing, but I need to use SocketAsyncEventArgs for high performance.
Is it possible to use SSL in an upper level, without implementing it in the server code?
Thanks in advance.

Not sure if anyone cares anymore since this is so old but I needed to do just that this week and could not find anything on the internet that met my needs. Maybe there is something new in the framework that does this that I was unable find... Regardless, I would post source code but since I wrote it for my company and they tend to frown on that, I'll just outline the approach I took:
Since SslStream takes a stream in the constructor, I implemented my own Stream subtype with an underlying MemoryStream for reads and another for writes. I also pass in the TcpClient to this object as well.
I used the TcpClient to do the handshake for setting up the SSL connection. After authenticating the server or client depending on how I am using it, I then use my two MemoryStreams for the SslStream read/writes.
So for Async writes, I first write my payload to the SslStream which populates my MemoryStream for writing with encrypted data. With the encrypted data from the MemoryStream, I populate the SocketAsyncEventArgs buffer and call the TcpClient SendAsync method. For reads, it's pretty much the opposite.
I can't say it particular excites me to move the data like that but as long as you don't let your MemoryBuffer objects get reallocated constantly, it's not a performance issue. At least this way, I can use just the framework and my own code without relying on third party software.

You can take third-party implementation of SSL/TLS protocol, such as our SecureBlackbox, and use it with any transport, including .NET Sockets in asynchronous mode. SSL server component of SecureBlackbox doesn't have it's own socket, instead it fires events, in whose handlers you write your socket-related code. This way you an plug any transport, even non-socket one.

I think I may have found a project that provides this.
https://sourceforge.net/projects/socketservers/
I'm still playing with it and am bumping into an issue loading the server certificate, however, looking through the source code it looks promising.
One aspect I'm unsure about is that it p/invokes to secur32.dll rather than being a pure c# implementation, so I'm not sure what the memory/performance impact of that is.
The details on the sourceforge project page are sparse as to what the goal of the project is.

Serial Comms programming structure in c# / net /

I'm an embedded programmer trying to do a little bit of coding for a communications app and need a quick start guide on the best / easiest way to do something.
I'm successfully sending serial data packets but need to impliment some form of send/ response protocol to avoid overflow on the target system and to ensure that the packet was received ok.
Right now - I have all the transmit code under a button click and it sends the whole lot without any control.
What's the best way to structure this code , i.e sending some packets - waiting for response .. sending more .. etc etc until it's all done, then carrying on with the main program.
I've not used threads or callbacks or suchlike in this environment before but will learn - I just need a pointer to the most straigtforward ways to do it.
Thanks
Rob

The .NET serialport uses buffers, learn to work with them.
Sending packets that are (far) smaller than the Send-buffer can be done w/o threading.
Receiving can be done by the DataReceived event but beware that that is called from another thread. You might as well start your own thread and use blocking reads from there.
The best approach depends on what your 'packets' and protocol look like.

I think to have a long experience about serial comm, both MCU and PC-based.
I strongly UNSUGGEST the single-thread based solution, although it is very straigthful for light-speed testing, but absolutely out for final releases.
Surely you may choose among several patterns, but they are mostly shaped around a dedicated thread for the comm process and a finite-state-machine to parse the protocol (during receiveing).
The prevoius answers give you an idea to how build a simple program, but it might depends on the protocol specification, target device, scope of the application, etc.

there are of course different ways.
I will describe a thread based and an async operation based way:
If you don't use threads, your app will block as long as the operation is performing. This is not what a user is expecting today. Since you are talking about a series of sending and receiveing commands, I would recommend starting the protocol as a thread and then waiting for it to finish. You might also place an Abort button if neccesary. Set the ReadTimeout values and at every receive be ready to catch the exception! An introducing into creating such a work thread is here
If you want to, use Async Send/Receive functions instead of a thread (e.g. NetworkStream.BeginRead etc.). But this is more difficult because you have to manage state between the calls: I recommend using a Finite State Machine then. In fact you create an enumeration (i.e. ProtocolState) and change the state whenever an operation has completed. You can then simply create a function that performs the next step of the protocol with a simple switch/case statement. Since you are working with a remote entity (in your case the serial target system), you always have to consider the device is not working or stops working during the protocol. Do this by starting a timeout timer (e.g. set to 2000ms) and start it after sending each command (assuming each command will get a reply in your protocol). Stop it if the command was received successfully or on timeout.

You could also implement low-level handshaking on the serial port; set the serial port's Handshake property to rts/cts or xon/xoff.
Otherwise (or in addition), use a background worker thread. For simple threads, I like a Monitor.Wait/Pulse mechanism for managing the thread.
I have some code that does read-only serial communications in a thread; email me and I'll be happy to send it to you.

I wasn't sure from your question if you were designing both the PC and embedded sides of the communication link, if you are you might find this SO question interesting.

TcpClient.GetStream().Read() vs. TcpClient.Client.Receive()

.NET allows two very similar ways to "read" from the network (assuming TCP connection):
1. TcpClient.GetStream().Read()
2. TcpClient.Client.Receive()
By looking at NetworkStream source code - it seems that it's an extra wrapper over the underlying socket, which eventually calls Socket methods.
Question: what's the benefit of using "indirect" NetworkStream variation (#1), instead of using direct wrapper provided by Socket implementation?
Thank you,
Boris.

There is, in fact, a pretty clear benefit of using the first option (TcpStream and not Socket). The benefit is that stream API is more flexible when different underlying implementations are needed at for the same program.
For example, a code which sometimes may use SSL and sometimes may not use it, can switch between SslStream and TcpStream with no changes to the calling code. This is something which is much harder to accomplish using only plain Socket API.

Nothing, really. It's just that sometimes it's more convenient to use a Stream.

To me, a successful Socket.Receive operation with zero bytes received tells you that connection is closed.

Listen for events in another application

Suppose I have two applications written in C#. The first is a third party application that raises an event called "OnEmailSent".
The second is a custom app that I've written that I would like to somehow subscribe to the "OnEmailSent" even of the first application.
Is there any way that I could somehow attach the second application to an instance of the first application to listen for "OnEmailSent" event?
So for further clarification, my specific scenario is that we have a custom third party application written in c# that raises an "OnEmailSent" event. We can see the event exists using reflector.
What we want to do is have some other actions take place when this component sends an email.
The most efficient way we can think of would be to be able to use some form of IPC as anders has suggested and listen for the OnEmailSent event being raised by the third party component.
Because the component is written in C# we are toying with the idea of writing another C# application that can attach itself to the executing process and when it detect the OnEmailSent event has been raise it will execute it's own event handling code.
I might be missing something, but from what I understand of how remoting works is that there would need to be a server defining some sort of contract that the client can subscribe to.
I was more thinking about a scenario where someone has written a standalone application like outlook for example, that exposes events that I would like to subscribe to from another application.
I guess the scenario I'm thinking of is the .net debugger and how it can attach to executing assemblies to inspect the code whilst it's running.

In order for two applications (separate processes) to exchange events, they must agree on how these events are communicated. There are many different ways of doing this, and exactly which method to use may depend on architecture and context. The general term for this kind of information exchange between processes is Inter-process Communication (IPC). There exists many standard ways of doing IPC, the most common being files, pipes, (network) sockets, remote procedure calls (RPC) and shared memory. On Windows it's also common to use window messages.
I am not sure how this works for .NET/C# applications on Windows, but in native Win32 applications you can hook on to the message loop of external processes and "spy" on the messages they are sending. If your program generates a message event when the desired function is called, this could be a way to detect it.
If you are implementing both applications yourself you can chose to use any IPC method you prefer. Network sockets and higher-level socket-based protocols like HTTP, XML-RPC and SOAP are very popular these days, as they allow you do run the applications on different physical machines as well (given that they are connected via a network).

You can try Managed Spy and for programmatic access ManagedSpyLib
ManagedSpyLib introduces a class
called ControlProxy. A ControlProxy is
a representation of a
System.Windows.Forms.Control in
another process. ControlProxy allows
you to get or set properties and
subscribe to events as if you were
running inside the destination
process. Use ManagedSpyLib for
automation testing, event logging for
compatibility, cross process
communication, or whitebox testing.
But this might not work for you, depends whether ControlProxy can somehow access the event you're after within your third-party application.
You could also use Reflexil
Reflexil allows
IL modifications by using the powerful
Mono.Cecil library written by Jb
EVAIN. Reflexil runs as Reflector plug-in and
is directed especially towards IL code
handling. It accomplishes this by
proposing a complete instruction
editor and by allowing C#/VB.NET code
injection.

You can either use remoting or WCF. See http://msdn.microsoft.com/en-us/library/aa730857(VS.80).aspx#netremotewcf_topic7.

What's the nature of that OnEmailSent event from that third party application? I mean, how do you know the application is triggering such an event?
If you are planning on doing interprocess communication, the first question you should ask yourself is: Is it really necessary?
Without questioning your motives, if you really need to do interprocess communication, you will need some sort of mechanism. The list is long, very long. From simple WM_DATA messages to custom TCP protocols to very complex Web services requiring additional infrastructures.
This brings the question, what is it you are trying to do exactly? What is this third party application you have no control over?
Also, the debugger has a very invasive way of debugging processes. Don't expect that to be the standard interprocess mechanism used by all other applications. As a matter of fact, it isn't.

You can implement a similar scenario with SQL Server 2005 query change notifications by maintaing a persistent SqlConnection with a .NET application that blocks until data changes in the database.
See http://www.code-magazine.com/article.aspx?quickid=0605061.

also WM_COPYDATA might be possible, see https://social.msdn.microsoft.com/Forums/en-US/eb5dab00-b596-49ad-92b0-b8dee90e24c8/wmcopydata-event-to-receive-data-in-form-application?forum=winforms
I'm using it for similar Purose (to notify that options have been changed)
In our C++/Cli-scenario (MFC-)programs communicate vith WM_COPYDATA with Information-String in COPYDATASTRUCT-Member lpData
(Parameterlist like "Caller=xyz Receiver=abc Job=dosomething"). also a C#-App can receive WM_COPYDATA-messages as shown in the link. Sending WM_COPYDATA from C# (to known Mainframe-Handle) is done by a cpp/cli-Assembly, (I didnt proove how sending WMCOPYDATA can bei done in C#).
PS in Cpp/Cli we send AfxGetMainWnd()->m_hWnd as WPARAM of WMCOPYDATA-Message and in C# (WndProc) m.WParam can be used as adress to send WM_COPYDATA