We Keep Coding

C# Force SerialPort.DataReceived event to fire

Like many people, I'm having issues with the DataReceived event not firing.
After working with it, I wrapped my handling processes under the the BytesToRead count, so if I miss a fire, I can pick up where I left off. Seemed like it would fix all my issues.
The problem is, sometimes it doesn't trigger even once. Depending on the packet being sent back, this could be absolutely critical, forcing me to restart the application and the setup process because it relies on being able to process a response.
Reading through some of the responses to similar questions hasn't gotten me any closer to guaranteeing that the event will fire at minimum requirements. Microsoft mentions the issue with DataReceived not being guaranteed to fire for every byte, but I noticed this above:
The DataReceived event is also raised if an Eof character is received, regardless of the number of bytes in the internal input buffer and the value of the ReceivedBytesThreshold property.
So my question is, can I force an EOF character through my serial connection to force the event to fire? What would this character be, 0x1A?
If I can't force an EOF character through serial, what would my options be? My first thought was maybe create a Task to keep a watch for the event triggering, and if it doesn't trigger, to trigger the actions through the Task.

So I was able to fix my issues, coming out a little wiser.
From my observations, the ReadBytesThreshold plays a critical part in how effective the event actually is. When the threshold is set too low, the serial port has a tendency to get itself confused, and will eventually throw up its hands and give up.
Setting this closer to the size of my expected data coming in appeared to help ease the burden enough to make the reading fairly consistent.
I didn't test my idea of using a Task, but reading further online appeared to answer my question about using 0x1A: from what I noticed, it will trigger if it receives this character (at least on a Windows machine).

C# SocketAsyncEventArgs stops firing completed event

I have noticed a problem where the .Completed event of a SocketAsyncEventArgs seems to stop firing. The same SAEA can fire properly and be replaced in the pool several times, but eventually all instances will stop firing, and because the code to replace them in the pool is in the event handler, the pool empties.
The following circumstances are also apparently true:
1) It seems to only occur when a server side socket sends data out to one of the connected clients. When the same class is connecting as a client, it doesn't appear to malfunction.
2) It seems to occur under high load. The thread count seems to creep up until eventually the error happens.
3) A test rig under similar stress appears never to malfunction. (It's only 20 messages per second, and the test rig has been proven to 20K)
I'm not going to be able to paste the rather complicated code, but here is a description of my code:
1) The main inspiration is this: http://vadmyst.blogspot.ch/2008/05/sample-code-for-tcp-server-using.html. It shows how to hook up a completion port using an event, how to get different sized messages over the TCP connection, and so on.
2) I have a byte buffer in which all SAEAs have a piece, that doesn't overlap.
3) I have an object pool of SAEAs, based on a blockingcollection. This throws if the pool is empty for too long.
4) As a server, I keep a collection of sockets returned from the AcceptAsync function, indexed by the endpoint of the client. A single process can use one instance as a server as well as multiple instances as clients (forming a web). They share the data buffer and pool of SAEAs.
I realise it's hard to explain this; I've been debugging it for an entire day and night. Just hoping someone has heard of this or has useful questions or suggestions.
At the moment, I am suspecting some sort of thread exhaustion, leading to the SAEAs not being able to call the completion. Alternatively, some sort of buffer problem on the outgoing buffer.

So, another day of debugging and finally I have an explanation.
1) The SAEAs were not firing the completed event because they were unable to send more. This is revealed by Wireshark to be due to the TCP window emptying. (TCP ZeroWindow)
2) The TCP window was emptying because the networking layer was passing an event up the stack that took too long to complete, ie there's no producer/consumer between the network layer and the UI. Thus the network op would have to wait for the screen draw before sending the ACK.
3) The event that took too long was a screen draw in an event handler on the GUI. The test rig was a console window (one that summarized incoming messages), so that's why it didn't cause a problem at much higher load. It's normal not to redraw the screen on each message, but this was happening because the project isn't quite done yet. The redraw rate would have been fixed later.
4) The short term solution is simply to make sure there's no GUIs holding up the show. A more robust solution might be to create a producer/consumer at the network layer.

Serial, RS422, In C#, TxDone Event Not Firing, No Data Being Received

I am writing an application that uses OpenNETCF.IO.Serial (open source, see serial code here) for its serial communication on a Windows CE 6.0 device. This application is coded in C# in the Compact Framework 2.0. I do not believe the issue I am about to describe is specifically related to these details, but I may be proven to be wrong in that regard.
The issue I am having is that, seemingly randomly (read as: intermittent issue I cannot reliably duplicate yet), data will fail to transmit or be received until the device itself is rebooted. The Windows CE device communicates with a system that runs an entirely different application. Rebooting this other system and disconnecting/reconnecting communication cables does not appear to resolve this issue, only rebooting the Windows CE device.
The only sign of this issue occurring is a lack of a TxDone event from OpenNETCF firing (look for "TxDone();" in OpenNETCF.IO.Serial class), and no data being received, when I know for a fact that the connected system is sending data.
Any character value from 1 - 255 (0x01 - 0xFF) can be sent and received in our serial communication. Null values are discarded.
My serial settings are 38400 Baud, 8 data bits, no parity, 1 stop bit (38400, 8n1). I've set the input and output buffer sizes to 256 bytes. DataReceived event happens whenever we receive 1 or more characters, and transmission occurs when there's 1 or more bytes in the output buffer, since messages are of variable length.
No handshaking is used. Since this is RS422, only 4 signals are being used: RX+, RX-, TX+, TX-.
I receive a "DataReceived" event, I read all data from the input buffer and make my own buffer in my code to parse through it at my leisure outside of the DataReceived event. When I receive a command message, I send an quick acknowledgment message back. When the other system receives a command message from the Windows CE device, it will send a quick acknowledgment message back. Acknowledgment messages get no further replies since they're intended as a simple "Yep, got it."
In my code, I receive/transmit through multiple threads, so I use the lock keyword so I'm not transmitting multiple messages simultaneously on multiple threads. Double checking through code has shown that I am not getting hung up on any locks.
At this point, I am wondering if I am continuously missing something obvious about how serial communication works, such as if I need to set some variable or property, rather than just reading from an input buffer when not empty and writing to a transmit buffer.
Any insight, options to check, suggestions, ideas, and so on are welcome. This is something I've been wrestling with on my own for months, I hope that answers or comments I receive here can help in figuring out this issue. Thank you in advance.
Edit, 2/24/2011:
(1) I can only seem to recreate the error on boot up of the system that the Windows CE device is communicating with, and not every boot up. I also looked at the signals, common mode voltage fluctuates, but amplitude of the noise that occurs at system boot up seems unrelated to if the issue occurs or not, I've seen 25V peak-to-peak cause no issue, when 5V peak-to-peak the issue reoccurred).
Issue keeps sounding more and more hardware related, but I'm trying to figure out what can cause the symptoms I'm seeing, as none of the hardware actually appears to fail or shutdown, at least where I've been able to reach to measure signals. My apologies, but I will not be able to give any sort of part numbers of hardware parts, so please don't ask the components being used.
(2) As per #ctacke's suggestion, I ensured all transmits were going through the same location for maintainability, the thread safety I put in is essentially as follows:
lock(transmitLockObj)
{
try
{
comPort.Output = data;
}
[various catches and error handling for each]
}
(3) Getting UART OVERRUN errors, in a test where <10 bytes were being sent and received on about a 300msec time interval at 38400 Baud. Once it gets an error, it goes to the next loop iteration, and does NOT run ReadFile, and does NOT run TxDone event (or any other line checking procedures). Also, not only does closing and reopening the port do nothing to resolve this, rebooting the software while the device is still running doesn't do anything, either. Only a hardware reboot.
My DataReceived event is as follows:
try
{
byte[] input = comPort.Input; //set so Input gets FULL RX buffer
lock(bufferLockObj)
{
for (int i = 0; i < input.Length; i++)
{
_rxRawBuffer.Enqueue(input[i]);
//timer regularly checks this buffer and parses data elsewhere
//there, it is "lock(bufferLockObj){dataByte = _rxRawBuffer.Dequeue();}"
//so wait is kept short in DataReceived, while remaining safe
}
}
}
catch (Exception exc)
{
//[exception logging and handling]
//hasn't gotten here, so no point in showing
}
However, instantly after the WriteFile call did timed out the first time in the test was when I started getting UART OVERRUN errors. I honestly can't see my code causing a UART OVERRUN condition.
Thoughts? Hardware or software related, I'm checking everything I can think to check.

Everything sounds right, but your observations kind of show that they're not.
Since you've stated that you're sending from multiple threads, the first thing I'd do is put in some sort of mechanism for sending where all send requests come into one location before calling out to the serial object instance. Sure, you say that you've ensured you have thread safety, but serializing these calls through one location would help reinforce that (and make the code a bit more maintainable/extensible).
Next I'd probably add some temp handling in the Serial lib to specifically set an event or break in the debugger when you've done a Tx but the TxDone event doesn't fire within some bounding period. It's always possible that the Serial lib has a bug in it (trust me, the author of that code is far from infallible) where some race condition is getting by.

Thank you everyone who responded. We've found that this actually appears to be hardware-related. I'm afraid I can't give more information than this, but I thank everyone who contributed possible solutions or troubleshooting steps.

Memory Leak using .net SerialPort, is there a better way?

I have a program that is used to talk to hardware over rs232. This software is used to display a stream of data that is pushed over the rs232 from the hardware as fast as it can be. The problem I am running into is that over time the private memory assigned to the program explodes, and will very rapidly crash the program. If I disable the hardware from sending data for about 2 minutes, then the software can clear out the memory, but only if I pause the data stream.
I am using the DataReceived event from the SerialPort, and this appears to be where the problem is at, because it will cause a memory spike even if the DataReceived function does nothing inside it. The only thing I can come up with is that every time this event is raised it creates a new thread to run, and it is happening so fast that the computer doesn't have time to run GC while the data is coming in.
Is there a more efficient way to pull data off a SerialPort object? I only care about a string when I receive a "NewLine"?
Thanks,
John Vickers

DataReceived is executed on a different thread. I had problems with really fast data and this event caused me problems. Because of that, I created one thread and read the data myself:
while (this.serialPort.IsOpen)
{
int b = this.serialPort.ReadByte();
if (b != -1)
{
// data is good here
}
}
But like the others said, without any code sample, there isn't much we can help you with.

This is very unusual but it is technically possible. SerialPort uses threadpool threads to call the DataReceived event handler. As soon as it receives one or more bytes, it grabs a TP thread to notify your app. There's a lock in the event generation code, only one thread can call your event handler at a time.
A potential failure mode here is that one of these calls, likely the first one, enters a loop in your code from which it never exits. If you haven't set up the Handshake property, the device can keep sending and triggering more TP calls, all of them blocking on that lock.
Diagnose this from the Debug + Windows + Threads window. If my guess is accurate then you should see a large number of threads listed here. One of them should be inside your DataReceived event handler, double-click it and look at the call stack to see where it is stuck. The memory you are seeing consumed is eaten by the stacks of these threads, one megabyte each.
Another possibility is that your DataReceived event handling code is very slow, possibly by calling Control.Invoke(). Slow enough to not be able to keep up with the device. You now really do need to use the Handshake property to setup flow control. Or fix whatever makes it so slow. There should also be a very large number of ErrorReceived events btw, be sure to implement it so you can see this stuff going wrong.
There's an upper limit on the number of TP threads that can be running at the same time. It is rather generous, 250 times the number of cores. That can easily consume half a gigabyte of memory on a typical dual-core machine.

Just to revive this issue.
I'm seeing a massive memory leak when using the DataReceived event.
I am using a USB 3G modem which provides a serial modem interface. I wrote a tiny program that just opens the serial port and connects to the DataReceived event. The event handler is just an empty method.
If you yank out the dongle memory starts to leak at about 10MB per second. No exception is thrown.
Spinning up a new thread and using the synchronous Read(...) method solved the problem for me. I now get an exception when I yank out the dongle that I can handle and no memory leaks.

I developed a state-driven serial port programming language in C# and I believe it really solves nearly all of the serial port problems that everybody encounters with.
Would you please try it with the following simple state and check memory leaks ?
state Init
recv();
$len = length($DATA_PACKET);
if("$len > 0") {
log($DATA_PACKET, Debug);
}
end state
Screen shots
Project homepage
Download
If you have any questions, please feel free to ask.

Multiple port asynchronous I/O over serialport in C#

I'm trying to write a test application for serial I/O (RS-232) with multiple units in C# and I'm running into an issue with my lack of threading experience so I'm soliciting feedback for a best known method.
I've got a pool of COM ports COM1-16 each of which can read/write at any time and I need to be able to manage them simultaneously. Is this a situation for a thread pool? What is some guidance on structuring this applet?
Edit:
Upon review I was wondering is I really even need to do asynchronous threads here, I could just maintain states for each COM-port and do flow logic (i.e., statemachine) for each COM-port individually.

Much of the difficulty surrounding the serial port is centered around an assumption. The assumption is that the serial port receives data in chunks that are convenient, and what is expected.
Here is an example. I know my GPS receiver sends lines (ends with CRLF). This is an example of one of the NMEA sentences:
$GPGSV,3,1,11,10,75,053,29,29,52,311,32,24,50,298,30,02,39,073,30*77
However, the serial ports DataReceived event handler might(usually does on my PC) fire several times with chunks of that data.
event fire - data
1 $
2 GPGSV,3,1,11,10
3 ,75,053,29,29,52,311,32,24,50,298,30,02,39,073,30*77
Instead of fighting this I created some routines that receive data whenever the event fires, and queue it up. When I need the data I call some other routines that put the data back together in chunk sizes I want. So using my example the first and second time I call read line(my readline) I get back an empty answer.
The third time I get the entire NMEA sentence back.
The bad news is that I don't C#. The code is here SerialPort
Depending on the speed of the ports delegates may not be a good choice. I tested my routines at near 1Mbps using delegates, and not using delegates. At those speeds not using delegates was a better choice.
Here are some tips from those in the know
Kim Hamilton

If you use the backgroundworker you won't be grabbing small
pieces of the data, you'll get the entire string. You need
something like:
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
try
{
serialPort1.Open();
if (serialPort1.IsOpen)
{
while (keepReading)
{
backgroundWorker1.ReportProgress(0, serialPort1.ReadLine());
//backgroundWorker1.ReportProgress(0, sentence.Split(','));
// split_gps_data();
}
}
}
}

The .NET way to handle this is to use Events and Delegates. This will end up creating multiple threads, but it will do so in a manner that means you don't explicitly create them. If you create an event handler and add it to the DataReceived event of each port, then when any port receives data, the event handler will be called on a separate thread. Naturally, this means the method must be reentrant, and that access to any shared data structures must be protected against concurrent access.
Your handler routine will do something like the following:
Call ReadExisting to get the data available.
Handle data.
Done.

You need to know at what speed the receiver is spitting out data. Is it once a second, twice a second, 5 times...
I had this same issue reading GPS data. You need to use the backgroundworker in C# to update your variables, screen.