We Keep Coding

What is the correct way to use USBHIDDRIVER for multiple writes?

I am writing an application that needs to write messages to a USB HID device and read responses. For this purpose, I'm using USBHIDDRIVER.dll (https://www.leitner-fischer.com/2007/08/03/hid-usb-driver-library/ )
Now it works fine when writing many of the message types - i.e. short ones.
However, there is one type of message where I have to write a .hex file containing about 70,000 lines. The protocol requires that each line needs to be written individually and sent in a packet containing other information (start, end byte, checksum)
However I'm encountering problems with this.
I've tried something like this:
private byte[] _responseBytes;
private ManualResetEvent _readComplete;
public byte[][] WriteMessage(byte[][] message)
{
byte[][] devResponse = new List<byte[]>();
_readComplete = new ManualResetEvent(false);
for (int i = 0; i < message.Length; i++)
{
var usbHid = new USBInterface("myvid", "mypid");
usbHid.Connect();
usbHid.enableUsbBufferEvent(UsbHidReadEvent);
if (!usbHid.write(message)) {
throw new Exception ("Write Failed");
}
usbHid.startRead();
if (!_readComplete.WaitOne(10000)) {
usbHid.stopRead();
throw new Exception ("Timeout waiting for read");
}
usbHid.stopRead();
_readComplete.Reset();
devResponse.Add(_responseBytes.ToArray());
usbHid = null;
}
return devResponse;
}
private void ReadEvent()
{
if (_readComplete!= null)
{
_readComplete.Set();
}
_microHidReadBytes = (byte[])((ListWithEvent)sender)[0];
}
This appears to work. In WireShark I can see the messages going back and forth. However as you can see it's creating an instance of the USBInterface class every iteration. This seems very clunky and I can see in the TaskManager, it starts to eat up a lot of memory - current run has it above 1GB and eventually it falls over with an OutOfMemory exception. It is also very slow. Current run is not complete after about 15 mins, although I've seen another application do the same job in less than one minute.
However, if I move the creation and connection of the USBInterface out of the loop as in...
var usbHid = new USBInterface("myvid", "mypid");
usbHid.Connect();
usbHid.enableUsbBufferEvent(UsbHidReadEvent);
for (int i = 0; i < message.Length; i++)
{
if (!usbHid.write(message)) {
throw new Exception ("Write Failed");
}
usbHid.startRead();
if (!_readComplete.WaitOne(10000)) {
usbHid.stopRead();
throw new Exception ("Timeout waiting for read");
}
usbHid.stopRead();
_readComplete.Reset();
devResponse.Add(_responseBytes.ToArray());
}
usbHid = null;
... now what happens is it only allows me to do one write! I write the data, read the response and when it comes around the loop to write the second message, the application just hangs in the write() function and never returns. (Doesn't even time out)
What is the correct way to do this kind of thing?
(BTW I know it's adding a lot of data to that devResponse object but this is not the source of the issue - if I remove it, it still consumes an awful lot of memory)
UPDATE
I've found that if I don't enable reading, I can do multiple writes without having to create a new USBInterface1 object with each iteration. This is an improvement but I'd still like to be able to read each response. (I can see they are still sent down in Wireshark)

C# winforms freezing: Serialport, Timer, Thread

Edit: Keeping the original question for continuity.
I then edited the question with replacement code for the ReadLine() method by using ReadExisting instead. It works however I still have the same freeze, where the app becomes unresponsive. Debug says it's locking (it takes a while to freeze, sometimes seconds, sometimes minutes) in the while () {} function where I wait for the complete message. More explanations below:
-- obsolete --
What is a good way to handle serialport.readtimeout exception?
try
{
serialPort1.Write(Command_);
if (!IsWriteComm_)
{
Response_ = serialPort1.ReadLine().Replace("\r", "");
}
}
catch (TimeoutException err)
{
DateTime d = DateTime.Now;
rtboxDiag.AppendText("\n" + d.ToString("HH:mm:ss") + ": ");
rtboxDiag.AppendText(err.Message);
if (!serialPort1.IsOpen)
InitConnection();
return Textbox_;
}
this bit of code is exectuted on a timer tick event.
I was having a weird "crash" of the app with an IO exception
"The I/O operation has been aborted because of either a thread exit or an application request."
no matter what I do I am not able to "recover" meaning, I am no longer able to poll data from the serial port.
I added this exception catch and it does log the exception. weirdly enough the test on !serialport.isopen is false (meaning the port is still open).
What might be a hint is: this error does STOP the timer somehow, this is not something I am doing in code. so I am suspecting something related to the timer, rather than the serialport, but I could be wrong.
Closing the port manually, and reconnecting does not fix the problem.
Disconnecting and reconnecting the USB does not fix the problem.
however, closing the app, and relaunching the app does fix the problem (without even disconnecting the MCU or power cycling the MCU/hardware).
-- /obsolete --
edit: the problem is appearing after a few seconds, sometimes minutes of flawless operations. I cannot repeat the issue using a serialport terminal polling the data the same way, at the same frequency. It seems the problem is not coming from the hardware itself.
cheers
Edit: I have yet to test the following modification, not sure if it will fix this problem (I doubt), but at least it's an attempt at not using .readline() which from what I've gathered is not good practice.
anyway here it is:
try
{
serialPort1.Write(Command_);
if (!IsWriteComm_)
{
while (!SerialRxCplt) ;
Response_ = SerialRxResponse.Replace("\r", "").Replace("\n", "");
SerialRxCplt = false;
//Response_ = serialPort1.ReadLine().Replace("\r", "");
}
}
catch (TimeoutException err)
{
DateTime d = DateTime.Now;
rtboxDiag.AppendText("\n" + d.ToString("HH:mm:ss") + ": ");
rtboxDiag.AppendText(err.Message);
if (!serialPort1.IsOpen)
InitConnection();
return Textbox_;
}
and I have the datareceived event enabled:
private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
var serialPort = (System.IO.Ports.SerialPort)sender;
string dataReceived = serialPort.ReadExisting();
ProcessSerialData(dataReceived);
}
and this is how I am processing the data, and manually "waiting" for the \n character which tells me when the data has been fully received.
private void ProcessSerialData(string data)
{
SerialRxBuffer += data;
if (SerialRxBuffer.Contains("\n"))
{
SerialRxCplt = true;
SerialRxResponse = SerialRxBuffer;
SerialRxBuffer = "";
}
else
{
SerialRxCplt = false;
}
}
any input is welcome.
I have added "stuff" for debugging inside that while loop and it does work fine for a while and then freezes, no error or exception is thrown there. For some reason I have a feeling it's not related to the serial port.
I have even added this:
try
{
serialPort1.Write(Command_);
if (!IsWriteComm_)
{
Stopwatch stopWatch = new Stopwatch();
stopWatch.Start();
while (!SerialRxCplt || Timer2StopWatchMilli > 5)
{
Timer2StopWatchMilli = stopWatch.Elapsed.TotalMilliseconds;
ExceptionMessage = Timer2StopWatchMilli.ToString();
IsException = true;
}
stopWatch.Stop();
if (!SerialRxCplt)
return Textbox_;
Response_ = SerialRxResponse.Replace("\r", "").Replace("\n", "");
SerialRxCplt = false;
//Response_ = serialPort1.ReadLine().Replace("\r", "");
}
}
the ExceptionMessage and IsException help me have an idea of what's happening in that loop. And in normal operations, it is what you would except, increments in the order of 0.0x milliseconds. Data is being processed correctly. When it freezes, nothing looks abnormal. I initially thought I was somehow getting "stuck" in an infinite loop but that || Timer2StopWatchMilli > 5 should get me out of it, acting as some sort of timeout.
one extra piece of info: when it freezes, the one CPU core is fully loaded. (I have a 6core CPU, and it's 16-17% in the task manager - memory usage is low < 30MB)
Any help is welcome

I fixed it by clearing RX/TX and stream buffers after each successful transaction.
I think data was being sent to the PC faster than it was able to read causing data to eventually accumulating on the Rx Buffer.
private void SerialPortClearBuffers()
{
serialPort1.DiscardOutBuffer();
serialPort1.DiscardInBuffer();
serialPort1.BaseStream.Flush();
}

Preventing UI from freezing while processing data

I am having some problems with BluetoothChat (that I beleive its the same code on bot Java and MonoForAndroid) example app. I have connected my Android to an microcontroller using a Bluetooth module. In case of sending messages (just raw bytes to microcontroller) it works just fine!
The microcontroller streams a constant serial message and I want to read that data. There is a class named MyHandler in BluetoothChat.cs app that has a code block like this:
case MESSAGE_READ:
byte[] readBuf = (byte[])msg.Obj;
// construct a string from the valid bytes in the buffer
var readMessage = new Java.Lang.String (readBuf, 0, msg.Arg1);
bluetoothChat.conversationArrayAdapter.Add(
bluetoothChat.connectedDeviceName + ": " + readMessage);
break;
So what I need to do is to process the incoming raw data and then change color of some buttons, So I made the follwing changes to the code above:
case MESSAGE_READ:
byte[] readBuf = (byte[])msg.Obj;
//I have just added this code and it blocks the UI
bluetoothChat.ProcessIncomingData(readBuff);
break;
And in the BluetootChat activity I have this method:
public void ProcessIncomingData(byte[] readBuf)
{
if (_logBox != null)
{
_logBox.Text += "\r\n"; //TextView
foreach (var b in readBuf)
{
_logBox.Text += (uint)b + " "; //Show the bytes as int value
}
}
}
`
But unfortunately the changes I made stops the UI and the app crashes after f short while.
Any ideas how can I do this neatly without freezing the UI?

You'll want to hand the work off to a background thread in order to keep the UI thread free to respond to input. I wrote up a post awhile back outlining some of the different methods available to you for doing background threads: Using Background Threads in Mono For Android Applications
One thing to be careful with when dealing with background threads is that if you want to make any changes to the UI, you must switch back to the UI thread. You can do this by using the RunOnUiThread() method.

Create a new thread for the process to take place in.
public static void threadProcess()
{
Thread thread = new Thread()
{
public void run()
{
// Process that will run in the thread
}
};
thread.start();
}

Cannot receive UDP packets inside Unity game

So, I have this game, written in Unity, which is supposed to receive data in real-time over UDP. The data will be coming over wireless from an android device, written in Java, while the Unity program is written in C#. My problem is, whenever I try to declare a UdpClient object, and call its Receive() method inside the Update() method of Unity, the game hangs. Here's the code that I am trying to put inside my Update() method -
UdpClient client = new UdpClient(9877);
IPEndPoint receivePoint = new IPEndPoint(IPAddress.Parse("192.168.1.105"), 9877);
byte[] recData = client.Receive(ref receivePoint);
But it's causing the game to hang.
I then tried a different approach - I tried to receive the data in a separate thread. Works like magic if all I have to do is receive the byte array. No issues. Except that I also need the data received to be used as parameters to functions used in the actual game (for now, let's just say I need to display the received data bytes as a string in the main game window). But, I do not have knowledge of how cross-threading works in Unity. I tried this -
string data = string.Empty;
private IPEndPoint receivePoint;
void OnGUI()
{
GUI.Box(new Rect(20, 20, 100, 40), "");
GUI.Label(new Rect(30, 30, 100, 40), data);
}
void Start()
{
LoadClient();
}
public void LoadClient()
{
client = new UdpClient(9877);
receivePoint = new IPEndPoint(IPAddress.Parse("192.168.1.105"), 9877);
Thread startClient = new Thread(new ThreadStart(StartClient));
startClient.Start();
}
public void StartClient()
{
try
{
while (true)
{
byte[] recData = client.Receive(ref receivePoint);
System.Text.ASCIIEncoding encode = new System.Text.ASCIIEncoding();
data = encode.GetString(recData);
}
}
catch { }
}
But my program hangs if I try the above. So, what exactly am I missing here?

The reason it hangs for you is because that's the way Receive is defined. It blocks your current code until there is data available on the network connection (i.e. the underlying socket). You are correct that you should use a background thread for that.
Please note though, that creating threads in your game object scripts can be dangerous business in case you for example attach the script to multiple objects at the same time. You don't want multiple version of this script running at the same time because they would all try to bind to the same socket address (which won't work).
You also need to pay attention to closing down the threads if the game object dies (this is not automatically done in C# - you have to stop threads).
That said, when you are using multiple threads you need to ensure thread safety. This means that you need to protect the data so that you cannot read it while it is being written to. The simplest way to do this is to use C# locks:
private readonly Object _dataLock = new Object();
private string _sharedData = String.Empty;
void OnGUI()
{
string text = "";
lock (_dataLock)
text = _sharedData;
}
void StartClient()
{
// ... [snip]
var data = Encoding.ASCII.GetString(recData);
lock (_dataLock)
_sharedData = data;
}
Note that locks can hurt performance a bit, especially if used frequently. There are other ways to protect data in c# that are more performant (but slightly more complex). See this guideline from Microsoft for a few examples.

Variable mysteriously changing value

EDIT: Ok I had a problem with one of the string concatenation functions, has nothing to do with threads, but knowing that it couldn't be a problem with threading lead me to the answer thank you for answering.
I am making a simple tcp/ip chat program for practicing threads and tcp/ip. I was using asynchronous methods but had a problem with concurrency so I went to threads and blocking methods (not asynchronous). I have two private variables defined in the class, not static:
string amessage = string.Empty;
int MessageLength;
and a Thread
private Thread BeginRead;
Ok so I call a function called Listen ONCE when the client starts:
public virtual void Listen(int byteLength)
{
var state = new StateObject {Buffer = new byte[byteLength]};
BeginRead = new Thread(ReadThread);
BeginRead.Start(state);
}
and finally the function to receive commands and process them, I'm going to shorten it because it is really long:
private void ReadThread(object objectState)
{
var state = (StateObject)objectState;
int byteLength = state.Buffer.Length;
while (true)
{
var buffer = new byte[byteLength];
int len = MySocket.Receive(buffer);
if (len <= 0) return;
string content = Encoding.ASCII.GetString(buffer, 0, len);
amessage += cleanMessage.Substring(0, MessageLength);
if (OnRead != null)
{
var e = new CommandEventArgs(amessage);
OnRead(this, e);
}
}
}
Now, as I understand it only one thread at a time will enter BeginRead, I call Receive, it blocks until I get data, and then I process it. The problem: the variable amessage will change it's value between statements that do not touch or alter the variable at all, for example at the bottom of the function at: if (OnRead != null) "amessage" will be equal to 'asdf' and at if (OnRead != null) "amessage" will be equal to qwert. As I understand it this is indicative of another thread changing the value/running asynchronously. I only spawn one thread to do the receiving and the Receive function is blocking, how could there be two threads in this function and if there is only one thread how does amessage's value change between statements that don't affect it's value. As a side note sorry for spamming the site with these questions but I'm just getting a hang of this threading story and it's making me want to sip cyanide.
Thanks in advance.
EDIT:
Here is my code that calls the Listen Method in the client:
public void ConnectClient(string ip,int port)
{
client.Connect(ip,port);
client.Listen(5);
}
and in the server:
private void Accept(IAsyncResult result)
{
var client = new AbstractClient(MySocket.EndAccept(result));
var e = new CommandEventArgs(client, null);
Clients.Add(client);
client.Listen(5);
if (OnClientAdded != null)
{
var target = (Control) OnClientAdded.Target;
if (target != null && target.InvokeRequired)
target.Invoke(OnClientAdded, this, e);
else
OnClientAdded(this, e);
}
client.OnRead += OnRead;
MySocket.BeginAccept(new AsyncCallback(Accept), null);
}
All this code is in a class called AbstractClient. The client inherits the Abstract client and when the server accepts a socket it create's it's own local AbstractClient, in this case both modules access the functions above however they are different instances and I couldn't imagine threads from different instances combining especially as no variable is static.

Well, this makes no sense the way you described it. Which probably means that what you think is going on is not what is really happening. Debugging threaded code is quite difficult, very hard to capture the state of the program at the exact moment it misbehaves.
A generic approach is to add logging to your code. Sprinkle your code with Debug.WriteLine() statements that shows the current value of the variable, along with the thread's ManagedId. You get potentially a lot of output, but somewhere you'll see it going wrong. Or you get enough insight in how thread(s) are interacting to guess the source of the problem.
Just adding the logging can in itself solve the problem because it alters the timing of code. Sucks when that happens.

I assume OnRead is firing an event dispatched on a thread pool thread. If any registered event handler is writing to amessage, its value could change any time you're in the reading loop.
Still not very clear where you are gettingthe value assigned to amessage in the loop. Should cleanmessage read content?