We Keep Coding

COM: Device only sends data when things are changing. How to deal with that?

So I want to connect to a device via Serial that only sends data when things are changing with the settings on the device (a measuring device).
I use C# and .Net's SerialPort.
I am able to read data and it looks kind of good. But there are a few problems I encountered.
I realized my reading- method (ReadExistingData()) so that it will constantly use the SerialDataReceivedEventHandler when there's new data.
Unfortunately, when I read it like that (probably because of varying package sizes) it will read very chaotically and thus I need to "catch" the first initiating byte (here it's 0xA5).
That means, I always check whether the byte I just received is a 0xA5 and if it is, I read the rest.
But I feel like that way I am missing some commands my device sends me and thats why I cant display the data consistently right, only from time to time.
On a side note: The device sends the device time and a value. The value is delayed and kind of unaccurate, but the time is always right on spot. The other parameters it sends are always weirded out and dont seem to be recognized and thus changed at all.
To display data I use the console for testing purposes, and the whole construct seems to be very reactive to Console outputs.
Here's a little code snippet:
class Device
{
private int stdMessageLengthInBytes = 5;
public DeviceData processedData;
byte[] dataBuffer;
int receivedMessages = 0;
public Device()
{
// Initialize BaseClass (SerialPort derivative)
this.port = new System.IO.Ports.SerialPort();
// Initialize Device
this.data = new byte[stdMessageLengthInBytes];
this.processedData = new P8005_Data();
dataBuffer = new byte[stdMessageLengthInBytes];
}
// Is supposed to read the data from serial port
protected override void ReadExistingData()
{
// Check whether buffer is empty -> Try to catch a 0xA5
if (dataBuffer[0] == 0x00)
{
port.Read(dataBuffer, 0, 1);
}
// If no 0xA5 was catched, restart
if (dataBuffer[0] != 0xA5)
{
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
return;
}
// Read next byte -> Command byte
port.Read(dataBuffer, 1, 1);
// If command was empty, restart
if (dataBuffer[1] == 0x00)
{
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
return;
}
// If its time that is communicated: Read 3 bytes
if (dataBuffer[1] == 0x06)
{
// 4 ms delay seems to be needed otherwise it wont function correctly somehow
System.Threading.Thread.Sleep(5);
port.Read(dataBuffer, 2, 3);
// Write buffer to actual raw data byte array
this.data = dataBuffer;
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
}
// Otherwise: Just read 2 bytes
System.Threading.Thread.Sleep(5); // Needed delay
port.Read(dataBuffer, 2, 2);
// Write buffer to actual raw data byte array
this.data = dataBuffer;
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
}
// Method called by SerialDataReceivedEventHandler
protected override void DataReceivedOnComPort(object sender, SerialDataReceivedEventArgs e)
{
bool valid = false;
ReadExistingData(); // Read data from COM- Port
lock (processedData)
{
switch (data[1]) // Check command byte
{
// Time (3 btyes)
case (0x06):
processedData.currentTime = String.Format("{0:D2}:{1:D2}:{2:D2}", DecodeBcd(data[2]), DecodeBcd(data[3]), DecodeBcd(data[4]));
valid = true;
receivedMessages++;
break;
// Value (2 bytes)
case (0x0D):
double val = 0;
val += DecodeBcd(data[2]) * 100;
val += DecodeBcd(data[3]);
val /= 10;
processedData.currentValue = val;
valid = true;
receivedMessages++;
break;
// ... here are various other hex- code that represent a command from the device (2 btyes)
default:
valid = false;
break;
}
}
// only to check when
if (valid)
{
Console.WriteLine("Received Valid Messages: {0}", receivedMessages);
ConsoleOutput();
}
}
}
On a note: The initialization of the port happens in another method from the base class and works fine.
Is there anything I am missing? How to deal with something like that? Are there any improvements that would help improving my performance? I thought about threading with locks, but I dont think that is the solution somehow... Or maybe everything is just a console problem?
EDIT:
I know changed my code (as #jdweng suggested) so that I put everything in a buffer (basically List<byte> mainBuffer. Then, I take all bytes in the buffer whenever its possbile and work with them, skimming it for 0xA5. When one is found, I read the command and determine how long the "message" has to be according to it (Time -> +3 bytes, Data -> +2 bytes, Other -> +1 byte). Then I can work off those messages (I put them into a List<byte[]>) and determine my output to my screen.
However, even after outsourcing the chopping up into messages and processing the messages, I still seem to either miss some messages, since some action are just not registered and have a big delay, or my processing is wrong. What I can think of is that because I lock my mainBuffer maybe some data isnt written to it.
Is this really this time critical? There is a software that comes with the device and it doesnt seem to have such big problems with delay and slightly wrong values...

Since you don't have the exact specs and/or an unreliable connection (which with serial data has to be expected) you need to sync to the 0xa5 at every message. I would just run every single byte you receive through a parser while keeping the state of the currently received message.
Make sure you validate your input since there are a bunch of things that can go wrong if you get messed up serial data. For example if there is an 0xa5 in the other message types, you might miss your next message. To prevent that I strongly recommend to either get to the specs if possible or code more logic based on data observations.
private const int MESSAGE_LENGTH = 5;
private const int VALUE_COMMAND = 0x0D;
private const int VALUE_SIZE = 4;
private const int TIME_COMMAND = 0x06;
private const int TIME_SIZE = 5;
private byte[] _message = new byte[MESSAGE_LENGTH];
private int _messagePos = 0;
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
var data = new byte[_serialPort.BytesToRead];
_serialPort.Read(data, 0, data.Length);
foreach (var b in data)
{
_message[_messagePos] = b;
if (_messagePos == 0 && b != 0xa5)
continue;
++_messagePos;
if (_messagePos > 2) // if command byte present, process command of any size
ProcessCommand(_message[1]);
}
}
private void ProcessCommand(byte command)
{
if (_messagePos == VALUE_SIZE && command == VALUE_COMMAND)
{
// parse value...
_messagePos = 0;
}
else if (_messagePos == TIME_SIZE && _message[1] == TIME_COMMAND)
{
// parse time...
_messagePos = 0;
}
}

IBuffer in UWP for TCP messages

I need to transfer a string over TCP connection. For this I serializable my object(over 10000 line list) in one stroke, without Intended. But large string won't transfer(As I understood due to buffer size). So MSDN, on this page (https://learn.microsoft.com/ru-ru/windows/uwp/networking/sockets) say me to use IBuffer for transfer my divided stroke. Here is a code:
// More efficient way to send packets.
// This way enables the system to do batched sends
IList<IBuffer> packetsToSend = PreparePackets();
var outputStream = stream.OutputStream;
int i = 0;
Task[] pendingTasks = new Tast[packetsToSend.Count];
foreach (IBuffer packet in packetsToSend)
{
pendingTasks[i++] = outputStream.WriteAsync(packet).AsTask();
}
// Now, wait for all of the pending writes to complete
await Task.WaitAll(pendingTasks);
What is the method PraparePackets()? How to prepare packets from my stroke?
Edit: I've found solution with DataReader and DataWriter, which has written in Albahari.(End of 16 chapter).

I've found solution with DataReader and DataWriter, which has written in Albahari.(End of 16 chapter).

How to record and playback with NAudio using AsioOut

I'm trying to get the sound input and send output directly with less latency possible with C#.
I'm using the library NAudio that supports ASIO for better latency.
In particular, I use the AsioOut object for recording and another for playback initialized with a BufferedWaveProvider, which is filled in a Callback function: OnAudioAvailable, which allows me to use the ASIO buffers.
The problem is that I hear the sound with various glitches and with a bit of delay. I think the problem is in the function OnAudioAvailable where the buffer is filled with data taken as input from the sound card.
Declarations:
NAudio.Wave.AsioOut playAsio;
NAudio.Wave.AsioOut recAsio;
NAudio.Wave.BufferedWaveProvider buffer;
Play procedure:
if (sourceList.SelectedItems.Count == 0) return;
int deviceNumber = sourceList.SelectedItems[0].Index;
recAsio = new NAudio.Wave.AsioOut(deviceNumber);
recAsio.InitRecordAndPlayback(null, 2, 44100); //rec channel = 1
NAudio.Wave.WaveFormat formato = new NAudio.Wave.WaveFormat();
buffer = new NAudio.Wave.BufferedWaveProvider(formato);
recAsio.AudioAvailable += new EventHandler<NAudio.Wave.AsioAudioAvailableEventArgs>(OnAudioAvailable);
//Collego l'output col buffer
playAsio = new NAudio.Wave.AsioOut(deviceNumber);
playAsio.Init(buffer);
//Registro
recAsio.Play();
//Playback
playAsio.Play();
OnAudioAvailable():
//Callback
private unsafe void OnAudioAvailable(object sender, NAudio.Wave.AsioAudioAvailableEventArgs e)
{
//Copio tutti gli elementi di InputBuffers in buf e li aggiungo in coda al buffer
byte[] buf = new byte[e.SamplesPerBuffer];
for (int i = 0; i < e.InputBuffers.Length; i++)
{
Marshal.Copy(e.InputBuffers[i], buf, 0, e.SamplesPerBuffer);
//Aggiungo in coda al buffer
buffer.AddSamples(buf, 0, buf.Length);
}
}
AsioAudioAvailableEventArgs definition:
public AsioAudioAvailableEventArgs(IntPtr[] inputBuffers, int samplesPerBuffer, AsioSampleType asioSampleType);
public float[] GetAsInterleavedSamples();
Does anyone know how to fix it?
Thank you all.

You should not be using two instances of AsioOut for the same device. I'm surprised this works at all. Just use the one, with InitRecordAndPlayback.
For absolute minimal latency for pass-through monitoring, in AudioAvailableEvent, copy directly to the OutputBuffers, and set WrittenToOutputBuffers = true. This means you don't need a BufferedWaveProvider.
Also remember that any glitches are simply due to you not managing to process the AudioAvailable event quickly enough. When you're working with ASIO, you can be at very low latencies (e.g. sub 10ms), and can be dealing with lots of data (e.g. 96kHz sample rates, 8 channels of input and output). So you need to do a lot of moving of data in a short time window. With .NET, you have to factor in the unfortunate fact that the garbage collector could kick in at any time and cause you to miss a buffer from time to time.

Windows Phone camera feed over UDP is "horribly" slow

I have been working on a private project where i wanted to learn how to program on a windows phone, and at a point i started to fiddle with sockets and the camera, and a great idea came to mind video feed (dumb me to even attempt).
but now I'm here, I have something that well, it works like a charm but a Lumia 800 cannot chug through the for-loop fast enough. It sends a frame per lets say 7-8 seconds something i think is strange since well, it should be strong enough. It feels and looks like watching porn on a 56k modem without the porn.
I also realized that a frame is 317000 pixels and that would sum up to roughly 1MB per frame I'm also sending xy coordinates so mine takes up 2.3MB per frame still working on a different way to solve this to keep it down. so I'm guessing i would need to do dome magic to make both position and pixel values of an acceptable size. because atm would i get5 it up at an acceptable speed it would require at least 60MB/s to get something like 30fps but thats a problem for another day.
//How many pixels to send per burst (1000 seems to be the best)
const int PixelPerSend = 1000;
int bSize = 7 * PixelPerSend;
//Comunication thread UDP feed
private void EthernetComUDP() //Runs in own thread
{
//Connect to Server
clientUDP = new SocketClientUDP();
int[] ImageContent = new int[(int)cam.PreviewResolution.Height * (int)cam.PreviewResolution.Width];
byte[] PacketContent = new byte[bSize];
string Pixel,l;
while (SendingData)
{
cam.GetPreviewBufferArgb32(ImageContent);
int x = 1, y = 1, SenderCount = 0;
//In dire need of a speedup
for (int a = 0; a < ImageContent.Length; a++) //this loop
{
Pixel = Convert.ToString(ImageContent[a], 2).PadLeft(32, '0');
//A - removed to conserve bandwidth
//PacketContent[SenderCount] = Convert.ToByte(Pixel.Substring(0, 8), 2);//0
//R
PacketContent[SenderCount] = Convert.ToByte(Pixel.Substring(8, 8), 2);//8
//G
PacketContent[SenderCount + 1] = Convert.ToByte(Pixel.Substring(16, 8), 2);//16
//B
PacketContent[SenderCount + 2] = Convert.ToByte(Pixel.Substring(24, 8), 2);//24
//Coordinates
//X
l = Convert.ToString(x, 2).PadLeft(16, '0');
//X bit(1-8)
PacketContent[SenderCount + 3] = Convert.ToByte(l.Substring(0, 8), 2);
//X bit(9-16)
PacketContent[SenderCount + 4] = Convert.ToByte(l.Substring(8, 8), 2);
//Y
l = Convert.ToString(y, 2).PadLeft(16, '0');
//Y bit(1-8)
PacketContent[SenderCount + 5] = Convert.ToByte(l.Substring(0, 8), 2);
//Y bit(9-16)
PacketContent[SenderCount + 6] = Convert.ToByte(l.Substring(8, 8), 2);
x++;
if (x == cam.PreviewResolution.Width)
{
y++;
x = 1;
}
SenderCount += 7;
if (SenderCount == bSize)
{
clientUDP.Send(ConnectToIP, PORT + 1, PacketContent);
SenderCount = 0;
}
}
}
//Close on finish
clientUDP.Close();
}
i have tried for simplicity to just send the pixels induvidialy using
BitConverter.GetBytes(ImageContent[a]);
instead of the string parsing mess i have created (to be fixed just wanted a proof of concept) but to do the simple BitConverter did not speed it up to much.
so now im on my last idea the UDP sender socket witch is rhoughly identical to the one on msdn's library.
public string Send(string serverName, int portNumber, byte[] payload)
{
string response = "Operation Timeout";
// We are re-using the _socket object that was initialized in the Connect method
if (_socket != null)
{
// Create SocketAsyncEventArgs context object
SocketAsyncEventArgs socketEventArg = new SocketAsyncEventArgs();
// Set properties on context object
socketEventArg.RemoteEndPoint = new DnsEndPoint(serverName, portNumber);
// Inline event handler for the Completed event.
// Note: This event handler was implemented inline in order to make this method self-contained.
socketEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(delegate(object s, SocketAsyncEventArgs e)
{
response = e.SocketError.ToString();
// Unblock the UI thread
_clientDone.Set();
});
socketEventArg.SetBuffer(payload, 0, payload.Length);
// Sets the state of the event to nonsignaled, causing threads to block
_clientDone.Reset();
// Make an asynchronous Send request over the socket
_socket.SendToAsync(socketEventArg);
// Block the UI thread for a maximum of TIMEOUT_MILLISECONDS milliseconds.
// If no response comes back within this time then proceed
_clientDone.WaitOne(TIMEOUT_MILLISECONDS);
}
else
{
response = "Socket is not initialized";
}
return response;
}
All in all i have ended up on 3 solutions
Accept defeat (but that wont happen so lets look at 2)
Work down the amount of data sent (destroys quality 640x480 is small enough i think)
Find the obvious problem (Google and friend's ran out of good ideas, thats why I'm here)

The problem is almost certainly the messing about with the data. Converting a megabyte of binary data into several megabytes of text and then extracting and sending individual characters will add a massive overhead per byte of source data. Looping through individual pixels to build a send buffer will take (relatively speaking) geological timescales.
The fastest way to do this is likely to be to grab a buffer of binary data from the camera, and send it with one UDP write. Only process or break up the data on the phone if you have to, and be careful to access the original binary data directly - don't convert it all to strings and back to binary. Every extra method call you add into this process will just add overhead. If you have to use a loop, try to pre-calculate as much as you can outside the loop to minimise the work that is done on each iteration.

A couple things come to mind: #1 Break up the raw image array into pieces to be sent over the wire. Not sure if Linq is available on Windows Phone but something like this. #2 Converting from int to string to byte will be very inefficient because of the processing time and memory usage. A better approach would be to bulk copy chunks of the int array to a byte array directly. Example

How to avoid the silent tickeling noises when streaming sound from microphone to speakers using DirectSound and C#?

I try to stream sound samples from my microphone to my speakers by using DirectSound and C#. It should be similar to 'listening to microphone', but later I want to use this for something else. By testing my approach I've noticed silent tickeling, cracking noises in the background. I would guess this has something to do with the delay between writing and playing the buffer, which must be greater than the latency to write the chunks.
If I set the delay between recording and playout to less than 50ms. Mostly it works but sometimes I get really loud cracking noises. So I've decided to a delay about at least 50ms. This works okay for me, but the delay of the systems "listen to device" seems to be much shorter. I would guess it is about 15-30ms, and nearly not noticeable. For 50ms I get at least a little reverb effect.
In the following I'll show you my microphone code (partially):
The initialisation is done like this:
capture = new Capture(device);
// Creating the buffer
// Determining the buffer size
bufferSize = format.AverageBytesPerSecond * bufferLength / 1000;
while (bufferSize % format.BlockAlign != 0) bufferSize += 1;
chunkSize = Math.Max(bufferSize, 256);
bufferSize = chunkSize * BUFFER_CHUNKS;
this.bufferLength = chunkSize * 1000 / format.AverageBytesPerSecond; // Redetermining the buffer Length that will be used.
captureBufferDescription = new CaptureBufferDescription();
captureBufferDescription.BufferBytes = bufferSize;
captureBufferDescription.Format = format;
captureBuffer = new CaptureBuffer(captureBufferDescription, capture);
// Creating Buffer control
bufferARE = new AutoResetEvent(false);
// Adding notifier to buffer.
bufferNotify = new Notify(captureBuffer);
BufferPositionNotify[] bpns = new BufferPositionNotify[BUFFER_CHUNKS];
for(int i = 0 ; i < BUFFER_CHUNKS ; i ++) bpns[i] = new BufferPositionNotify() { Offset = chunkSize * (i+1) - 1, EventNotifyHandle = bufferARE.SafeWaitHandle.DangerousGetHandle() };
bufferNotify.SetNotificationPositions(bpns);
The capturing will run like this in an extra thread:
// Initializing
MemoryStream tempBuffer = new MemoryStream();
// Capturing
while (isCapturing && captureBuffer.Capturing)
{
bufferARE.WaitOne();
if (isCapturing && captureBuffer.Capturing)
{
captureBuffer.Read(currentBufferPart * chunkSize, tempBuffer, chunkSize, LockFlag.None);
ReportChunk(applyVolume(tempBuffer.GetBuffer()));
currentBufferPart = (currentBufferPart + 1) % BUFFER_CHUNKS;
tempBuffer.Dispose();
tempBuffer = new MemoryStream(); // Reset Buffer;
}
}
// Finalizing
isCapturing = false;
tempBuffer.Dispose();
captureBuffer.Stop();
if (bufferARE.WaitOne(bufferLength + 1)) currentBufferPart = (currentBufferPart + 1) % BUFFER_CHUNKS; // That on next start the correct bufferpart will be read.
stateControlARE.Set();
While capturing ReportChunk takes the data to the speaker as an event that could be subscribed. The speaker part is initialized like this:
// Creating the dxdevice.
dxdevice = new Device(device);
dxdevice.SetCooperativeLevel(hWnd, CooperativeLevel.Normal);
// Creating the buffer
bufferDescription = new BufferDescription();
bufferDescription.BufferBytes = bufferSize;
bufferDescription.Format = input.Format;
bufferDescription.ControlVolume = true;
bufferDescription.GlobalFocus = true; // That sound doesn't stop if the hWnd looses focus.
bufferDescription.StickyFocus = true; // - " -
buffer = new SecondaryBuffer(bufferDescription, dxdevice);
chunkQueue = new Queue<byte[]>();
// Creating buffer control
bufferARE = new AutoResetEvent(false);
// Register at input device
input.ChunkCaptured += new AInput.ReportBuffer(input_ChunkCaptured);
The data is put by the event method into the queue, simply by:
chunkQueue.Enqueue(buffer);
bufferARE.Set();
Filling the playbackbuffer and starting/stopping the playback buffer is done by another thread:
// Initializing
int wp = 0;
bufferARE.WaitOne(); // wait for first chunk
// Playing / writing data to play buffer.
while (isPlaying)
{
Thread.Sleep(1);
bufferARE.WaitOne(BufferLength * 3); // If a chunk is played and there is no new chunk we try to continue and may stop playing, else may the buffer runs out.
// Note that this may fails if the sender was interrupted within one chunk
if (isPlaying)
{
if (chunkQueue.Count > 0)
{
while (chunkQueue.Count > 0) wp = writeToBuffer(chunkQueue.Dequeue(), wp);
if (buffer.PlayPosition > wp - chunkSize * 3 / 2) buffer.SetCurrentPosition(((wp - chunkSize * 2 + bufferSize) % bufferSize));
if (!buffer.Status.Playing)
{
buffer.SetCurrentPosition(((wp - chunkSize * 2 + bufferSize) % bufferSize)); // We have 2 chunks buffered so we step back 2 chunks and play them while getting new chunks.
buffer.Play(0, BufferPlayFlags.Looping);
}
}
else
{
buffer.Stop();
bufferARE.WaitOne(); // wait for a filling chunk
}
}
}
// Finalizing
isPlaying = false;
buffer.Stop();
stateControlARE.Set();
writeToBuffer simply writes the enqueued chunk to the buffer by this.buffer.Write(wp, data, LockFlag.None); and caring about bufferSize and chunkSize and wp, which represents the last writing position. I think this is everything that is important about my code. Maybe the definitions are missing and at least there is another method that starts/stops=controls the threads.
I've posted this code in case I've made a mistake in filling the buffer or my initialisation is wrong. But I would guess that this problem occurs because the execution of C# bytecode is too slow or something like that. But in the end my question is still open: My question is how to reduce the latency and how to avoid noises that shouldn't be there?

I know the reason of your problem and the way that you can solve it, but I can't implement it in C# and .Net, so I will explain it in hope that you can find your way.
Audio will be recorded by your mic. with an specified frequency( for example 44100 ) and then played on the sound card at same sample rate( again 44100 ), the problem is the crystal that count the time in input device( mic. for example ) is not same as the crystal that play sound in sound card.
also the difference is so small they are not the same( there is no 2 exact same crystal in entire world ) so after a while there will be a gap in your playback routines.
Now the solution is re-sample data to match the sample rate of the output but I don't know how to do that in C# and .Net

A long time ago I figured out, that this problem was caused by the Thread.Sleep(1); in combination with high CPU usage. Due the windows timerresolution is 15,6ms by default, this sleep doesn't mean sleep for 1ms, but sleep until the next clock interrupt is reached. (For more read this paper) Combined with high CPU usage it may stacks up to the length of a chunk or even more.
For example: If my chunksize is 40ms, this could be about 46,8ms (3 * 15,6ms) and this causes the tickeling. One solution for that is setting the resolution down to 1ms. That can be done in this way:
[DllImport("winmm.dll", EntryPoint="timeBeginPeriod", SetLastError=true)]
private static extern uint timeBeginPeriod(uint uiPeriod);
[DllImport("winmm.dll", EntryPoint="timeEndPeriod", SetLastError=true)]
private static extern uint timeEndPeriod(uint uiPeriod);
void routine()
{
Thead.Sleep(1); // May takes about 15,6ms or even longer.
timeBeginPeriod(1); // Should be set at the startup of the application.
Thead.Sleep(1); // May takes about 1, 2 or 3 ms depending on the CPU usage.
// ... time depending routines goes here ...
timeEndPeriod(1); // Should end at application shutdown.
}
As far as I know this should be already done by directx. But due this setting is a global setting, other parts of the application or other applications maybe change it. This shouldn't happen if an application sets and revokes the setting once. But somehow it seems to happen caused by any dirty programmed part or other running application.
One more thing that needs to be watched, is whether you're still using the correct position of the directx buffer, if you're skipping one chunk for any reason. In this case a resynchronization is required.