I'm trying to play a wave file with C# using the portaudiosharp bindings for the portaudio C library and am having trouble envisioning the proper way to go about doing it. I will paste the code that I am using at the moment. It works somewhat but I don't think it is the proper way of doing things.
This is my callback function:
public PortAudio.PaStreamCallbackResult myPaStreamCallback(
IntPtr input,
IntPtr output,
uint frameCount,
ref PortAudio.PaStreamCallbackTimeInfo timeInfo,
PortAudio.PaStreamCallbackFlags statusFlags,
IntPtr userData)
{
short[] mybuffer = (short[])myQ.Dequeue();
Marshal.Copy(mybuffer, 0, output, (int)frameCount * 2);
return PortAudio.PaStreamCallbackResult.paContinue;
}
And then I have a 'main loop':
PortAudio.Pa_Initialize();
IntPtr stream;
IntPtr userdata = IntPtr.Zero;
PortAudio.Pa_OpenDefaultStream(out stream, 1, 2, 8,
48000, NUM_SAMPLES/2, new PortAudio.PaStreamCallbackDelegate(myPaStreamCallback), userdata);
PortAudio.Pa_StartStream(stream);
while (readerPosition < reader.Length)
{
short[] qBuffer = new short[NUM_SAMPLES];
read = reader.Read(buffer, 0, NUM_SAMPLES * 2); //read a block out from my wave file
Buffer.BlockCopy(buffer, 0, qBuffer, 0, read); //copy them to the short buffer
myQ.Enqueue(qBuffer);
readerPosition += read;
}
while(PortAudio.Pa_IsStreamActive(stream) == 0)
{
//this while loop never gets entered -- why??
Console.WriteLine("waiting");
}
System.Threading.Thread.Sleep(5000); //need this so that the callback function fires
PortAudio.Pa_StopStream(stream);
I have tried to implement a FIFO buffer but I think I may have done it in a silly way as basically what happens is the queue gets filled up until there are no more samples left to fit in there and only then does the PA callback start firing.
What is the better way of doing this? How do I make my main loop yield so the callback function can fire without having to sleep?
I am using a NAudio wavreader to read from a wave file but I don't think that's important. I can post more details about that if it is though.
A few obvious things:
You should fill your ring buffer before calling StartStream()
You want your main loop to keep the buffer full by writing data to it whenever it isn't full. You could do this the wrong way by polling and sleeping. If the queue is big enough you can sleep for a second at a time and the overhead won't be so big.
The "right" way to do it is to use an Event object and signal it from the callback every time the queue is becomes "not full". The main() loop blocks on that Event with WFSO and wakes whenever it can write data to the queue. (hint: use an auto-reset Event).
If all you want to do is play a soundfile you could use PA's WriteStream() API which does all this internally.
Other notes:
It is non-trivial to write a correct atomic FIFO queue. You haven't shown your code for this.
Your callback doesn't deal with the case where the queue is empty. In that case it should probably output silence.
You might not want to be newing up a new buffer for every block. Consider returning used blocks to the main thread via a second queue and reusing them.
You probably want to bound the size of the queue (3-5 seconds of audio is more than enough for most scenarios) -- this is what I mean by "not full" above. Another way to think about this is in terms of a high watermark: PA callback drains the buffer while it is non-empty, main() fills the buffer while it is less full than the (e.g. 5 second duration) watermark. Callback wakes main whenever the buffer is lower than the watermark.
Related
When I query
fileSystemWatcher.InternalBufferSize
It will give the total Internal buffer size allocated to the Watcher. But I want to know (during debugging) how much buffer size for the Watcher is left and can be used and when I use the above statement in the Event handler method (say for the write operation) it always gives me the total buffer allocated size to the Watcher. Is there any way to obtain the remaining size of the buffer?
Other Questions:
From this answer, it is clear that event is handled on the separate thread than the thread which received the event. Suppose we have many concurrent events coming for a single Watcher which is watching the file. What I think (correct me if I am wrong) the main thread which received the event information will spawn a new thread for each event and processing of events will happen on different threads. So I want to ask:
Will the main thread wait to finish the processing of all the events?
Which thread will clear the internal buffer associated with the Watcher and when?
I have read at lots of places that the handler method should take as minimum time as possible or we can get InternalBufferOverflow Exception.
So, is it safe to assume that the Internal Buffer for the Watcher is only cleaned up when the thread(s) (I can't say one or all, but want to ask from you) which are processing the handler method has processed the method?
No, you can't know how much buffer is left.
It is an implementation detail hidden in an internal class called FSWAsyncResult. If you get hold of an instance of that class and the buffer byte array it contains you still can't reliably answer how much space is left as that byte array only acts as reserved memory for the result of a call to ReadDirectoryChangesW
Find at the bottom of this answer a stripped down, reverse engineered version version of watching a folder for filechanges. Its logic and code matches what you'll find in the real FileSystemWatcher. I didn't bother to replace the magic constants with their proper meaning. It just works™. Don't forget to change the build setting unsafe as the code fiddles with pointers and native structures a lot. And I stripped out all error handling ...
If you follow below code, you'll notice that there is only one place where the byte[] buffer is created, and that only happens once. That same buffer is re-used. Reading the documentation, blogs and worker and I/O threads I understand that ReadDirectoryChangesW is used to issue a callback in an I/O completion fashion. It doesn't matter much for the managed world, that is just another thread.
The callback is scheduled on managed threadpool thread. Sometimes you'll get the same managed id you had before, when it is busy you get several. On that thread CompletionStatusChanged is executed. And that method is responsible for processing all the events that are present in the current byte buffer. Notice that I included a sizeused variable so you can see the actual size of valid data that was present in the buffer. For each event it found it raises/calls the subcribers of the events synchronously (so on the same thread). Once that is complete it calls Monitor again with the same byte[] buffer it just processed. Any file changes during the time CompletionStatusChanged is executing are kept by the OS and send the next time CompletionStatusChanged is called.
tl;dr;
Here is a recap of the answers to your questions:
... I want to know (during debugging) how much buffer size for the Watcher is left and can be used
There is only one buffer used and it makes no sense to know how much is used or how much is left. Once your eventhandlers are called the buffer is reset and starts at 0 again. It raises an exception when there are more events then the byte buffer can handle.
Will the main thread wait to finish the processing of all the events?
The OS will issue an asynchronous callback via an IOCompletionPort but that will manifest itself as normal managed threadpool thread. That thread will handle all events in the current buffer and call the eventhandlers.
Which thread will clear the internal buffer associated with the Watcher and when?
The thread that executes the CompletionStatusChanged method. Notice in my testing the buffer was never cleared (as in filled with zeroes). Data was just overwritten.
I have read at lots of places that the handler method should take as minimum time as possible or we can get InternalBufferOverflow Exception. So, is it safe to assume that the Internal Buffer for the Watcher is only cleaned up when the thread(s) (I can't say one or all, but want to ask from you) which are processing the handler method has processed the method?
You should keep your processing as short as possible because there is only one thread that will call all eventhandlers and in the end it has to call ReadDirectoryChangesW again. During this time it will keep track of filechanges. When those filechange events don't fit in the buffer it will raise an InternalBufferOverflow the next time the completion method is called.
Setup
A simple console app, with a ReadLine to keep it running while waiting for events.
static object instance = new object(); // HACK
static SafeFileHandle hndl; // holds our filehandle (directory in this case)
static void Main(string[] args)
{
// the folder to watch
hndl = NativeMethods.CreateFile(#"c:\temp\delete", 1, 7, IntPtr.Zero, 3, 1107296256, new SafeFileHandle(IntPtr.Zero, false));
// this selects IO completion threads in the ThreadPool
ThreadPool.BindHandle(hndl);
// this starts the actual listening
Monitor(new byte[4096]);
Console.ReadLine();
}
Monitor
This method is responsible for creating the Native structures and an instance of a helper class to act as IAsyncResult implementation.
This method also calls ReadDirectoryChangesW and it chooses the combination of parameters that sets it up for asynchronous completion, with IOCompletinPorts. More background on those options can be found in Understanding ReadDirectoryChangesW - Part 1
static unsafe void Monitor(byte[] buffer)
{
Overlapped overlapped = new Overlapped();
// notice how the buffer goes here as instance member on AsyncResult.
// Arrays are still Reference types.
overlapped.AsyncResult = new AsyncResult { buffer = buffer };
// CompletionStatusChanged is the method that will be called
// when filechanges are detected
NativeOverlapped* statusChanged = overlapped.Pack(new IOCompletionCallback(CompletionStatusChanged), buffer);
fixed (byte* ptr2 = buffer)
{
int num;
// this where the magic starts
NativeMethods.ReadDirectoryChangesW(hndl,
new HandleRef(instance, (IntPtr)((void*)ptr2)),
buffer.Length,
1,
(int)(NotifyFilters.FileName | NotifyFilters.LastAccess | NotifyFilters.LastWrite | NotifyFilters.Attributes),
out num,
statusChanged,
new HandleRef(null, IntPtr.Zero));
}
}
CompletionStatusChanged
The CompletionStatusChanged method is called by the OS as soon as a filechange is detected. In the Overlapped structure we will find, after unpacking, our earlier ResultAsync instance with a filled buffer. The remainder of the method then decodes the byte array by reading the offset for any following events in the array as well as flags and the filename.
// this gets called by a ThreadPool IO Completion thread
static unsafe void CompletionStatusChanged(uint errorCode, uint numBytes, NativeOverlapped* overlappedPointer)
{
var sb = new StringBuilder();
Overlapped overlapped = Overlapped.Unpack(overlappedPointer);
var result = (AsyncResult) overlapped.AsyncResult;
var position = 0;
int offset;
int flags;
int sizeused = 0;
string file;
// read the buffer,
// that can contain multiple events
do
{
fixed (byte* ptr = result.buffer)
{
// process FILE_NOTIFY_INFORMATION
// see https://msdn.microsoft.com/en-us/library/windows/desktop/aa364391(v=vs.85).aspx
offset = ((int*)ptr)[position / 4];
flags = ((int*)ptr + position / 4)[1];
int len = ((int*)ptr + position / 4)[2];
file = new string((char*)ptr + position / 2 + 6, 0, len / 2);
sizeused = position + len + 14;
}
sb.AppendFormat("#thread {0}, event: {1}, {2}, {3}, {4}\r\n", Thread.CurrentThread.ManagedThreadId, position, offset, flags, file);
// in the real FileSystemWatcher here the several events are raised
// so that uses the same thread this code is on.
position += offset;
} while (offset != 0);
// my own logging
sb.AppendFormat(" === buffer used: {0} ==== ", sizeused);
Console.WriteLine(sb);
// start again, reusing the same buffer:
Monitor(result.buffer);
}
}
Helper methods
The AsyncResult implements IAsyncResult (all empty) and holds the member to the byte array buffer.
The NativeMethods are exactly what they are called: entry points for Native calls into the WinAPI.
class AsyncResult : IAsyncResult
{
internal byte[] buffer;
// default implementation of the interface left out
// removed default implementation for brevity
}
static class NativeMethods
{
[DllImport("kernel32.dll", BestFitMapping = false, CharSet = CharSet.Auto)]
public static extern SafeFileHandle CreateFile(string lpFileName, int dwDesiredAccess, int dwShareMode, IntPtr lpSecurityAttributes, int dwCreationDisposition, int dwFlagsAndAttributes, SafeFileHandle hTemplateFile);
[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public unsafe static extern bool ReadDirectoryChangesW(SafeFileHandle hDirectory, HandleRef lpBuffer, int nBufferLength, int bWatchSubtree, int dwNotifyFilter, out int lpBytesReturned, NativeOverlapped* overlappedPointer, HandleRef lpCompletionRoutine);
}
I have two high speed USB3 cameras (Ximea) and want to code an application for image recording. Framerates are up to 500fps at VGA resolution but I also want to use the 2Mpx resolution at 170fps.
Their .Net SDK tells me that I should simply "get" the images in a loop.
My problem is that I have no idea how to get the images and save them while still showing a live preview. Everytime I add some code to actually update the picturebox the frame rate drops drastically.
At the moment I utilize a recording function that is called with
Task.Run(() => Record());
and inside the Record() I have a loop getting the bitmaps
while(record == true)
{
Camera.GetImage(out myImage, timeout); //From Ximea .Net SDK
Info = Camera.GetLastImageParams();
Timestamp = Info.GetTimeStamp();
ThreadPool.QueueUserWorkItem(state => SaveImage(myImage, filepath, Timestamp));
}
with the SaveImage being
private void SaveImage(Bitmap myImage, string filepath, double Timestamp)
{
try
{
lock(myImage)
{
myImage.Save(filepath + Timestamp.ToString("0.00000") + ".tif");
}
}
catch{}
}
How can I show a live preview while recording and how can I make the entire code more stable (at the moment there are some dropped frames because of "object already in use"-errors or "generic error in GDI+" at the Image.Save() call, that I skip with the try/catch statement)?
I believe you can tell the Ximea API how many image buffers you want in the incoming queue... use XI_PRM_BUFFER_POLICY and XI_PRM_BUFFERS_QUEUE_SIZE appropriately to make that queue length somewhat long.
Then, have a thread that, when activated, copies an image out of a XI_IMG struct into your own buffer. Activate that thread every n frames (based on the size of the Ximea image buffer queue size)... but don't do any memory copies in the loop that actually calls xiGetImage. You should probably block in your thread to avoid tearing (because the Ximea code could get back around to using the same buffer again if you're not fast enough at copying the data out)... but you could then dynamically adjust the number of buffers so you can finish your copy within the time you have. Also, you may consider copying the image data to ANOTHER buffer if you're doing something that takes a long time...
pseudo-code (sorry, it's C-ish):
// sync objects and a global image buffer pointer
CRITICAL_SECTION cs;
void *buf;
HANDLE ev;
int CopyImageThreadProc(...)
{
while (true)
{
if (WaitOnSingleObject(ev) == WAIT_OBJ_0)
{
EnterCriticalSection(cs);
// copy the image data at buf where ever you want
LeaveCriticalSection(cs);
}
}
}
int main(...)
{
// set up ximea api with appropriate buffering
// create event and critsec, start thread
while (!done)
{
XI_IMG img;
xiGetImage(dev, 10, &img);
// every 15 frames, tell your thread to go...
// if you find that the critsec is causing a hiccup, you can adjust this
// but remember to adjust the queue length, too
// if you change this to TRY entercriticalsection, you can determine that
if ((img.acq_nframe % 15) == 0)
{
EnterCriticalSection(cs);
buf = img.bp;
SetEvent(ev);
LeaveCriticalSection(cs);
}
}
// clean up
}
Add each captured frame to a queue, then have a worker thread that takes those images, one at a time, and saves them. Trying to write multiple images to disk at the same time will most likely be slower. Also, always Dispose of any GDI objects or you will run into trouble really fast. I would think not doing that is what is giving you the exceptions.
As for showing the images, make sure you are not trying to display every image. Your monitor most likely runs at 60 Hz, so anything faster than that will be a waste. I also suspect that (with the performance of GDI), you won't necessarily be able to achieve even that. So I suggest you have a second queue with images to display, and if you see that queue getting too large, your program will need to slow down a bit and not push as many frames to the queue.
Edit: And of course, as #Franck mentions, if your disk can't keep up, your queue/buffer will fill up quickly. Compressing the images might help, assuming they have suitable content for compression and that your processor can keep up.
Edit: What you need is a producer-consumer pattern. There are many ways of doing this, but one might be something like:
// blocking collection
private BlockingCollection<Bitmap> m_Queue = ...
// camera thread
while( run )
{
var bitmap = GrabFrame();
m_Queue.Add( bitmap );
}
// worker thread
try
{
while( true )
{
// Take() will block if the queue is empty
var bitmap = m_Queue.Take();
bitmap.Save( ... );
bitmap.Dispose();
}
catch( InvalidOperationException )
{
// you'll end up here if you call `m_Queue.CompleteAdding()`
// (after the queue has been emptied, of course)
}
As for displaying the images, you could probably use something similar, with the addition of having some code that determines if it's time to push a new image or not.
I am pretty new to coding with some experience in ASM and C for PIC. I am still learning high level programming with C#.
Question
I have a Serial port data reception and processing program in C#. To avoid losing data and knowing when it was coming, I set a DataReceived event and loop into the handling method until there were no more bytes to read.
When I attempted this, the loop continued endlessly and blocked my program from other tasks (such as processing the retrieved data) when I continuously received data.
I read about threading in C#, I created a thread that constantly checks for SerialPort.Bytes2Read property so it will know when to retrieve available data.
I created a second thread that can process data while new data is still being read. If bytes have been read and ReadSerial() has more bytes to read and the timeout (restarted every time a new byte is read from the serial) they can still be processed and the frames assembled via a method named DataProcessing() which reads from the same variable being filled by ReadSerial().
This gave me the desired results, but I noticed that with my solution (both ReadSerial() and DataProcessing() threads alive), CPU Usage was skyrocketed all the way to 100%!
How do you approach this problem without causing such high CPU usage?
public static void ReadSerial() //Method that handles Serial Reception
{
while (KeepAlive) // Bool variable used to keep alive the thread. Turned to false
{ // when the program ends.
if (Port.BytesToRead != 0)
{
for (int i = 0; i < 5000; i++)
{
/* I Don't know any other way to
implement a timeout to wait for
additional characters so i took what
i knew from PIC Serial Data Handling. */
if (Port.BytesToRead != 0)
{
RxList.Add(Convert.ToByte(Port.ReadByte()));
i = 0;
if (RxList.Count > 20) // In case the method is stuck still reading
BufferReady = true; // signal the Data Processing thread to
} // work with that chunk of data.
BufferReady = true; // signals the DataProcessing Method to work
} // with the current data in RxList.
}
}
}
I can not understand completely what you are meaning with the "DataReceived" and the "loop". I am also working a lot with Serial Ports as well as other interfaces. In my application I am attaching to the DataReceived Event and also reading based on the Bytes to read, but I dont use a loop there:
int bytesToRead = this._port.BytesToRead;
var data = new byte[bytesToRead];
this._port.BaseStream.Read(data , 0, bytesToRead);
If you are using a loop to read the bytes I recommend something like:
System.Threading.Thread.Sleep(...);
Otherwise the Thread you are using to read the bytes is busy all the time. And this will lead to the fact that other threads cannot be processed or your CPU is at 100%.
But I think you don't have to use a loop for polling for the data if you are using the DataReceived event. If my undertanding is not correct or you need further information please ask.
From outside SerialPort object, it seems to make no difference what the size of the write buffer is, and whether or not it is full.
Using synchronous writing, the write method blocks until all the data has been sent and the buffer is empty.
Using async writing, the data is queued and the program continues on going. The callback method is not called until the write operation is completed and the data is out of the buffer.
The behavior of the serialport object seems to be the same regardless of how much data is in the buffer and whether or not the buffer is full. No errors seem to happen when the write buffer is full.
So, why be able to check BytesToWrite and WriteBufferSize at all? Is there any way that SerialPort behaves differently when the write buffer is full?
Buffers are a mechanism designed to allow whoever processes the buffer to do it in their own way, at their own time.
When i send data, i want it to be pushed at the maximal rate of the port, but i don't want to busywait on the port and wait for each byte to be sent before i push the next one. So you have a processing buffer that feeds the hardware, and can be passed in chunks.
As for the why you'd want to check the BytesToWrite - you are often interested to know if all your data was sent before moving on to the next thing you want to do, you might expect a response after a given period of time, you might want to know what the actual transfer rate is etc'.
The C# SerialPort.BytesToWrite property corresponds to the unmanaged Win32 COMSTAT.cbOutQue field which is described as:
The number of bytes of user data remaining to be transmitted for all write operations. This value will be zero for a nonoverlapped write.
This seems to suggest you could observe the write buffer being consumed as it is sent with async writing before the write complete callback is called.
I wanted to create a test utility that constantly sends out 0xAA out the serial port, with no gaps, forever. I don't care about RX.
I used a timer to keep the buffer full, and monitored BytesToWrite to wait below it was below threshold, before I wrote more data into the buffer.
I could have alternatively not used a timer, but refreshed the serial port in the AsyncCallback, but I wanted to do it this way for fun. You can view the label11 to see the buffer fill and empty.
Note you can get away with BeginWrite without EndWrite for a short time, but eventually you will run out resources. I am basically just putting in a dummy EndWrite.
private void checkBox2_CheckedChanged(object sender, EventArgs e)
{
timerFill.Enabled = checkBox2.Checked;
}
private void timerFill_Tick(object sender, EventArgs e)
{
GenerateSquareWave();
}
int const bufferSize = 256;
void GenerateSquareWave()
{
int k = serialPort1.BytesToWrite;
label11.Text = k.ToString();
if (k < bufferSize)
{
byte[] data = new byte[bufferSize];
for (int i = 0; i < bufferSize; i++)
{
data[i] = 0xAA;
}
serialPort1.BaseStream.BeginWrite(data, 0, data.Length, new AsyncCallback((IAsyncResult ar) => serialPort1.BaseStream.EndWrite(ar)), null);
}
}
I'm developing a simple application to send files over TCP using the TCPListener and TCPClient classes. Here's the code that sends the file.
Stop is a volatile boolean which helps stopping the process at any time and WRITE_BUFFER_SIZE might be changed in runtime (another volatile)
while (remaining > 0 && !stop)
{
DateTime current = DateTime.Now;
int bufferSize = WRITTE_BUFFER_SIZE;
buffer = new byte[bufferSize];
int readed = fileStream.Read(buffer, 0, bufferSize);
stream.Write(buffer, 0, readed);
stream.Flush();
remaining -= readed;
// Wait in order to guarantee send speed
TimeSpan difference = DateTime.Now.Subtract(current);
double seconds = (bufferSize / Speed);
int wait = (int)Math.Floor(seconds * 1000);
wait -= difference.Milliseconds;
if (wait > 10)
Thread.Sleep(wait);
}
stream.Close();
and this is the code that handles the receiver side:
do
{
readed = stream.Read(buffer, 0, READ_BUFFER_SIZE);
// write to .part file and flush to disk
outputStream.Write(buffer, 0, readed);
outputStream.Flush();
offset += readed;
} while (!stop && readed > 0);
Now, when the speed is low (about 5KBps) everything works ok but, as I increase the speed the receiver size becomes more prone to raise a SocketException when reading from the stream. I'm guessing it has to do with the remote socket being closed before all data can be read, but What's the correct way to do this? When should I close the sending client?
I haven't found any good examples of file transmission on google, and the ones that I've found have a similar implementation of what I'm doing so I guess I'm missing something.
Edit: I get this error "Unable to read data from the transport connection". This is an IOException whose inner exception is a SocketException.
I've added this in the sender function, still I get the same error, the code never reaches the stream.close() and of course the tcpclient never really get closed... so I'm completely lost now.
buffer = new byte[1];
client.Client.Receive(buffer);
stream.Close();
Typically you want to set the LINGER option on the socket. Under C++ this would be SO_LINGER, but under windows this doesn't actually work as expected. You really want to do this:
Finish sending data.
Call shutdown() with the how parameter set to 1.
Loop on recv() until it returns 0.
Call closesocket().
Taken from: http://tangentsoft.net/wskfaq/newbie.html#howclose
C# sharp may have corrected this in its libraries, but I doubt it since they are built on top of the winsock API.
Edit:
Looking at your code in more detail. I see that you are sending no header across at all, so on the receiving side you have no idea of how many bytes you are actually supposed to read. Knowing the number of bytes to read of the socket makes this a much easier problem to debug. Keep in mind that shutting down the socket can still snip of the last bit of data if you don't close it properly.
Additionally having your buffer size be volatile is not thread safe and really doesn't buy you anything. Using stop as a volatile is safe, but don't expect it to be instant. In other words the loop could run several more times before it gets the updated value of stop. This is especially true on multiprocessor machines.
Edit_02:
For the TCPClientClass you want to do the following (as far as I can tell without having access to a C# at the moment).
// write all the bytes
// Then do the following
client.client.Shutdown(Shutdown.Send) // This assumes you have access to this protected member
while (stream.read(buffer, 0, READ_BUFFER_SIZE) != 0);
client.close()