Threading dilemma, atleast i think it is - c#

I am trying to output audio samples, and do so with cswavplay from http://www.codeproject.com/KB/audio-video/cswavplay.aspx which in turn seem to use DllImports from winmm.dll.
I did get it to play using 8-bit samples, however it fails miserably when I try to feed it 16-bit samples. I dug through the code as best as I can and I understand it as this:
I get a pointer to a buffer to fill each time cswavplay finished playing the last buffer. It works for one iteration, it plays back one buffer, sometimes...
I get all sorts of funny exceptions, AccessViolationException just now for instance when I tried to use a buffer size of 44100 to hear more clearly how much gets played. But when I put breakpoints in various places inside the WaveOut class (part of cswavplay) it seems none of the objects it uses, like the buffers and an instance of AutoResetEvent, are still alive the second iteration. My best guess is that these problems are related to threading or GC. The exceptions seem pretty random, and I am far too inexperienced to comprehend fully what's going on.
I'm asking for either of the following:
1) Wild guesses as to what could be the problem
2) Educated guesses as to what could be the problem
3) Pointers to an alternative way of outputting sound in realtime using C#
I'm not asking for a thorough bug tracking of software i didn't write, so don't mind cswavplay...
At the end of the day, i might be doing something wrong here, but it's hard to know when i don't get a relevant exception (along the lines of BufferAllocationException or something)...
EDIT:
Thanks for all the suggestions about other sound API's. They all seem to assume a .wav file. I'm sorry for not being clear, i'm not playing .wav files, i synthesize samples in realtime.

DirectSound and for .NET the XNA framework comes to my mind. There are many very high quality samples out there how to play sound and animate graphics at the same time with .NET.

Related

Beat Detection Algorithm

I'm currently working on an idea for a game i have that involves beat detction. Th engine im working with is Unity, and I've never had any experience with audio, coding wise, so be gentle :)
I've looked at several articles and tested out several algorithms including some of my own, but none we're really successful nor accurate enough, and i feel like I've been getting something wrong this entire time.
Specifically I've tried implementing the idea's presented here:
http://archive.gamedev.net/archive/reference/programming/features/beatdetection/index.html
but with little success, i still think im skipping over something and i cant quite pinpoint it.
If someone could provide an explanation about how to make an actually accurate beat detector i would be very grateful.
EDIT:
some people were confused as to what im having trouble with. Here is my latest try at detecting beats, i still dont understand why it's so inaccurate:
http://pastebin.com/BD8y9tfz
in this i used (R1) equation in the link i posted above to compute the instant energy from the 1024 samples i took, and then i used (R3) to calculate the local average sound energy from the buffer containing all the previous instant energy calculations, then i checked if there is a significant rise in instant energy compared to the average local sound energy, if there is, it means there is a beat, if there isn't, the program continues as usual.
(stupid reputation system doesnt let me post links and pictures ): ).
Edit 2:
added implementation for R4,R5 and R6, still not working though.
added a bit of debug, and for some reason the constant is ridicolosely small, numbers like:
Constant: -103416
and Constant: -54793.28, ive got no clue why im getting these numbers, any help?

Mitigating suspension and/or termination while encoding video in an app

I have a Windows Store app with an option to export certain data in a video file format. My app is in C#, but the encoding itself is handled by dropping into a C++ library adapted from this sample by David Catuhe and is working well. The problem is that I have found is that the encoding process can take a long time when run at high quality, and if the screen times out (say, on a Surface RT) or the user switches apps, the process fails. I'm not entirely sure what the source of the failure is and am working to verify it, but even if the process were able to survive suspension without changes, I don't know how to handle being tombstoned.
I can live with the encoding being interrupted in certain situations. What I don't want is to have to start over from scratch if the app goes away for some reason.
As far as I can tell, it isn't feasible to simply close the stream without finalizing the video and resume writing to it later. In light of this, I have considered a few options, but I can't tell which, if any, might actually work. I'd be very grateful for some direction.
1) If possible, it'd be great to be able to simply close the stream and reopen it later, picking up where I left off. At the moment I haven't been able to get this to work, but if it SHOULD work I'd love to know.
2) Push the encode process to a background task, either from the start or only when tombstoned. But is there a way to pass an open stream from my app to a background task? If not, is there a way to get my app's background task to run without CPU/memory limitations at least while my app is in the foreground? Because doing a whole encode within the very tight constraints that normally bind background tasks would take years.
3) Render segments of the video progressively while the app is in the foreground and then stitch the parts together at the end. This way, if the encode is interrupted I can pick up at the most recent segment. From my reading this should be possible in theory (I think it falls under the category of remuxing, which would avoid the need to re-encode the video). But I haven't found any samples that cover this scenario, not even in C++ (which I have almost no experience with). The Transcode API doesn't seem to cover joining multiple samples. I've looked into using SharpDX to do it, but the most likely candidate for what I'd want to use (a Media Session) is only exposed for desktop apps.
4) Push the work off to either a desktop or server app. The problem is I want to have this run on Windows RT (so desktop is out) and I don't currently have a business model that can support servers capable of handling such intensive work on my customers' behalf.
So my question is, what is my best line of attack here? Is there any way to hold onto my stream across suspension? And if, as I suspect, option #3 is my best bet, do you know of any samples or guides on how to do it? Obviously C# options would be very much preferred, so I hope I am overlooking one. C++ might be OK (as it was with Mr. Catuhe's sample that got me this far), but I'm afraid I'd need some pretty specific guidance. The MSDN documentation on this, incidentally, is so high-level that I have only a vague idea of even which pieces I would need to assemble and what each requires, let alone how to write the actual program in C++.
Any help you could offer would be very much appreciated.
Unfortunately I don't have enough reputation points on SO to just comment so I have to give this as an answer.
You could consider a combination of #3 and #4. Render in segments within your app and then upload the segments for stitching together. This would bring you back into the realms of using a commodity solution to create your final output.

Frequencies and amplitudes of sound file in C#

I'm trying to create a program which gets the various "notes" in a sound file (WAV or MP3) and can get the frequency and amplitude of each. I've been searching around for this, and of course there is the problem of distinguishing individual "notes" in a music file which isn't a MIDI, but it seems that something along these lines can be done with NAudio or DirectSound. Any ideas?
Thanks!
What you are asking to do is extremely difficult.
Step one would be to convert your audio from a time domain to a frequency domain. That is, you take a number of samples, and do a Fourier transform (implemented in your software as FFT).
Next, you begin deciding what you call a note or not. This is as not as simple as picking out the loudest of the frequencies! Different instruments have different timbre, which is created by various harmonics. If you had a song of nothing but sine waves, this would be much simpler. However, you'll find that you'll start seeing notes where your ear tells you they don't exist.
Now, psychoacoustics come into play. It is entirely possible for humans to "hear" notes that do not even have a fundamental. This is particularly true in a musical context. If I were to take a trombone and start playing a scale downward, at some point, the fundamental disappears or is mostly gone. However, you will still perceive that scale as going downward, when in fact the fundamental sound has all-but disappeared. Things get really tricky at this point.
To answer your question, start with an FFT. Maybe this is sufficient for your needs. If not, begin reading the significant amount of technical literature on the subject.

How to disable output buffering in Process.StandardOutput

This question has been asked more than once before, but I have not found a satisfactory answer in any of those discussions.
I am launching a command-line process that produces a real-time measurement to STDOUT, producing a new result approximately every second. Using System.Diagnostics.Process.StandardOutput results in completely unacceptable lag (over 20 seconds) as the STDOUT data works through the 4k buffer in the Process.StandardOutput StreamReader, and there doesn't seem to be any way to get around this.
Calling Process.StandardOutput.BaseStream.Flush() doesn't work.
I've tried doing a byte-by-byte synchronous read of Process.StandardOutput, but I'm still 4k behind the actual output.
Can anyone at least verify for me that it is possible to somehow overcome all of the buffering issues I am having with redirecting STDOUT, and receive the data in my application as soon as it would have appeared in the shell window? Can I inherit from the Process class and change how the StandardOutput streamreader behaves? Do I need to be looking at raw WINAPI calls?
Somehow, this has to get done, even if I end up writing unmanaged C++ to launch the task and consume the output, and linking that in. Any help is much appreciated; I'm at my wit's end...
Edit: It appears that what I need is a .Net implementation of the "expect" libraries that are available for C/C++, Perl, Python, and Java (those are the only ones I have found so far). Does anyone know if such a beast exists?
"[I]s there a way to launch it such [that] it doesn't realize it is being redirected?" YES: that is exactly the domain of Expect. I know of no .Net implementation; it's surely feasible, though ...

using C# for real time applications

Can C# be used for developing a real-time application that involves taking input from web cam continuously and processing the input?
You cannot use any main stream garbage collected language for “hard real-time systems”, as the garbage collect will sometimes stop the system responding in a defined time. Avoiding allocating object can help, however you need a way to prove you are not creating any garbage and that the garbage collector will not kick in.
However most “real time” systems don’t in fact need to always respond within a hard time limit, so it all comes down do what you mean by “real time”.
Even when parts of the system needs to be “hard real time” often other large parts of the system like the UI don’t.
(I think your app needs to be fast rather than “real time”, if 1 frame is lost every 100 years how many people will get killed?)
I've used C# to create multiple realtime, high speed, machine vision applications that run 24/7 and have moving machinery dependent on the application. If something goes wrong in the software, something immediately and visibly goes wrong in the real world.
I've found that C#/.Net provide pretty good functionality for doing so. As others have said, definitely stay on top of garbage collection. Break up to processing into several logical steps, and have separate threads working each. I've found the Producer Consumer programming model to work well for this, perhaps ConcurrentQueue for starters.
You could start with something like:
Thread 1 captures the camera image, converts it to some format, and puts it into an ImageQueue
Thread 2 consumes from the ImageQueue, processing the image and comes up with a data object that is put onto a ProcessedQueue
Thread 3 consumes from the ProcessedQueue and does something interesting with the results.
If Thread 2 takes too long, Threads 1 and 3 are still chugging along. If you have a multicore processor you'll be throwing more hardware at the math. You could also use several threads in place of any thread that I wrote above, although you'd have to take care of ordering the results manually.
Edit
After reading other peoples answers, you could probably argue my definition of "realtime". In my case, the computer produces targets that it sends to motion controllers which do the actual realtime motion. The motion controllers provide their own safety layers for things like timing, max/min ranges, smooth accel/decelerations and safety sensors. These controllers read sensors across an entire factory with a cycle time of less than 1ms.
Absolutely. The key will be to avoid garbage collection and memory management as much as possible. Try to avoid new-ing objects as much as possible, using buffers or object pools when you can.
Of course, someone has even developed a library to do that: AForge.NET
As with any real-time application and not just C#, you'll have to manage the buffers well as #David suggested.
Not only that, there're also the XNA Framework (for things like 3D games) and you can program DirectX using C# as well which are very real-time.
And did you know that, if you want, you can do pointer manipulations in C# too?
It depends on how 'real-time' it needs to be; ie, what your timing constraints are, and how quickly you need to 'do something'.
If you can handle 'doing something' maybe every 300ms or so in .NET, say on a timer event, I've found Windows to work okay. Note that this is something I found true on multiple systems of different ages and different speeds. As always, YMMV.
But that number is awfully long for a lot of applications. Maybe not for yours.
Do some research, make sure your app responds quickly enough for your application.

Categories