I need to be able to play a short[] array filled with 16-bit PCM values (after decode) in real time.
The array keeps getting longer and the player should be able to play it while the array is "growing".
I an writing in C#.
thanks ahead,
Ofek
EDIT:
I used the BufferedWaveProvider and the code looks like this:
Class Player
{
Private WaveFormat waveForamt=new WaveFormat(8000,16,1);
private BufferedWaveProvider bufferedWaveProvider;
public Player()
{
bufferedWaveProvider=new BufferedWaveProvider(waveFormat);
}
public void AddSamples(byte[] array)
{
bufferedWaveProvider.AddSamples(array,0,array.length);
WaveOut waveout=new WaveOut();
waveout.Init(bufferedWaveProvider);
waveout.Play();
}
}
the array that the AddSamples func gets is 32000 long I can see the parameters
bufferedWaveProvider.BufferedBytes = 32000
bufferedWaveProvider.BufferedBytes = 80000
bufferedWaveProvider.BufferedDuration = 00:00:02
bufferedWaveProvider.BufferDuration = 00:00:05
when I record the array I speak for 2 seconds and say: "one, two, three..."
The porblem is that when I play it, I just hear a quick sound, and nothing like what I said..
any ideas about what went wrong?
Thnaks, Ofek
EDIT 2:
Now my code looks like this, I call the AddSamples function from the Capture class.
after each time the waveout.Play() function is called - I clear the buffer and wait for it to fill up again. once it is full again I play it, and so on. the problem is that in the second time I play the buffer the sound starts fast and then it slows down... I heard something about working with two buffers.. do you know anything about this?
Thnaks a lot!
Here is the class that calls the player.AddSamples
You basically need to be able to set up a soundcard driver to read from an audio buffer and then write to that same buffer. The BufferedWaveProvider class in naudio could help. Take a look at http://naudio.codeplex.com/.
Edit : Your class works fine for me in an empty 'forms' project, playing 2 seconds of audio:
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
var player = new Player();
var rand = new Random();
var bytes = new byte[32000];
rand.NextBytes(bytes);
player.AddSamples(bytes);
}
}
Are you calling it in a background thread, or maybe a console app? It May be a threading or callback issue - see the 1) WaveOut section in http://naudio.codeplex.com/wikipage?title=Audio%20Output%20Driver%20Selection. You may need to pass different callback options into WaveOut()?
Related
I'm going to make a rhythm game.
music playing time is variable which change by music playing in millisecond
List is a float list which have the time(float second) I filled , let me to access it and use for compare with music playing time to instance object .
when list time = music time then instant object.
but sometimes will missing(not found value in list but value is really exist in list).
Here is my assumption.
I think List.indexOf performance are not good and thread have some delay.
Music playing time change every millisecond , a little delay from unity or others will cause missing(doesn't entering if statement)
I don't know if it's correct.
Could any one help me.
Here is my code.
IEnumerator Execute(MethodDelegate Start,MethodDelegate Stop)
{
while (true) {
int res = result.IndexOf ((float)System.Math.Round (GameObject.Find ("Music").GetComponent<AudioSource> ().time, DigitalAdjust)-TimeAdjust);
if (res!=-1) {
if (res == result.Count-2) {
Stop.Invoke ();
print ("CoroutineStop");
StopCoroutine (_Execute);
}
//execute
num=Positoin[res];
print (res);
Start.Invoke();
}
yield return null;
}
}
Thanks.
Chances are that you are correct. You might miss some if statement because you don't match the millisecond exactly. Here are some things that could help:
It you game reaches 60 FPS (the usual rate for a smooth rendering), each frame will take around 16 milliseconds. If you have events that must trigger at exact milliseconds you will miss some because your Execute function calls are separated by around 16ms (a coroutine is called once per frame).
A solution to this is remember the last time the Execute function was called and check everything in between:
private float _lastTime;
IEnumerator Execute(MethodDelegate Start,MethodDelegate Stop)
{
while (true) {
// here your code must check all the event between _lastTime and Time.time
var lastTimeMilliseconds = (int)(_lastTime*1000);
var currentTimeMilliseconds = (int)(Time.time*1000);
for(int time = lastTimeMilliseconds+1; time <= currentTimeMillisedons; time++)
{
// let's say last frame was at time 1000ms
// This frame occurs at time 1016ms
// we have to check your list for events that have occured since last frame (at time 1001, 1002, 1003, ...)
// so we have a for loop starting at 1001 until 1016 and check the list
int res = result.IndexOf ((float)System.Math.Round (time, DigitalAdjust)-TimeAdjust);
if (res!=-1)
{
if (res == result.Count-2)
{
Stop.Invoke ();
print ("CoroutineStop");
StopCoroutine (_Execute);
}
//execute
num=Positoin[res];
print (res);
Start.Invoke();
}
}
// At the end, remember the time:
_lastTime = Time.time;
yield return null;
}
}
Check the Time class, you also have access to Time.deltaTime to know the time elapsed between two frames, if that helps.
EDIT:
As you requested in comment, I added some bit of code from your example to explain better this idea. Note that I don't know what your variables do so you will have to adapt. For instance, Time.time gives that time since app start. You will likely need to adapt this use the time since you started the audio
Another important thing:
GameObject.Find must look in all the objects. It is really slow and shouldn't be used every frame
GetComponent looks for all your scripts and is slow as well. It shouldn't be used every frame.
Instead, do this:
private AudioSource _audioSource;
private void Start()
{
_audioSource = GameObject.Find ("Music").GetComponent<AudioSource> ();
}
This will retrieve the source only once. Then in your code you simply call
_audioSource.time;
I am developing C# WPF Auto Number Plate Recognition Using an OCR.
The Flow is, i am getting a pictures from a video stream MJPEG and this images should be passed to the OCR to get the plate number and other details.
The problem is : the Video stream is producing about 30 Frame/second and the CPU can't handle this much of processing also it will take around 1 Sec to process 1 frame, Also when i will get many frames on the Queue the CPU will be 70% used (Intel I7 4th G).
Can anyone suggest solution and better implementation.
//This is the queue where it will hold the frames
// produced from the video streaming(video_Newfram1)
private readonly Queue<byte[]> _anpr1Produces = new Queue<byte[]>();
//I am using AForg.Video to read the MJPEG Streaming
//this event will be triggered for every frame
private void video_NewFrame1(object sender, NewFrameEventArgs eventArgs)
{
var frameDataAnpr = new Bitmap(eventArgs.Frame);
AnprCam1.Source = GetBitmapimage(frameDataAnpr);
//add current fram to the queue
_anpr1Produces.Enqueue(imgByteAnpr);
//this worker is the consumer that will
//take the frames from the queue to the OCR processing
if (!_workerAnpr1.IsBusy)
{
_workerAnpr1.RunWorkerAsync(imgByteAnpr);
}
}
//This is the consumer, it will take the frames from the queue to the OCR
private void WorkerAnpr1_DoWork(object sender, DoWorkEventArgs e)
{
while (true)
{
if (_anpr1Produces.Count <= 0) continue;
BgWorker1(_anpr1Produces.Dequeue());
}
}
//This method will process the frames that sent from the consumer
private void BgWorker1(byte[] imageByteAnpr)
{
var anpr = new cmAnpr("default");
var objgxImage = new gxImage("default");
if (imageByteAnpr != null)
{
objgxImage.LoadFromMem(imageByteAnpr, 1);
if (anpr.FindFirst(objgxImage) && anpr.GetConfidence() >= Configs.ConfidanceLevel)
{
var vehicleNumber = anpr.GetText();
var vehicleType = anpr.GetType().ToString();
if (vehicleType == "0") return;
var imagename = string.Format("{0:yyyy_MMM_dd_HHmmssfff}", currentDateTime) + "-1-" +
vehicleNumber + ".png";
//this task will run async to do the rest of the process which is saving the vehicle image, getting vehicle color, storing to the database ... etc
var tsk = ProcessVehicle("1", vehicleType, vehicleNumber, imageByteAnpr, imagename, currentDateTime, anpr, _anpr1Produces);
}
else
{
GC.Collect();
}
}
}
What you should do is this:
First, figure out if a frame is worth processing or not. If you're using a compressed video stream, you can usually quickly read the frame's compressed size. It stores the difference between the current frame and the previous one.
When it's small, not much changed (i.e: no car drove by).
That's a low-tech way to do motion detection, without even having to decode a frame, and it should be extremely fast.
That way, you can probably decide to skip 80% of the frames in a couple of milliseconds.
Once and a while you'll get frames that need processing. Make sure that you can buffer enough frames so that you can keep recording while you're doing your slow processing.
The next thing to do is find a region of interest, and focus on those first. You could do that by simply looking at areas where the color changed, or try to find rectangular shapes.
Finally, one second of processing is SLOW if you need to process 30 fps. You need to make things faster, or you'll have to build up a gigantic buffer, and hope that you'll ever catch up if it's busy on the road.
Make sure to make proper use of multiple cores if they are available, but in the end, knowing which pieces of the image are NOT relevant is the key to faster performance here.
I have a code to stream mp3 from url. I want to show the current time of the mp3 player in UI. I tried it using WaveOut.GetPosition, but it couldn't work for me.
How can I do that?
My code:
do
{
//..codes to get url stream,to create BufferedWaveProvider
int decompressed = decompressor.DecompressFrame(frame, buffer, 0);
provider.AddSamples(buffer, 0, decompressed);
if (provider.BufferedDuration.Seconds > 2 && waveOut == null)
{
waveOut = new WaveOut();
waveOut.Init(provider);
waveOut.Play();
}
if (waveOut != null)
{
currentTime = (int)(waveOut.GetPosition() * 1d / AvgBytesPerSec);
}
}
while (bytesRead > 0 || waveOut.PlaybackState == PlaybackState.Playing);
I think the only change you need to make to your code is to use waveOut.OutputWaveFormat.AverageBytesPerSecond instead of the AvgBytesPerSec property you are currently using. The IWavePosition interface (which is what you are actually using here) "thinks" in hardware terms, so if you are using a format that has to be converted before the hardware can use it, the hardware byte rate will be different than your source byte rate.
Note that the position returned by GetPosition() is only since playback was last started. If waveOut.Stop() is called, the position is reset to 0 when playback is started again. Mapping the position to the source is up to the caller (which is really simple; just track where you last started playback on the source and add it to the position returned. Buffering makes it more complicated, but still completely doable).
I wrote the original IWavePosition interface & implementations for NAudio. It works great in the project I built it for. :)
did you try the property current time and property position?
mp3Reader = new Mp3FileReader("example.mp3");
waveOut.Init(mp3Reader);
waveOut.Play();
// reposition to five seconds in
mp3Reader.CurrentTime = TimeSpan.FromSeconds(5.0);
so mp3Reader.CurrentTime should give you what you need I think
Hope this will help you
Hi im very new to this thing so please bear with me. I am trying to convert a WAV file to a spectrogram but arent sure how to begin with. I read on something that says to read the PCM data(which i think is my WAV file) and store it in an array in the WavReader class before apply the FFT on it and converting it to GUI. Im currently using Naudio to achieve this but could not find anything that shows how to convert the WAV file to a spectrogram. Thanks
Edit :
I found out about converting PCM to FFT with Naudio and im stuck.
using (var reader = new AudioFileReader("test1.wav"))
{
// test1.wav is my file to process
// test0.wav is my temp file
IWaveProvider stream16 = new WaveFloatTo16Provider(reader);
using (WaveFileWriter converted = new WaveFileWriter("test0.wav", stream16.WaveFormat))
{
// buffer length needs to be a power of 2 for FFT to work nicely
// however, make the buffer too long and pitches aren't detected fast enough
// successful buffer sizes: 8192, 4096, 2048, 1024
// (some pitch detection algorithms need at least 2048)
byte[] buffer = new byte[8192];
int bytesRead;
do
{
bytesRead = stream16.Read(buffer, 0, buffer.Length);
converted.WriteData(buffer, 0, bytesRead);
} while (bytesRead != 0 && converted.Length < reader.Length);
}
}
Edit : I would also like to know if it is possible to compare 2 spectrograms of 2 different files programmatically.
You could also use BASS.NET library which natively provides all these features and is free.
The Visuals.CreateSpectrum3DVoicePrint Method does exactly that.
Feel free to ask for assistance if you're having a hard time using it.
EDIT : here's a quick and dirty sample
public partial class Form1 : Form
{
private int _handle;
private int _pos;
private BASSTimer _timer;
private Visuals _visuals;
public Form1()
{
InitializeComponent();
}
private void timer_Tick(object sender, EventArgs e)
{
bool spectrum3DVoicePrint = _visuals.CreateSpectrum3DVoicePrint(_handle, pictureBox1.CreateGraphics(),
pictureBox1.Bounds, Color.Cyan, Color.Green,
_pos, false, true);
_pos++;
if (_pos >= pictureBox1.Width)
{
_pos = 0;
}
}
private void Form1_Load(object sender, EventArgs e)
{
string file = "..\\..\\mysong.mp3";
if (Bass.BASS_Init(-1, 44100, BASSInit.BASS_DEVICE_DEFAULT, Handle))
{
_handle = Bass.BASS_StreamCreateFile(file, 0, 0, BASSFlag.BASS_DEFAULT);
if (Bass.BASS_ChannelPlay(_handle, false))
{
_visuals = new Visuals();
_timer = new BASSTimer((int) (1.0d/10*1000));
_timer.Tick += timer_Tick;
_timer.Start();
}
}
}
}
EDIT 2
You can provide a file name but you can also provide your own audio data using the other overload that accepts an IntPtr or use Bass.BASS_StreamCreatePush with Bass.BASS_StreamPutData.
Regarding comparing spectrograms you could do the following :
Resize the image to a smaller size, reduce information by dithering it to 8-bit (with a good algorithm however)
Compare the two images
However for comparing audio data I'd strongly suggest you to use fingerprints, it roughly does that but is much more robust than my suggestion.
Here's a fingerprinting library that is free to use :
http://www.codeproject.com/Articles/206507/Duplicates-detector-via-audio-fingerprinting
Not entirely sure it would work for small samples, though.
EDIT 3
I'm afraid I can't find the link where I've read that but that's what they do: reducing data and comparing images such as the example below (last image):
(note : not to compare at all with image 1, it's something else but just to show why using a lower resolution might give better yields)
(from http://blog.echonest.com/post/545323349/the-echo-nest-musical-fingerprint-enmfp)
Now a very basic explanation of the process:
Comparison source A:
Comparison source B: (I've just changed a region of A)
Comparison result:
(done with Paint.Net by adding the former images as layers and setting 2nd layer blending to difference instead of normal)
If the fingerprints were to be identical the resulting image would be completely black.
And by reducing data to a 8-bit image you are easing the comparison process but keep in mind you will need a good dithering algorithm.
This is one is quite good :
http://www.codeproject.com/Articles/66341/A-Simple-Yet-Quite-Powerful-Palette-Quantizer-in-C
Well it's not on par with Photoshop or Hypersnap's one (which IMO is exceptional) but that might be enough for the task.
And avoid at all costs Floyd–Steinberg dithering or something that does error diffusion.
Here some attempts on creating dithering algorithms : http://bisqwit.iki.fi/story/howto/dither/jy/
Take this with caution as I'm not an expert in the field but that's roughly how it's done.
Go to https://dsp.stackexchange.com/ and ask a few questions there, you might get useful hints on achieving this.
I'm using XAudio2 with SlimDX and I've managed to get it playing a short (~8second) wav file on loop, however as it approaches the end of the first loop, the audio begins to stutter and the stuttering continues into the next loop, getting worse and worse as time goes on.
I turned on the debug profile and in the output window I get these errors:
XAUDIO2: WARNING: Spent 5.63ms in the OnVoiceProcessingPassStart callback
XAUDIO2: WARNING: Spent 5.60ms in the OnVoiceProcessingPassStart callback
XAUDIO2: WARNING: Spent 5.59ms in the OnVoiceProcessingPassStart callback
XAUDIO2: WARNING: Spent 5.69ms in the OnVoiceProcessingPassStart callback
And these coincide with when the stuttering occurs. I am doing nothing in these callbacks (I haven't even added anything to the events), and yet it's slowing down. I've added my code below for reference:
Wave class for holding the data stream and the buffer:
public class Wave
{
public WaveStream Data { get; private set; }
public AudioBuffer Buffer { get; private set; }
public Wave(string path, bool repeating)
{
Data = new WaveStream(path);
Buffer = new AudioBuffer();
Buffer.AudioBytes = (int)Data.Length;
Buffer.AudioData = Data;
if (repeating)
{
Buffer.Flags = BufferFlags.EndOfStream;
}
else
{
Buffer.Flags = BufferFlags.None;
}
Buffer.PlayBegin = 0;
Buffer.PlayLength = 0;
Buffer.LoopBegin = 0;
Buffer.LoopCount = 100;
Buffer.LoopLength = 0;
}
}
Sound class for holding the XAudio engine and the voices, and to cover adding/removing voices:
public class Sound
{
private XAudio2 audio;
private MasteringVoice master;
private List<SourceVoice> sources;
public Sound()
{
audio = new XAudio2(XAudio2Flags.DebugEngine, ProcessorSpecifier.AnyProcessor);
master = new MasteringVoice(audio);
sources = new List<SourceVoice>();
}
public void AddSound(Wave wave)
{
SlimDX.Multimedia.WaveFormat format = wave.Data.Format;
SourceVoice source = new SourceVoice(audio, format);
source.Start();
source.SubmitSourceBuffer(wave.Buffer);
sources.Add(source);
}
}
And to run it, I use:
Wave wave = new Wave("Music2/untitled.wav", true);
Sound sound = new Sound();
sound.AddSound(wave);
Rather embarrassingly this one's my own fault. I had a D3D buffer being recreated and destroyed every frame that I'd forgotten to change to a dynamic buffer. This was causing my memory usage to balloon to several gigabytes in size, which meant there probably wasn't enough to allocate for the music.
Once I removed the memory leak, the music sounded fine.
I'm not sure if the XAudio2 exception is very well-worded though...