I'm building a fairly simple sampler in C#. I've got the basic sound structure down (sounds pitched, stopping sounds mid play, etc). But my problem arises when I try to record and play the sounds the user inputs.
When recording, I save the sound into a dictionary with the starting time as the key (Class 'time' starts from 0), along with the length of the sound.
When playing back the recorded sound, I'm currently using a timer to simulate time in the system. I set the timer interval to the time difference between the current sound and the next sounds every time I play a sound.
It mostly starts out fine, but the sound usually goes completely out of sync, sounds are cut short or start too late, etc etc, I assume the problem is with my use of a timer, but I have no idea of another way of doing it.
I'm using Bass.Net for the sounds.
What timers are you using? Most timers in .NET would be too inaccurate for this task, you should look into P/Invoking Multimedia Timers (see this for example) or building your own using Stopwatch.
Related
So, Unity does not do a lot of rhythm games on android. I decided to find out why, and program one as assignment (the basics of it anyway). My most important hurdle is user input. As we know that input is based on frame rate in unity, and a music game (i assume) would prefer the smallest possible delay between button press and action.
If we look at music, at around 15 to 20ms of delay, the human ear hears something is "off beat".
I heard Android Unity games run at 30FPS (since 60FPS sucks the battery dry), simple math indicates: 1000/30 = 33ms per frame. Calculating in the 15ms we can probably not notice, we are at 18ms of possible disaster. assuming we always reach this 30FPS at any given moment.
When i get input from a user, i can play a sound on that input on the exact same frame. However, we could be 18ms off.
Now there is a way to get DIRECT controls from mouse and keyboard, which uses OnGui() instead of Update(), to get events of the keyboard or mouse clicks on the spot. The problem is, android probably doesn't work with this, (this doesn't work for gamepads either) and the methods sounds downright strange, especially when we try to play sounds from the OnGui() method.
My question:
What would you do, and why? Should we just accept the possible 18ms off, and assume we reach the 30FPS, or should we look for a reliable way to get input directly, instead of waiting for an update to come by?
Thanks for any insight you can give me, i have not found any articles on this that are useful just yet.
-Smiley
EDIT
I just did some basic testing with a metronome, running at 100FPS in the editor (which should be 10ms per frame) tapping my spacebar on a metronome inside unity. The results i got were just horrible.
Tapping rapidly: I get as close as 20ms to my metronome tick, but nothing closer.
Tapping on the beat: I got at least 200ms off my target tick. Unless i am confused with this rhythm, this is just wrong.
Currently i use Debug.Log to get my test data to the log. Can anyone please confirm for me if this may be the cause (causes some long delay? i know debug isn't that optimized), or is the timing actually that bad on it?
Thanks in advance,
-Smiley
First, I'd like to add a few things to your comments and analysis:
There is a hell of a lot more to measuring the latency between the tactile input and what your eyes perceive.
Probably the biggest one I'm seeing missing in your tests is the latency between the graphics card and the PC monitor you're testing on. Many common LCD monitors these days have a latency of between 15-30ms in processing lag. I don't know how much this relates to mobile screens and hardware, but I would suggest you take the time to perform additional tests on your target hardware before drawing further conclusions.
To more directly answer your question:
I would continue to use Unity however I would continue to research the best methods of taking the input and feeding it back to the player as fast as possible. In the above comments #rutter has pointed you to what appears to be a pretty good thread on the issue.
Of specific note I would look at using FixedUpdate() to decouple the framerate of the game from the input processing speed.
I think it is also worth putting time into researching the psychology of the perception of latency. For example, if your game is a sort of Guitar Hero game of matching the playing song, you could simply take into account the lag you know is there and in your game logic take that into account when checking input.
I think you are over-complicating this, and that the accuracy issue is no where near as bad as you think.
People usually hit the buttons a little early in order to sync what they are seeing and hearing.
It also depends alot on if you have some kind of scrolling display that they are trying to match up to... if the display is scrolling smoothly at 30fps (without big jumps) they they are still able to make their timing presses fairly accurate.
I would surmise that although people can hear when their timing is off, their actual timing of hitting the buttons at exactly the right time is not that accurate anyway.
Here is one other simple solution... which I think is what rock band and guitar hero often do...
You start playing the note/sound at the correct time anyway.... then change it to a broken sound if you detect they missed it or goofed up.
I'm currently building a game in windows phone 7 using xna
i'm trying to get beat per minute from song that played in background song,
i also not quite sure if what i want is BPM, what i want is something like pace or tempo in music, faster the pace ,faster the sprites is moving. What i'm thinking right now, BPM is how much a frequency from music hits a range of defined constant, e.g 20 Mhz - 30 MHz,
Feel free to correct me if i'm wrong, i'm not really familiar with audio thing, i have tried using VisualizationData from MediaLibrary XNA, but after some googling they said that VisualizationData doesn't work with WP7, i also had tried it and the output is 256 length float array contains 0 value,,or if i could do some fft with it,i'll give it a try
Thank you...
Like you were saying, as for the beats you can't get it directly but you'll have to interpret this data. If you personally can preprocess this music and ship it with your title it would be your best bet
In XNA you really only have MediaPlayer.GetVisualizationData to work with. There isn't anything built in that allows you to predetermine this sort of thing. It's used like the following and gets you information about the different frequencies that are playing.
MediaPlayer.IsVisualizationEnabled = true;
VisualizationData visData = new VisualizationData();
MediaPlayer.GetVisualizationData(visData);
So how do you take this frequency stuff and make it worthwhile for your application? There's a great breakdown of how you can do this that's on the App Hub forums in this thread called "Audio Analysis" in the reply by jwatte. Essentially, you're going to look at the low frequencies and try to figure out when the beats are coming in. Nothing perfect, but hopefully you'll get something that you approve of.
Good luck!
When I am using the BackgroundAudioPlayer in my Windows Phone 7 application, it takes a lot of time to load the first time I want to play a song. Is there any way of preinitializing the BackgroundAudioPlayer before playing the first track, so that when I start playing, it starts right along? I have googled it, but no luck. I am just using BackgroundAudioPlayer.Instance when I e.g. want to play, pause, stop etc an audiotrack. Is there something other i could do to fix this?
You could just call BackgroundAudioPlayer.Instance.Stop(); in your App constructor and then discard the first occurance of UserAction.Stop in the OnUserAction method in your implementation of AudioPlayerAgent
This might be looking at this from a simple angle, but could you not call play and then instantly pause it until you are ready to play?? I'm not hugely familiar with the control but looking here:
http://msdn.microsoft.com/en-us/library/microsoft.phone.backgroundaudio.backgroundaudioplayer_members(v=vs.92).aspx
Upfront you could possibly check BufferingProgress and PlayerState to check when the track is ready to play, and then pause until you're ready to continue.
It's a bit brute force but may work. Worth a try?
I am building a small MIDI composer application, using the Stanford Library. It is almost complete, except for a small problem: sound vanishes after the NoteOn command, but I want notes to keep on playing until I release the button.
With MIDI, you should hear the note continue forever until you send a NoteOff command. If you're hearing a note begin but then fade out (without having sent a NoteOff message), then it might be that the channel is set to an instrument that naturally fades out on its own - like a marimba or steel drum sound.
That would be down to the MIDI device that you are addressing. You might be able to tell it to increase the sustain level of the envelope via MIDI NRPN or SysEx, but this would be device specific. Generally a string or pad voice will maintain a high sustain, whereas a piano or other percussive sound will consistently fade to zero regardless of sustain level.
So, you are saying that when you play a note it decays normally (as if you head down a piano key) and you don't want that?
That is a feature of the synthesizer and has nothing to do with MIDI. You will need to pick a patch that doesn't do this. Organ patches should work fine.
If instead you are saying that your note stops immediately (as if you hit the key and immediately released) then you have something sending a note-off command or a note command with 0 velocity. Check to make sure you are in fact sending 7-bit values for velocity and what not.
i have an application that records a date/time when the space bar is pressed
unfortunately, the window doesn't always have focus, for various reasons. eg virus checker popup, itunes update, windows update, etc
so what i though would be a cool idea is to use a joystick button, then regardless of which window has focus, i can always detect the button press event and record the date/time it was pressed.
now i had a look at a few game tutorials, but they seem to have the joystick polling tied into the screen refresh event, because in a game i guess if the computer is slow and the screen doesn't refresh often, having the joystick button pressed is going to make no difference anyway.
That doesn't really suit my need, so i am looking at running a thread that will poll the joystick every so often
My questions are
is a background thread the best solution
how often should i poll the joystick
what kind of timer should i use to determine the polling frequency.
I would like the responses to be at least equivalent to a keyboard. also need to make sure it doesn't auto-repeat if trigger is held down.
any sample code (C#) appreciated
thanks
alex
If you want to go for polling the joystick just because you are not able to capture the keyboard event, I think you need to re-work your strategy.
Even if your form doesn't have the focus there are ways that you can make sure you know of the keyboard state.
You could set up a system wide keyboard hook which can be done in C# using the SetWindowsHookEx API call.
This is delving a little in unmanaged code and Windows API, but I think it's a better idea than polling a joystick cause you won't need to add a joystick to your computer just because you need to capture datetime at the press of a spacebar.
Do a little search on google for keyboard hooks in C#. It's easier than many people believe.
I don't think using a timer object to do this is a good idea at all. Cause I believe you said you need to do it at the press of a key.
Your best bet would be to create a Timer object on your form, and tie the joystick check to the Tick event.
I've also found that running simple stuff like this every 100ms (10 times a second) has a negligible effect on CPU usage.
Use something like AHK (Auto HotKey) it is a simple language that can be compiled to an EXE and is designed for automating the keyboard and mouse, I'm not 100% sure how well it works with a joystick, but I'm sure it's possible they managed to write some stuff for the wii remotes.
Also the IRC Channel and Forums always have people willing to help if need be.