Get audio buffer in xamarin for ios - c#

I want to port an application i wrote in adobe air to xamarin. the reason is that i was not able to get the mp3s from the ipod library with the flex framework. Now i can get the list of the audio files but i have problems to extract the audio buffer from the assets. In AS3 it's a black box but it's always export the data as a byte array with with this properties:
Sampling Rate 44100Hz
32 Bits
floating point
stereo
Because i don't want to rewrite all my algorithms, i want to "convert" all the songs in the same format as in AIR. so i wrote this
AVAsset urlAsset = AVUrlAsset.FromUrl (url);
NSError error = null;
AVAssetReader reader = new AVAssetReader (urlAsset, out error);
AVAssetTrack[] tracks = urlAsset.TracksWithMediaType (AVMediaType.Audio);
NSMutableDictionary audioReadSettings = new NSMutableDictionary ();
audioReadSettings.SetValueForKey (NSNumber.FromObject(AudioFormatType.LinearPCM), AVAudioSettings.AVFormatIDKey);
audioReadSettings.SetValueForKey (NSNumber.FromBoolean(false), AVAudioSettings.AVLinearPCMIsBigEndianKey);
audioReadSettings.SetValueForKey (NSNumber.FromBoolean(true), AVAudioSettings.AVLinearPCMIsFloatKey);
audioReadSettings.SetValueForKey (NSNumber.FromBoolean(false), AVAudioSettings.AVLinearPCMIsNonInterleaved);
audioReadSettings.SetValueForKey ((NSNumber)32, AVAudioSettings.AVLinearPCMBitDepthKey);
/*
audioReadSettings.SetValueForKey ((NSNumber)2, AVAudioSettings.AVNumberOfChannelsKey);
audioReadSettings.SetValueForKey ((NSNumber)44100.0, AVAudioSettings.AVSampleRateKey);
*/
AVAssetTrack track = tracks[0];
AVAssetReaderTrackOutput outputReader = AVAssetReaderTrackOutput.FromTrack (track, audioReadSettings);
if (!reader.CanAddOutput(outputReader)){
return;
}
reader.AddOutput(outputReader);
outputReader.Dispose();
double sampleRate = 0;
int channelCount = 0;
foreach (CMFormatDescription formatDescription in track.FormatDescriptions){
if (formatDescription.AudioFormats.Length != 0){
AudioFormat format = formatDescription.AudioFormats[0];
sampleRate = format.AudioStreamBasicDescription.SampleRate;
channelCount = format.AudioStreamBasicDescription.ChannelsPerFrame;
break;
}
}
int bytesPerSample = 2 * channelCount;
outputReader = (AVAssetReaderTrackOutput)reader.Outputs[0];
if (!reader.StartReading()){
return;
}
List flatBuffer = new List();
List> buffers = new List>();
string bufferString = string.Empty;
for (int i = 0; i < 16; i++){
CMSampleBuffer sampleBuffer = outputReader.CopyNextSampleBuffer();
List buffer = extractBuffer(sampleBuffer, i, bytesPerSample, channelCount);
flatBuffer.AddRange (buffer);
sampleBuffer.Dispose();
}
and the extractBuffer methode looks like this
private List extractBuffer(CMSampleBuffer sampleBuffer, int index, int bytesPerSamples, int channelCount){
CMBlockBuffer blockBuffer = sampleBuffer.GetDataBuffer ();
int dataLength = (int)blockBuffer.DataLength;
NSMutableData data = NSMutableData.FromLength (dataLength);
blockBuffer.CopyDataBytes (0, (uint)dataLength, data.MutableBytes);
int sampleCount = sampleBuffer.NumSamples;
float[] resultBuffer = new float[sampleCount];
unsafe{
float* samples = (float*)data.MutableBytes;
for (int i = 0; i < sampleCount; i++){
float left = (float)*samples++;
float right = left;
if (channelCount == 2){
right = (float)*samples++;
}
resultBuffer [i] = (float)((left + right) / 2);
}
}
List result = new List(resultBuffer);
resultBuffer = null;
blockBuffer.Dispose ();
blockBuffer = null;
return result;
}
note that for the analysis later on I retransform the signal as a mono one.
Now i should get for the same mp3 the same data in xamarin as in adobe air, but it is not the case. I get most of the time the same values but not at the same indexes. I presume that the engineers at adobe knows what there are doing so what did I miss here?
Thanks for your help

Related

NAudio splitting wav file produces 0 second wav files

I am using NAudio to split an full Album into several audio files. The program runs smooth and theres no exceptions, but in the end, the output audio files are not listenable and they get shown as 0 seconds.
Here is my code; albumPath describes the path to a folder where the single files will be stored, filePath is the path to the input file.
private void splitWavPrep(string albumPath,string filePath)
{
MediaFoundationReader reader = new MediaFoundationReader(filePath);
int bytesPerMilliSecond = reader.WaveFormat.AverageBytesPerSecond / 1000;
for (int i = 0; i < tracks.Count; i++)
{
string trackName = tracks[i].Name;
int startMilliSeconds = (int) tracks[i].StartSeconds * 1000;
int duration;
if (i< tracks.Count - 1)
{
//if there's a next track
duration = (int)(tracks[i + 1].StartSeconds*1000) - startMilliSeconds;
}
else
{
//this is the last track
duration = (int) reader.TotalTime.TotalMilliseconds - startMilliSeconds;
}
int startPos = startMilliSeconds * bytesPerMilliSecond;
startPos = startPos - startPos % reader.WaveFormat.BlockAlign;
int endMilliSeconds = startMilliSeconds + duration;
int endBytes = (endMilliSeconds - startMilliSeconds) * bytesPerMilliSecond;
endBytes = endBytes - endBytes % reader.WaveFormat.BlockAlign;
int endPos = startPos + endBytes;
string trackPath = Path.Combine(albumPath, trackName + ".wav");
splitWav(trackPath, startPos, endPos, reader);
}
}
private async void splitWav(string trackPath, int startPos, int endPos, MediaFoundationReader reader)
{
int progress = 0;
WaveFileWriter writer = new WaveFileWriter(trackPath, reader.WaveFormat);
reader.Position = startPos;
byte[] buffer = new byte[1024];
while (reader.Position < endPos)
{
int bytesRequired = (int)(endPos - reader.Position);
if (bytesRequired > 0)
{
int bytesToRead = Math.Min(bytesRequired, buffer.Length);
int bytesRead = reader.Read(buffer, 0, bytesToRead);
if (bytesRead > 0)
{
await writer.WriteAsync(buffer, 0, bytesRead);
progress += bytesRead;
}
}
}
}
I never dealt with audio files before and I don't know about bitrates and all that stuff.
If you have an idea or a suggestion, please tell me because I'm stranded here.
You need to Dispose the WaveFileWriter for the WAV header to get formatted correctly.
Update your splitWav method:
using(WaveFileWriter writer = new WaveFileWriter(trackPath, reader.WaveFormat))
{
// your code here
}
Also, you need to use the Write method instead of WriteAsync, as Write will keep track of the number of bytes written to the data chunk

How to normalize wav files volume in C#?

I'm using NAudio to open a wav file.
After I have used the SimpleCompressor class I also must do some normalizing the volume of the file to 0db, but I have no idea how to do that.
At the moment I have this:
string strCompressedFile = "";
byte[] WaveData = new byte[audio.Length];
SimpleCompressorStream Compressor = new SimpleCompressorStream(audio);
Compressor.Enabled = true;
if (Compressor.Read(WaveData, 0, WaveData.Length) > 0)
{
//doing the normalizing now
}
How can I get the volume from the new byte array WaveData and how can I change it?
In WaveData is the entire wav file including the file header.
You definitely can change individual sample value so that it fits maximum level:
string strCompressedFile = "";
byte[] WaveData = new byte[audio.Length];
SimpleCompressorStream Compressor = new SimpleCompressorStream(audio);
Compressor.Enabled = true;
byte maxLevel = 20;
if (Compressor.Read(WaveData, 0, WaveData.Length) > 0)
{
for (int i = 0; i < audio.Length; i++)
{
if (WaveData[i] > maxLevel)
{
WaveData[i] = maxLevel;
}
}
}
I've added a loop which iterates through all the samples and if it's value is higher that maxLevel we set it to maxLevel.

Xamarin: Playing sound for a minute

I am trying to play one minute of sound. I can do that, but the way I was doing it would generate the sound buffer 1st then play it and that was taking too much time, there was a big delay. so now I am trying to generate the sound buffer while playing it.so there would be no delay. It plays for like 4 secs and then stops, but I can't click my button again for like a min. so I think the code is still running even though I can't hear anything.
here is my code:
public void play()
{
short[] buffer = new short[44100];
track = new AudioTrack (Stream.Music, 44100, ChannelOut.Mono, Encoding.Pcm16bit, 5292000, AudioTrackMode.Static);
track.SetPlaybackRate (44100);
while (n2 < 5292000) {
for (int n = 0; n < 44100; n++) {
float temp1 = (float)( Amplitude * Math.Sin ((2 * Math.PI * Frequency * n2) / 44100D));
byte[] bytes = BitConverter.GetBytes (temp1);
buffer [n] = (short)(temp1 * short.MaxValue);
n2++;
}
int buffer_number = track.Write (buffer, 0, buffer.Length);
track.Play();
}
track.Release();
}
I tried to use AudioTrackMode Stream
but when the program get to Track.play is throw a runtime error:
play() called on uninitialized AudioTrack.
so what am I doing wrong?
Here is a condensed version of how I got it to work in c# based on the android sample at:
http://marblemice.blogspot.fr/2010/04/generate-and-play-tone-in-android.html
public void playSound()
{
var duration = 1;
var sampleRate = 8000;
var numSamples = duration * sampleRate;
var sample = new double[numSamples];
var freqOfTone = 1900;
byte[] generatedSnd = new byte[2 * numSamples];
for (int i = 0; i < numSamples; ++i) {
sample[i] = Math.Sin(2 * Math.PI * i / (sampleRate/freqOfTone));
}
int idx = 0;
foreach (double dVal in sample) {
short val = (short) (dVal * 32767);
generatedSnd[idx++] = (byte) (val & 0x00ff);
generatedSnd[idx++] = (byte) ((val & 0xff00) >> 8);
}
var track = new AudioTrack (global::Android.Media.Stream.Music, sampleRate, ChannelOut.Mono, Encoding.Pcm16bit, numSamples, AudioTrackMode.Static);
track.Write(generatedSnd, 0, numSamples);
track.Play ();
}
Adjust durationand `freqOfTone``acording to your needs.

get image pixels into array

i am trying to rewrite following code from silverlight to wpf. found here https://slmotiondetection.codeplex.com/
my problem is that WritaeableBitmap.Pixels is missing from wpf. how to achieve that? i understand how it works but i started with C# like week ago.
could you please point me to right direction?
public WriteableBitmap GetMotionBitmap(WriteableBitmap current)
{
if (_previousGrayPixels != null && _previousGrayPixels.Length > 0)
{
WriteableBitmap motionBmp = new WriteableBitmap(current.PixelWidth, current.PixelHeight);
int[] motionPixels = motionBmp.Pixels;
int[] currentPixels = current.Pixels;
int[] currentGrayPixels = ToGrayscale(current).Pixels;
for (int index = 0; index < current.Pixels.Length; index++)
{
byte previousGrayPixel = BitConverter.GetBytes(_previousGrayPixels[index])[0];
byte currentGrayPixel = BitConverter.GetBytes(currentGrayPixels[index])[0];
if (Math.Abs(previousGrayPixel - currentGrayPixel) > Threshold)
{
motionPixels[index] = _highlightColor;
}
else
{
motionPixels[index] = currentPixels[index];
}
}
_previousGrayPixels = currentGrayPixels;
return motionBmp;
}
else
{
_previousGrayPixels = ToGrayscale(current).Pixels;
return current;
}
}
public WriteableBitmap ToGrayscale(WriteableBitmap source)
{
WriteableBitmap gray = new WriteableBitmap(source.PixelWidth, source.PixelHeight);
int[] grayPixels = gray.Pixels;
int[] sourcePixels = source.Pixels;
for (int index = 0; index < sourcePixels.Length; index++)
{
int pixel = sourcePixels[index];
byte[] pixelBytes = BitConverter.GetBytes(pixel);
byte grayPixel = (byte)(0.3 * pixelBytes[2] + 0.59 * pixelBytes[1] + 0.11 * pixelBytes[0]);
pixelBytes[0] = pixelBytes[1] = pixelBytes[2] = grayPixel;
grayPixels[index] = BitConverter.ToInt32(pixelBytes, 0);
}
return gray;
}
`
In order to get the bitmap's raw pixel data you may use one of the BitmapSource.CopyPixels methods, e.g. like this:
var bytesPerPixel = (source.Format.BitsPerPixel + 7) / 8;
var stride = source.PixelWidth * bytesPerPixel;
var bufferSize = source.PixelHeight * stride;
var buffer = new byte[bufferSize];
source.CopyPixels(buffer, stride, 0);
Writing to a WriteableBitmap can be done by one of its WritePixels methods.
Alternatively you may access the bitmap buffer by the WriteableBitmap's BackBuffer property.
For converting a bitmap to grayscale, you might use a FormatConvertedBitmap like this:
var grayscaleBitmap = new FormatConvertedBitmap(source, PixelFormats.Gray8, null, 0d);

c# Pitch shift of wave files

I'm currently trying to do pitch shifting of a wave file using this algorithm
https://sites.google.com/site/mikescoderama/pitch-shifting
Here my code which use the above implementation, but with no luck. The outputted wave file seems to be corrupted or not valid.
The code is quite simple, except for the pitch shift algorithm :)
It load a wave file, it reads the wave file data and put it in a
byte[] array.
Then it "normalize" bytes data into -1.0f to 1.0f format (as
requested by the creator of the pitch shift algorithm).
It applies the pitch shift algorithm and then convert back the
normalized data into a bytes[] array.
Finally saves a wave file with the same header of the original wave
file and the pitch shifted data.
Am I missing something?
static void Main(string[] args)
{
// Read the wave file data bytes
byte[] waveheader = null;
byte[] wavedata = null;
using (BinaryReader reader = new BinaryReader(File.OpenRead("sound.wav")))
{
// Read first 44 bytes (header);
waveheader= reader.ReadBytes(44);
// Read data
wavedata = reader.ReadBytes((int)reader.BaseStream.Length - 44);
}
short nChannels = BitConverter.ToInt16(waveheader, 22);
int sampleRate = BitConverter.ToInt32(waveheader, 24);
short bitRate = BitConverter.ToInt16(waveheader, 34);
// Normalized data store. Store values in the format -1.0 to 1.0
float[] in_data = new float[wavedata.Length / 2];
// Normalize wave data into -1.0 to 1.0 values
using(BinaryReader reader = new BinaryReader(new MemoryStream(wavedata)))
{
for (int i = 0; i < in_data.Length; i++)
{
if(bitRate == 16)
in_data[i] = reader.ReadInt16() / 32768f;
if (bitRate == 8)
in_data[i] = (reader.ReadByte() - 128) / 128f;
}
}
//PitchShifter.PitchShift(1f, in_data.Length, (long)1024, (long)32, sampleRate, in_data);
// Backup wave data
byte[] copydata = new byte[wavedata.Length];
Array.Copy(wavedata, copydata, wavedata.Length);
// Revert data to byte format
Array.Clear(wavedata, 0, wavedata.Length);
using (BinaryWriter writer = new BinaryWriter(new MemoryStream(wavedata)))
{
for (int i = 0; i < in_data.Length; i++)
{
if(bitRate == 16)
writer.Write((short)(in_data[i] * 32768f));
if (bitRate == 8)
writer.Write((byte)((in_data[i] * 128f) + 128));
}
}
// Compare new wavedata with copydata
if (wavedata.SequenceEqual(copydata))
{
Console.WriteLine("Data has no changes");
}
else
{
Console.WriteLine("Data has changed!");
}
// Save modified wavedata
string targetFilePath = "sound_low.wav";
if (File.Exists(targetFilePath))
File.Delete(targetFilePath);
using (BinaryWriter writer = new BinaryWriter(File.OpenWrite(targetFilePath)))
{
writer.Write(waveheader);
writer.Write(wavedata);
}
Console.ReadLine();
}
The algorithm here works fine
https://sites.google.com/site/mikescoderama/pitch-shifting
My mistake was on how i was reading the wave header and wave data. I post here the fully working code
WARNING: this code works only for PCM 16 bit (stereo/mono) waves. Can be easily adapted to works with PCM 8 bit.
static void Main(string[] args)
{
// Read header, data and channels as separated data
// Normalized data stores. Store values in the format -1.0 to 1.0
byte[] waveheader = null;
byte[] wavedata = null;
int sampleRate = 0;
float[] in_data_l = null;
float[] in_data_r = null;
GetWaveData("sound.wav", out waveheader, out wavedata, out sampleRate, out in_data_l, out in_data_r);
//
// Apply Pitch Shifting
//
if(in_data_l != null)
PitchShifter.PitchShift(2f, in_data_l.Length, (long)1024, (long)10, sampleRate, in_data_l);
if(in_data_r != null)
PitchShifter.PitchShift(2f, in_data_r.Length, (long)1024, (long)10, sampleRate, in_data_r);
//
// Time to save the processed data
//
// Backup wave data
byte[] copydata = new byte[wavedata.Length];
Array.Copy(wavedata, copydata, wavedata.Length);
GetWaveData(in_data_l, in_data_r, ref wavedata);
//
// Check if data actually changed
//
bool noChanges = true;
for (int i = 0; i < wavedata.Length; i++)
{
if (wavedata[i] != copydata[i])
{
noChanges = false;
Console.WriteLine("Data has changed!");
break;
}
}
if(noChanges)
Console.WriteLine("Data has no changes");
// Save modified wavedata
string targetFilePath = "sound_low.wav";
if (File.Exists(targetFilePath))
File.Delete(targetFilePath);
using (BinaryWriter writer = new BinaryWriter(File.OpenWrite(targetFilePath)))
{
writer.Write(waveheader);
writer.Write(wavedata);
}
Console.ReadLine();
}
// Returns left and right float arrays. 'right' will be null if sound is mono.
public static void GetWaveData(string filename, out byte[] header, out byte[] data, out int sampleRate, out float[] left, out float[] right)
{
byte[] wav = File.ReadAllBytes(filename);
// Determine if mono or stereo
int channels = wav[22]; // Forget byte 23 as 99.999% of WAVs are 1 or 2 channels
// Get sample rate
sampleRate = BitConverter.ToInt32(wav, 24);
int pos = 12;
// Keep iterating until we find the data chunk (i.e. 64 61 74 61 ...... (i.e. 100 97 116 97 in decimal))
while(!(wav[pos]==100 && wav[pos+1]==97 && wav[pos+2]==116 && wav[pos+3]==97)) {
pos += 4;
int chunkSize = wav[pos] + wav[pos + 1] * 256 + wav[pos + 2] * 65536 + wav[pos + 3] * 16777216;
pos += 4 + chunkSize;
}
pos += 4;
int subchunk2Size = BitConverter.ToInt32(wav, pos);
pos += 4;
// Pos is now positioned to start of actual sound data.
int samples = subchunk2Size / 2; // 2 bytes per sample (16 bit sound mono)
if (channels == 2)
samples /= 2; // 4 bytes per sample (16 bit stereo)
// Allocate memory (right will be null if only mono sound)
left = new float[samples];
if (channels == 2)
right = new float[samples];
else
right = null;
header = new byte[pos];
Array.Copy(wav, header, pos);
data = new byte[subchunk2Size];
Array.Copy(wav, pos, data, 0, subchunk2Size);
// Write to float array/s:
int i=0;
while (pos < subchunk2Size)
{
left[i] = BytesToNormalized_16(wav[pos], wav[pos + 1]);
pos += 2;
if (channels == 2)
{
right[i] = BytesToNormalized_16(wav[pos], wav[pos + 1]);
pos += 2;
}
i++;
}
}
// Return byte data from left and right float data. Ignore right when sound is mono
public static void GetWaveData(float[] left, float[] right, ref byte[] data)
{
// Calculate k
// This value will be used to convert float to Int16
// We are not using Int16.Max to avoid peaks due to overflow conversions
float k = (float)Int16.MaxValue / left.Select(x => Math.Abs(x)).Max();
// Revert data to byte format
Array.Clear(data, 0, data.Length);
int dataLenght = left.Length;
int byteId = -1;
using (BinaryWriter writer = new BinaryWriter(new MemoryStream(data)))
{
for (int i = 0; i < dataLenght; i++)
{
byte byte1 = 0;
byte byte2 = 0;
byteId++;
NormalizedToBytes_16(left[i], k, out byte1, out byte2);
writer.Write(byte1);
writer.Write(byte2);
if (right != null)
{
byteId++;
NormalizedToBytes_16(right[i], k, out byte1, out byte2);
writer.Write(byte1);
writer.Write(byte2);
}
}
}
}
// Convert two bytes to one double in the range -1 to 1
static float BytesToNormalized_16(byte firstByte, byte secondByte)
{
// convert two bytes to one short (little endian)
short s = (short)((secondByte << 8) | firstByte);
// convert to range from -1 to (just below) 1
return s / 32678f;
}
// Convert a float value into two bytes (use k as conversion value and not Int16.MaxValue to avoid peaks)
static void NormalizedToBytes_16(float value, float k, out byte firstByte, out byte secondByte)
{
short s = (short)(value * k);
firstByte = (byte)(s & 0x00FF);
secondByte = (byte)(s >> 8);
}
sorry to revive this but I tried that pitchshifter class and, while it works, I get crackles in the audio while pitching down(0.5f). You work out a way around that?

Categories