We Keep Coding

How to convert base64 mpeg to AudioClip in Unity? (part 2)

(continuing to the question here... )
After some research, I have got the following:
private float[] Normalize(float[] data) {
float max = float.MinValue;
for (int i = 0; i < data.Length; i++){
if (System.Math.Abs(data[i]) > max) max = System.Math.Abs(data[i]);
}
for (int i = 0; i < data.Length; i++) data[i] = data[i] / max;
return data;
}
private float[] ConvertByteToFloat(byte[] array){
float[] floatArr = new float[array.Length / 4];
for (int i = 0; i < floatArr.Length; i++){
if (System.BitConverter.IsLittleEndian) System.Array.Reverse(array, i * 4, 4);
floatArr[i] = System.BitConverter.ToSingle(array, i * 4) ;
}
return Normalize(floatArr);
}
byte[] bytes = System.Convert.FromBase64String(data);
float[] f = ConvertByteToFloat(bytes);
qa[i] = AudioClip.Create("qjAudio", f.Length, 2, 44100, false);
qa[i].SetData(f,0);
However, all I heard was some random noise.
Someone suggested converting it to a file first:
[SerializeField] private AudioSource _audioSource;
private void Start()
{
StartCoroutine(ConvertBase64ToAudioClip(EXAMPLE_BASE64_MP3_STRING, _audioSource));
}
IEnumerator ConvertBase64ToAudioClip(string base64EncodedMp3String, AudioSource audioSource)
{
var audioBytes = Convert.FromBase64String(base64EncodedMp3String);
var tempPath = Application.persistentDataPath + "tmpMP3Base64.mp3";
File.WriteAllBytes(tempPath, audioBytes);
UnityWebRequest request = UnityWebRequestMultimedia.GetAudioClip(tempPath, AudioType.MPEG);
yield return request.SendWebRequest();
if (request.result.Equals(UnityWebRequest.Result.ConnectionError))
Debug.LogError(request.error);
else
{
audioSource.clip = DownloadHandlerAudioClip.GetContent(request);
audioSource.Play();
}
File.Delete(tempPath);
}
However, for this to work on Android devices, I will need to request Android permissions, which can discourage players to try my game. It will also create a lot of sophistication as I will need to handle cases when the player has already rejected them once so that the request dialogs won't appear again.
How can I correctly convert a base64 mp3 string into AudioClip without resorting to using the physical storage?

The code should be simple an not require any byte swapping. See following : https://docs.unity.cn/540/Documentation/ScriptReference/WWW.GetAudioClip.html
byte[] bytes = System.Convert.FromBase64String(data);
MemoryStream ms = new MemoryStream(bytes);
ms.Position = 0;
AudioClip GetAudioClip(bool threeD, bool stream, AudioType.MPEG);

Unity Converting Microphone input into Hertz

I'm working on a Unity app that has some Microphone controls. At one point, I have to convert the Microphone input into Hertz (Hz) values and show them to the user. Now, I did some research and I made the following script for this purpose:
int amountSamples = 1024;
void Start ()
{
_fSample = AudioSettings.outputSampleRate;
}
void Update() {
if (focused && Initialized) {
if (Microphone.IsRecording(selectedDevice) && recording) {
spectrumData = GetSpectrumAnalysis();
if (spectrumCurve.keys.Length <= spectrumData.Length) {
float keyTimeValue = 0;
float currentHighestKeyTime = 0;
//create a curvefield if none exists
spectrumCurve = new AnimationCurve();
for (int t = 0; t < spectrumData.Length; t++) {
spectrumCurve.AddKey(1 / spectrumData.Length + t, spectrumData[t]);
spectrumCurve.MoveKey(1 / spectrumData.Length + t, new Keyframe(1 / spectrumData.Length + t, keyTimeValue = spectrumData[t])); //update keyframe value
if (keyTimeValue > currentHighestKeyTime) {
currentHighestKeyTime = keyTimeValue;
}
}
HighestKeyTimeValue = currentHighestKeyTime;
float freqN = HighestKeyTimeValue;
float f = freqN * (_fSample / 2) / amountSamples;
Debug.Log(f); //hz
}
}
}
audioSource.volume = 1;
}
And the GetSpectrumAnalysis()
public float[] GetSpectrumAnalysis ()
{
float[] dataSpectrum = new float[amountSamples];
audioSource.GetSpectrumData (dataSpectrum, 0, FFTWindow.BlackmanHarris);
for (int i = 0; i <= dataSpectrum.Length - 1; i++)
{
dataSpectrum[i] = Mathf.Abs (dataSpectrum[i] * sensitivity);
}
return dataSpectrum;
}
Now, with this code, the Hz value should be calculated in float f, it does work but the Hz values aren't too accurate, for example, I'm getting 400-500 Hz where I should get around 880 Hz. Similarly I'm getting 130 Hz instead of 220 Hz, etc.. So, I have 2 issues: I'm getting less Hz then I should and the Hz value is jumping too much and too fast so it's not consistent even if the sound playing is constant. Any idea how to improve this code? Where did I made a mistake?
EDIT
Check my answer for the solution.

Ok, nevermind, I found the solution, maybe this will help someone stumbling across this thread, change GetSpectrumAnalysis method to this:
public float test() {
float Threshold = 0.02f;
float[] dataSpectrum = new float[amountSamples];
audioSource.GetSpectrumData(dataSpectrum, 0, FFTWindow.BlackmanHarris); //Rectangular
float maxV = 0;
var maxN = 0;
for (int i = 0; i < amountSamples; i++) {
if (!(dataSpectrum[i] > maxV) || !(dataSpectrum[i] > Threshold)) {
continue;
}
maxV = dataSpectrum[i];
maxN = i; // maxN is the index of max
}
float freqN = maxN; // pass the index to a float variable
if (maxN > 0 && maxN < amountSamples - 1) { // interpolate index using neighbours
var dL = dataSpectrum[maxN - 1] / dataSpectrum[maxN];
var dR = dataSpectrum[maxN + 1] / dataSpectrum[maxN];
freqN += 0.5f * (dR * dR - dL * dL);
}
return freqN * (_fSample / 2) / amountSamples; // convert index to frequency
}
Then just call this in the update method like this:
Text.text = test().ToString("00");
For more info check out this thread: Unity answers

Finding bitmap inside a bitmap in C# - Doesn't work on a server?

I am trying to automate something with my C# application, for which I use a bitmap detection system to detect if an icon has appeared on screen. This works perfectly on a PC. However, when I put the application on a server, it never works. I am using a Google Cloud instance with a Tesla K80, 2 vcpus running Windows server 2012.
Here is my code:
// Capture the current screen as a bitmap
public static Bitmap CaptureScreen()
{
// Bitmap format
Bitmap ScreenCapture = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
// Capture screen
Graphics GFX = Graphics.FromImage(ScreenCapture);
GFX.CopyFromScreen(Screen.PrimaryScreen.Bounds.X,
Screen.PrimaryScreen.Bounds.Y, 0, 0,
ScreenCapture.Size, CopyPixelOperation.SourceCopy);
return ScreenCapture;
}
// Find a list of all the points of a bitmap within another bitmap
public static List<Point> FindBitmapsEntry(Bitmap SourceBitmap, Bitmap SearchedBitmap)
{
#region Arguments check
if (SourceBitmap == null || SearchedBitmap == null)
throw new ArgumentNullException();
if (SourceBitmap.PixelFormat != SearchedBitmap.PixelFormat)
throw new ArgumentException("Pixel formats aren't equal.");
if (SourceBitmap.Width < SearchedBitmap.Width || SourceBitmap.Height < SearchedBitmap.Height)
throw new ArgumentException("Size of SearchedBitmap is bigger than SourceBitmap!");
#endregion
var PixelFormatSize = Image.GetPixelFormatSize(SourceBitmap.PixelFormat) / 8;
// Copy SourceBitmap to byte array
var SourceBitmapData = SourceBitmap.LockBits(new Rectangle(0, 0, SourceBitmap.Width, SourceBitmap.Height),
ImageLockMode.ReadOnly, SourceBitmap.PixelFormat);
var SourceBitmapByteLength = SourceBitmapData.Stride * SourceBitmap.Height;
var SourceBytes = new byte[SourceBitmapByteLength];
Marshal.Copy(SourceBitmapData.Scan0, SourceBytes, 0, SourceBitmapByteLength);
SourceBitmap.UnlockBits(SourceBitmapData);
// Copy SearchedBitmap to byte array
var SearchingBitmapData =
SearchedBitmap.LockBits(new Rectangle(0, 0, SearchedBitmap.Width, SearchedBitmap.Height),
ImageLockMode.ReadOnly, SearchedBitmap.PixelFormat);
var SearchingBitmapByteLength = SearchingBitmapData.Stride * SearchedBitmap.Height;
var SearchingBytes = new byte[SearchingBitmapByteLength];
Marshal.Copy(SearchingBitmapData.Scan0, SearchingBytes, 0, SearchingBitmapByteLength);
SearchedBitmap.UnlockBits(SearchingBitmapData);
var PointsList = new List<Point>();
// Searching entries, minimizing searching zones
// SourceBitmap.Height - SearchedBitmap.Height + 1
for (var MainY = 0; MainY < SourceBitmap.Height - SearchedBitmap.Height + 1; MainY++)
{
var SourceY = MainY * SourceBitmapData.Stride;
for (var MainX = 0; MainX < SourceBitmap.Width - SearchedBitmap.Width + 1; MainX++)
{
// MainY & MainX - pixel coordinates of SourceBitmap
// SourceY + SourceX = pointer in array SourceBitmap bytes
var SourceX = MainX * PixelFormatSize;
var IsEqual = true;
for (var c = 0; c < PixelFormatSize; c++)
{
// Check through the bytes in pixel
if (SourceBytes[SourceX + SourceY + c] == SearchingBytes[c])
continue;
IsEqual = false;
break;
}
if (!IsEqual) continue;
var ShouldStop = false;
// Find first equation and search deeper
for (var SecY = 0; SecY < SearchedBitmap.Height; SecY++)
{
var SearchY = SecY * SearchingBitmapData.Stride;
var SourceSecY = (MainY + SecY) * SourceBitmapData.Stride;
for (var SecX = 0; SecX < SearchedBitmap.Width; SecX++)
{
// SecX & SecY - coordinates of SearchingBitmap
// SearchX + SearchY = pointer in array SearchingBitmap bytes
var SearchX = SecX * PixelFormatSize;
var SourceSecX = (MainX + SecX) * PixelFormatSize;
for (var c = 0; c < PixelFormatSize; c++)
{
// Check through the bytes in pixel
if (SourceBytes[SourceSecX + SourceSecY + c] == SearchingBytes[SearchX + SearchY + c]) continue;
// Not equal - abort iteration
ShouldStop = true;
break;
}
if (ShouldStop) break;
}
if (ShouldStop) break;
}
if (!ShouldStop) // Bitmap is found
{
PointsList.Add(new Point(MainX, MainY));
}
}
}
return PointsList;
}
And here is how I use it:
Bitmap HighlightBitmap = new Bitmap(Resources.icon);
Bitmap CurrentScreen = CaptureScreen();
List<Point> HighlightPoints = FindBitmapsEntry(CurrentScreen, HighlightBitmap);
with this HighlightPoints[0] is supposed to give me the first point the two bitmaps (icon, screenshot) collide. But as mentioned before, it just doesn't work on the server.
Thanks in advance!
P.S. I am using the server with a RDP so it does have a visual interface to work with

How to adjust the brightness of an Image efficiently

Does anyone know of a more efficient way of adjusting the brightness of an image at runtime in UWP?
I found this question which works fine but runs terribly slow.
However, I can't find any documentation online suggesting there is an alternative method.
Here is my problematic code.
// TODO Make Image Brightness Slider quicker and more intuitive.
private WriteableBitmap ChangeBrightness(WriteableBitmap source, int increment)
{
var dest = new WriteableBitmap(source.PixelWidth, source.PixelHeight);
byte[] color = new byte[4];
using (var srcBuffer = source.PixelBuffer.AsStream())
using (var dstBuffer = dest.PixelBuffer.AsStream())
{
while (srcBuffer.Read(color, 0, 4) > 0)
{
for (int i = 0; i < 3; i++)
{
var value = (float)color[i];
var alpha = color[3] / (float)255;
value /= alpha;
value += increment;
value *= alpha;
if (value > 255)
{
value = 255;
}
color[i] = (byte)value;
}
dstBuffer.Write(color, 0, 4);
}
}
return dest;
}

This might work. I didn't test it:
private async Task<WriteableBitmap> ChangeBrightness(WriteableBitmap source, float increment)
{
var canvasBitmap = CanvasBitmap.CreateFromBytes(CanvasDevice.GetSharedDevice(), source.PixelBuffer,
source.PixelWidth, source.PixelHeight, DirectXPixelFormat.B8G8R8A8UIntNormalized);
var brightnessFx = new BrightnessEffect
{
Source = canvasBitmap,
BlackPoint = new Vector2(0, increment)
};
var crt = new CanvasRenderTarget(CanvasDevice.GetSharedDevice(), source.PixelWidth, source.PixelHeight, 96);
using (var ds = crt.CreateDrawingSession())
{
ds.DrawImage(brightnessFx);
}
crt.GetPixelBytes(source.PixelBuffer);
return source;
}
You have to reference win2d nuget

play audio periodically from a memory stream c#

I have some buffer containing samples of sinus which I want to be played from a memory stream periodically. Is there any efficient way to play it without having gaps in time? I try to make my own signal generator (I know that are some libraries providing that but I want to generate it by myself).
The platform is Windows Phone 8.1 silverlight
Update: the code is taken from this forum somewhere
public static void PlayBeep(UInt16 frequency, int msDuration, UInt16 volume = 16383)
{
var mStrm = new MemoryStream();
BinaryWriter writer = new BinaryWriter(mStrm);
const double TAU = 2 * Math.PI;
int samplesPerSecond = 44100;
{
double theta = frequency * TAU / (double)samplesPerSecond;
// 'volume' is UInt16 with range 0 thru Uint16.MaxValue ( = 65 535)
// we need 'amp' to have the range of 0 thru Int16.MaxValue ( = 32 767)
double amp = volume >> 2; // so we simply set amp = volume / 2
for (int step = 0; step < samples; step++)
{
short s = (short)(amp * Math.Sin(theta * (double)step));
writer.Write(s);
}
}
}

Here how I did it - just create a new SoundEffectInstance object and set it to the return value of SoundEffect.CreateInstance.
https://msdn.microsoft.com/en-us/library/dd940203.aspx
SoundEffect mySoundPlay = new SoundEffect(mStrm.ToArray(), 16000,AudioChannels.Mono);
SoundEffectInstance instance = mySoundPlay.CreateInstance();
instance.IsLooped = true;
instance.Play();