We Keep Coding

Scrolling through a waveform

I have a waveform visualiser I am trying to make for some audio editing, and need to be able to scroll through the wave form. The code I'm currently using comes from this question and works after I made some modification to allow the specifying of a start audio time and end audio time:
public Texture2D PaintWaveformSpectrum(AudioClip audio, int textWidth, int textHeight, int audioStart, int audioEnd, Color col) {
Texture2D tex = new Texture2D(textWidth, textHeight, TextureFormat.RGBA32, false);
float[] samples = new float[audioLength];
float[] waveform = new float[textWidth];
audio.GetData(samples, 0);
int packSize = ((audioEnd - audioStart) / textWidth) + 1;
if (audioStart != 0) {
audioStart += packSize % audioStart;
}
int s = 0;
for (int i = audioStart; i < audioEnd; i += packSize) {
waveform[s] = Mathf.Abs(samples[i]);
s++;
}
for (int x = 0; x < textWidth; x++) {
for (int y = 0; y < textHeight; y++) {
tex.SetPixel(x, y, Color.gray);
}
}
for (int x = 0; x < waveform.Length; x++) {
for (int y = 0; y <= waveform[x] * ((float)textHeight * .75f); y++) {
tex.SetPixel(x, (textHeight / 2) + y, col);
tex.SetPixel(x, (textHeight / 2) - y, col);
}
}
tex.Apply();
return tex;
}
The issue here however, is that when I'm scrolling through the audio, the waveform changes. It does indeed scroll, but the issue is that it is now showing different values in the waveform. This is because there are significantly more samples than pixels, so there is a need to down sample. At the moment, every nth sample is chosen, but the issue is with a different start point, different samples will be chosen. Images below for comparison (additionally, here's a video. This is what I want the scroll to look like):
As you can see they are slightly different. The overall structure is there but the waveform is ultimately different.
I thought this would be an easy fix - shift the start audio value to the nearest packSize (ie, audioStart += packSize % audioStart when audioStart != 0) but this didn't work. The same issue still occurred.
If anyone has any suggestions on how I can keep the waveform consistent while scrolling it would be much appreciated.

Despite years of programming experience, I still can't seem to correctly round a number. It was as simple as that.
The line
if (audioStart != 0) {
audioStart += packSize % audioStart;
}
should be
audioStart = (int) Mathf.Round(audioStart / packSize) * packSize;
Adding 1 extra byte to waveform is also necessary as half the time the rounding will cause there to be one extra sample included. As such, waveform should be defined as:
float[] waveform = new float[textWidth+1];
This solves the issue and the samples are chosen consistently. I'm not quite sure how programs like audacity manage to get nice looking waveforms that aren't super noisy (comparison below for the same song: mine on top, audacity below) but that's for another question.

How to detect where the image content ends?

I receive images of the same size but with different amounts of information. Examples below (red borders are mine). The background is always white.
I am trying to detect where the information on the image ends - at what pixel height (and crop accordingly). In other words, find the first non-white pixel from the bottom.
Is there a better way to do this other than extract BitmapData out of Image object and loop through all the pixels?

Just to add a suggestion having looked over your images and your solution (below) and your method is fine but you may be able to improve efficiency.
The more you know about your image the better; you're confident the background is always white (according to your post, the code is a more generic utility but the following suggestion can still work); can you be confident on the furthest point in a non-white pixel will be found if the row is not empty?
For example; in your two pictures the furthest in non-white pixel on a row is about 60px in. If this is universally true for your data then you don't need to scan the whole line of the image, which would make your for loop:
for (int y = bitmap.Height - 1; y >= 0; y--) {
for (int x = 0; x < 60; x++) {
Color color = bitmap.GetPixel(x, y);
if (color.R != backColor.R || color.G != backColor.G || color.B != backColor.B) {
foundContentOnRow = y;
break;
}
}
}
(You could make it a parameter on the function so you can easily control it if needed).
Imagine for example that the first non-white row was 80px down. To find it currently you do 640 x 300 = 192,000 checks. If you could confidently say that you would know a row was blank within 100 pixels (an over-estimate based on the data presented) then this would be 100 * 300 = 30,000 checks per image.
If you always knew that the first 10 pixels of the image were always blank you could shave a little bit more off (say 3000 checks).
Musing on a setup where you knew that the first non-white pixel was between 10 and 60 pixels in (range of 50) you could find it at row 80 in 50 x 300 = 15,000 checks which is a good reduction.
Of course the downside about assumptions is that if things change your assumptions may not be valid, but if the data is going to remain fairly constant then it may be worthwhile, especially if you do this for a lot of images.

I've ended up using the following code to trim the image. Hopefully someone finds this useful.
class Program {
static void Main(string[] args) {
Image full = Image.FromFile("foo.png");
Image cropped = full.TrimOnBottom();
}
}
public static class ImageUtilities {
public static Image TrimOnBottom(this Image image, Color? backgroundColor = null, int margin = 30) {
var bitmap = (Bitmap)image;
int foundContentOnRow = -1;
// handle empty optional parameter
var backColor = backgroundColor ?? Color.White;
// scan the image from the bottom up, left to right
for (int y = bitmap.Height - 1; y >= 0; y--) {
for (int x = 0; x < bitmap.Width; x++) {
Color color = bitmap.GetPixel(x, y);
if (color.R != backColor.R || color.G != backColor.G || color.B != backColor.B) {
foundContentOnRow = y;
break;
}
}
// exit loop if content found
if (foundContentOnRow > -1) {
break;
}
}
if (foundContentOnRow > -1) {
int proposedHeight = foundContentOnRow + margin;
// only trim if proposed height smaller than existing image
if (proposedHeight < bitmap.Height) {
return CropImage(image, bitmap.Width, proposedHeight);
}
}
return image;
}
private static Image CropImage(Image image, int width, int height) {
Rectangle cropArea = new Rectangle(0, 0, width, height);
Bitmap bitmap = new Bitmap(image);
return bitmap.Clone(cropArea, bitmap.PixelFormat);
}
}

Draw a fixed size bitmap image on canvas

I am creating a card game, for this i have created a custom surface view, in which images are getting load. Since images are downloaded from internet, they are of different sizes and looks visually bad on screen. I want to achieve two things here.
Load images of fixed size or resize the images dynamically.
Draw images from bottom of screen in upward direction.
For 1st point i used CreateBitmap method but getting below exception.
java.lang.OutOfMemoryError: Failed to allocate a 1915060280 byte allocation with 4194304 free bytes and 123MB until OOM error
To fixed the issue i thought of using Glide/Picasso based on this question and this, but i found out that Glide/Picasso load images only on imageview, but i don't have any imageview, i only got a custom surfaceview inside a linearlayout.
For 2nd point i used rotation of image. Following is the code of that.
public void Render(Canvas paramCanvas)
{
try
{
// paramCanvas.DrawColor(Android.Graphics.Color.Blue);
int i = 0;
Down_Card_Gap = 0;
foreach (Cards localcard in FaceDownDeck.ToList())
{
Bitmap localimage = BitmapFactory.DecodeResource(Resources, localcard.GetImageId(context));
Bitmap rotatedimage = RotateBitmap(localimage, 180);
paramCanvas.DrawBitmap(rotatedimage, (Screen_Center_X - Card_Width / 2)+Down_Card_Gap, (Screen_Height - Card_Height), null);
// paramCanvas.DrawBitmap(localimage, (Screen_Center_X - Card_Width / 2), (Screen_Center_Y - Card_Height), null);
if (i++ == 7)
{ break; }
if (Down_Card_Gap > 0)
{
Down_Card_Gap += Card_Width / 2;
}
else
{
Down_Card_Gap -= Card_Width / 2;
}
Down_Card_Gap *= -1;
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
}
private Bitmap RotateBitmap(Bitmap localimage, float angle)
{
Matrix matrix = new Matrix();
matrix.PostRotate(angle);
matrix.PostScale(Card_Width, Card_Height);
Bitmap resized= Bitmap.CreateBitmap(localimage, 0, 0, localimage.Width, localimage.Height, matrix, true);
localimage.Recycle();
return resized;
}
I want to know if it is a right approach, or is there any better method achieve the functionality.

Load images of fixed size or resize the images dynamically.
About the fixed size and resize, you can refer to this, find decodeFile method:
protected Bitmap decodeFile(File f) {
try {
//decode image size
BitmapFactory.Options o = new BitmapFactory.Options();
o.inJustDecodeBounds = true;
BitmapFactory.decodeStream(new FileInputStream(f), null, o);
//Find the correct scale value. It should be the power of 2.
final int REQUIRED_SIZE = 150;
int width_tmp = o.outWidth, height_tmp = o.outHeight;
int scale = 1;
while (true) {
if (width_tmp / 2 < REQUIRED_SIZE || height_tmp / 2 < REQUIRED_SIZE)
break;
width_tmp /= 2;
height_tmp /= 2;
scale *= 2;
}
//decode with inSampleSize
BitmapFactory.Options o2 = new BitmapFactory.Options();
o2.inSampleSize = scale;
return BitmapFactory.decodeStream(new FileInputStream(f), null, o2);
} catch (FileNotFoundException e) {
}
return null;
}
You can see, it uses BitmapFactory.Options.inJustDecodeBounds= true to preload the bitmap, and scale the bitmap. Also you can refer to official document. Read this to compress bitmap's quality.
Except from this, you also need consider the picture cache.This talks about how to build an efficient memory cache for bitmaps.

Iterate the bitmap rows and column

i know it may be easy but i couldn't able to find a solution for this.
i need to get the next row of bitmap after using all the bitmap of current bitmap
I'm making a stenography program where I'm hiding a text file inside image.
Each character is stored inside 8 different bytes.
so after hiding text inside 1st column i need to get next column and so on.
I'm very weak in this. I tried this for 1st row, but don't know for other rows according to text length.
private void HIDE(){
if (textBox1.Text != "")
{
Bitmap bitmap = (Bitmap)pictureBox1.Image;
int next,width=0;
for (int t = 0; t < textBox1.Text.Length; t++)
{
next = 8 * t;
for (int i = 0; i < 8; i++)
{
if (i * t <= bitmap.Width/8)
{
//hiding code for 1st row
}
else
{
//hiding code for 2nd row
}
}
}
}}

How about this?
private void HIDE(){
if (textBox1.Text != "")
{
Bitmap bitmap = (Bitmap)pictureBox1.Image;
// Starting point for embedding secret
int x0=0, y0=0;
// Error checking, to see if text can fit in image
int imageSize = bitmap.Width * bitmap.Height;
if (imageSize - x0 * bitmap.Width - y0 < 8 * textBox1.Text.Length)
{
// Deal with error
}
// Ready to embed
for (int t = 0; t < textBox1.Text.Length; t++)
{
for (int i = 0; i < 8; i++)
{
// Check if y0 has exceeded image width
// so to wrap around to the new row
if (y0 == bitmap.Width)
{
x0++;
y0=0;
}
// x0, y0 are now the current pixel coordinates
//
// EMBED MESSAGE HERE
//
y0++; // Move to the next pixel for the next bit
}
}
}}
For example, if you have a width of 10, these should be the coordinates for the first two letters of your text:
Letter 1:
(0, 0)
...
(0, 7)
Letter 2:
(0, 8)
(0, 9)
(1, 0)
...
IMPORTANT NOTE: It looks like you don't hide the length of the text and the decoding process won't know how many pixels to read to retrieve the message.
If you have dealt with it somewhere else, or the decoder always knows the length, that's fine. Otherwise, you want to use a set number of pixels at a fixed location (usually the first 32), to encode in binary the length of your text. For example,
int secretLength = 8 * textBox1.Text.Length;
string binary = Convert.ToString(secretLength, 2);
For the text "Hello World", binary will be 00000000000000000000000001011000. Now you can embed these in your 32 specific pixels and the actual secret message afterwards. Remember that in this case your image must have at least 8 * TextBox1.Text.Length + 32 number of pixels to accommodate the whole secret.
Using 4 bytes for the length of the text is such an overkill, considering, above all, image size limitations. You can use fewer bytes if you can always guarantee the text size will never exceed a specific length, or a more dynamic approach, like this
Reference: Integer to binary string conversion borrowed from here.

Two image compare using win32 api c#

i have two images and i want to compare two image and want to get difference. i search google and found a link from where i copy paste the code for image comparison using win32 api.
so this is the url
http://blog.bobcravens.com/2009/04/create-a-remote-desktop-viewer-using-c-and-wcf/
here i am pasting the code.
private void button1_Click(object sender, EventArgs e)
{
Bitmap _prevBitmap = new Bitmap(#"d:\prev.jpg");
Bitmap _newBitmap = new Bitmap(#"d:\current.jpg");
Rectangle bounds = GetBoundingBoxForChanges(_prevBitmap, _newBitmap);
if (bounds == Rectangle.Empty)
{
}
Bitmap diff = new Bitmap(bounds.Width, bounds.Height);
Graphics g = Graphics.FromImage(diff);
g.DrawImage(_newBitmap, 0, 0, bounds, GraphicsUnit.Pixel);
g.Dispose();
// Set the current bitmap as the previous to prepare
// for the next screen capture.
//
diff.Save(#"d:\diff.bmp");
//return diff;
}
private Rectangle GetBoundingBoxForChanges(Bitmap _prevBitmap, Bitmap _newBitmap)
{
// The search algorithm starts by looking
// for the top and left bounds. The search
// starts in the upper-left corner and scans
// left to right and then top to bottom. It uses
// an adaptive approach on the pixels it
// searches. Another pass is looks for the
// lower and right bounds. The search starts
// in the lower-right corner and scans right
// to left and then bottom to top. Again, an
// adaptive approach on the search area is used.
//
// Note: The GetPixel member of the Bitmap class
// is too slow for this purpose. This is a good
// case of using unsafe code to access pointers
// to increase the speed.
//
// Validate the images are the same shape and type.
//
if (_prevBitmap.Width != _newBitmap.Width ||
_prevBitmap.Height != _newBitmap.Height ||
_prevBitmap.PixelFormat != _newBitmap.PixelFormat)
{
// Not the same shape...can't do the search.
//
return Rectangle.Empty;
}
// Init the search parameters.
//
int width = _newBitmap.Width;
int height = _newBitmap.Height;
int left = width;
int right = 0;
int top = height;
int bottom = 0;
BitmapData bmNewData = null;
BitmapData bmPrevData = null;
try
{
// Lock the bits into memory.
//
bmNewData = _newBitmap.LockBits(
new Rectangle(0, 0, _newBitmap.Width, _newBitmap.Height),
ImageLockMode.ReadOnly, _newBitmap.PixelFormat);
bmPrevData = _prevBitmap.LockBits(
new Rectangle(0, 0, _prevBitmap.Width, _prevBitmap.Height),
ImageLockMode.ReadOnly, _prevBitmap.PixelFormat);
// The images are ARGB (4 bytes)
//
int numBytesPerPixel = 4;
// Get the number of integers (4 bytes) in each row
// of the image.
//
int strideNew = bmNewData.Stride / numBytesPerPixel;
int stridePrev = bmPrevData.Stride / numBytesPerPixel;
// Get a pointer to the first pixel.
//
// Note: Another speed up implemented is that I don't
// need the ARGB elements. I am only trying to detect
// change. So this algorithm reads the 4 bytes as an
// integer and compares the two numbers.
//
System.IntPtr scanNew0 = bmNewData.Scan0;
System.IntPtr scanPrev0 = bmPrevData.Scan0;
// Enter the unsafe code.
//
unsafe
{
// Cast the safe pointers into unsafe pointers.
//
int* pNew = (int*)(void*)scanNew0;
int* pPrev = (int*)(void*)scanPrev0;
// First Pass - Find the left and top bounds
// of the minimum bounding rectangle. Adapt the
// number of pixels scanned from left to right so
// we only scan up to the current bound. We also
// initialize the bottom & right. This helps optimize
// the second pass.
//
// For all rows of pixels (top to bottom)
//
for (int y = 0; y < _newBitmap.Height; ++y)
{
// For pixels up to the current bound (left to right)
//
for (int x = 0; x < left; ++x)
{
// Use pointer arithmetic to index the
// next pixel in this row.
//
if ((pNew + x)[0] != (pPrev + x)[0])
{
// Found a change.
//
if (x < left)
{
left = x;
}
if (x > right)
{
right = x;
}
if (y < top)
{
top = y;
}
if (y > bottom)
{
bottom = y;
}
}
}
// Move the pointers to the next row.
//
pNew += strideNew;
pPrev += stridePrev;
}
// If we did not find any changed pixels
// then no need to do a second pass.
//
if (left != width)
{
// Second Pass - The first pass found at
// least one different pixel and has set
// the left & top bounds. In addition, the
// right & bottom bounds have been initialized.
// Adapt the number of pixels scanned from right
// to left so we only scan up to the current bound.
// In addition, there is no need to scan past
// the top bound.
//
// Set the pointers to the first element of the
// bottom row.
//
pNew = (int*)(void*)scanNew0;
pPrev = (int*)(void*)scanPrev0;
pNew += (_newBitmap.Height - 1) * strideNew;
pPrev += (_prevBitmap.Height - 1) * stridePrev;
// For each row (bottom to top)
//
for (int y = _newBitmap.Height - 1; y > top; y--)
{
// For each column (right to left)
//
for (int x = _newBitmap.Width - 1; x > right; x--)
{
// Use pointer arithmetic to index the
// next pixel in this row.
//
if ((pNew + x)[0] != (pPrev + x)[0])
{
// Found a change.
//
if (x > right)
{
right = x;
}
if (y > bottom)
{
bottom = y;
}
}
}
// Move up one row.
//
pNew -= strideNew;
pPrev -= stridePrev;
}
}
}
}
catch (Exception ex)
{
int xxx = 0;
}
finally
{
// Unlock the bits of the image.
//
if (bmNewData != null)
{
_newBitmap.UnlockBits(bmNewData);
}
if (bmPrevData != null)
{
_prevBitmap.UnlockBits(bmPrevData);
}
}
// Validate we found a bounding box. If not
// return an empty rectangle.
//
int diffImgWidth = right - left + 1;
int diffImgHeight = bottom - top + 1;
if (diffImgHeight < 0 || diffImgWidth < 0)
{
// Nothing changed
return Rectangle.Empty;
}
// Return the bounding box.
//
return new Rectangle(left, top, diffImgWidth, diffImgHeight);
}
when GetBoundingBoxForChanges() call then i am getting error and error message is Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
error occur at this code if ((pNew + x)[0] != (pPrev + x)[0])
so i am not being able to find out the reason. how to fix this error. please guide. thanks

bmNewData = _newBitmap.LockBits(...., _newBitmap.PixelFormat);
This algorithm implicitly assumes that a pixel has 4 bytes and can be addressed with an int*. It however fails to provide that guarantee. Asking for _newBitmap.PixelFormat in LockBits() is not sufficient, that just asks for the same format that the original image used. You'll get a hard crash if the images are 24bpp for example, very common.
Explicitly ask for 32bppArgb instead.

Instead of the Win32 API you could use a managed image processing library such as AForge.NET. In the documentation look for the AForge.Imaging.Filters.Difference class. It works with Bitmap objects so you will have to make minimal changes to your program.
Bitmap overlayImage;
Bitmap sourceImage;
//ToDo: Load the two images.
// Create filter.
Difference filter = new Difference(overlayImage);
// Apply the filter and return a new bitmap that is the difference between the source and overlay images.
Bitmap resultImage = filter.Apply(sourceImage);
// If you don't want a new image the you can apply the filter directly to the source image.
filter.ApplyInPlace(sourceImage);

This is the mechanism that I use to calculate image differences in C#. Note that it need to be compiled with the unsafe directive. Hope it helps:
using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using System.Drawing.Imaging;
using log4net;
namespace ImageDiff
{
public class ImageDifferences
{
private static ILog mLog = LogManager.GetLogger("ImageDifferences");
public static unsafe Bitmap PixelDiff(Image a, Image b)
{
if (!a.Size.Equals(b.Size)) return null;
if (!(a is Bitmap) || !(b is Bitmap)) return null;
return PixelDiff(a as Bitmap, b as Bitmap);
}
public static unsafe Bitmap PixelDiff(Bitmap a, Bitmap b)
{
Bitmap output = new Bitmap(
Math.Max(a.Width, b.Width),
Math.Max(a.Height, b.Height),
PixelFormat.Format32bppArgb);
Rectangle recta = new Rectangle(Point.Empty, a.Size);
Rectangle rectb = new Rectangle(Point.Empty, b.Size);
Rectangle rectOutput = new Rectangle(Point.Empty, output.Size);
BitmapData aData = a.LockBits(recta, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
BitmapData bData = b.LockBits(rectb, ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
BitmapData outputData = output.LockBits(rectOutput, ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
try
{
byte* aPtr = (byte*)aData.Scan0;
byte* bPtr = (byte*)bData.Scan0;
byte* outputPtr = (byte*)outputData.Scan0;
int len = aData.Stride * aData.Height;
for (int i = 0; i < len; i++)
{
// For alpha use the average of both images (otherwise pixels with the same alpha won't be visible)
if ((i + 1) % 4 == 0)
*outputPtr = (byte)((*aPtr + *bPtr) / 2);
else
*outputPtr = (byte)~(*aPtr ^ *bPtr);
outputPtr++;
aPtr++;
bPtr++;
}
return output;
}
catch (Exception ex)
{
mLog.Error("Error calculating image differences: " + ex.Message);
return null;
}
finally
{
a.UnlockBits(aData);
b.UnlockBits(bData);
output.UnlockBits(outputData);
}
}
}
}