We Keep Coding

Adding multiple elements to a single line of a List

Hi I am trying to add sets of coordinates to a single line of a List (so x, y as a single list line). I am currently generating the x and y coordinates as I break up an image and display the chunks in a grid. I am hitting a brick wall in being able to add both x and y to the same List line. Here is the code I have so far -
float sf = 1f;
int x = 0;
int y = 0;
int width = tileImage.Width / tileNumber;
int height = tileImage.Height / tileNumber;
int placeXValue = (width / 10);
int placeYValue = (height / 10);
int placeX = placeXValue;
int placeY = placeYValue;
Rectangle tileRect = tileImage.Bounds;
tileRect.Width = width;
tileRect.Height = height;
coordinates = new List<int>();
for (int i = 0; i < tileNumber; i++)
{
for (int j = 0; j < tileNumber; j++)
{
tileRect.X = x;
tileRect.Y = y;
_spriteBatch.Draw(tileImage, new Vector2((x + placeX) * sf, (y + placeY) * sf), tileRect, Color.White, 0, new Vector2(0, 0), sf, SpriteEffects.None, 0);
placeX += placeXValue;
x += width;
coordinates.Add(x,y);
}
x = 0;
y += height;
placeX = placeXValue;
placeY += placeYValue;
}

I found out how to do what I was trying to in the question. The first thing that I did was change my - private List<int> coordinates; - to a tuple list like this - List<Tuple<int, int>> coordinates; - This then allowed me to add my x and y elements to a new list by using the - coordinates.Add(new Tuple<int, int>(x,y)); - tuple functionality. Making this change has allowed me to add my generated coordinate pairs to my list.

Using fastai (pytorch) in c# code, how to normalize Bitmap with mean and std?

A few days ago i switched from tensorflow to fastai for my c# Project. But now i am facing a problem with my normalisation. For both i use an onnx pipeline to load the model and the data.
var onnxPipeline = mLContext.Transforms.ResizeImages(resizing: ImageResizingEstimator.ResizingKind.Fill, outputColumnName: inputName,
imageWidth: ImageSettings.imageWidth, imageHeight: ImageSettings.imageHeight,
inputColumnName: nameof(ImageInputData.Image))
.Append(mLContext.Transforms.ExtractPixels(outputColumnName: inputName, interleavePixelColors: true, scaleImage: 1 / 255f))
.Append(mLContext.Transforms.ApplyOnnxModel(outputColumnName: outputName, inputColumnName: inputName, modelFile: onnxModelPath));
var emptyData = mLContext.Data.LoadFromEnumerable(new List<ImageInputData>());
var onnxModel = onnxPipeline.Fit(emptyData);
with
class ImageInputData
{
[ImageType(ImageSettings.imageHeight, ImageSettings.imageWidth)]
public Bitmap Image { get; set; }
public ImageInputData(byte[] image)
{
using (var ms = new MemoryStream(image))
{
Image = new Bitmap(ms);
}
}
public ImageInputData(Bitmap image)
{
Image = image;
}
}
After using fastai i learned, that the models get better accuracy if the data is normalized with a specific mean and standard deviation (because i used the resnet34 model it should be means { 0.485, 0.456, 0.406 } stds = { 0.229, 0.224, 0.225 } respectively).
So the pixelvalues (for each color ofc.) have to be transformed with those values to match the trainings images. But how can i achive this in C#?
What i tried so far is:
int imageSize = 256;
double[] means = new double[] { 0.485, 0.456, 0.406 }; // used in fastai model
double[] stds = new double[] { 0.229, 0.224, 0.225 };
Bitmap bitmapImage = inputBitmap;
Image image = bitmapImage;
Color[] pixels = new Color[imageSize * imageSize];
for (int x = 0; x < bitmapImage.Width; x++)
{
for (int y = 0; y < bitmapImage.Height; y++)
{
Color pixel = bitmapImage.GetPixel(x, y);
pixels[x + y] = pixel;
double red = (pixel.R - (means[0] * 255)) / (stds[0] * 255); // *255 to scale the mean and std values to the Bitmap
double gre = (pixel.G - (means[1] * 255)) / (stds[1] * 255);
double blu = (pixel.B - (means[2] * 255)) / (stds[2] * 255);
Color pixel_n = Color.FromArgb(pixel.A, (int)red, (int)gre, (int)blu);
bitmapImage.SetPixel(x, y, pixel_n);
}
}
Ofcourse its not working, because the Colorvalues can`t be negative (which i realised only later).
But how can i achive this normalisation between -1 and 1 for my model in C# with the onnx-model?
Is there a different way to feed the model or to handle the normalisation?
Any help would be appreciated!

One way to solve this problem is to switch from an onnx pipeline to an onnx Inferencesession, which is in my view simpler and better to understand:
public List<double> UseOnnxSession(Bitmap image, string onnxModelPath)
{
double[] means = new double[] { 0.485, 0.456, 0.406 };
double[] stds = new double[] { 0.229, 0.224, 0.225 };
using (var session = new InferenceSession(onnxModelPath))
{
List<double> scores = new List<double>();
Tensor<float> t1 = ConvertImageToFloatData(image, means, stds);
List<float> fl = new List<float>();
var inputMeta = session.InputMetadata;
var inputs = new List<NamedOnnxValue>()
{
NamedOnnxValue.CreateFromTensor<float>("input_1", t1)
};
using (var results = session.Run(inputs))
{
foreach (var r in results)
{
var x = r.AsTensor<float>().First();
var y = r.AsTensor<float>().Last();
var softmaxScore = Softmax(new double[] { x, y });
scores.Add(softmaxScore[0]);
scores.Add(softmaxScore[1]);
}
}
return scores;
}
}
// Create your Tensor and add transformations as you need.
public static Tensor<float> ConvertImageToFloatData(Bitmap image, double[] means, double[] std)
{
Tensor<float> data = new DenseTensor<float>(new[] { 1, 3, image.Width, image.Height });
for (int x = 0; x < image.Width; x++)
{
for (int y = 0; y < image.Height; y++)
{
Color color = image.GetPixel(x, y);
var red = (color.R - (float)means[0] * 255) / ((float)std[0] * 255);
var gre = (color.G - (float)means[1] * 255) / ((float)std[1] * 255);
var blu = (color.B - (float)means[2] * 255) / ((float)std[2] * 255);
data[0, 0, x, y] = red;
data[0, 1, x, y] = gre;
data[0, 2, x, y] = blu;
}
}
return data;
}
Also i have to use my own Softmax method on these scores to get the real probabilities out of my model:
public double[] Softmax(double[] values)
{
double[] ret = new double[values.Length];
double maxExp = values.Select(Math.Exp).Sum();
for (int i = 0; i < values.Length; i++)
{
ret[i] = Math.Round((Math.Exp(values[i]) / maxExp), 4);
}
return ret;
}
Hope this helps someone who has a similar Problem.

Trimming Bitmap Causes Cutting of Text at Bottom

I'm using the following code to remove whitespace around an image.
static Bitmap TrimBitmap(Bitmap source)
{
Rectangle srcRect = default(Rectangle);
BitmapData data = null;
try
{
data = source.LockBits(new Rectangle(0, 0, source.Width, source.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
byte[] buffer = new byte[data.Height * data.Stride];
Marshal.Copy(data.Scan0, buffer, 0, buffer.Length);
int xMin = int.MaxValue,
xMax = int.MinValue,
yMin = int.MaxValue,
yMax = int.MinValue;
bool foundPixel = false;
// Find xMin
for (int x = 0; x < data.Width; x++)
{
bool stop = false;
for (int y = 0; y < data.Height; y++)
{
byte alpha = buffer[y * data.Stride + 4 * x + 3];
if (alpha != 0)
{
xMin = x;
stop = true;
foundPixel = true;
break;
}
}
if (stop)
break;
}
// Image is empty...
if (!foundPixel)
return null;
// Find yMin
for (int y = 0; y < data.Height; y++)
{
bool stop = false;
for (int x = xMin; x < data.Width; x++)
{
byte alpha = buffer[y * data.Stride + 4 * x + 3];
if (alpha != 0)
{
yMin = y;
stop = true;
break;
}
}
if (stop)
break;
}
// Find xMax
for (int x = data.Width - 1; x >= xMin; x--)
{
bool stop = false;
for (int y = yMin; y < data.Height; y++)
{
byte alpha = buffer[y * data.Stride + 4 * x + 3];
if (alpha != 0)
{
xMax = x;
stop = true;
break;
}
}
if (stop)
break;
}
// Find yMax
for (int y = data.Height - 1; y >= yMin; y--)
{
bool stop = false;
for (int x = xMin; x <= xMax; x++)
{
byte alpha = buffer[y * data.Stride + 4 * x + 3];
if (alpha != 0)
{
yMax = y;
stop = true;
break;
}
}
if (stop)
break;
}
srcRect = Rectangle.FromLTRB(xMin, yMin, xMax , yMax);
}
finally
{
if (data != null)
source.UnlockBits(data);
}
Bitmap dest = new Bitmap(srcRect.Width, srcRect.Height);
Rectangle destRect = new Rectangle(0, 0, srcRect.Width, srcRect.Height);
using (Graphics graphics = Graphics.FromImage(dest))
{
graphics.DrawImage(source, destRect, srcRect, GraphicsUnit.Pixel);
}
return dest;
}
I'm trying to trim a Bitmap with Text Drawn on it.The Proper Image should look like this after trimming
But after trimming i get the following result ..the bottom portion clipped off
What i'm i doing wrong? Please advice..

This is actually a problem with Rectangle.FromLTRB !
Looking closer at the images you will find that you have actually lost only one row of pixels. (The strong magnification had me fooled for a while..)
The algorithm to determine the Height (and Width) of the rectangle is basically right, but off by one.
If you use this
srcRect = Rectangle.FromLTRB(xMin, yMin, xMax + 1 , yMax + 1);
or this:
srcRect = new Rectangle(xMin, yMin, xMax - xMin + 1 , yMax - yMin + 1);
it should work.
You can test with pen and paper: Say: 1st pixel row with a color = 4, first from bottom on a 10 pixel square: 8, that makes 5 not 4 net data: 4,5,6,7,8.
Do note that this issue is inherent in FromLTRB:
Rectangle myRectangle = Rectangle.FromLTRB(0, 0, 10, 10);
..results in a Rectangle with Height=10 even though 0..10 should cover 11 pixels rows! So the Right-Bottom coordinate is actually excluded from the result!!
I think the whole issue with rectangles being off by one stems from legacy ways to use a Pen with its alignment. When using the same rectangles to fill with a Brush all works as expected.

Looping through pixels excluding an area

I have the following code for looping pixels in an image:
Bitmap myBitmap = new Bitmap(pictureBox1.Image);
for (x = 0; x < pictureBox1.Image.Width; x += 1)
{
for (y = 0; y < pictureBox1.Image.Height; y += 1)
{
Color pixelColor = myBitmap.GetPixel(x, y);
aws = pixelColor.GetBrightness();
}
}
The above works. What if i want to exclude an area of the image. The area is an area selected by the user,for example:
from x=200&y=30 to x=250&y=30
from x=200&y=35 to x=250&y=35
from x=200&y=40 to x=250&y=40
from x=200&y=45 to x=250&y=45
How i loop all pixels except the one i wrote above?
Thank you!

I think you can do something like the following
Rectangle rectToExclude = new Rectangle(200, 30, 250, 30);//rectangle to be excluded
Bitmap myBitmap = new Bitmap(pictureBox1.Image);
for (x = 0; x < pictureBox1.Image.Width; x += 1)
{
for (y = 0; y < pictureBox1.Image.Height; y += 1)
{
if (rectToExclude.Contains(x, y)) continue; // if in the exclude rect then continue without doing any effects
Color pixelColor = myBitmap.GetPixel(x, y);
aws = pixelColor.GetBrightness();
}
}

Maybe write a function to test x and y like this:
public static bool IsInArea(int testx, int testy, int x1, int y1, int x2, int y2)
{
if (testx > x1 && testx < x2 && testy > y1 && testy < y2)
{
return true;
}
else
{
return false;
}
}
and then use it like this:
BitmapData bmd = myBitmap.LockBits(new Rectangle(0, 0, myBitmap.Width, myBitmap.Height),
System.Drawing.Imaging.ImageLockMode.ReadWrite,
myBitmap.PixelFormat);
int PixelSize = 4;
unsafe
{
for (int y = 0; y < bmd.Height; y++)
{
byte* row = (byte*)bmd.Scan0 + (y * bmd.Stride);
for (int x = 0; x < bmd.Width; x++)
{
if (!IsInArea(x, y, 200, 30, 250, 30))
{
int b = row[x * PixelSize]; //Blue
int g = row[x * PixelSize + 1]; //Green
int r = row[x * PixelSize + 2]; //Red
int a = row[x * PixelSize + 3]; //Alpha
Color c = Color.FromArgb(a, r, g, b);
float brightness = c.GetBrightness();
}
}
}
}
myBitmap.UnlockBits(bmd);

ERROR XNA Framework VertexBuffer IndexBuffer usage

I am creating a game after working through a XNA 4.0 book. It will be 3D, but I am already stuck in creating the terrain...
UPDATE: Everything starting from here is an update...
Terrain Update:
public void Update(Matrix view, Matrix projection)
{
View = view;
Projection = projection;
World = Matrix.CreateTranslation(-Width / 2f, 0, Height / 2f);
}
Terrain Draw:
public void Draw(GraphicsDevice g)
{
effect.CurrentTechnique = effect.Techniques["ColoredNoShading"];
effect.Parameters["xView"].SetValue(View);
effect.Parameters["xProjection"].SetValue(Projection);
effect.Parameters["xWorld"].SetValue(World);
foreach (EffectPass pass in effect.CurrentTechnique.Passes)
{
pass.Apply();
//g.DrawUserIndexedPrimitives(PrimitiveType.TriangleList, vertices, 0, vertices.Length, indices, 0, indices.Length / 3, VertexPositionColorNormal.VertexDeclaration);
g.DrawIndexedPrimitives(PrimitiveType.TriangleList, 0, 0, vertices.Length, 0, indices.Length / 3);
}
}
The commented line is working, in the both cases I am able to see the terrain...
The following code is to initialize Vertex and Index Buffer:
private void SetUpVertices(GraphicsDevice g)
{
float currentH;
int currentI;
vertices = new VertexPositionColorNormal[Width * Height];
for (int x = 0; x < Width; x++)
{
for (int y = 0; y < Height; y++)
{
currentH = heightData[x,y];
currentI = x + y * Width;
vertices[currentI].Position = new Vector3(x, currentH , -y);
if (currentH < minH + (maxH - minH) / 3)
vertices[currentI].Color = Color.ForestGreen;
else if (currentH < maxH - (maxH - minH) / 3)
vertices[currentI].Color = Color.LawnGreen;
else
vertices[currentI].Color = Color.White;
}
}
SetUpIndices(g);
}
private void SetUpIndices(GraphicsDevice g)
{
indices = new int[(Width - 1) * (Height - 1) * 6];
int counter = 0;
for (int y = 0; y < Height - 1; y++)
{
for (int x = 0; x < Width - 1; x++)
{
int lowerLeft = x + y * Width;
int lowerRight = (x + 1) + y * Width;
int topLeft = x + (y + 1) * Width;
int topRight = (x + 1) + (y + 1) * Width;
indices[counter++] = topLeft;
indices[counter++] = lowerRight;
indices[counter++] = lowerLeft;
indices[counter++] = topLeft;
indices[counter++] = topRight;
indices[counter++] = lowerRight;
}
}
SetUpNormals(g);
}
private void SetUpNormals(GraphicsDevice g)
{
for (int i = 0; i < vertices.Length; i++)
{
vertices[i].Normal = Vector3.Zero;
}
int[] index = new int[3];
Vector3 s1, s2, n;
for (int i = 0; i < vertices.Length / 3; i++)
{
for (int y = 0; y < 3; y++)
index[y] = indices[i * 3 + y];
s1 = vertices[index[0]].Position - vertices[index[2]].Position;
s2 = vertices[index[0]].Position - vertices[index[1]].Position;
n = Vector3.Cross(s1, s2);
for (int y = 0; y < 3; y++)
{
vertices[index[y]].Normal += n;
vertices[index[y]].Normal.Normalize();
}
}
FillBuffers(g);
}
private void FillBuffers(GraphicsDevice g)
{
VertexBuffer = new VertexBuffer(g, VertexPositionColorNormal.VertexDeclaration, vertices.Length, BufferUsage.WriteOnly);
VertexBuffer.SetData(vertices);
IndexBuffer = new IndexBuffer(g, typeof(int), indices.Length, BufferUsage.WriteOnly);
IndexBuffer.SetData(indices);
g.Indices = IndexBuffer;
g.SetVertexBuffer(VertexBuffer);
}
I don't think, that there is a mistake, because it is working with the other line. Might there be an error with the .fx file I am using. If you think so, I am going to switch to BasicEffects...
(You might notice, that the code is from http://www.riemers.net/eng/Tutorials/XNA/Csharp/series1.php )
Thanks for your help...
Yours,
Florian

(Answer to original revision of the question.)
You're not setting your vertex buffer and index buffer onto the graphics device. These two lines of code (untested) should do what you need:
g.GraphicsDevice.Indices = indexBuffer;
g.GraphicsDevice.SetVertexBuffer(vertexBuffer);
Place them just after you set the parameters on your effect (ef), before the loop.
The vertex buffer provides the vertex declaration that the exception message is asking for.
Edit after question update: In your new version you're setting the vertex and index buffers - but it's in the wrong place. You need to set them onto the graphics device each frame. Your code would only work if nothing changes them after you set them in FillBuffers. But I'm guessing that stuff is being drawn outside your class's Draw method?
If that something else is a SpriteBatch, even it works using vertex buffers and index buffers. So it will reset your settings. (It's worth adding that it also sets render states - in which case you might need to see this article.)