I'm trying to get an entire dark "D" from this image :
With this code, I get this result :
```
Cv2.Threshold(im_gray, threshImage, 80, 255, ThresholdTypes.BinaryInv); // Threshold to find contour
Cv2.FindContours(
threshImage,
out contours,
out hierarchyIndexes,
mode: RetrievalModes.Tree,
method: ContourApproximationModes.ApproxSimple
);
double largest_area = 0;
int largest_contour_index = 0;
Rect rect = new Rect();
//Search biggest contour
for (int i = 0; i <contours.Length; i++)
{
double area = Cv2.ContourArea(contours[i]); // Find the area of contour
if (area > largest_area)
{
largest_area = area;
largest_contour_index = i; //Store the index of largest contour
rect = Cv2.BoundingRect(contours[i]); // Find the bounding rectangle for biggest contour
}
}
Cv2.DrawContours(finalImage, contours, largest_contour_index, new Scalar(0, 0, 0), -1, LineTypes.Link8, hierarchyIndexes, int.MaxValue);
```
But when I save finalImage, I get the result below :
How can I get entire black letter?
Apart from my comment, I tried out another way. I used the statistical property of the gray scale image.
For this image I used the mean value of the image to select an optimal threshold value.
The optimal value is selected by choosing all pixel values under 33% of the mean value
med = np.mean(gray)
optimal = int(med * 1.33) #selection of the optimal threshold
ret,th = cv2.threshold(gray, optimal, 255, 0)
This is what I got as a result:
Now invert this image and obtain the largest contour and there you go you will have the Big D
EDIT:
For some images with drastic histogram, you can use the median of the gray scale image instead of the mean.
Related
This is the method i'm using to read and set the pixels :
public void ReadSetPixels(Bitmap image1, Bitmap image2)
{
int tolerance = 64;
for (int x = 0; x < image1.Width; x++)
{
for (int y = 0; y < image1.Height; y++)
{
Color pixelColor = image1.GetPixel(x, y);
// just average R, G, and B values to get gray. Then invert by 255.
int invertedGrayValue = 255 - (int)((pixelColor.R + pixelColor.G + pixelColor.B) / 3);
if (invertedGrayValue > tolerance) { invertedGrayValue = 255; }
// this keeps the original pixel color but sets the alpha value
image1.SetPixel(x, y, Color.FromArgb(invertedGrayValue, pixelColor));
}
}
// composite image1 on top of image2
using (Graphics g = Graphics.FromImage(image2))
{
g.CompositingMode = CompositingMode.SourceOver;
g.CompositingQuality = CompositingQuality.HighQuality;
g.DrawImage(image1, new Point(0, 0));
}
image2.Save(#"d:\mynewbmp.bmp");
image1.Dispose();
image2.Dispose();
}
This is how i'm using it in form1 :
RadarPixels rp = new RadarPixels();
rp.ReadSetPixels(new Bitmap(#"d:\resized1.png"),
new Bitmap(#"d:\clean_radar_image11.jpg"));
The image resized1 is size 512x512 Bit depth 32
This is the image :
resized image
This is the base image that the resized1 should be over it.
This image is size 512x512 72 dpi Bit depth 24
base image
The result is the image mynewbmp.bmp :
result
UPDATE :
i found the problem.
i found the problem but not the solution. the problem is that the setpixel set it in the wrong position. it is the right position by the code 0,0 when making new Point(0, 0) but it's not what i want. iwant that the clouds the readed pixels to be around the center of the base image clean_radar_image11 i changed it to format bmp and all the clouds are show just not in the right position.
This image i created now show the image on the left where the clouds pixels in the position at 0,0 on the right side is how the clouds pixels should be positioned. the right image is just example to show how the clouds should be positioned on the left image.
example of how the clouds pixels should be set in position
A possible problem is that the images dpi does not match. Try calling SetResolution for both images
image1.SetResolution(96, 96);
image2.SetResolution(96, 96);
You should be able to set any resolution, as long as they are the same, but 96 tend to be the default on windows.
You might also try using DrawImageUnscaled, since that should draw the image according to physical/pixel size.
maybe image2's format is .jpg image,its have not alpha value
I would like to do grabcut which uses a depth map that cuts away far objects, that is used in mixed reality application. So I would like to show just the front of what I see and the background as virtual reality scene.
The problem right now I tried to adapt so code and what I get is front which is cut but in black color, the mask actually.
I don't know where is the problem settle.
The input is a depth map from zed camera.
here is a picture of the behaviour:
My trial:
private void convertToGrayScaleValues(Mat mask)
{
int width = mask.rows();
int height = mask.cols();
byte[] buffer = new byte[width * height];
mask.get(0, 0, buffer);
for (int x = 0; x < width; x++)
{
for (int y = 0; y < height; y++)
{
int value = buffer[y * width + x];
if (value == Imgproc.GC_BGD)
{
buffer[y * width + x] = 0; // for sure background
}
else if (value == Imgproc.GC_PR_BGD)
{
buffer[y * width + x] = 85; // probably background
}
else if (value == Imgproc.GC_PR_FGD)
{
buffer[y * width + x] = (byte)170; // probably foreground
}
else
{
buffer[y * width + x] = (byte)255; // for sure foreground
}
}
}
mask.put(0, 0, buffer);
}
For Each depth frame from Camera:
Mat erodeElement = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(4, 4));
Mat dilateElement = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(7, 7));
depth.copyTo(maskFar);
Core.normalize(maskFar, maskFar, 0, 255, Core.NORM_MINMAX, CvType.CV_8U);
Imgproc.cvtColor(maskFar, maskFar, Imgproc.COLOR_BGR2GRAY);
Imgproc.threshold(maskFar, maskFar, 180, 255, Imgproc.THRESH_BINARY);
Imgproc.dilate(maskFar, maskFar, erodeElement);
Imgproc.erode(maskFar, maskFar, dilateElement);
Mat bgModel = new Mat();
Mat fgModel = new Mat();
Imgproc.grabCut(image, maskFar, new OpenCVForUnity.CoreModule.Rect(), bgModel, fgModel, 1, Imgproc.GC_INIT_WITH_MASK);
convertToGrayScaleValues(maskFar); // back to grayscale values
Imgproc.threshold(maskFar, maskFar, 180, 255, Imgproc.THRESH_TOZERO);
Mat foreground = new Mat(image.size(), CvType.CV_8UC4, new Scalar(0, 0, 0));
image.copyTo(foreground, maskFar);
Utils.fastMatToTexture2D(foreground, texture);
In this case, the graph cut on the depth image might not be the correct method to solve all of your issue.
If you insist the processing should be done in the depth image. To find everything that is not on the table and filter out the table part. You may first apply the disparity based approach for finding the object that's is not on the ground. Reference: https://github.com/windowsub0406/StereoVision
Then based on the V disparity output image, find the locally connected component that is grouped together. You may follow this link how to do this disparity map in OpenCV which is asking the similar way to find the objects that's not on the ground
If you are ok with RGB based approaches, then use any deep learning-based method to recognize the monitor should be the correct approaches. It can directly detect the mointer bounding box. By apply this bounding box to the depth image, you may have what you want. For deep learning based approaches, there are many available package such as Yolo series. You may find one that is suitable for you. reference: https://medium.com/#dvshah13/project-image-recognition-1d316d04cb4c
Please Note - This is a Math question essentially. However, i have also tagged C#
as this is the language i am working in
Summary
I'm looking for an algorithm (or name thereof) that can find the Negative Space (or space) in an image. The closest i have found Dijkstra's algorithm (which is seemingly close), yet its actually a subset of the actual problem. Namely, to walk through a Cartesian Plane traversing every coordinate that isn't filled (or black in my case) to find a mask. Example below
Example of Dijkstra's Algorithm
The background
I need to tidy up 10's of thousands of images that have artefacts in them. By cleaning up i mean these things specifically :
Using Edge Detection to find the edges of the objects in the images
Masking the Negative Space so i can covert the image backgrounds to plain white
Cropping the images to their optimal size.
Currently i'm using Canny Edge Detection to find the most important part of the image. I can crop the image fairly well (shown below), and also find all the images that have the problem. However i am having trouble locating the best algorithm (or name thereof) to find the negative space.
Example of the original image
As you can see the image looks pretty clean, however its not
Example of the accentuated problem
The image has lots of artefacts in the background and they need to be removed
Example of Canny Edge Detection
This does a wonderful job of cleaning up the image
The Problem
Dijkstra's algorithms premise is it looks for all the possible paths, its basically a solves the Travelling Sales man problem
The problems is; The algorithm actually does much more than i need to do with regards to the weighing and the distance measures , and it stops when it has the shortest path (where i need it to complete the image).
The pseudo code
1 function Dijkstra(Graph, source):
2
3 create vertex set Q
4
5 for each vertex v in Graph: // Initialization
6 dist[v] ← INFINITY // Unknown distance from source to v
7 prev[v] ← UNDEFINED // Previous node in optimal path from source
8 add v to Q // All nodes initially in Q (unvisited nodes)
9
10 dist[source] ← 0 // Distance from source to source
11
12 while Q is not empty:
13 u ← vertex in Q with min dist[u] // Node with the least distance
14 // will be selected first
15 remove u from Q
16
17 for each neighbor v of u: // where v is still in Q.
18 alt ← dist[u] + length(u, v)
19 if alt < dist[v]: // A shorter path to v has been found
20 dist[v] ← alt
21 prev[v] ← u
22
23 return dist[], prev[]
Can anyone suggest an Algorithm or modify the Pseudo Code to Dijkstra's Algorithms to achieve this?
The answer to the question was simply the Flood-fill Algorithm.
However, to solve the entire problem of cleaning subtle artefacts from images, the total solution was as follows.
Use Canny Edge Detection with appropriate thresholds to get the outline of objects in the image
Use a Gaussian Blur to Blur the Canny results enough so the flood full wont bleed
Use a flood fill to create the Mask and apply it back to the original image
Some traps for your for young players.
PixelFormats, you need to make sure everything is talking the same format
Not editing the bitmap directly by using scanlines or locked pixels
paralleling algorithms where possible, in this case the flood fill and Blur where good candiates
Update
Even a faster method was just to use Parallel FloodFill with a Color Threshold value
Color Threshold
public static bool IsSimilarColor(this Color source, Color target, int threshold)
{
int r = source.R - target.R, g = source.G - target.G, b = source.B - target.B;
return (r * r + g * g + b * b) <= threshold * threshold;
}
Parallel FloodFill
public static Bitmap ToWhiteCorrection(this Bitmap source, Color sourceColor, Color targetColor, Color maskColor, int threshold, Size tableSize, int cpu = 0)
{
using (var dbMask = new DirectBitmap(source))
{
using (var dbDest = new DirectBitmap(source))
{
var options = new ParallelOptions
{
MaxDegreeOfParallelism = cpu <= 0 ? Environment.ProcessorCount : cpu
};
// Divide the image up
var rects = dbMask.Bounds.GetSubRects(tableSize);
Parallel.ForEach(rects, options, rect => ProcessWhiteCorrection(dbMask, dbDest, rect, sourceColor, targetColor, maskColor, threshold));
return dbDest.CloneBitmap();
}
}
}
private static void ProcessWhiteCorrection(this DirectBitmap dbMask, DirectBitmap dbDest, Rectangle rect, Color sourceColor, Color targetColor, Color maskColor, int threshold)
{
var pixels = new Stack<Point>();
AddStartLocations(dbMask, rect, pixels, sourceColor, threshold);
while (pixels.Count > 0)
{
var point = pixels.Pop();
if (!rect.Contains(point))
{
continue;
}
if (!dbMask[point]
.IsSimilarColor(sourceColor, threshold))
{
continue;
}
dbMask[point] = maskColor;
dbDest[point] = targetColor;
pixels.Push(new Point(point.X - 1, point.Y));
pixels.Push(new Point(point.X + 1, point.Y));
pixels.Push(new Point(point.X, point.Y - 1));
pixels.Push(new Point(point.X, point.Y + 1));
}
}
Worker
private static void ProcessWhiteCorrection(this DirectBitmap dbMask, DirectBitmap dbDest, Rectangle rect, Color sourceColor, Color targetColor, Color maskColor, int threshold)
{
var pixels = new Stack<Point>();
// this basically looks at a 5 by 5 rectangle in all 4 corners of the current rect
// and looks to see if we are all the source color
// basically it just picks good places to start the fill
AddStartLocations(dbMask, rect, pixels, sourceColor, threshold);
while (pixels.Count > 0)
{
var point = pixels.Pop();
if (!rect.Contains(point))
{
continue;
}
if (!dbMask[point].IsSimilarColor(sourceColor, threshold))
{
continue;
}
dbMask[point] = maskColor;
dbDest[point] = targetColor;
pixels.Push(new Point(point.X - 1, point.Y));
pixels.Push(new Point(point.X + 1, point.Y));
pixels.Push(new Point(point.X, point.Y - 1));
pixels.Push(new Point(point.X, point.Y + 1));
}
}
Direct bitmap
public class DirectBitmap : IDisposable
{
public DirectBitmap(int width, int height, PixelFormat pixelFormat = PixelFormat.Format32bppPArgb)
{
Width = width;
Height = height;
Bounds = new Rectangle(0, 0, Width, Height);
Bits = new int[width * height];
BitsHandle = GCHandle.Alloc(Bits, GCHandleType.Pinned);
Bitmap = new Bitmap(width, height, width * 4, PixelFormat.Format32bppPArgb, BitsHandle.AddrOfPinnedObject());
using (var g = Graphics.FromImage(Bitmap))
{
g.Clear(Color.White);
}
}
public DirectBitmap(Bitmap source)
{
Width = source.Width;
Height = source.Height;
Bounds = new Rectangle(0, 0, Width, Height);
Bits = new int[source.Width * source.Height];
BitsHandle = GCHandle.Alloc(Bits, GCHandleType.Pinned);
Stride = (int)GetStride(PixelFormat, Width);
Bitmap = new Bitmap(source.Width, source.Height, Stride, PixelFormat.Format32bppPArgb, BitsHandle.AddrOfPinnedObject());
using (var g = Graphics.FromImage(Bitmap))
{
g.DrawImage(source, new Rectangle(0, 0, source.Width, source.Height));
}
}
...
I have a image that contains a layout for a level, and I want to load the level in the game by reading each pixels color from the image, and drawing the corresponding block. I am using this code:
public void readLevel(string path, GraphicsDevice graphics)
{
//GET AN ARRAY OF COLORS
Texture2D level = Content.Load<Texture2D>(path);
Color[] colors = new Color[level.Width * level.Height];
level.GetData(colors);
//READ EACH PIXEL AND DRAW LEVEL
Vector3 brickRGB = new Vector3(128, 128, 128);
int placeX = 0;
int placeY = 0;
foreach (Color pixel in colors)
{
SpriteBatch spriteBatch = new SpriteBatch(graphics);
spriteBatch.Begin();
if (pixel == new Color(brickRGB))
{
Texture2D brick = Content.Load<Texture2D>("blocks/brick");
spriteBatch.Draw(brick, new Rectangle(placeX, placeY, 40, 40), Color.White);
}
if(placeX == 22)
{
placeX = 0;
placeY++;
}
else
spriteBatch.End();
}
}
But it just shows a blank screen. Help would be appreciated!
EDIT: PROBLEM FIXED! (Read htmlcoderexe's answer below) Also, there was another problem with this code, read here.
Your code seems to draw each sprite at one pixel offset from the previous, but your other parameter suggests they are 40 pixel wide. placeX and placeY will need to be multiplied by the stride of your tiles (40).
Also, in the bit where you compare colours, you might be having a problem with floating point colour values (0.0f-1.0f) and byte colours being used together.
new Color(brickRGB)
This translates to:
new Color(new Vector3(128f,128f,128f))
So it tries constructing a colour from the 0.0f-1.0f range, clips it down to 1f (the allowed maximum for float input for Color), and you end up with a white colour (255,255,255), which is not equal to your target colour (128,128,128).
To get around this, try changing
Vector3 brickRGB = new Vector3(128, 128, 128);
to
Color brickRGB = new Color(128, 128, 128);
and this part
if (pixel == new Color(brickRGB))
to just
if (pixel == brickRGB)
You will also need to create your drawing rectangle with placeX and placeY multiplied by 40, but do not write to the variable - just use placeX*40 for now and replace it with a constant later.
I have thresholding image :
I want to know, can i detect "white zones" and draw rectangle around them (save data also wanted)
Or can i draw parallelepiped (polygon) and "say" area inside it is white?
Thanks.
So in order to detect the white zones, just get the contours of the image. This can be done with:
vector<vector<Point>>contours;
vector<Vec4i> hierarchy;
findContours(blackWhiteImage,
contours,
hierarchy,
CV_RETR_TREE,
CV_CHAIN_APPROX_SIMPLE,
Point(0,0));
Then, you can generate the approximate bounding box that models every contour you extracted with:
vector<vector<Point> > contours_poly( contours.size() );
vector<Rect> boundRect( contours.size() );
for( int i = 0; i < contours.size(); i++ ){
approxPolyDP( Mat(contours[i]),
contours_poly[i],
3,
true );
//3 is epsilon tuning param for bias-variance trade off
//true denotes contours are closed
boundRect[i] = boundingRect( Mat(contours_poly[i]) );
}
Once this is done, you can access the boundingRect objects in your boundingRect array just like you would access any other array.
Simular code for EmguCV (C#) without approximation:
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Mat hierarchy;
CvInvoke.FindContours(binMat, contours, hierarchy, RetrType.External, ChainApproxMethod.ChainApproxSimple);
for (int i = 0; i < contours.Size; ++i)
{
if (CvInvoke.ContourArea(contours[i]) > 8)
{
Rectangle rc = CvInvoke.BoundingRectangle(contours[i]);
CvInvoke.Rectangle(srcMat, rc, new MCvScalar(0, 0, 255));
}
}