Have spent the last week trying to have my C# program show both the depth feed and the RGB feed (similar to how /Samples/Bin64/Release/NiViewer64.exe shows both feeds in a window).
Project specs: C# - VS2013 Express OpenNI - Using a modified
SimpleViewer.net (has two feeds of depth). Asus Xtion Pro Live
I would like one of the feeds to become a normal camera feed instead of the depth feed.
I'm guessing it has something to do with this:
MapOutputMode mapMode = this.depth.MapOutputMode;
this.bitmap = new Bitmap((int)mapMode.XRes, (int)mapMode.YRes,System.Drawing.Imaging.PixelFormat.Format24bppRgb);
Any ideas?
Finally figured it out, thanks to another programmer.
image = context.FindExistingNode(NodeType.Image) as ImageGenerator;
ImageMetaData imd = image.GetMetaData();
lock (this)
{
//**************************************//
//***********RGB Camera Feed************//
//**************************************//
Rectangle rect = new Rectangle(0, 0, this.bitmap.Width, this.bitmap.Height);
BitmapData data = this.camera_feed.LockBits(rect, ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
byte* pDest = (byte*)data.Scan0.ToPointer();
byte* imstp = (byte*)image.ImageMapPtr.ToPointer();
// set pixels
for (int i = 0; i < imd.DataSize; i += 3, pDest += 3, imstp += 3)
{
pDest[0] = imstp[2];
pDest[1] = imstp[1];
pDest[2] = imstp[0];
}
and declare this somewhere:
public ImageGenerator image;
Related
I have a project in which I'm using IronOCR to read an area define by OpenCVSharp4 but the problem I'm encountering is IronOCrs CropRectangle method, it uses System.drawing.rectangle and for some reason my OpenCvSharp.Rect cannot be converted to it, by this I mean when I Finally uses IronOCRs Input.Add(Image, ContentArea) the results I get are not what is expected.
Below the code I have attached a picture of what the code currently produces.
Don't worry about IronOCR not getting the correct letters I believe it has to do with it creating a weird box and some letters getting cut off, it works if I made the area larger for crop rectangle width and height
var Ocr = new IronTesseract();
String[] splitText;
using (var Input = new OcrInput())
{
//OpenCv
OpenCvSharp.Rect rect = new OpenCvSharp.Rect(55, 107, 219, 264);
//IronOCR
Rectangle ContentArea = new Rectangle() { X = rect.TopLeft.X, Y = rect.TopLeft.Y, Height = rect.Height, Width = rect.Width };
CropRectangle r = new CropRectangle(ContentArea);
CordBox.Text = r.Rectangle.ToString();
//OpenCv
resizedMat.Rectangle(rect.TopLeft, rect.BottomRight, Scalar.Blue, 3);
resizedMat.Rectangle(new OpenCvSharp.Point(55, 107), new OpenCvSharp.Point(219, 264), Scalar.Brown, 3);
Cv2.ImShow("resizedMat", resizedMat);
//IronOCR
Input.Add(#"C:\Projects\AnExperiment\WpfApp1\Images\TestSave.PNG", r);
Input.EnhanceResolution();
var Result = Ocr.Read(Input);
ResultBox.Text = Result.Text;
splitText = ResultBox.Text.Split('\n');
}
SO here is the solution I came up with.
This problem is a OpenCvSharp4 one where OpenCvSharp4.Rectangle for some reason does have matching coordinates to System.Drawing.Rectangle. I have posted this on the gitHub for OpenCvSHarp4 and he says its fine, but its not.
So I switched over to Emgu NuGet package its better for C# applications and is a OpenCv Wrapper made for C# (I was just scared of giving it a try before because i never really understood it.)
Emgu uses System.Drawing.Rectangle by default instead of something like OpenCvSharp4.Rectangle so everything matches up nicely.
Mat testMat = new Mat();
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", testMat );
After finding this out the rest was pretty easy so the final code is below on how it was transformed. (For reference Emgu.CV.CVInvoke is how its called and Emgu.CV.BitmapExtension is its own separate NuGet package)
// Get the original Image
fullPage = CvInvoke.Imread(#"C:\Projects\AnExperiment\WpfApp1\Images\TestImageFinalFilled.png");
// Resize it so it works with the cordinates stored previously in a json file
CvInvoke.Resize(fullPage, resizedMat, EmguSetResolution(fullPage, dpi));
// Save the small version so iron ocr doesnt mess up
var bitmap = Emgu.CV.BitmapExtension.ToBitmap(resizedMat);
bitmap.Save(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
// Let user select box
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", resizedMat);
CvInvoke.DestroyWindow("main");
// Draw Rect for debugging
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
// Read section we highlighted by pulling the saved resuze imag as a reference
var Ocr = new IronTesseract();
IronOcr.OcrResult ocrResult;
Ocr.UseCustomTesseractLanguageFile(#"C:\Projects\AnExperiment\WpfApp1\tessdata_best-main\eng.traineddata");
using (var Input = new OcrInput())
{
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
IronOcr.CropRectangle contentArea = new CropRectangle(roi);
Input.AddImage(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG", contentArea);
Input.EnhanceResolution();
Input.Sharpen();
Input.Contrast();
ocrResult = Ocr.Read(Input);
}
File.Delete(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
CvInvoke.Imshow("m", resizedMat);
After all this I have some functions that spit the ocrResult.Text into the textbox and separate certain things I needed from it.
I am attempting to load images into an Picture Box using the following code:
private void button1_Click(object sender, EventArgs e)
{
//Clear the Image Area
if (templateArea.Image != null)
{ templateArea.Image = null; }
//Set Base Variables
Bitmap img = new Bitmap(Resources.BlankBackground);
Graphics gpx = Graphics.FromImage(img);
Bitmap newBitmap = null;
string bitmapToLoad;
ResourceManager rm;
//Loop Lists
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
rm = Resources.ResourceManager;
newBitmap = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
gpx.Dispose();
}
//Set Image Area to newBitmap
templateArea.Image = newBitmap;
}
when trying to run the code I get errors on the second loop where i = 1 in this portion:
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
//given error: System.ArgumentException: 'Parameter is not valid.' //Does not point to anything specific.
the comps list contains lists formatted as such:
//These listed items are defaulted at program start but more are added while running
// p elementName type sub type x y sX sY R G B O i
eleComps = new List<Object> {0,"Background","Static","BlankBackground",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Border", "Static","CardBorder_02", 0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Cut Line", "Static","00_Card_CutLine",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
Each possible "sub type" is a bitmap file in resources, and there are more than just those listed above.
What I have tried:
I have consulted the miracle that is Google and YouTube and haven't been able to solve this one. Although I feel like I'm missing something basic I have been looking at it all day and may need some fresh eyes on it to correct something that may be obvious but for some reason I seem to be missing.
What I am looking for:
I am trying to create an image viewer that will overlap images or parts of images (.png) at specific coordinates in an Picture Box. The comps list will be added to and removed from actively during application use and as such using a variable to reference the Resources is necessary and that portion of the code has given me the most difficult time out of the supplied code samples.
UPDATE 1: changed image box to picture box to reduce confusion.
UPDATE 2: NOTE: This sample achieves the desired end result, but does not allow me to use the variables as needed to change positions, scale, color, and opacity.
//Clear Img Area
templateArea.InitialImage = null;
//Find Bitmap(s)
string p = #"C:\Users\david\Desktop\Test Fields\Default\";
Bitmap img1 = new Bitmap(p + "00_Empty.png", true);
Bitmap img2 = new Bitmap(p + "CardBorder_02.png", true);
Bitmap img3 = new Bitmap(p + "00_Card_CutLine.png", true);
//Set initial image for image area
Graphics gpx = Graphics.FromImage(img1);
//place an image at x, y, Width, Height
gpx.DrawImage(img2, 0, 0, img2.Width, img2.Height);
gpx.DrawImage(img3, 0, 0, img3.Width, img3.Height);
//clear gpx cache
gpx.Dispose();
//set image area to modified img1
templateArea.Image = img1;
Also please let me know if I'm just plain going about this in the wrong way, and what I need to do to get it right.
UPDATE 3: I seem to have gotten it to work properly with this variation, using the originally supplied list of listed parameters. Thanks to Idle_Mind's pointers I was able to figure it out.
Bitmap newBitmap_01 = new Bitmap(templateArea.Width, templateArea.Height);
Graphics gpx = Graphics.FromImage(newBitmap_01);
string bitmapToLoad;
ResourceManager rm = Resources.ResourceManager;
Bitmap newBitmap_02;
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
newBitmap_02 = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap_02,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap_02.Width,
newBitmap_02.Height);
}
gpx.Dispose();
templateArea.Image = newBitmap_01;
The error with the following code is, it only find, partially, the first image itself.
Let me be more clear, I took a screenshot of an image, cropped it and putted it on the desktop. Then I saved the actual original image in full size (not cropped) on desktop as well.
When I run the code, it correctly detects all the coordinates of the cropped image on the desktop, however, when I open the full size image and hover it, it won't detect anything and return null, while it should be able to detect the pixels location inside the full size image as well.
I'm gonna post a video of what I mean, to be more clear:
https://www.youtube.com/watch?v=Ha3eGxWcAF8
As you see, it does not detect the pixels on the main image.
What's wrong on there?
Why it detects only the loaded cropped image, instead of a position on the screenshot based on the actual full fize image?
Code below:
public static Point? Find(Bitmap first, Bitmap second)
{
if (null == first || null == second)
{
return null;
}
if (first.Width < second.Width || first.Height < second.Height)
{
return null;
}
var firstArray = GetPixelArray(first);
var secondArray = GetPixelArray(second);
foreach (var firstLineMatchPoint in FindMatch(firstArray.Take(first.Height - second.Height), secondArray[0]))
{
if (IssecondPresentAtLocation(firstArray, secondArray, firstLineMatchPoint, 1))
{
return firstLineMatchPoint;
}
}
return null;
}
private static int[][] GetPixelArray(Bitmap bitmap)
{
var result = new int[bitmap.Height][];
var bitmapData = bitmap.LockBits(new Rectangle(0, 0, bitmap.Width, bitmap.Height), ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
for (int y = 0; y < bitmap.Height; ++y)
{
result[y] = new int[bitmap.Width];
Marshal.Copy(bitmapData.Scan0 + y * bitmapData.Stride, result[y], 0, result[y].Length);
}
bitmap.UnlockBits(bitmapData);
return result;
}
private static IEnumerable<Point> FindMatch(IEnumerable<int[]> firstLines, int[] secondLine)
{
var y = 0;
foreach (var firstLine in firstLines)
{
for (int x = 0, n = firstLine.Length - secondLine.Length; x < n; ++x)
{
if (ContainSameElements(firstLine, x, secondLine, 0, secondLine.Length))
{
yield return new Point(x, y);
}
}
y += 1;
}
}
private static bool ContainSameElements(int[] first, int firstStart, int[] second, int secondStart, int length)
{
for (int i = 0; i < length; ++i)
{
if (first[i + firstStart] != second[i + secondStart])
{
return false;
}
}
return true;
}
private static bool IssecondPresentAtLocation(int[][] first, int[][] second, Point point, int alreadyVerified)
{
//we already know that "alreadyVerified" lines already match, so skip them
for (int y = alreadyVerified; y < second.Length; ++y)
{
if (!ContainSameElements(first[y + point.Y], point.X, second[y], 0, second.Length))
{
return false;
}
}
return true;
}
static void Main(string[] args)
{
int counter = 1;
while (true)
{
Console.WriteLine("Executing pixel search");
Bitmap bitmap = new Bitmap(Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height);
Graphics graphics = Graphics.FromImage(bitmap);
graphics.CopyFromScreen(0, 0, 0, 0, bitmap.Size);
Bitmap bitmap2 = (Bitmap)Image.FromFile(#"path_to_desktop\cropped.jpg", true);
if (Find(bitmap, bitmap2) == null)
{
Console.WriteLine("Not found.");
}
else {
Console.WriteLine($"{Find(bitmap, bitmap2)}.");
}
System.Threading.Thread.Sleep(500); // loop to detect new position
counter++;
}
}
Edit 7.11.2020: I published a Sample Project and Video as a result of this question:
Sample:
https://github.com/DataJuggler/SubImageCreator
Could be called SubImageSearcher also.
Video:
How To Search Images Using C# And DataJuggler.PixelDatabase
https://youtu.be/JKc7QtdaxWY
Search for a sub image in a larger image using C#.
Nuget Updates
DataJuggler.PixelDatabase .Net Core
DataJuggler.PixelDatabase.Net .Net Framework
// Return a sub image after a PixelDatabase is loaded
Bitmap subImage = CreateSubImage(Point topLeft, Rectangle size);
// Search for a sub image inside a larger image
SearchResult result = PixelDatabase.SearchForSubImage(bitmap, searchDepth);
Search Result contains a Point of the top left and a score. A score of zero is a perfect match.
The sample project draws a little yellow rectangle on the Top Left hand corner if the sub image is found inside the larger image.
--
End Edit
Are the images something you can share? If not a sample image might give something to work with. If yes, tell me what you are trying to find and maybe I can find "where the pixels are you are searching for".
Another thing to watch out for is if either of the images were stretched, any difference like that could distort exact pixel colors and locations.
Are you using .Net Framework or .Net Core?
I have a Nuget package that might help you with this:
Nuget:
DataJuggler.PixelDatabase .Net Core
DataJuggler.PixelDatabase.Net .Net Framework
Both packages were updated yesterday.
Showing a .Net Core example
using DataJuggler.PixelDatabase;
// local
DataJuggler.PixelDatabase.PixelDatabase pixelDatabase = null;
// Load the pixelDatabase (you can also pass in a Bitmap or an Image also)
pixelDatabase = PixelDatabaseLoader.LoadPixelDatabase(FullImagePath, null);
// get information about a pixel at the coordinates given
PixelInformation pixel = PixelDatabase.GetPixel(x, y);
The pixel information object contains the Red Green and Blue colors, plus the X, Y and a lot of read only properties such as Total, BlueRed, MinMaxDifference, etc., that make querying and updating Bitmaps pretty simple.
Another thing you could do is if you know of a certain color you are looking for:
// color
Color color = Color.FromArgb(128, 55, 92);
// get the pixels that match a certain color
List<PixelInformation> pixels = PixelDatabase.GetPixels(color);
That might help you locate certain groups of pixels in the image.
The same code that powers the Nuget packages is also used on my site:
https://PixelDatabase.Net
The site and the Nuget packages use a language called Bitmap Query Language, or BQL for short. This is very similar to SQL for databases if you have learned that yet.
There are lots of help and videos, but if you do any Pixel querying, I think you will find it worth the price of free.
The full source code is on Git Hub:
https://github.com/DataJuggler/PixelDatabase.Net .Net Framework
https://github.com/DataJuggler/PixelDatabase .Net Core
And Help is located here:
https://pixeldatabase.net/Help
And lots of videos here:
https://www.youtube.com/playlist?list=PLKrW5tXCPiX2PxrLPszDzlcEZwQG-Qb8r
I've been unable to find much documentation for properly using the MatrixBox from the Emgu.CV.UI.
I'm using EmguCV version 3.4.1 and I would like to use the MatrixBox to show a live update of pixel values from a video I'm streaming.
Winforms Matrixbox Control
private void BufferReceiver_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
Matrix<UInt16> matrix = new Matrix<UInt16>(mat.Rows, mat.Cols, mat.NumberOfChannels);
Matrix<UInt16> subMatrix = matrix.GetSubRect(new Rectangle(0, 0, 10, 10));
matrixBox1.Matrix = subMatrix;
matrixBox1.Refresh();
}
I'm able to display a 10x10 matrix of the pixel values, but when the above method executes again, the matrix grows to 20x20, but replaces the 10x10 matrix with the updated values.
First Run
Second Run
I can see from the OpenCv source that the MatrixBox is just updating a DataGridView, but I can't find a way to clear the MatrixBox before updating the values.
You may try
Size sz = matrixBox1.Size;
Point lc = matrixBox1.Location;
int ti = matrixBox1.TabIndex;
matrixBox1.Dispose();
matrixBox1 = New Emgu.CV.UI.MatrixBox;
matrixBox1.Parent = this;
matrixBox1.Location = lc;
matrixBox1.Matrix = Nothing;
matrixBox1.Name = "MatrixBox1";
matrixBox1.Size = sz;
matrixBox1.TabIndex = ti;
matrixBox1.Matrix = subMatrix;
I am new to DirectX. I am trying to write a custom IP camera video player and for which I am using DirectX11 to render the decoded image with Wpf Gui as my front end.
I am a c# developer and have used managed directx which is no longer updated by microsoft hence moved to wpf and directx11.
All parts of my application up to the rendering of the frames is working fine.
I have managed to create a D3DImage source which will be used in Wpf app, successfully create my viewports and my device including my shared resource since D3DImage only works Directx9. I am using SharpDX as the wrapper for DirectX API.
Now my problem is I can't seem to find a way to create a texture/update a texture from decoded image bytes or what would be the correct way to do so in order to render the decoded image from the bytes received.
Any help on this would be great or if someone can direct me to the right direction as to how this is to be approached?
Thanks.
After nearly 2 weeks of searching and trying to find the solution to my stated problem, i have finally found it as below.
However, this does display the image but not as expected but i believe it is a start for me as the code below answers my question originally asked.
Device.ImmediateContext.ClearRenderTargetView(this.m_RenderTargetView, Color4.Black);
Texture2DDescription colordesc = new Texture2DDescription
{
BindFlags = BindFlags.ShaderResource,
Format = m_PixelFormat,
Width = iWidth,
Height = iHeight,
MipLevels = 1,
SampleDescription = new SampleDescription(1, 0),
Usage = ResourceUsage.Dynamic,
OptionFlags = ResourceOptionFlags.None,
CpuAccessFlags = CpuAccessFlags.Write,
ArraySize = 1
};
Texture2D newFrameTexture = new Texture2D(this.Device, colordesc);
DataStream dtStream = null;
DataBox dBox = Device.ImmediateContext.MapSubresource(newFrameTexture, 0, MapMode.WriteDiscard, 0, out dtStream);
if (dtStream != null)
{
int iRowPitch = dBox.RowPitch;
for (int iHeightIndex = 0; iHeightIndex < iHeight; iHeightIndex++)
{
//Copy the image bytes to Texture
dtStream.Position = iHeightIndex * iRowPitch;
Marshal.Copy(decodedData, iHeightIndex * iWidth * 4, new IntPtr(dtStream.DataPointer.ToInt64() + iHeightIndex * iRowPitch), iWidth * 4);
}
}
Device.ImmediateContext.UnmapSubresource(newFrameTexture, 0);
Device.ImmediateContext.CopySubresourceRegion(newFrameTexture, 0, null, this.RenderTarget, 0);
var shaderRescVw = new ShaderResourceView(this.Device, this.RenderTarget);
Device.ImmediateContext.PixelShader.SetShaderResource(0, shaderRescVw);
Device.ImmediateContext.Draw(6, 0);
Device.ImmediateContext.Flush();
this.D3DSurface.InvalidateD3DImage();
Disposer.SafeDispose(ref newFrameTexture);
With the code above i am now able to populate the texture with the new images data i receive but the images are not being rendered in correct colors/pixels as shown within the red box in the image below.
Screenshot of the rendered image:
The image bytes are received via decoder in BGRA32 pixel format.
Any suggestion to resolve this would be very helpful.