We Keep Coding

Converting OpenCVSharp4 Rectangle to IronOCR CropRectangle(System.Drawing.Rectangle)

I have a project in which I'm using IronOCR to read an area define by OpenCVSharp4 but the problem I'm encountering is IronOCrs CropRectangle method, it uses System.drawing.rectangle and for some reason my OpenCvSharp.Rect cannot be converted to it, by this I mean when I Finally uses IronOCRs Input.Add(Image, ContentArea) the results I get are not what is expected.
Below the code I have attached a picture of what the code currently produces.
Don't worry about IronOCR not getting the correct letters I believe it has to do with it creating a weird box and some letters getting cut off, it works if I made the area larger for crop rectangle width and height
var Ocr = new IronTesseract();
String[] splitText;
using (var Input = new OcrInput())
{
//OpenCv
OpenCvSharp.Rect rect = new OpenCvSharp.Rect(55, 107, 219, 264);
//IronOCR
Rectangle ContentArea = new Rectangle() { X = rect.TopLeft.X, Y = rect.TopLeft.Y, Height = rect.Height, Width = rect.Width };
CropRectangle r = new CropRectangle(ContentArea);
CordBox.Text = r.Rectangle.ToString();
//OpenCv
resizedMat.Rectangle(rect.TopLeft, rect.BottomRight, Scalar.Blue, 3);
resizedMat.Rectangle(new OpenCvSharp.Point(55, 107), new OpenCvSharp.Point(219, 264), Scalar.Brown, 3);
Cv2.ImShow("resizedMat", resizedMat);
//IronOCR
Input.Add(#"C:\Projects\AnExperiment\WpfApp1\Images\TestSave.PNG", r);
Input.EnhanceResolution();
var Result = Ocr.Read(Input);
ResultBox.Text = Result.Text;
splitText = ResultBox.Text.Split('\n');
}

SO here is the solution I came up with.
This problem is a OpenCvSharp4 one where OpenCvSharp4.Rectangle for some reason does have matching coordinates to System.Drawing.Rectangle. I have posted this on the gitHub for OpenCvSHarp4 and he says its fine, but its not.
So I switched over to Emgu NuGet package its better for C# applications and is a OpenCv Wrapper made for C# (I was just scared of giving it a try before because i never really understood it.)
Emgu uses System.Drawing.Rectangle by default instead of something like OpenCvSharp4.Rectangle so everything matches up nicely.
Mat testMat = new Mat();
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", testMat );
After finding this out the rest was pretty easy so the final code is below on how it was transformed. (For reference Emgu.CV.CVInvoke is how its called and Emgu.CV.BitmapExtension is its own separate NuGet package)
// Get the original Image
fullPage = CvInvoke.Imread(#"C:\Projects\AnExperiment\WpfApp1\Images\TestImageFinalFilled.png");
// Resize it so it works with the cordinates stored previously in a json file
CvInvoke.Resize(fullPage, resizedMat, EmguSetResolution(fullPage, dpi));
// Save the small version so iron ocr doesnt mess up
var bitmap = Emgu.CV.BitmapExtension.ToBitmap(resizedMat);
bitmap.Save(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
// Let user select box
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", resizedMat);
CvInvoke.DestroyWindow("main");
// Draw Rect for debugging
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
// Read section we highlighted by pulling the saved resuze imag as a reference
var Ocr = new IronTesseract();
IronOcr.OcrResult ocrResult;
Ocr.UseCustomTesseractLanguageFile(#"C:\Projects\AnExperiment\WpfApp1\tessdata_best-main\eng.traineddata");
using (var Input = new OcrInput())
{
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
IronOcr.CropRectangle contentArea = new CropRectangle(roi);
Input.AddImage(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG", contentArea);
Input.EnhanceResolution();
Input.Sharpen();
Input.Contrast();
ocrResult = Ocr.Read(Input);
}
File.Delete(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
CvInvoke.Imshow("m", resizedMat);
After all this I have some functions that spit the ocrResult.Text into the textbox and separate certain things I needed from it.

Having issues when attempting to load images into a bitmap/Picture Box Visual Studio 2019

I am attempting to load images into an Picture Box using the following code:
private void button1_Click(object sender, EventArgs e)
{
//Clear the Image Area
if (templateArea.Image != null)
{ templateArea.Image = null; }
//Set Base Variables
Bitmap img = new Bitmap(Resources.BlankBackground);
Graphics gpx = Graphics.FromImage(img);
Bitmap newBitmap = null;
string bitmapToLoad;
ResourceManager rm;
//Loop Lists
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
rm = Resources.ResourceManager;
newBitmap = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
gpx.Dispose();
}
//Set Image Area to newBitmap
templateArea.Image = newBitmap;
}
when trying to run the code I get errors on the second loop where i = 1 in this portion:
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
//given error: System.ArgumentException: 'Parameter is not valid.' //Does not point to anything specific.
the comps list contains lists formatted as such:
//These listed items are defaulted at program start but more are added while running
// p elementName type sub type x y sX sY R G B O i
eleComps = new List<Object> {0,"Background","Static","BlankBackground",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Border", "Static","CardBorder_02", 0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Cut Line", "Static","00_Card_CutLine",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
Each possible "sub type" is a bitmap file in resources, and there are more than just those listed above.
What I have tried:
I have consulted the miracle that is Google and YouTube and haven't been able to solve this one. Although I feel like I'm missing something basic I have been looking at it all day and may need some fresh eyes on it to correct something that may be obvious but for some reason I seem to be missing.
What I am looking for:
I am trying to create an image viewer that will overlap images or parts of images (.png) at specific coordinates in an Picture Box. The comps list will be added to and removed from actively during application use and as such using a variable to reference the Resources is necessary and that portion of the code has given me the most difficult time out of the supplied code samples.
UPDATE 1: changed image box to picture box to reduce confusion.
UPDATE 2: NOTE: This sample achieves the desired end result, but does not allow me to use the variables as needed to change positions, scale, color, and opacity.
//Clear Img Area
templateArea.InitialImage = null;
//Find Bitmap(s)
string p = #"C:\Users\david\Desktop\Test Fields\Default\";
Bitmap img1 = new Bitmap(p + "00_Empty.png", true);
Bitmap img2 = new Bitmap(p + "CardBorder_02.png", true);
Bitmap img3 = new Bitmap(p + "00_Card_CutLine.png", true);
//Set initial image for image area
Graphics gpx = Graphics.FromImage(img1);
//place an image at x, y, Width, Height
gpx.DrawImage(img2, 0, 0, img2.Width, img2.Height);
gpx.DrawImage(img3, 0, 0, img3.Width, img3.Height);
//clear gpx cache
gpx.Dispose();
//set image area to modified img1
templateArea.Image = img1;
Also please let me know if I'm just plain going about this in the wrong way, and what I need to do to get it right.
UPDATE 3: I seem to have gotten it to work properly with this variation, using the originally supplied list of listed parameters. Thanks to Idle_Mind's pointers I was able to figure it out.
Bitmap newBitmap_01 = new Bitmap(templateArea.Width, templateArea.Height);
Graphics gpx = Graphics.FromImage(newBitmap_01);
string bitmapToLoad;
ResourceManager rm = Resources.ResourceManager;
Bitmap newBitmap_02;
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
newBitmap_02 = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap_02,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap_02.Width,
newBitmap_02.Height);
}
gpx.Dispose();
templateArea.Image = newBitmap_01;

Dynamically display an array of PictureBoxes - Performance issue

I'd like to display coverarts for each album of an MP3 library, a bit like Itunes does (at a later stage, i'd like to click one any of these coverarts to display the list of songs).
I have a form with a panel panel1 and here is the loop i'm using :
int i = 0;
int perCol = 4;
int disBetWeen = 15;
int width = 250;
int height = 250;
foreach(var alb in mp2)
{
myPicBox.Add(new PictureBox());
myPicBox[i].SizeMode = System.Windows.Forms.PictureBoxSizeMode.StretchImage;
myPicBox[i].Location = new System.Drawing.Point(disBetWeen + (disBetWeen * (i % perCol) +(width * (i % perCol))),
disBetWeen + (disBetWeen * (i / perCol))+ (height * (i / perCol)));
myPicBox[i].Name = "pictureBox" + i;
myPicBox[i].Size = new System.Drawing.Size(width, height);
myPicBox[i].ImageLocation = #"C:/Users/Utilisateur/Music/label.jpg";
panel1.Controls.Add(myPicBox[i]);
i++;
}
I'm using the same picture per picturebox for convenience, but i'll use the coverart embedded in each mp3 file eventually.
It's working fine with an abstract of the library (around 50), but i have several thousands of albums. I tried and as expected, it takes a long time to load and i cannot really scroll afterward.
Is there any way to load only what's displayed ? and then how to assess what is displayed with the scrollbars.
Thanks

Winforms really isn't suited to this sort of thing... Using standard controls, you'd probably need to either provision all the image boxes up front and load images in as they become visible, or manage some overflow placeholder for the appropriate length so the scrollbars work.
Assuming Winforms is your only option, I'd suggest you look into creating a custom control with a scroll bar and manually driving the OnPaint event.
That would allow you to keep a cache of images in memory to draw the current view [and a few either side], while giving you total control over when they're loaded/unloaded [well, as "total" as you can get in a managed language - you may still need tune garbage collection]
To get into some details....
Create a new control
namespace SO61574511 {
// Let's inherit from Panel so we can take advantage of scrolling for free
public class ImageScroller : Panel {
// Some numbers to allow us to calculate layout
private const int BitmapWidth = 100;
private const int BitmapSpacing = 10;
// imageCache will keep the images in memory. Ideally we should unload images we're not using, but that's a problem for the reader
private Bitmap[] imageCache;
public ImageScroller() {
//How many images to put in the cache? If you don't know up-front, use a list instead of an array
imageCache = new Bitmap[100];
//Take advantage of Winforms scrolling
this.AutoScroll = true;
this.AutoScrollMinSize = new Size((BitmapWidth + BitmapSpacing) * imageCache.Length, this.Height);
}
protected override void OnPaint(PaintEventArgs e) {
// Let Winforms paint its bits (like the scroll bar)
base.OnPaint(e);
// Translate whatever _we_ paint by the position of the scrollbar
e.Graphics.TranslateTransform(this.AutoScrollPosition.X,
this.AutoScrollPosition.Y);
// Use this to decide which images are out of sight and can be unloaded
var current_scroll_position = this.HorizontalScroll.Value;
// Loop through the images you want to show (probably not all of them, just those close to the view area)
for (int i = 0; i < imageCache.Length; i++) {
e.Graphics.DrawImage(GetImage(i), new PointF(i * (BitmapSpacing + BitmapWidth), 0));
}
}
//You won't need a random, just for my demo colours below
private Random rnd = new Random();
private Bitmap GetImage(int id) {
// This method is responsible for getting an image.
// If it's already in the cache, use it, otherwise load it
if (imageCache[id] == null) {
//Do something here to load an image into the cache
imageCache[id] = new Bitmap(100, 100);
// For demo purposes, I'll flood fill a random colour
using (var gfx = Graphics.FromImage(imageCache[id])) {
gfx.Clear(Color.FromArgb(255, rnd.Next(0, 255), rnd.Next(0, 255), rnd.Next(0, 255)));
}
}
return imageCache[id];
}
}
}
And Load it into your form, docking to fill the screen....
public Form1() {
InitializeComponent();
this.Controls.Add(new ImageScroller {
Dock = DockStyle.Fill
});
}
You can see it in action here: https://www.youtube.com/watch?v=ftr3v6pLnqA (excuse the mouse trails, I captured area outside the window)

How to display real time updates to an image in Unity?

I am trying to create an application that generates a bitmap image every frame based on user actions and have it display that image to the screen. I would like the application to also be able to update that image in unity in real time as soon as the user makes another action.
I have created an application that does this and it works. However, it is veryyyy slow. My Update() method is attached below.
My idea was:
Capture user data (mouse location).
Convert that data into a special signal format that another program recognizes.
Have that program return a bitmap image.
Use that bitmap as a texture and update the existing texture with the new image.
Code:
UnityEngine.Texture2D oneTexture;
Bitmap currentBitmap;
private int frameCount = 0;
void Update()
{
// Show mouse position in unity environment
double xValue = Input.mousePosition.x;
double yValue = Screen.height - Input.mousePosition.y;
myPoints = "" + xValue + "," + yValue + Environment.NewLine;
// Show heatmap being recorded.
signals = Program.ConvertStringToSignalsList(myPoints);
currentBitmap = Program.CreateMouseHeatmap(Screen.width, Screen.height, signals);
// Update old heatmap texture.
UpdateTextureFromBitmap();
ri.texture = oneTexture;
ri.rectTransform.sizeDelta = new Vector2(Screen.width, Screen.height);
frameCount++;
// Write points to Database.
StartCoroutine(WriteToDB(xValue, yValue)); // <<<<< Comment out when playback.
}
private void UpdateTextureFromBitmap()
{
// Convert Bitmap object into byte array instead of creating actual
// .bmp image file each frame.
byte[] imageBytes = ImageToBytes(currentBitmap);
BMPLoader loader = new BMPLoader();
BMPImage img = loader.LoadBMP(imageBytes);
// Only initialize the Texture once.
if (frameCount == 0)
{
oneTexture = img.ToTexture2D();
}
else
{
Color32[] imageData = img.imageData;
oneTexture.SetPixels32(imageData);
oneTexture.Apply();
}
}
I was wondering if someone could help me improve the rate at which the image updates to the screen? I know that it is possible to make this program much faster but I am so new to unity and C# that I don't know how to make that happen. Also if there is a completely different way that I should be going about doing this then I am open to that too. Any help would be appreciated. Thanks!
Also, below is a screenshot of the Profiler showing the breakdown of CPU Usage. Currently it looks like every frame is taking about 500ms.

OpenNI show RGB camera feed

Have spent the last week trying to have my C# program show both the depth feed and the RGB feed (similar to how /Samples/Bin64/Release/NiViewer64.exe shows both feeds in a window).
Project specs: C# - VS2013 Express OpenNI - Using a modified
SimpleViewer.net (has two feeds of depth). Asus Xtion Pro Live
I would like one of the feeds to become a normal camera feed instead of the depth feed.
I'm guessing it has something to do with this:
MapOutputMode mapMode = this.depth.MapOutputMode;
this.bitmap = new Bitmap((int)mapMode.XRes, (int)mapMode.YRes,System.Drawing.Imaging.PixelFormat.Format24bppRgb);
Any ideas?

Finally figured it out, thanks to another programmer.
image = context.FindExistingNode(NodeType.Image) as ImageGenerator;
ImageMetaData imd = image.GetMetaData();
lock (this)
{
//**************************************//
//***********RGB Camera Feed************//
//**************************************//
Rectangle rect = new Rectangle(0, 0, this.bitmap.Width, this.bitmap.Height);
BitmapData data = this.camera_feed.LockBits(rect, ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
byte* pDest = (byte*)data.Scan0.ToPointer();
byte* imstp = (byte*)image.ImageMapPtr.ToPointer();
// set pixels
for (int i = 0; i < imd.DataSize; i += 3, pDest += 3, imstp += 3)
{
pDest[0] = imstp[2];
pDest[1] = imstp[1];
pDest[2] = imstp[0];
}
and declare this somewhere:
public ImageGenerator image;