I've created a dll to pass two images from unity to opencv and return a struct. But I'm getting a run time c++ error.
This is the code I used in C++:
struct rigidTransform
{
rigidTransform(double eular_angle_x, double eular_angle_y, double eular_angle_z, double eular_dis_x, double eular_dis_y, double eular_dis_z) : Eular_angle_x(eular_angle_x), Eular_angle_y(eular_angle_y), Eular_angle_z(eular_angle_z), Eular_dis_x(eular_dis_x), Eular_dis_y(eular_dis_y), Eular_dis_z(eular_dis_z) {}
double Eular_angle_x;
double Eular_angle_y;
double Eular_angle_z;
double Eular_dis_x;
double Eular_dis_y;
double Eular_dis_z;
};
struct Color32
{
uchar r;
uchar g;
uchar b;
uchar a;
};
extern "C" rigidTransform __declspec(dllexport) __stdcall findPose(Color32* img1, Color32* img2, int width, int height)
{
Mat Img1(height, width, CV_8UC4, img1), Img2(height, width, CV_8UC4, img2);
//my pose estimation code here
return Tr;
}
And my C# code used in unity is:
using System.Collections;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using UnityEngine;
public struct regidTransform
{
public double eular_angle_x;
public double eular_angle_y;
public double eular_angle_z;
public double eular_dis_x;
public double eular_dis_y;
public double eular_dis_z;
}
public class UI : MonoBehaviour
{
[DllImport("test2", EntryPoint = "findPose", CallingConvention = CallingConvention.Cdecl)]
public unsafe static extern regidTransform findPose(Color32[] img1, Color32[] img2, int width, int height);
public regidTransform results_;
public WebCamTexture webcamTexture;
public Color32[] img1;
public Color32[] img2;
void Start()
{
results_ = new regidTransform();
webcamTexture = new WebCamTexture();
webPlane.GetComponent<MeshRenderer>().material.mainTexture = webcamTexture;
webcamTexture.Play();
if(webcamTexture.isPlaying)
{
img1 = webcamTexture.GetPixels32();
System.Array.Reverse(img1);
img2 = webcamTexture.GetPixels32();
System.Array.Reverse(img2);
unsafe
{
results_ = findPose(img1, img2, webcamTexture.width, webcamTexture.height);
}
}
}
void Update()
{
}
}
When I run this code, it throws an error saying:
Runtime Error!
Program:
This Application has requested the Runtime to terminate it in an
unusual way. Please contact the application support team.
Kindly assist me on what wrong I've made in this code.
Thanking You in advance!
UPDATED:
With the help of #Programmer I found that the code is getting crashed when calling calcOpticalFlowPyrLK(). And when checked if I'm getting the image correctly using imshow() function, I found it results a complete black image.
Finally I figured it out.
Yes that code worked well. and there is no error in my C++ code or C# code. The reason that resulted a complete black image for me is, I call my dll fuction in side strat() function of unity, which only execute once. So I found in my case, that the typical 1st image acquired to dll will mostly a black one, this was proved when I tested with imshow() function inside the dll. The reason it throws error is because I'm passing that black image to calcOpticalFlowPyrLK() function, which doesn't have any feature points to track.
So I created another function inside dll to check if the true image is acquired before passing it to my code by checking for feature points.
Something like:
featureDetection(frame_1, points1);
if (points1.size() > MIN_NUM_FEATURES)
{
//Pass the frame to my code.
}
Now things will perfectly work. My sincere thanks to the #Programmer who helped me to figure this out.
Related
I have made an OpenCV c++ plugin for unity as shown below:
bool resultShare(unsigned char* data, int width, int height)
{
Mat img = Mat(height, width, CV_8UC3, data);
if (img.empty())
{
a = false;
}
else if (!img.empty())
{
a = true;
namedWindow("Image", WINDOW_NORMAL);
imshow("Image", img);
waitKey(0);
}
return a;
}
But I don't know how to pass Image from unity to that C++ plugin
I have tried many things but I am not able to understand the pointers in c#
unsafe, fixed, etc ...
Please help me to solve this problem!!
I want to get the screen size of the primary screen, without adding any references (e.g. WinForms or Presentation). I found a similar question here, however there is no solution which doesn't include downloading or something like that.
But I want to make a method, which can be executed in the C# interactive on any other pc. Therefore I need a solution that doesn't reference other stuff than the standard (E.g. System, System.Core, ... is allowed).
I do know this is possible with
System.Windows.Forms.Screen.PrimaryScreen.Bounds;
but as this requires the System.Windows.Forms reference, it's not suitable for me. But basically the result of this snippet is what I want to get without references.
Here's an example I came up with.
I have noticed that it does not work correctly on High-DPI Screens. It will report the apparent resolution, not the actual resolution.
static void Main(string[] args)
{
var size = GetScreenSize();
Console.WriteLine(size.Length + " x " + size.Width);
Console.ReadLine();
}
static Size GetScreenSize()
{
return new Size(GetSystemMetrics(0), GetSystemMetrics(1));
}
struct Size
{
public Size(int l, int w)
{
Length = l;
Width = w;
}
public int Length { get; set; }
public int Width { get; set; }
}
[DllImport("User32.dll", ExactSpelling = true, CharSet = CharSet.Auto)]
public static extern int GetSystemMetrics(int nIndex);
If you don't want to use those libraries, You'll probably need to use native methods. I can't think of a way around this, as you'll need to communicate with the system any way you go.
A good place to start would be the source of System.Windows.Forms.Screen
Here's a link to the source. I bet you could strip down their code, and get the bare minimum.
I actually found a solution to this:
using System;
using System.Runtime.InteropServices;
static void Main()
{
EnumWindows(E, IntPtr.Zero);
Console.Write($"{_.Item1}x{_.Item2}");
}
struct R
{
int l;
int t;
public int r;
public int b;
public override string ToString() => $"{l},{t},{r},{b}";
public bool i() => l == 0 && r != 00;
}
static (int, int) _;
static bool E(IntPtr w, IntPtr l)
{
var r = new R();
GetWindowRect(w, ref r);
if (r.i() && _.Item1 == 0)
_ = (r.r, r.b);
return true;
}
delegate bool P(IntPtr w, IntPtr l);
[DllImport("user32.dll")]
static extern bool EnumWindows(P e, IntPtr l);
[DllImport("user32.dll")]
static extern bool GetWindowRect(IntPtr w, ref R r);
Main()
Paste this into your interactive and it should output the screen resolution - at least it does for me.
Don't ask me how it works, it's stumped together from different tutorials and pressed into the interactive. Therefore I can't guarantee this will work on every pc.
Could somebody else please test this?
I see a lot of programmers wanting to converting stuff INTO Bitmap, but I'm unable to find a suiting solution to the opposite issue.
I'm using AForge.net with Unity, and I'm trying to test it out by applying my processed image to a cube.
My current code looks like this:
using UnityEngine;
using System.Collections;
using System.Drawing;
using AForge;
using AForge.Imaging;
using AForge.Imaging.Filters;
public class Test : MonoBehaviour {
// Use this for initialization
public Renderer rnd;
public Bitmap grayImage;
public Bitmap image;
public UnmanagedImage final;
public byte[] test;
Texture tx;
void Start () {
image = AForge.Imaging.Image.FromFile("rip.jpg");
Grayscale gs = new Grayscale (0.2125, 0.7154, 0.0721);
grayImage = gs.Apply(image);
final = UnmanagedImage.FromManagedImage(grayImage);
rnd = GetComponent<Renderer>();
rnd.enabled = true;
}
// Update is called once per frame
void Update () {
rnd.material.mainTexture = final;
}
}
I get the following error in the line rnd.material.mainTexture = final;:
Cannot implicitly convert type 'AForge.Imaging.UnmanagedImage' to 'UnityEngine.Texture'
I'm unclear if the Managed to Unmanaged convertion is needed.
By reading your code, the question should be "How to convert UnmanagedImage to Texture or Texture2D" since UnmanagedImage(final variable) stores the converted image from UnmanagedImage.FromManagedImage.
UnmanagedImage has a property called ImageData which returns IntPtr.
Luckily, Texture2D, has at least, two functions that loads textures from IntPtr.
Your final variable is a type of UnmanagedImage.
1.Use Texture2D's constructor Texture2D.CreateExternalTexture and it's complimentary function UpdateExternalTexture.
Texture2D convertedTx;
//Don't initilize Texture2D in the Update function. Do in the Start function
convertedTx = Texture2D.CreateExternalTexture (1024, 1024, TextureFormat.ARGB32 , false, false, final.ImageData);
//Convert UnmanagedImage to Texture
convertedTx.UpdateExternalTexture(final.ImageData);
rnd.material.mainTexture = convertedTx;
2.Use Texture2D's LoadRawTextureData and it's complimentary function Apply.
Texture2D convertedTx;
//Don't initilize Texture2d in int the Update function. Do in the Start function
convertedTx = new Texture2D(16, 16, TextureFormat.PVRTC_RGBA4, false);
int w = 16;
int h = 16;
int size = w*h*4;
//Convert UnmanagedImage to Texture
convertedTx.LoadRawTextureData(final.ImageData, size);
convertedTx.Apply(); //Must call Apply after calling LoadRawTextureData
rnd.material.mainTexture = convertedTx;
I'm trying to use some native functions in C# over OpenGL.
I need to draw a Sphere and I read about gluSphere and looked at it on GL\glu.h, but receives a struct object as parameter:
typedef struct GLUquadric GLUquadric;
void APIENTRY gluSphere(GLUquadric *qobj,GLdouble radius,GLint slices,GLint stacks);
I need to create a struct so I can send it to gluSphere.
Is there some place or information of how is GLUquadric defined so I can write it and send it to gluSphere ?
[StructLayout(LayoutKind.Sequential)]
struct GLUquadric{
//What's here???
}
The bigger question is, if you actually want to use GLU at all. GLU has not been maintained for decades and didn't keep up with the development of the OpenGL API. GLU is not part of OpenGL proper, it's a companion library developed by SGI alongside OpenGL and published together with the OpenGL-1.1 specification. This first and latest version of GLU still assumes the presence of a fixed function pipeline and immediate drawing modes. Both have been removed from modern OpenGL.
I need to create a struct so I can send it to gluNewQuadric.
Actually there's no need to know what's inside this struct. It's defined as a opaque pointer type. Think of it as a class instance handle to which you don't have the interface; you can still pass it into the module implementing the class and call global methods on it, but you can't look inside. From the C# perspective it's an unmanaged pointer to something.
EDIT a code example (that I hope is valid C#)
[DllImport("glu32.dll")]
static extern IntPtr gluNewQuadric();
[DllImport("glu32.dll")]
static extern void gluDeleteQuadric(IntPtr quadric);
[DllImport("glu32.dll")]
static extern void gluSphere(IntPtr quadric, double radius, int slices, int stacks);
IntPtr quadric = gluNewQuadric();
gluSphere(quadric, 1, 10, 10);
gluDeleteQuadric(quadric);
That being said if you accept these caveats I wonder if it wouldn't make more sense to port some GLU implementation over to .net/CLI so that it can be used natively from C#.
You can of course also access GLU through unmanaged interfaces. Now my personal experience with C# is little (I've got more experience with F#) and I never left the managed grounds doing unmanaged things. But from my understanding what you have to do there is just define an integer variable large enough to hold a native pointer (and if I'm not mistaken, there already should be such a integer type for holding unmanaged pointers) and use that for the GLUQuadric* type.
Looks like searching in google for
"struct GLUquadric"
didn't give any information or clue... But searching for
"struct GLUquadric{ "
took me to the place I wanted:
OGLES_GLU.h
Struct I found and used and WORKS is:
[StructLayout(LayoutKind.Sequential)]
public struct GLUquadric
{
int normals;
bool textureCoords;
int orientation;
int drawStyle;
}
So now I can use:
[DllImport("glu32.dll")]
static extern void gluSphere(ref GLUquadric qobj, double radius, int slices, int stacks);
public static void Sphere(ref GLUquadric qobject, double Radius, int Slices, int Stacks)
{
gluSphere(ref qobject, Radius, Slices, Stacks);
}
OpenGL draws spheres now.
NOTE: When drawing spheres in imported openGL function, DO NOT CALL gluDeleteQuadric();
Let GC do it's work, just declare a new GLUQuadric() and send it as a ref to gluSphere, else you will have memory problems in your program.
To compliment datenwolf's answer which is not valid in my case:
The implementation of my program is this way:
[StructLayout(LayoutKind.Sequential)]
public struct GLUquadric
{
int normals;
bool textureCoords;
int orientation;
int drawStyle;
public void Init(int norm, int draw, int orient, bool textCoor)
{
normals = norm;
drawStyle = draw;
orientation = orient;
textureCoords = textCoor;
}
}
Use is:
public static void DrawSphere(T Radius, Int32 Slices, Int32 Stacks,
GLU.QuadricDrawStyles Style, GLU.QuadricNormals Normal, Color color)
{
OpenGL.SetColor(color);
GLU.GLUquadric quadric = new GLU.GLUquadric();
quadric.Init((int)Normal, (int)Style, 0, false);
GLU.Sphere(ref quadric, (dynamic)Radius, Slices, Stacks);
}
Implementation is full OO, so every Sphere is isolated from static GL Function as gluQuadricDrawStyle and gluQuadricNormals so leaving struct empty is NOT valid since it will draw nothing.
Asked a few questions about a project I was working on, got some good feedback and made some progress. The idea is to create an application that generates images of fractals, accelerated by CUDA. I am creating the ui in C# and having a DLL do the heavy lifting.
Basically, I am allocating a byte array in C#, passing that to the dll to fill with pixel data, and then using that to create a Bitmap and display that with a Windows Forms PictureBox in the ui. Previous questions have helped - was using dll to allocate memory before, now using consistent calling convention between dll and c#, but the code still gives an System.ArgumentException at "img = new Bitmap(...)
Relevant Code:
C++
extern "C" __declspec(dllexport) void __cdecl generateBitmap(void *bitmap)
{
int width = 1920;
int height = 1080;
int *dev_bmp;
gpuErrchk(cudaMalloc((void**)&dev_bmp, (3*width*height*sizeof(int))));
kernel<<<BLOCKS_PER_GRID, THREADS_PER_BLOCK>>>(dev_bmp, width, height);
gpuErrchk(cudaPeekAtLastError());
gpuErrchk(cudaDeviceSynchronize());
gpuErrchk(cudaMemcpy(bitmap, dev_bmp, (width*height*3), cudaMemcpyDeviceToHost));
cudaFree(dev_bmp);
}
c#
public unsafe class NativeMethods
{
[DllImport(#"C:\Users\Bill\Documents\Visual Studio 2012\Projects\FractalMaxUnmanaged\Debug\FractalMaxUnmanaged.dll", CallingConvention=CallingConvention.Cdecl)]
public static extern void generateBitmap(void *bitmap);
public static Bitmap create()
{
byte[] buf = new byte[1920 * 1080 * 3];
fixed (void* pBuffer = buf)
{
generateBitmap(pBuffer);
}
IntPtr unmanagedPtr = Marshal.AllocHGlobal(buf.Length);
Marshal.Copy(buf, 0, unmanagedPtr, buf.Length);
Bitmap img = new Bitmap(1920, 1080, 3, PixelFormat.Format24bppRgb, unmanagedPtr);
Marshal.FreeHGlobal(unmanagedPtr);
return img;
}
}
//...
private unsafe void mandlebrotButton_Click(object sender, EventArgs e)
{
FractalBox1.Image = (Image)NativeMethods.create();
}
What am I still doing wrong? As far as I can tell, all the parameters are invalid, but I get an invalid parameter exception in System.Drawing when I try to create the bitmap.
I am not sure what happens exactly in your case cause you didn't specify which parameter is invalid in the exception. I see that your stride must not be correct.
stride Type: System.Int32
Integer that specifies the byte offset between the beginning of one
scan line and the next. This is usually (but not necessarily) the
number of bytes in the pixel format (for example, 2 for 16 bits per
pixel) multiplied by the width of the bitmap. The value passed to this
parameter must be a multiple of four..
So your constructor should be like this:
Bitmap img = new Bitmap(1920, 1080, 1920 * 3, PixelFormat.Format24bppRgb, unmanagedPtr);