We Keep Coding

How to use a dll that returns array pointer in python?

I'm trying to get out an array in a pythonic format [numpy or list] form a DLLwritten in c#/c++. This DLL gives out the pointer to the memory and not the evaluated array. How can I access the memory in a FAST way (I wrote the DLL to speedup the code)?
I have used the pythonnet module and i managed to get the memory pointer as a but I'm failing in extracting the memory pointer as an integer or binary and then to get the data of the output vector.
Here the python code:
import numpy as np
import clr
clr.AddReference(R'...\PyDLL.dll')
from PyDLL import StringLibrary
my_instance = StringLibrary()
x = my_instance.arrayShift(np.array([1,2,3,4,5,6,7,8,9,0]).reshape(-1))
And then the C# code
namespace PyDLL
{
public class StringLibrary
{
public static double[] arrayShift(double[] array)
{
int b = array.Length;
double[] newArray = new double[b];
for(int i=0; i<b; i++)
{
newArray[i] = array[b-i-1];
}
return newArray;
}
}
}
I expect that the output x in python is a list or a np array, while now it is a System.Double[] object.
the expected output of x is [0,9,8,7,6,5,4,3,2,1]
Thank you for any help

After many tries and researches, I managed to solve the problem in this way:
Cpp part:
Open Visual Studio, then create a new project (Dynamic-Link Library).
Name the project and add a new file in the project (Header file) and name it with the same project name.
Insert the following code in the header:
// headerName.h - Contains declarations of math functions
#pragma once
#ifdef HEADERNAME_EXPORTS
#define HEADERNAME_API __declspec(dllexport)
#else
#define HEADERNAME_API __declspec(dllimport)
#endif
// Define the functions that will be called using the DLL
extern "C" HEADERNAME_API void funName1(...);
extern "C" HEADERNAME_API void funName2(...);
extern "C" HEADERNAME_API void funName3(...);
...;
In the folder 'Source Files' add a new file with extension .cpp (call it with the same name as the project), then write the function in this .cpp file. Here an example code:
#include "pch.h" // use stdafx.h in Visual Studio 2017 and earlier COMPULSORY TO ADD
#include "HEADERNAME.h" // COMPULSORY TO ADD
#include lib#1
#include lib#2
#include lib#3
...
using namespace std;
void funName1(...)
{
...
}
void funName2(...)
{
...
}
void funName3(...)
{
...
}
To be able to pass the arrays between python and c++, in my solution, the use of pointers is needed and no return must be used. The output variable is passed as an input variable, initialized in python, using a pointer. In the DLL the memory cells of the output variables are re-wrote and, doing so, when the DLL completes the function, the output is computed and already usable by python without return.
Here an example of how to write a function using pointers for input and output arrays:
void funName(double* inputArray_pointer, double* outputArray_pointer, int extraVariable)
{
double inner_var = 100;
for (int i = 0; i < extraVariable; i++)
{
inner_var = min(inner_var, *(inputArray_pointer+i));
*(outputArray_pointer + i) = inner_var;
}
}
Then rebuild the DLL.
Python part
To use correctly a DLL in the python code, the modules 'ctypes' and 'numpy' must be imported.
Here follows an example of the python code (This will use the example DLL of point (9)):
import numpy as np
import ctypes
cpp_fun = ctypes.CDLL(R'pathToDll/DLLname.dll')
# Define the types of the arguments of the DLL (pointer, int, double, ecc)
cpp_fun.funName.argtypes = [np.ctypeslib.ndpointer(),
np.ctypeslib.ndpointer(),
ctypes.c_int]
# Allocate or define the input variables using numpy or the standard scalar of python (int, float, ecc..)
inputArray = np.array([0,1,2,3,4,5,6,7,8,9,...,1000]).reshape(-1).astype('f8')
outputArray = np.zeros([inputArray.shape[0]])astype('f8')
extraVariable = inputArray.shape[0]
cpp_fun.funName(inputArray, outputArray, extraVariable)
Doing so, in the variable 'outputArray' we will find the result of the calculations performed inside the DLL as a np array structure.

PHP like serialized string in .NET

I have a legacy PHP application which is using PHP's serialize() method to serialize the class and store it to a file and there are other applications using this string using unserialize() method to create object of the serialized class and use the data associated. I need to achieve similar serialization technique in .NET so that the output is same as PHP. I am thinking to use Reflection for this but still need to identify how I will make use of that. I have also tried the .NET serialization but the output is totally different.
Is there any other way I can do this?
consider example below -
sample php class
class SerializeTest
{
var $test = 0;
var $test1 = "testing";
var $test2 = 1;
var $test3 = "testing1";
public function __construct()
{
echo serialize($this);
}
}
serialized string
O:13:"SerializeTest":4:{s:4:"test";i:0;s:5:"test1";s:7:"testing";s:5:"test2";i:1;s:5:"test3";s:8:"testing1";}
sample .NET Class
class SerializeDemo
{
internal int test = 0;
internal string test1 = "testing";
internal int test2 = 1;
internal string test3 = "testing3";
}
required serialized string
O:13:"SerializeTest":4:{s:4:"test";i:0;s:5:"test1";s:7:"testing";s:5:"test2";i:1;s:5:"test3";s:8:"testing1";}

Try the Sharp Serialization Library
Sharp Serialization Library serializes and deserializes primitives,
ArrayLists and Hashtables, compatible with PHP serialize(). Use it for
SOAP/Web Services communications where Hashtables cannot be passed
otherwise, or saving to a file readable by php.

How to pass a String pointer/reference from C++ to C#?

I'm writing a C# com library. I need a function that takes string from C++ and modifies string, after I have modified this string in my library modified value should be usable by C++ code. Here is my function (it's not working),
C# Interface,
Int32 testString([param: MarshalAs(UnmanagedType.LPStr)] ref string strData);
C# Implementation,
class Test:ITest
{
public int testString(ref string strData)
{
strData = strData + strData;
Console.WriteLine("strData in lib :"+strData);
return 0;
}
}
C++ code (I missed the parts, registering com lib, importing tlb etc.),
ITest * mInterface = NULL;
//...initilizing interface , skipped since this parts has no problem
LPSTR tt = "OLA";
mInterface ->testString(&tt); //It crashes when comes here ..
cout << tt <<endl;

I have found a solution to my problem , I am not sure but maybe we cannot initilized object from c++ to c# using com interop.If a use a null pointer to c# com , and initilize that pointer inside c# library code it worked .You can see the sample below ;
c# library interface
int testString(out string val);
c# library imp
class Test
{
int testString(out string val)
{
val = "Hello world"; //now user of this method can use val
return 0;
}
}
c++ usage
#pragma warning (disable: 4278)
#import "../yourtlb.tlb" no_namespace named_guids
ITestInterface * fpInterface = NULL;
int retval = 1;
CoInitialize(NULL);
HRESULT hr = CoCreateInstance(CLSID_Test ,
NULL, CLSCTX_INPROC_SERVER,
IID_ITestInterface, reinterpret_cast<void**>(&fpInterface));
if(!FAILED(hr))
{
BSTR val = NULL;
int ret = fpInterface->testString(val);
cout <<val<<endl;
}
Of course before using code in c++ , dont forget to generate tlb file from your c# project , and dont forget to edit registry files.Here you can see a good sample

How to access .h file information from C#

When using the NI tools to create a FPGA bit file, LabView generates a C .h file that contains information for communicating to the FPGA application such as the signature (a const) and the register addresses (as enums).
The challenge is that my application is written in C# and I need access to those register addresses.
The only options I can think of are:
Cut-and-paste the contents of the .h file into a .cs file ~~~~ ... gives me the shivers, C# is one developer and FPGA is another.
Parse the .h file ~~~~ ... gives me the shivers too - writing my own language parser :(
There has got to be a better way!!!
Sample .h:
/*
* Generated with the FPGA Interface C API Generator 12.0.0
* for NI-RIO 12.0.0 or later.
*/
#ifndef __NiFpga_M_OTRM_OVDFPGA_h__
#define __NiFpga_M_OTRM_OVDFPGA_h__
#ifndef NiFpga_Version
#define NiFpga_Version 1200
#endif
#include "NiFpga.h"
/**
* The filename of the FPGA bitfile.
*
* This is a #define to allow for string literal concatenation. For example:
*
* static const char* const Bitfile = "C:\\" NiFpga_M_OTRM_OVDFPGA_Bitfile;
*/
#define NiFpga_M_OTRM_OVDFPGA_Bitfile "NiFpga_M_OTRM_OVDFPGA.lvbitx"
/**
* The signature of the FPGA bitfile.
*/
static const char* const NiFpga_M_OTRM_OVDFPGA_Signature = "D0751ADE3A0EC976606D63C425B35E85";
typedef enum
{
NiFpga_M_OTRM_OVDFPGA_IndicatorBool_FallingEdge = 0x8132,
NiFpga_M_OTRM_OVDFPGA_IndicatorBool_StopSignal = 0x813A,
} NiFpga_M_OTRM_OVDFPGA_IndicatorBool;
....
#endif

given its likely only a tiny subset of things you need to parse, in fact, it won't have to parse but probably just do a few text manipulations / search replaces. It shouldn't be too hard. Also, you can use C++.NET to wrap the things into an API you can use from C#.
Also you may want to look at something like 'SWIG'
http://www.swig.org/

I solved the problem by adding a C++/CLI project to the solution that includes the .h file and exposes the information through managed static methods.
public ref class OTRM_Ovd_FPGA
{
public:
static System::String^ GetFilename() { return gcnew System::String(NiFpga_M_OTRM_OVDFPGA_Bitfile);};
static System::String^ GetSignature() { return gcnew System::String(NiFpga_M_OTRM_OVDFPGA_Signature);};
enum class RegisterAddress : System::UInt16
{
InitialPulseAmplitude = NiFpga_M_OTRM_OVDFPGA_IndicatorU16_ExtPulseAmplitudeDisplay,
CurrentPulseAmplitude = NiFpga_M_OTRM_OVDFPGA_IndicatorU16_ExtPulseAmplitudeDisplay,
PulseAmplitudeDecrement = NiFpga_M_OTRM_OVDFPGA_ControlU16_Decrement_Step,
PollPeriod = NiFpga_M_OTRM_OVDFPGA_ControlI16_Frequency_ms,
HvprotState = NiFpga_M_OTRM_OVDFPGA_ControlI16_HVProt,
PaceThreshold = NiFpga_M_OTRM_OVDFPGA_ControlI16_VVIThreshold,
};
};

Is it possible to serialize a C# code block?

I'm using C# with .NET 3.5. Is it possible to serialize a block of code, transmit it somewhere, deserialize it, and then execute it?
An example usage of this would be:
Action<object> pauxPublish = delegate(object o)
{
if (!(o is string))
{
return;
}
Console.WriteLine(o.ToString());
};
Transmitter.Send(pauxPublish);
With some remote program doing:
var action = Transmitter.Recieve();
action("hello world");
My end goal is to be able to execute arbitrary code in a different process (which has no prior knowledge of the code).

YES!!!
We have done this for a very real case of performance. Doing this at runtime or using a DSL was not an option due to performance.
We compile the code into an assembly, and rip the IL out of the method. We then get all the metadata associated with this method and serialize the whole mess via XML, compress it, and put it in our database.
At re-hydration time, we re-constitute the IL with the metadata using the DynamicMethod class, and execute it.
We do this because of speed. We have thousands of little blocks of code. Unfortunately, to compile a block of code and run it on the fly takes at least 250 ms, which is way too slow for us. We took this approach, and it is working REALLY well. At run-time, it takes an unmeasurable amount of time to reconstitute the method and run it.
Only thing to keep an eye on... Signed assemblies and Unsigned assemblies cannot mix the serialized method data.

You could try to use IronPython in your project. It's trivial to do what you are asking in Python. The Python code could call your C# methods. As for security, you could execute the code in a restricted environment of some kind (one example is RestrictedPython).

Generally speaking that sounds like a really bad idea and a big security hole.
You don't want another process to execute any code. Understand what you really need another process to do and build a little DSL around it.

You could also send it as a string then use the CodeDomProvider to compile it, same result. I have an example bit of code thus:
using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
using Microsoft.CSharp;
namespace DynamicCodeApplication
{
class azCodeCompiler
{
private List<string> assemblies;
public azCodeCompiler()
{
assemblies = new List<string>();
scanAndCacheAssemblies();
}
public Assembly BuildAssembly(string code)
{
CodeDomProvider prov = CodeDomProvider.CreateProvider("CSharp");
string[] references = new string[] { }; // Intentionally empty, using csc.rsp
CompilerParameters cp = new CompilerParameters(references)
{
GenerateExecutable = false,
GenerateInMemory = true
};
string path = System.Runtime.InteropServices.RuntimeEnvironment.GetRuntimeDirectory();
cp.CompilerOptions = "#" + path + #"\csc.rsp";
CompilerResults cr = prov.CompileAssemblyFromSource(cp, code);
foreach (CompilerError err in cr.Errors)
{
Console.WriteLine(err.ToString());
}
return cr.CompiledAssembly;
}
public object ExecuteCode(string code,
string namespacename, string classname,
string functionname, bool isstatic, params object[] args)
{
object returnval = null;
Assembly asm = BuildAssembly(code);
object instance = null;
Type type = null;
if (isstatic)
{
type = asm.GetType(namespacename + "." + classname);
}
else
{
instance = asm.CreateInstance(namespacename + "." + classname);
type = instance.GetType();
}
MethodInfo method = type.GetMethod(functionname);
returnval = method.Invoke(instance, args);
return returnval;
}
private void scanAndCacheAssemblies()
{
/*
foreach (string str in Directory.GetFiles(#"C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727"))
{
if (str.Contains(".dll"))
{
foreach (string st in str.Split(new char[] { '\\' }))
{
if (st.Contains(".dll"))
{
assemblies.Add(st);
}
}
}
}
* */
assemblies.Add("Accessibility.dll");
assemblies.Add("AspNetMMCExt.dll");
assemblies.Add("cscompmgd.dll");
assemblies.Add("CustomMarshalers.dll");
assemblies.Add("IEExecRemote.dll");
assemblies.Add("IEHost.dll");
assemblies.Add("IIEHost.dll");
assemblies.Add("Microsoft.Build.Conversion.dll");
assemblies.Add("Microsoft.Build.Engine.dll");
assemblies.Add("Microsoft.Build.Framework.dll");
assemblies.Add("Microsoft.Build.Tasks.dll");
assemblies.Add("Microsoft.Build.Utilities.dll");
assemblies.Add("Microsoft.Build.VisualJSharp.dll");
assemblies.Add("Microsoft.CompactFramework.Build.Tasks.dll");
assemblies.Add("Microsoft.JScript.dll");
assemblies.Add("Microsoft.VisualBasic.Compatibility.Data.dll");
assemblies.Add("Microsoft.VisualBasic.Compatibility.dll");
assemblies.Add("Microsoft.VisualBasic.dll");
assemblies.Add("Microsoft.VisualBasic.Vsa.dll");
assemblies.Add("Microsoft.Vsa.dll");
assemblies.Add("Microsoft.Vsa.Vb.CodeDOMProcessor.dll");
assemblies.Add("Microsoft_VsaVb.dll");
assemblies.Add("mscorlib.dll");
assemblies.Add("sysglobl.dll");
assemblies.Add("System.configuration.dll");
assemblies.Add("System.Configuration.Install.dll");
assemblies.Add("System.Data.dll");
assemblies.Add("System.Data.OracleClient.dll");
assemblies.Add("System.Data.SqlXml.dll");
assemblies.Add("System.Deployment.dll");
assemblies.Add("System.Design.dll");
assemblies.Add("System.DirectoryServices.dll");
assemblies.Add("System.DirectoryServices.Protocols.dll");
assemblies.Add("System.dll");
assemblies.Add("System.Drawing.Design.dll");
assemblies.Add("System.Drawing.dll");
assemblies.Add("System.EnterpriseServices.dll");
assemblies.Add("System.Management.dll");
assemblies.Add("System.Messaging.dll");
assemblies.Add("System.Runtime.Remoting.dll");
assemblies.Add("System.Runtime.Serialization.Formatters.Soap.dll");
assemblies.Add("System.Security.dll");
assemblies.Add("System.ServiceProcess.dll");
assemblies.Add("System.Transactions.dll");
assemblies.Add("System.Web.dll");
assemblies.Add("System.Web.Mobile.dll");
assemblies.Add("System.Web.RegularExpressions.dll");
assemblies.Add("System.Web.Services.dll");
assemblies.Add("System.Windows.Forms.dll");
assemblies.Add("System.XML.dll");
assemblies.Add("vjscor.dll");
assemblies.Add("vjsjbc.dll");
assemblies.Add("vjslib.dll");
assemblies.Add("vjslibcw.dll");
assemblies.Add("vjssupuilib.dll");
assemblies.Add("vjsvwaux.dll");
assemblies.Add("vjswfc.dll");
assemblies.Add("VJSWfcBrowserStubLib.dll");
assemblies.Add("vjswfccw.dll");
assemblies.Add("vjswfchtml.dll");
assemblies.Add("Accessibility.dll");
assemblies.Add("AspNetMMCExt.dll");
assemblies.Add("cscompmgd.dll");
assemblies.Add("CustomMarshalers.dll");
assemblies.Add("IEExecRemote.dll");
assemblies.Add("IEHost.dll");
assemblies.Add("IIEHost.dll");
assemblies.Add("Microsoft.Build.Conversion.dll");
assemblies.Add("Microsoft.Build.Engine.dll");
assemblies.Add("Microsoft.Build.Framework.dll");
assemblies.Add("Microsoft.Build.Tasks.dll");
assemblies.Add("Microsoft.Build.Utilities.dll");
assemblies.Add("Microsoft.Build.VisualJSharp.dll");
assemblies.Add("Microsoft.CompactFramework.Build.Tasks.dll");
assemblies.Add("Microsoft.JScript.dll");
assemblies.Add("Microsoft.VisualBasic.Compatibility.Data.dll");
assemblies.Add("Microsoft.VisualBasic.Compatibility.dll");
assemblies.Add("Microsoft.VisualBasic.dll");
assemblies.Add("Microsoft.VisualBasic.Vsa.dll");
assemblies.Add("Microsoft.Vsa.dll");
assemblies.Add("Microsoft.Vsa.Vb.CodeDOMProcessor.dll");
assemblies.Add("Microsoft_VsaVb.dll");
assemblies.Add("mscorlib.dll");
assemblies.Add("sysglobl.dll");
assemblies.Add("System.configuration.dll");
assemblies.Add("System.Configuration.Install.dll");
assemblies.Add("System.Data.dll");
assemblies.Add("System.Data.OracleClient.dll");
assemblies.Add("System.Data.SqlXml.dll");
assemblies.Add("System.Deployment.dll");
assemblies.Add("System.Design.dll");
assemblies.Add("System.DirectoryServices.dll");
assemblies.Add("System.DirectoryServices.Protocols.dll");
assemblies.Add("System.dll");
assemblies.Add("System.Drawing.Design.dll");
assemblies.Add("System.Drawing.dll");
assemblies.Add("System.EnterpriseServices.dll");
assemblies.Add("System.Management.dll");
assemblies.Add("System.Messaging.dll");
assemblies.Add("System.Runtime.Remoting.dll");
assemblies.Add("System.Runtime.Serialization.Formatters.Soap.dll");
assemblies.Add("System.Security.dll");
assemblies.Add("System.ServiceProcess.dll");
assemblies.Add("System.Transactions.dll");
assemblies.Add("System.Web.dll");
assemblies.Add("System.Web.Mobile.dll");
assemblies.Add("System.Web.RegularExpressions.dll");
assemblies.Add("System.Web.Services.dll");
assemblies.Add("System.Windows.Forms.dll");
assemblies.Add("System.XML.dll");
assemblies.Add("vjscor.dll");
assemblies.Add("vjsjbc.dll");
assemblies.Add("vjslib.dll");
assemblies.Add("vjslibcw.dll");
assemblies.Add("vjssupuilib.dll");
assemblies.Add("vjsvwaux.dll");
assemblies.Add("vjswfc.dll");
assemblies.Add("VJSWfcBrowserStubLib.dll");
assemblies.Add("vjswfccw.dll");
assemblies.Add("vjswfchtml.dll");
return;
}
}
}

Compile it into a separate assembly, send the assembly, have the other process load it.
You might want to consider security implications.
Update: another idea would be to generate an expression tree and use this library to serialize it:
http://www.codeplex.com/metalinq/

It is an interesting challenge, but you should probably describe why you want to do this, since there is a lot of different approaches depending on your objective. As humpohl points out, there is also some pretty serious security issues.
"Serialized code" could just be source code or a compiled assembly, depending on your requirements. You probably don't need to use a seperate code serialization format.
If you want to generate code dynamically and pass that on, you could generate code using CodeDOM and compile it. However, you most likely dont need to generate completely arbitrary code.

Another option is using the DLR, and constraining the code to execute...