We Keep Coding

Undefined symbols when trying to use native C++ .so in Mono Pinvoke

Recently I have been trying to get some Point Cloud Library functionality going in my .NET framework application, and considering that there is no completely functional wrapper for PCL for C#, I made my own for a few functions as a test. Something like this:
[DllImport(DllFilePath, CallingConvention = CallingConvention.Cdecl)]
public extern static IntPtr StatisticalOutlierFilter(IntPtr data, int length, int meanK = 50, float mulThresh = 1.0f);
Which calls a function from a C++ library, such as this:
EXPORT VectorXYZ* StatisticalOutlierFilter(VectorXYZ* data, int length, int meanK, float mulThresh) {
auto processedCloud = process.StatisticalOutlierFilter(data, length, meanK, mulThresh);
auto processedVector = convert.ToVectorXYZ(processedCloud);
return processedVector;
}
Where EXPORT is defined such for gcc:
#define EXPORT extern "C" __attribute__ ((visibility ("default")))
And relevant processing function from PCL is implemented such in a class (note that the returned is a boost shared pointer):
PointCloud<PointXYZ>::Ptr Processors::StatisticalOutlierFilter(VectorXYZ* data, int length, int meanK, float mulThresh) {
auto cloud = PrepareCloud(data, length);
PointCloud<PointXYZ>::Ptr cloud_filtered(new PointCloud<PointXYZ>);
StatisticalOutlierRemoval<PointXYZ> sor;
sor.setInputCloud(cloud);
sor.setMeanK(meanK);
sor.setStddevMulThresh(mulThresh);
sor.filter(*cloud_filtered);
return cloud_filtered;
}
This procedure works well with a dll built w/MSVC and running the whole thing on Windows, though the final target is gcc/Linux/Mono, where I get several errors of the following type (this is from mono debug):
'libavpcl_dll.so': '/usr/lib/libavpcl_dll.so: undefined symbol: _ZN3pcl7PCLBaseINS_8PointXYZEE13setInputCloudERKN5boost10shared_ptrIKNS_10PointCloudIS1_EEEE'.
I have investigated quite a bit so far, and have set my CmakeLists.txt to set(CMAKE_CXX_VISIBILITY_PRESET hidden) , therefore, I imagine, only functions I defined as EXPORT should be visible and imported - however, that is not the case, and I get the aforementioned errors. PCL was installed on Windows via vcpkg and on Xubuntu via apt. I am somewhat stumped as to what is the error source, considering the code runs well on windows, and builds without issue on Linux. Thanks.

I've been running into the same issue as you. I solved it by adding each reference library into the CMakeLists.txt file (I was missing the reference files which gave me the similar missing symbol issues).
I'm at the 'I don't know why this worked' stage but I can give you step by step implementation (I'm also trying to use DllImport into .NET on Linux).
Started with this:
https://medium.com/#xaviergeerinck/how-to-bind-c-code-with-dotnet-core-157a121c0aa6
Then added my in-scope files thanks to the main comment here: How to create a shared library with cmake?:
add_library(mylib SHARED
sources/animation.cpp
sources/buffers.cpp
[...]
)
run cmake .
run make -j$(grep -c ^processor /proc/cpuinfo)
copy path to .so file
DllImport path from above to my c# app

Calling a function of a 32-bit DLL from a 64-bit Application

I have a 32bit dll (no source code) that I need to access from 64bit C# application. I've read this article and took a look into the corresponding code from here. I've also read this post.
I'm not sure that I'm asking the right question, so please help me.
There are 3 projects: dotnetclient, x86Library and x86x64. The x86x64 has x86LibraryProxy.cpp which loads the x86library.dll and calls the GetTemperature function:
STDMETHODIMP Cx86LibraryProxy::GetTemperature(ULONG sensorId, FLOAT* temperature)
{
*temperature = -1;
typedef float (__cdecl *PGETTEMPERATURE)(int);
PGETTEMPERATURE pFunc;
TCHAR buf[256];
HMODULE hLib = LoadLibrary(L"x86library.dll");
if (hLib != NULL)
{
pFunc = (PGETTEMPERATURE)GetProcAddress(hLib, "GetTemperature");
if (pFunc != NULL)
dotnetclient calls that GetTemperature function and print the result:
static void Main(string[] args)
{
float temperature = 0;
uint sensorId = 2;
var svc = new x86x64Lib.x86LibraryProxy();
temperature = svc.GetTemperature(sensorId);
Console.WriteLine($"temperature of {sensorId} is {temperature}, press any key to exit...");
This all works if I build all projects either as x86 or x64. The result for the temperature I get is 20. But, the whole idea was to use 32bit x86x64Lib.dll. That means that dotnetclient should be built as x64 and x86Library and x86x64 as x86, right? If I do this I get -1 as a result.
Should I build x86Library and x86x64 as x86 and dotnetclient as x64? If I do, so what can be the problem that I get -1?
CLARIFICATION
It seems that the provided example only works when both client and server are build in 32 or 64 bit. But not when the client build in 64bit and the server in 32bit. Can someone take a look please?

IMHO, the easiest way to do this is to use COM+ (Component Services) which is part of Windows for like 20 years or so (previous versions used to be called MTS...). It provides the surrogate infrastructure for you with tools, UI, and everything you need.
But that means you'll have to use COM, so it's good to know a bit of COM for this.
First create an x86 COM DLL. I've used ATL for that. Created an ATL project, added an ATL simple object to it, added the method to the IDL and implementation.
.idl (note the [out, retval] attributes so the temperature is considered a return value for higher level languages including .NET):
import "oaidl.idl";
import "ocidl.idl";
[
object,
uuid(f9988875-6bf1-4f3f-9ad4-64fa220a5c42),
dual,
nonextensible,
pointer_default(unique)
]
interface IMyObject : IDispatch
{
HRESULT GetTemperature(ULONG sensorId, [out, retval] FLOAT* temperature);
};
[
uuid(2de2557f-9bc2-42ef-8c58-63ba77834d0f),
version(1.0),
]
library x86LibraryLib
{
importlib("stdole2.tlb");
[
uuid(b20dcea2-9b8f-426d-8d96-760276fbaca9)
]
coclass MyObject
{
[default] interface IMyObject;
};
};
import "shobjidl.idl";
Method implementation for testing purposes:
STDMETHODIMP GetTemperature(ULONG sensorId, FLOAT* temperature)
{
*temperature = sizeof(void*); // should be 4 in x86 :-)
return S_OK;
}
Now, you must register this component in the 32-bit registry (in fact, if you're running Visual Studio w/o admin rights, it will complain at compile time that the component cannot be registered, that's expected), so on a 64-bit OS, you must run something like this (note SysWow64) with admin rights:
c:\Windows\SysWOW64\regsvr32 x86Library.dll
Once you've done that, run "Component Services", browse "Computers/My Computer/COM+ Applications", right click and create a New Application. Choose a name and a "Server application". It means your component will be hosted in COM+ surrogate process.
Once you've done that, browse "Components", right click and create a New Component. Make sure you select "32-bit registry". You should see your object's ProgId. In my case when I created my ATL project I added "MyObject" as a Progid, but otherwise it could be named something like "x86Library.MyObject" or "x86LibraryLib.MyObject"... If it's not there, than you made some mistake earlier.
That's it. Now, this .NET program will always be able to run, compiled as AnyCpu or x86 or x64:
class Program
{
static void Main(string[] args)
{
var type = Type.GetTypeFromProgID("MyObject"); // the same progid
dynamic o = Activator.CreateInstance(type);
Console.WriteLine(o.GetTemperature(1234)); // always displays 4
}
}
You can use Component Services UI to configure your surrogate (activation, shutdown, etc.). It also has an API so you can create COM+ apps programmatically.

You are not going to be able to directly call 32-bit code from 64-bit code (or the other way around), it simply is not going to happen.
There are alternatives, such as creating a 32-bit COM host program that then forwards calls to the DLL. Coupled with that you use DCOM standard marshalling so your 64-bit process can connect to the 32-bit host.
But if recompiling the 32-bit DLL is at all an option that is almost certainly your best option.

How to make CUDA dll that can be used in C# application?

It would be good if you could give me a brief tutorial instead of a few words.
My CUDA application is working as I wanted. Now, the problem is how to export CUDA code to C# as I would like to make front end and everything else in C#.
From this link:
http://www.codeproject.com/Articles/9826/How-to-create-a-DLL-library-in-C-and-then-use-it-w
I know how to make a library in C language that can be imported into C# application as Win32 dll.
But my question is, how to make CUDA application dll (or some other extension) that can be shipped to C# and used from C# application?
It would be good if there is somewhere tutorial for CUDA like the one for C library to C# app(above link).
I am using Win7 64 bit, Visual Studio 2010 Ultimate, Cuda Toolikt 5.0 and NSight 2.2.012313

ManagedCUDA is perfect for this type of thing. First you need to follow the instructions in the documentation to set up your Visual Studio Project.
Here is an example of a solution:
test.cu (compiles to test.ptx)
#if !defined(__CUDACC__)
#define __CUDACC__
#include <host_config.h>
#include <device_launch_parameters.h>
#include <device_functions.h>
#include <math_functions.h>
#endif
extern "C"
{
__global__ void test(float * data)
{
float a = data[0];
float b = data[1];
float c = data[2];
data[0] = max(a, max(b, c));
}
}
and here is the C# code:
private static void Test()
{
using (CudaContext ctx = new CudaContext())
{
CudaDeviceVariable<float> d = new CudaDeviceVariable<float>(3);
CUmodule module = ctx.LoadModulePTX("test.ptx");
CudaKernel kernel = new CudaKernel("test", module, ctx)
{
GridDimensions = new dim3(1, 1),
BlockDimensions = new dim3(1, 1)
};
kernel.Run(d.DevicePointer);
}
}
This is just a proof of concept, the device memory is not even initialized and the result is not read but is enough to illustrate how to do it.
You have several options how to distribute your application. In this case i opted for compiling the .cu file into PTX and load it inside the C# project from filesystem.
You could also embed the PTX as a resource directly into your C# application.
You could also compile into a cubin and load or embed that instead of PTX.

G-WAN + Phalanger

OK i have this crazy idea, since php does not play nice with G-WAN, maybe the solution is to use phalanger to compile php code to c# mono assembly and then use it from g-wan?
Anyone has any experience with this combination and could help?
OR maybe i'm wrong and G-wan can run php?

Did someone tried PH7 ?
PH7 is a PHP engine which allow the host application to compile and execute PHP scripts in-process.
As an embedded interpreter, it allows multiple interpreter states to coexist in the same program, without any interference between them.
PH7 is threadsafe.
But in order to be thread-safe, PH7 must be compiled with the PH7_ENABLE_THREADS compile time directive defined.

Well, I did contact the people behind Phalanger (and a few other solutions) to add support for PHP. And their reply (at the time) was that Phalanger was no longer developed.
Now it has been re-emplemented as a CLR language this might give PHP a second life. While I have used the G-WAN 3.9 beta I did not yet try to play with the various languages supported by the Mono runtime.
Regarding the genuine PHP library, I wrote the code below to make it run:
// ----------------------------------------------------------------------------
// php.c: G-WAN using PHP scripts
//
// To build PHP5:
//
// CFLAGS="-O3" ./configure --enable-embed --enable-maintainer-zts --with-tsrm-pthreads --without-pear
// make clean
// make
// sudo make install
/* Installing PHP SAPI module: embed
Installing PHP CLI binary: /usr/local/bin/
Installing PHP CLI man page: /usr/local/php/man/man1/
Installing PHP CGI binary: /usr/local/bin/
Installing build environment: /usr/local/lib/php/build/
Installing header files: /usr/local/include/php/
Installing helper programs: /usr/local/bin/
program: phpize
program: php-config
Installing man pages: /usr/local/php/man/man1/
page: phpize.1
page: php-config.1
Installing PEAR environment: /usr/local/lib/php/
[PEAR] Archive_Tar - already installed: 1.3.7
[PEAR] Console_Getopt - already installed: 1.3.0
[PEAR] Structures_Graph- already installed: 1.0.4
[PEAR] XML_Util - already installed: 1.2.1
[PEAR] PEAR - already installed: 1.9.4
Wrote PEAR system config file at: /usr/local/etc/pear.conf
You may want to add: /usr/local/lib/php to your php.ini include_path
/home/pierre/Downloads/PHP/php5.4-20/build/shtool install -c ext/phar/phar.phar /usr/local/bin
ln -s -f /usr/local/bin/phar.phar /usr/local/bin/phar
Installing PDO headers: /usr/local/include/php/ext/pdo/ */
/*
enabling the 'thread safety' --enable-maintainer-zts option results in:
error: 'tsrm_ls' undeclared (first use in this function)
*/
/*
tsrm_ls
TSRM local storage - This is the actual variable name being passed around
inside the TSRMLS_* macros when ZTS is enabled. It acts as a pointer to
the start of that thread's independent data storage block.
TSRM
Thread Safe Resource Manager - This is an oft overlooked, and seldom if
ever discussed layer hiding in the /TSRM directory of the PHP source code.
By default, the TSRM layer is only enabled when compiling a SAPI which
requires it (e.g. apache2-worker). All Win32 builds have this layer
enabled enabled regardless of SAPI choice.
ZTS
Zend Thread Ssafety - Often used synonymously with the term TSRM.
Specifically, ZTS is the term used by ./configure
( --enable-experimental-zts for PHP4, --enable-maintainer-zts for PHP5),
and the name of the #define'd preprocessor token used inside the engine
to determine if the TSRM layer is being used.
TSRMLS_??
A quartet of macros designed to make the differences between ZTS and
non-ZTS mode as painless as possible. When ZTS is not enabled, all
four of these macros evaluate to nothing. When ZTS is enabled however,
they expand out to the following definitions:
TSRMLS_C tsrm_ls
TSRMLS_D void ***tsrm_ls
TSRMLS_CC , tsrm_ls
TSRMLS_DC , void ***tsrm_ls
PHP relies on global variables from resource type identifiers, to
function callback pointers, to request specific information such as
the symbol tables used to store userspace variables. Attempting to
pass these values around in the parameter stack would be more than
unruly, it'd be impossible for an application like PHP where it's
often necessary to register callbacks with external libraries which
don't support context data.
So common information, like the execution stack, the function and
class tables, and extension registries all sit up in the global
scope where they can be picked up and used at any point in the
application.
For single-threaded SAPIs like CLI, Apache1, or even Apache2-prefork,
this is perfectly fine. Request specific structures are initialized
during the RINIT/Activation phase, and reset back to their original
values during the RSHUTDOWN/Deactivation phase in preparation for
the next request. A given webserver like Apache1 can serve up multiple
pages at once because it spawns multiple processes each in their own
process space with their own independant copies of global data.
The trouble starts with threaded webservers like Apache2-worker, or IIS
where two or more threads trying to run the a request at the same time.
Each thread wants to use the global scope to store its request-specific
information, and tries to do so by writing to the same
storage space. At the least, this would result in userspace variables
declared in one script showing up in another. In practice, it leads to
quick and disasterous segfaults and completely unpredictable behavior as
memory is double freed or written with conflicting information by separate
threads.
*/
#pragma include "/usr/local/include/php"
#pragma include "/usr/local/include/php/main"
#pragma include "/usr/local/include/php/TSRM"
#pragma include "/usr/local/include/php/Zend"
#pragma link "/usr/local/lib/libphp5.so"
#include "gwan.h" // G-WAN exported functions
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/syscall.h>
#include <php/sapi/embed/php_embed.h>
#include <php/Zend/zend_stream.h>
static pid_t gettid(void) { return syscall(__NR_gettid); }
// PHP
static int ub_write(const char *str, unsigned int str_len TSRMLS_DC)
{
puts(str); // this is the stdout output of a PHP script
return 0;
}
static void log_message(char * message)
{
printf("log_message: %s\n", message);
}
static void sapi_error(int type, const char * fmt, ...) { }
static void php_set_var(char *varname, char *varval)
{
zval *var;
MAKE_STD_ZVAL(var);
ZVAL_STRING(var, varval, 1);
zend_hash_update(&EG(symbol_table), varname, strlen(varname) + 1,
&var, sizeof(zval*), NULL);
}
static char *php_get_var(char *varname)
{
zval **data = NULL;
char *ret = NULL;
if(zend_hash_find(&EG(symbol_table), varname, strlen(varname) + 1,
(void**)&data) == FAILURE)
{
printf("Name not found in $GLOBALS\n");
return "";
}
if(!data)
{
printf("Value is NULL (not possible for symbol_table?)\n");
return "";
}
ret = Z_STRVAL_PP(data);
return ret;
}
static int php_init(void)
{
static int once = 0;
if(once) return 0;
once = 1;
static char *myargv[2] = {"toto.php", NULL};
php_embed_module.log_message = log_message;
php_embed_module.sapi_error = sapi_error;
php_embed_module.ub_write = ub_write;
if(php_embed_init(1, myargv PTSRMLS_CC) == FAILURE)
{
printf("php_embed_init error\n");
return 1;
}
return 0;
}
static void php_shutdown()
{
php_embed_shutdown(TSRMLS_C);
}
static int php_exec(char *str)
{
zval ret_value;
int exit_status;
zend_first_try
{
PG(during_request_startup) = 0;
// run the specified PHP script file
// sprintf(str, "include (\"% s \ ");", scriptname);
zend_eval_string(str, &ret_value, "toto.php" TSRMLS_CC);
exit_status = Z_LVAL(ret_value);
} zend_catch
{
exit_status = EG(exit_status);
}
zend_end_try();
return exit_status;
}
__thread char reply_num[8] = {0};
__thread pid_t tid = 0;
int main(int argc, char *argv[])
{
if(!tid)
{
tid = gettid();
s_snprintf(reply_num, 8, "%u", tid);
php_init();
}
xbuf_t *reply = get_reply(argv);
//php_set_var("argv", argv[0]);
php_set_var(reply_num, "");
char fmt[] = //"print(\"from php [$test]\n\");\n"
"$reply%s = \"Hello World (PHP)\";\n";
char php[sizeof(fmt) + sizeof(reply_num) + 2];
s_snprintf(php, sizeof(php), fmt, reply_num);
php_exec(php);
xbuf_cat(reply, php_get_var(reply_num));
return 200;
}
If anybody can make this code work with more than one worker thread without crashing the PHP runtime, then PHP will be added to G-WAN.
Here is what G-WAN produces with one single worker thread:
-----------------------------------------------------
weighttp -n 100000 -c 100 -t 1 -k "http://127.0.0.1:8080/?php.c"
finished in 0 sec, 592 millisec, **168744 req/s**, 48283 kbyte/s
requests: 100000 total/started/done/succeeded, 0 failed/errored
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 29299985 bytes total, 27599985 bytes http,
1700000 bytes data
-----------------------------------------------------
That would be great to resolve this PHP threading issue. Thanks for helping anyone!

Calling c# dll (which parses XML file) from my c++ application to return array / list to c++

I have a simple c++ Application. This Application is just printing text out.
I have also a c# .dll NET 3.5 which parses complex xml files, extracts values and saves them into a List. Its like 2 Classes with 4 methods. They open a file, parse the xml and store it into a List. When the c# .dll is done, it has a List with 10000 values;
Since i do not want to write the complex parsing XML in c++, i would like to use my c# xml parsing .dll.
Is it possible for me, to call my c# .dll from inside my c++ application, let the c# .dll parse a specified xml file, and return that created List, with the parsed xml values, to my c++ application?
In my c++ application i would proceed to modify the data within the returned list.
edit: i would be using vc++ (Microsoft Visual Studio 2010)
edit2: the vc++ app would be an expension/plug-in to another bigger Application. I would register the plug-in to that bigger application, and every time i press the icon in the menu, my vc++ application would be started
edit3: Has anyone experience with such a task? I kinda need a clear yes or no if it is possible.
edit4: i do want to avoid reading files that have been written by my .net .dll. I want my c++ app to send a string to my .net .dll and receive back a list/array object. Or is this a bad idea and i should do the xml parsing in c++ itself?

Yes, you can do it, but its kinda hard. One way to do it is to use C++ to load up CLR and execute your function, something like this:
code inspired by Blizzhackers.cc
void StartNET()
{
DWORD result;
ICLRRuntimeHost* pCLR = NULL;
CorBindToRuntimeEx(NULL, L"wks", NULL, CLSID_CLRRuntimeHost, IID_ICLRRuntimeHost, (LPVOID*)&pCLR);
pCLR->Start();
pCLR->ExecuteInDefaultAppDomain(L"C:\\myNET.dll", L"myNet.Program", L"Main", L"arg", &result);
pCLR->Stop();
}
This C++ code will execute the int Main(string arg) function from namespace myNet and class Program, by that I mean:
myNET.dll:
namespace myNet
{
class Program
{
int Main(string arg)
{
//and here you can run your XML parser:
List<string> myList = XMLParse();
FileStream fs = new Filestream("xmllist.txt");
StreamWriter sw = new StreamWriter(fs);
foreach(string s in myList)
sw.WriteLine(s);
sw.Close();
fs.Close();
return 1;
}
List<string> XMLParse()
{
//Your code here
return aList;
}
}
}
And after this you could use c++ to get the files from xmllist.txt, something like:
vector<char[]> getList()
{
vector<char[]> *myVector = new vector<char>;
ifstream cin("xmllist.txt");
while(!cin.eof())
{
char line[100];
cin >> line;
myVector.push(line);
}
cin.close();
return myVector;
}
I don't know if this last function works, but you get the general idea.

I created a managed/unmanaged dll in C++ some time ago. (Note though that C++.net has not the easiest syntax.)
Actually, I do not know where to start now, maybe this has the information http://channel9.msdn.com/Forums/TechOff/101918-Mixing-Managed-and-Unmanaged-C-in-a-DLL . It certainly was well possible with.. VS 2005? With that, you can have both managed an unmanaged code in one binary/assembly, and thus call the C# dll.
Apart from that, you can wrap your C# objects as COM objects. http://msdn.microsoft.com/en-us/library/ms404285.aspx . Then you can use COM interop.
In each case, marshalling the input/output would require some trial and error - it is not that obvious if you have never done it before.
Actually, this link should cover it all: http://msdn.microsoft.com/en-us/library/ms973872.aspx .