We Keep Coding

Passing String from C# to C++ DLL different Text Encoding on different return Types

I hope somebody can explain what exactly the difference is:
In my C# Programm I want to pass an String to an C++ Method.
My Code in C# looks like this:
[DllImport("Test.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.StdCall)]
public static extern String DoIt(String filename);
The inside of my C++ DLL looks like this:
__declspec(dllexport) CString DoIt(char* szFilename)
{
....
}
If I pass the String (like: C:\temp\my.txt) it becomes malformed =>"Ôœ&°8é-".
Now comes the confusings part I can't literly understand. If I change the return Type from CString to char* everything is fine.
__declspec( dllexport ) char* DoIt(char* filename)
Why is that so? The CharSet in C# is already set to Ansi to Marshal the String into the right Type. I cannot figure out where the connection between the return Type and my passing String is.
If you need more information just let me know.

Both versions of your code are wrong. You certainly can't pass CString as an interop type. You to use simple types for interop, not C++ classes.
The error when you use char* is more subtle. In that scenario there are two problems.
Firstly, with a string return type on the C# side, the pinvoke marshaller assumes that the returned string was allocated with CoTaskMemAlloc and will call CoTaskMemFree to deallocate it once it has been copied.
Secondly, although we can't see it, your C++ code almost certainly returns a pointer to a buffer owned by a local variable in the C++ function. Obviously this local variable goes out of scope when the function returns and so the pointer becomes invalid.
Some options:
Have the caller locate the string buffer and let the callee populate it.
Allocate the returned char* using CoTaskMemAlloc and so meet the expectations of the C# code.
Use the COM BSTR type which the pinvoke marshaller does understand.

Understanding Piece of C# code

I have been coding in c# for a bit but I came across some piece of code which does not make any sense to me at all. It looks something like below:
[DllImport(DllName, SetLastError = true,
CallingConvention=CallingConvention.Cdecl)]
static extern byte QLIB_UploadCEFS_File(UInt32 handle, string sFileName,
string sPartitionFileName);
I have no idea how to interpret this code. Can anyone explain me what are they trying to achieve here?

The is a P/Invoke declaration. It declares a function, external to this module, implemented in an unmanaged DLL.
The DllImport attribute specifies the name of the unmanaged DLL, DllName in this instance. The other properties of the DllImport attribute specify, in this instance, the calling convention, and that the function sets the Win32 last error variable.
The function declaration itself specifies the signature of the unmanaged function. In this case the parameters mean that the function has the following unmanaged declaration:
unsigned char QLIB_UploadCEFS_File(
unsigned int handle,
const char *sFileName,
const char *sPartitionFileName
);
From the perspective of the managed code that calls the function, it's just like any other function. You call it passing parameters as specified in the code in your question.
For what it is worth, I do suspect that the declaration is incorrect. The first parameter is a handle and these are almost always pointer sized. So, whilst the code is probably just fine under 32 bit, it is likely to break under 64 bit. I would expect to see that first parameter declared as IntPtr. Of course, this is speculation because I cannot see the actual unmanaged function declaration.

P/Invoke AccessViolationException

I have an unmanaged function call that is throwing this exception when I try to pass it the path to a file name.
I've read that this is likely caused by the DLL itself but I don't think that is the case since the DLL is used in another application, so the problem is likely in my method calling the function.
The specification:
libvlc_media_new_path (libvlc_instance_t *p_instance, const char *path)
Description:
p_instance the instance
path local filesystem path
And my method:
[DllImport("libvlc", EntryPoint = "libvlc_media_new_path")]
public static extern IntPtr NewMedia(IntPtr instance,
[MarshalAs(UnmanagedType.LPStr)] string path);
I think I'm missing the convention call but what would that likely be? Or would it be something else causing this exception?
EDIT: Based on some comments I did some poking around and found... well, nothing. The struct for the instance is opaque, which means I have no idea in Laymans terms. My guess is that it means you don't need to reconstruct it in the application that is using it?
In a blind guess based on this, I replaced the return value that I had been using with the function responsible for setting the *p_instance value to a long instead of an IntPtr since when it was an IntPtr it was returning 0, and with a long I was seeing a value. Again, what an IntPtr is I don't really know. I was pretty happy to see something not 0 in the instance variable but when I ran it past that, it errored out again.
EDIT: I've expanded the question to here.

Based on the exception you're seeing and the declaration you've provided for the native function,
libvlc_media_new_path (libvlc_instance_t *p_instance, const char *path)
your p/invoke declaration is incorrect. You've mismatched the calling conventions. The default for the .NET p/invoke system is stdcall (to match the Windows API functions), but the default for C and C++ code is cdecl. You have to tell .NET explicitly that your function uses the cdecl calling convention.
So change it to look like this:
[DllImport("libvlc", EntryPoint = "libvlc_media_new_path", CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr NewMedia(IntPtr instance,
[MarshalAs(UnmanagedType.LPStr)] string path);
Of course, I'm guessing that you're right about the return value being a pointer. The native function declaration you've shown is missing the return type.
As for your question about the instance parameter, and whether you are correctly using the IntPtr type: The parameter is a pointer to a libvlc_instance_t, so you have two basic ways of making that work using p/invoke. First is to declare the parameter as an IntPtr, which gets it marshalled like a raw pointer value. This is not particularly useful for cases where the pointer needs to be anything other than opaque (i.e. retrieved from one native function, stored, and then passed to another native function). Second is to declare a managed structure that mirrors the native structure, and then write the p/invoke declaration to use this structure so that the marshaller will handle things automatically. This is most useful if you actually need to interact with the values stored in the structure pointed to by the pointer.
In this case, after a Google search, it looks like you're using one of the VLC APIs. Specifically this one. That also tells us what an libvlc_instance_t is: it is an opaque structure that represents a libvlc instance. So declaring a managed structure is not an option here, because the structure is treated as opaque even by the native code. All you really need is the pointer, passed back and forth; a perfect case for the first method I talked about above. So the declaration shown above is your winner.
The only battle now is obtaining a valid pointer to a libvlc instance that you can pass to the function whenever you call it. Chances are good that will come from a prior call to a function like libvlc_new, which is documented as creating and intializing a new libvlc instance. Its return value is exactly what you need here. So unless you've already created a libvlc instance (in which case, use that pointer), you will also need to call this function and store its result.
If the documentation is correct about the required values for the libvlc_new function's parameters, you can declare it very simply:
[DllImport("libvlc", EntryPoint = "libvlc_new", CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr NewCore(int argc, IntPtr argv);
And call it thus:
IntPtr pLibVlc = NewCore(0, IntPtr.Zero);
// now pLibVlc should be non-zero
And of course, I know nothing about the VLC APIs, but my general knowledge of API design tells me that you will probably need to call the libvlc_release function with that same pointer to an instance once you're finished with it.

Try without the [MarshalAs(UnmanagedType.LPStr)], usually works for me.

Calling a Delphi method in a dll from c#

I am trying to call a method in a Delphi DLL with the following signature:
function SMap4Ovr(const OverFileName : ShortString ;
const Aclay : Integer ;
const Acarbon : Double ;
out errstr : ShortString): WordBool;
I am using the following import in C#:
[DllImport("SMap.dll", CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Ansi)]
public static extern bool SMap4Ovr(
string OverFileName,
int Aclay,
double Acarbon,
out string errstr
);
But I am getting a AccessViolationException.
I seem to be able to call into a couple of simpler methods in the DLL that have string parameters but not ints or doubles.
I have also tried with the CallingConvention = CallingConvention.Cdecl but this gives me the same error.

When writing interop code it is critical that both sides of the interface match in every way. Here are the main issues that you must make agree on both sides:
Calling conventions.
Parameters lists.
Parameter types and semantics.
The first observation is that your calling conventions do not match. You have register on the Delphi side and stdcall on the C# side. The Delphi register convention is private to Delphi and so you should use stdcall.
Secondly, your string parameter types do not match. The Delphi shortstring is a data type that became legacy when Delphi 2 was released and should be considered a relic from the previous century. It was never a valid interop type, and there's nothing in the p/invoke framework that can be used to match it. Whilst you could attempt to do the marshalling by hand, this is a lot of work that is simply not needed when there are simple solutions available. You should try to forget all about shortstring.
You need to use a string type that both sides of the interface can work with. You could use null-terminated C strings, but a better and simpler choice is the COM BSTR which is WideString in Delphi.
So, the final result is as follows.
Delphi
function SMap4Ovr(
OverFileName: WideString;
Aclay: Integer;
Acarbon: Double;
out errstr: WideString
): WordBool; stdcall;
C#
[DllImport("SMap.dll")]
public static extern bool SMap4Ovr(
[MarshalAs(UnmanagedType.BStr)]
string OverFileName,
int Aclay,
double Acarbon,
[MarshalAs(UnmanagedType.BStr)]
out string errstr
);
I did not bother specifying the calling convention on the DllImport since the default is stdcall. If you prefer you can be explicit about this.
Be careful when using WideString that you don't attempt to use it as a return value. Because Delphi uses non-standard semantics for return values, you can only use simple types that fit into a register as return values.

Default calling convention in Delphi is register, not stdcall. It seems calling conventions details show us that Microsoft fastcall is not the same as Borland fastcall (register)
And C# string type differs from Delphi ShortString (it contains internally one byte length + string body)

How to natively call this function from a DLL?

I have used DLL Export viewer to try and find the functions that are in this DLL, I have found a list of functions and here it is:
public: int __thiscall CSTVdsDisk::GetPartitionCount(void);
the question is within in C# I am not able to call the function using either:
[DllImport("Some.dll",
ExactSpelling = true,
EntryPoint = "GetPartitionCount",
CallingConvention = CallingConvention.StdCall,
SetLastError = true)]
or:
[DllImport("Some.dll",
ExactSpelling = true,
EntryPoint = "CSTVdsDisk::GetPartitionCount",
CallingConvention = CallingConvention.StdCall,
SetLastError = true)]
private static extern int GetPartitionSize();
They all fail. Is there something that I am doing wrong? Can anyone help? Thanks!

You can't call that function using P/Invoke. The __thiscall calling reference means that this function is a class member function. It is a member function of the CSTVdsDisk class.
To be able to call the function you will have to create an instance of the CSTVdsDisk class and call GetPartitionCount from that instance.
You'll have to do that in C++ or C++/CLR as you can't create a C++ class in C#. See also Create unmanaged c++ object in c#.

Based on the name, that appears to be a C++ class method. That is going to make it very difficult to call that method directly from P/Invoke for two reasons:
You need to find the "real" name; the Export Viewer is apparently showing up the unmangled name, but the actual C++ function name will look much uglier, like #0GetPartitionCount#CSTVdsDisk##QPBAEXA or similar. You may need to use a lower-level tool like dumpbin to find that name.
You need to fake the "thiscall" style call; this means you need to pass an instance of a C++ class as the first parameter. This is only going to work if the C++ class constructor is also exposed from the DLL; in which case, you can call the class constructor, store the result in an IntPtr, and pass it to every subsequent call. (If the constructor is exposed as a DLL export its mangled name will start with ??, like `??0CSTVdsDisk##QAE#ABV0##Z
This CodeProject article shows you how to do most of that, but it's pretty fragile, so expect problems. I'd strongly suggest you look for a non-C++ library that does something similar, or at least one that's designed to be usable from C code.

In your native code, make sure you're exporting the function. By default you function wont be listed in the Exports table so you need to mark it so that the compiler knows to export it. You also need to mark the function as
extern "C"
so that the compiler doesn't mangle the name.
Typically I define the following macro:
#define DLLEXPORT extern "C" __declspec(dllexport)
to handle all of this, and then simply declare exported functions like:
DLLEXPORT __cdecl int Example(int x, int y)
If you find you're still having troubles with the name, try using a free PE explorer program on the dll and checking the exported function table for the correct name.