We Keep Coding

How to convert IStream(C++) to System::IO::Stream(c#) and vice versa

I have 2 libraries one is c++ and the other one c#.
Name of C++ library->LibA
Name of C#->LibB
In LibA, 2 main APIs will be there:
Serialize-> Serialize API will generate IStream as output with the given inputs.
Deserialize-> Deserialize API will take IStream as input and deserializes the stream and gets actual data from it.
#pragma once
struct GPosition
{
double x;
double y;
double z;
};
struct GUnitVector
{
double x;
double y;
double z;
};
struct GLine
{
GPosition m_rootPoint; /* Point on the line, should be in region of interest */
GUnitVector m_direction; /* Gradient */
double m_start; /* Parameter of start point, must be <= m_end */
double m_end; /* Parameter of end point */
};
class GraphicSerializer
{
public:
GUID objectID;
uint32_t aspectID;
uint32_t controlFlag;
vector<const GLine *> geomVector;
void Serialize(IStream &pStream);
void Deserialize(IStream* pStream);
};
In LibB, 4 APIs will be there:
Object GetObjectFromStream(Stream s)-> Takes stream as input and deserializes it and returns Objects
PutToDB(Object A)-> persists the given object to DB
Stream GetStreamFromObject(Object a)-> Takes object as input serializes it and returns it.
Object GetFromDB(objectID)-> Gets the object from DB based on id
public class CommonBase
{
public Guid id { get; set; };
public byte[] Bytes { get; set; } //contains aspect, control flag, vec<GLine>
};
public interface IGraphicRepository
{
CommonBase Get(Guid guid);
bool Put(CommonBase graphic);
}
public static class GraphicStreamUtility
{
public static CommonBase GetCommonBaseFromStream(Stream stream);
public static void SerializeCommonBaseToStream(CommonBase obj, Stream stream);
}
Now I'm writing C++/CLI to use stream generated by libA in libB and vice versa. So that I can persist and retrieve the objects to and from DB.
Can anyone please let me know how to convert IStream to .Net Stream and .Net stream to IStream.

Stream^ CGDMStreamCLI::ConvertIStreamToStream(IStream *pStream)
{
ULARGE_INTEGER ulSize{};
IStream_Size(pStream, &ulSize);
size_t size = ulSize.QuadPart;
auto buffer = std::make_unique<char[]>(size);
ULONG numRead{};
HRESULT hr = pStream->Read(buffer.get(), size, &numRead);
if (FAILED(hr) || size != numRead)
{
throw std::exception("Failed to read the stream");
}
cli::array<Byte>^ byteArray = gcnew cli::array<Byte>(numRead+2);
// convert native pointer to System::IntPtr with C-Style cast
Marshal::Copy((IntPtr)buffer.get(), byteArray, 0, numRead);
return gcnew System::IO::MemoryStream(byteArray);
}

Passing Array of Structures from C#(.NET Core) to C++(unamnaged)

So I've read the documentation and countless examples online how to marshal array of structures. I've marshalled array of int's, I've marshalled structures, but now I'm completely stuck and can't get it to work no matter what I've try. Been stuck on it for over a day now.
Structure/class, tried as both
[StructLayout(LayoutKind.Sequential,CharSet = CharSet.Unicode)]
public class SaveDetails
{
[MarshalAs(UnmanagedType.LPWStr)]
public string Log;
public FILETIME FileTime;
[MarshalAs(UnmanagedType.Bool)]
public bool Saved;
}
Pinvoke and call delegate
public class LogSaveFiles : IDisposable
{
[UnmanagedFunctionPointer(CallingConvention.Winapi,CharSet = CharSet.Unicode)]
private delegate Status DLogSaveFiles([ In, Out] SaveDetails[] logsToSave, string destinationPath);
private static DLogSaveFiles _dLogSaveFiles;
private IntPtr PLogSaveFiles { get; set; }
public bool LogSaveFilesAvailable => PLogSaveFiles != IntPtr.Zero;
public LogSaveFiles(Importer importer)
{
if (importer.dllLibraryPtr!= IntPtr.Zero)
{
PLogSaveFiles = Importer.GetProcAddress(importer.dllLibrary, "LogSaveFiles");
}
}
public Status SaveFiles(SaveDetails[] logsToSave,string destinationPath)
{
Status result = Status.FunctionNotAvailable;
if (LogSaveFilesAvailable)
{
_dLogSaveFiles = (DLogSaveFiles)Marshal.GetDelegateForFunctionPointer(PLogSaveFiles, typeof(DLogSaveFiles));
result = _dLogSaveFiles(logsToSave, destinationPath);
}
return result;
}
public void Dispose()
{
}
}
Call
private void SaveLogs()
{
var logsToSave = new[]{
new SaveDetails{
FileTime = new FILETIME {dwHighDateTime = 3,dwLowDateTime = 5},
Log = LogTypes.logDeviceLog,
Saved = true},
new SaveDetails{
FileTime = new FILETIME {dwHighDateTime = 1,dwLowDateTime = 2},
Log = LogTypes.logDeviceLog,
Saved = false}
};
var pathToSave = "C:\\Logs";
_logSaveFiles.SaveFiles(logsToSave, pathToSave);
}
c++ exposed call
typedef struct _LOG_SAVE_DETAILS
{
LPTSTR szLog;
FILETIME fromFileTime;
BOOL bSaved;
} LOG_SAVE_DETAILS, *PLOG_SAVE_DETAILS;
/* Function definitions */
ULY_STATUS _API LogSaveFiles (PLOG_SAVE_DETAILS ppLogs [],
LPCTSTR szDestinationPath);
Path to destination gets passed properly, but array of structures never goes through resulting in access violation when trying to access it. At first I thought it was issue with LPTSTR not going through properly but I've implemented other calls with it on its own and succeeded marshalling it through.
I've read everything on https://learn.microsoft.com/en-us/dotnet/framework/interop/marshaling-data-with-platform-invoke , it all indicates that my approach is correct, but it doesn't work.
Any help is appreciated.

Simple solution: C side change PLOG_SAVE_DETAILS ppLogs [] to LOG_SAVE_DETAILS ppLogs [], then C#-side change public class SaveDetails to public struct SaveDetails.
Marshaling array of objects seems to be difficult (I wasn't able to do it). Marshaling array of structs works. An alternative is to do the marshaling manually, but it is a pain.
The "pain" of manual marshaling (only modified lines of code):
[UnmanagedFunctionPointer(CallingConvention.Winapi, CharSet = CharSet.Unicode)]
private delegate Status DLogSaveFiles(IntPtr[] logsToSave, string destinationPath);
and then
public Status SaveFiles(SaveDetails[] logsToSave, string destinationPath)
{
Status result = Status.FunctionNotAvailable;
if (LogSaveFilesAvailable)
{
if (_dLogSaveFiles == null)
{
_dLogSaveFiles = (DLogSaveFiles)Marshal.GetDelegateForFunctionPointer(PLogSaveFiles, typeof(DLogSaveFiles));
}
int size = Marshal.SizeOf(typeof(SaveDetails));
IntPtr basePtr = IntPtr.Zero;
IntPtr[] ptrs = new IntPtr[logsToSave.Length + 1];
try
{
basePtr = Marshal.AllocHGlobal(size * logsToSave.Length);
for (int i = 0; i < logsToSave.Length; i++)
{
ptrs[i] = IntPtr.Add(basePtr, (i * size));
Marshal.StructureToPtr(logsToSave[i], ptrs[i], false);
}
result = _dLogSaveFiles(ptrs, destinationPath);
}
finally
{
if (basePtr != IntPtr.Zero)
{
for (int i = 0; i < logsToSave.Length; i++)
{
if (ptrs[i] != IntPtr.Zero)
{
Marshal.DestroyStructure(ptrs[i], typeof(SaveDetails));
}
}
Marshal.FreeHGlobal(basePtr);
}
}
}
return result;
}
Important: this is a marshaler C#->C++. The C++ mustn't modify the received array in any way or there will be a memory leak.

How should I marshall a Rust function with a &str, Option, and String and use it in C#?

The following code is in Rust:
#[no_mangle]
#[cfg(not(target_arch = "wasm32"))]
pub extern fn generate_work(input_hash: &str, max_iters: Option<u64>) -> Option<String> {
let bytes = Vec::from_hex(input_hash).unwrap();
generate_work_internal(&bytes[..], max_iters)
}
I have the following code in C#:
[DllImport("mydll.dll")]
private static extern string generate_work(string input_hash, ulong[] max_iters);
I'm getting the error:
System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'
I have tried other signatures, but none has worked.

As described in my FFI Omnibus, you cannot pass complicated Rust-specific types via FFI functions. You must use only types that are known to the C ABI. In your example, these types are not known:
&str
Option
String
Instead, you will be required to only use basic C pointers. A NULL pointer can represent None for a string, but for an Option<u64>, you will need to break it up into two values, a boolean and the real value. Your FFI function will look something like:
extern crate libc;
use std::ffi::{CStr, CString};
use std::ptr;
#[no_mangle]
#[cfg(not(target_arch = "wasm32"))]
pub extern "C" fn generate_work(
input_hash: *const libc::c_char,
max_iters_present: bool,
max_iters: u64,
) -> *const libc::c_char {
let input_hash = if input_hash.is_null() {
return ptr::null();
} else {
unsafe { CStr::from_ptr(input_hash) }
};
let input_hash = match input_hash.to_str() {
Ok(s) => s,
Err(_) => return ptr::null(),
};
let max_iters = if max_iters_present {
Some(max_iters)
} else {
None
};
let result = inner_code(input_hash, max_iters);
match result {
Some(s) => {
match CString::new(s) {
Ok(s) => s.into_raw(),
Err(_) => ptr::null(),
}
},
None => ptr::null(),
}
}
Note that this returns an allocated string that you need to pass back to Rust to deallocate. Again, as described in the FFI Omnibus, you'll need something like
#[no_mangle]
pub extern fn free_a_string(s: *mut c_char) {
unsafe {
if s.is_null() { return }
CString::from_raw(s)
};
}
Converting a string to Rust is easy:
[DllImport("string_arguments", EntryPoint="how_many_characters")]
public static extern uint HowManyCharacters(string s);
Returning a string requires a lot more trickery, sadly:
internal class Native
{
[DllImport("string_return")]
internal static extern ThemeSongHandle theme_song_generate(byte length);
[DllImport("string_return")]
internal static extern void theme_song_free(IntPtr song);
}
internal class ThemeSongHandle : SafeHandle
{
public ThemeSongHandle() : base(IntPtr.Zero, true) {}
public override bool IsInvalid
{
get { return false; }
}
public string AsString()
{
int len = 0;
while (Marshal.ReadByte(handle, len) != 0) { ++len; }
byte[] buffer = new byte[len];
Marshal.Copy(handle, buffer, 0, buffer.Length);
return Encoding.UTF8.GetString(buffer);
}
protected override bool ReleaseHandle()
{
Native.theme_song_free(handle);
return true;
}
}
See also:
CStr
CString
Taking string arguments via FFI
Returning strings via FFI
Using Unicode strings in DllImport with a DLL written in Rust

Unique structs in child classes (C#)

I am developing an app that generates binary blobs from its input. As there are many types binary blob, I want to define a parent class called Config, and then multiple child classes. The parent class will have a method that generates the binary blob, with each child class having a unique struct that defines the format of the blob.
Parent:
class Config
{
public struct BinaryFormat
{
public UInt16 _config_length;
}
// compile parameters to binary format
public byte[] CompileToBinary()
{
// find the binary format struct
??????
foreach (var field in typeof(BinaryFormat).GetFields(BindingFlags.Instance | BindingFlags.Public))
{
string identifier = field.Name;
if (identifier == "_config_length")
continue;
ConfigParameter param = FindParameterByIdentifier(identifier);
if (param == null)
throw new Exception("Struct field with no matching parameter (" + identifier + ")");
field.SetValue(null, param.value);
}
int size = Marshal.SizeOf(cfg);
cfg._config_length = (UInt16)size;
byte[] buffer = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(cfg, ptr, true);
Marshal.Copy(ptr, buffer, 0, size);
Marshal.FreeHGlobal(ptr);
return buffer;
}
}
Child:
class ChildConfig : Config
{
[StructLayout(LayoutKind.Sequential)]
public struct BinaryFormat
{
public UInt16 _config_length;
public sbyte config1;
public sbyte config2;
public sbyte config3;
}
}
In CompileToBinary() if I just create a new variable of type BinaryFormat it uses the struct from the parent class. How can I use the struct from the child class?
Or is this entirely the wrong way to go about this?
Solution, based on hints and code from m.rogalski.
Parent:
class Config
{
public dynamic binary_struct;
// compile parameters to binary format
public byte[] CompileToBinary()
{
FieldInfo[] fieldinfo = Type.GetType(GetType().FullName + "+BinaryFormat").GetFields(BindingFlags.Instance | BindingFlags.Public);
dynamic cfg = binary_struct;
foreach (var field in fieldinfo)
{
string identifier = field.Name;
if (identifier == "_config_length")
continue;
ConfigParameter param = FindParameterByIdentifier(identifier);
if (param == null)
throw new Exception("Struct field with no matching parameter (" + identifier + ")");
field.SetValue(cfg, Convert.ChangeType(param.value, field.FieldType));
}
int size = Marshal.SizeOf(cfg);
cfg._config_length = (UInt16)size;
byte[] buffer = new byte[size];
IntPtr ptr = Marshal.AllocHGlobal(size);
Marshal.StructureToPtr(cfg, ptr, true);
Marshal.Copy(ptr, buffer, 0, size);
Marshal.FreeHGlobal(ptr);
return buffer;
}
}
Child:
class ChildConfig : Config
{
public cfgSettings()
{
binary_struct = new BinaryFormat();
}
[StructLayout(LayoutKind.Sequential)]
private struct BinaryFormat
{
public UInt16 _config_length;
public sbyte Config1;
public sbyte Config2;
public sbyte Config3;
}
}

So first of all your ChildConfig class will throw very annoying warning about hiding member something...
class ChildConfig : Config
{
[StructLayout(LayoutKind.Sequential)]
public new struct BinaryFormat // now it wont
{
public UInt16 _config_length;
public sbyte config1;
public sbyte config2;
public sbyte config3;
}
}
Second thing is that in C# you can get type by specifying it's "path":
namespace.class.insideclass+inside_structure
But the best thing is that most of the methods are invoked in to "top-most" object so for example if you call GetType() from Config it will return Config's class Type but if called from ChildConfig it will return ChildConfig's class Type ( brilliant as ... ).
Knowing thease things you should consider rebuilding your logic a bit:
// this was originaly
//foreach (var field in typeof(BinaryFormat).GetFields(BindingFlags.Instance | BindingFlags.Public))
// this should work with every BinaryFormat you want
foreach (var field in Type.GetType(GetType().FullName + "+BinaryFormat").GetFields(BindingFlags.Instance | BindingFlags.Public))
To create an instance you can add method inside the base Config object :
public object CreateInstance()
{
return Activator.CreateInstance(Type.GetType(GetType().FullName + "+BinaryFormat"));
}
And then use it inside your derived classes:
class ChildConfig : Config
{
private BinaryFormat _format = (BinaryFormat)CreateInstance();
}
Check it out here

Use virtual/override or abstract/override method
Define a method which depends on the BinayFormat either as a parameter or use the protected variable. Then implement it in the child class. This way the functionality will be accessed only from the child class with the child's BinayFormattype.
But this has a downside, you wont be able to access the parent struct type. Will you ever use the parent struct type? If you want to use them interchangeably you might want both of them depend on an common interface.

Copying a string to a fixed length byte buffer in a structure

given this structure in c#:
[StructLayout(LayoutKind.Sequential)]
unsafe public struct AppVPEntry
{
public int Num;
public fixed byte CompName[256];
public int VPBeginAddress;
}
Whats the easiest way to copy a string ("c:\path\file.txt") to the fixed length buffer 'CompName'. This is in a structure thats being sent over to an archaic DLL that we've got no choice but to use. Ideally I'd love to use a .NET function but since it's fixed which implies 'unsafe' I know I'm limited here. A more generic function would help since we've got strings like this all over the DLL import space.

// C# to convert a string to a byte array.
public static byte[] StrToByteArray(string str)
{
System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
return encoding.GetBytes(str);
}
You probably want to check to see if the size of the string isn't longer than the size of the buffer.

Try this out. Use an IntPtr in your DllImport wherever you might pass a VPEntry. Pass the "unmanaged" field wherever you call your DLL method.
public sealed class AppVPEntry : IDisposable {
[StructLayout(LayoutKind.Sequential, Size = 264)]
internal struct _AppVPEntry {
[MarshalAs(UnmanagedType.I4)]
public Int32 Num;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 256)]
public Byte[] CompName;
[MarshalAs(UnmanagedType.I4)]
public Int32 VPBeginAddress;
}
private readonly IntPtr unmanaged;
private readonly _AppVPEntry managed = new _AppVPEntry();
public AppVPEntry(Int32 num, String path, Int32 beginAddress) {
this.managed.Num = num;
this.managed.CompName = new byte[256];
Buffer.BlockCopy(Encoding.ASCII.GetBytes(path), 0, this.managed.CompName, 0, Math.Min(path.Length, 256));
this.managed.VPBeginAddress = beginAddress;
this.unmanaged = Marshal.AllocHGlobal(264);
Marshal.StructureToPtr(this.managed, this.unmanaged, false);
}
public void Dispose() {
Marshal.FreeHGlobal(this.unmanaged);
}
}