We Keep Coding

How to convert ReadOnlyMemory<char> to ReadOnlyMemory<byte> [duplicate]

We can cast Span<T> and ReadOnlySpan<T> to another using MemoryMarshal.Cast method overloads. Like :
Span<byte> span = stackalloc byte[4];
var singleIntSpan = MemoryMarshal.Cast<byte, int>(span);
But is there any way to cast Memory<T> to another? for example cast Memory<byte> to Memory<ushort>.

You can't do it directly; however, if you really need, you can create a custom MemoryManager<T> (presumably actually a MyMemoryManager<TFrom, TTo> : MemoryManager<TTo> that performs the cast as part of the GetSpan() override. This is slightly non-trivial, and demands another allocation - unlike a Span<T> cast, which is allocation-free.
If you need a concrete example of that, I can whip one up (I actually do exactly this in some existing code), but: to be honest, you probably want to reconsider the scenario instead.
Edit: something like this:
using System;
using System.Buffers;
using System.Runtime.InteropServices;
class Program
{
static void Main()
{
Memory<byte> bytes = new byte[1024];
Memory<ushort> typed = Utils.Cast<byte, ushort>(bytes);
Console.WriteLine(typed.Length); // 512
// note CPU endianness matters re the layout
typed.Span[0] = 0x5432;
Console.WriteLine(bytes.Span[0]); // 50 = 0x32
Console.WriteLine(bytes.Span[1]); // 84 = 0x54
}
}
static class Utils
{
public static Memory<TTo> Cast<TFrom, TTo>(Memory<TFrom> from)
where TFrom : unmanaged
where TTo : unmanaged
{
// avoid the extra allocation/indirection, at the cost of a gen-0 box
if (typeof(TFrom) == typeof(TTo)) return (Memory<TTo>)(object)from;
return new CastMemoryManager<TFrom, TTo>(from).Memory;
}
private sealed class CastMemoryManager<TFrom, TTo> : MemoryManager<TTo>
where TFrom : unmanaged
where TTo : unmanaged
{
private readonly Memory<TFrom> _from;
public CastMemoryManager(Memory<TFrom> from) => _from = from;
public override Span<TTo> GetSpan()
=> MemoryMarshal.Cast<TFrom, TTo>(_from.Span);
protected override void Dispose(bool disposing) { }
public override MemoryHandle Pin(int elementIndex = 0)
=> throw new NotSupportedException();
public override void Unpin()
=> throw new NotSupportedException();
}
}
If you really want to support pin/unpin, that should be possible - you'll just need to compute the relative ranges and offsets from the competing TFrom/TTo, though - presumably using Unsafe.SizeOf<T> etc, and using MemoryMarshal.TryGetMemoryManager to get the underlying memory manager (if one - note that naked arrays don't have a memory manager). Unless you're going to extensively test that option, throwing is probably safer than getting it wrong.

I don't think you can, however I guess you can return a span from it, though i doubt it will help
Memory.Span Property
Returns a span from the current instance.
var array = new int[4];
var mem = array.AsMemory();
var span = MemoryMarshal.Cast<int, byte>(mem.Span);

You can use Unsafe.As to cast Memory<TFrom> to Memory<TTo> but you would need to take care of the correct length of Memory<TTo>:
using System.Runtime.CompilerServices;
// test data
var floatData = new float[5];
var floatMemory = floatData.AsMemory();
floatMemory.Span.Fill(1.99f);
// cast
var resizeFactor = sizeof(double) / sizeof(float);
var doubleMemory = Unsafe
.As<Memory<float>, Memory<double>>(ref floatMemory)
.Slice(0, floatMemory.Length / resizeFactor);
Edit: this works as long as the byte size of TTo is larger or equal to TFrom

How can I cast Memory<T> to another

We can cast Span<T> and ReadOnlySpan<T> to another using MemoryMarshal.Cast method overloads. Like :
Span<byte> span = stackalloc byte[4];
var singleIntSpan = MemoryMarshal.Cast<byte, int>(span);
But is there any way to cast Memory<T> to another? for example cast Memory<byte> to Memory<ushort>.

You can't do it directly; however, if you really need, you can create a custom MemoryManager<T> (presumably actually a MyMemoryManager<TFrom, TTo> : MemoryManager<TTo> that performs the cast as part of the GetSpan() override. This is slightly non-trivial, and demands another allocation - unlike a Span<T> cast, which is allocation-free.
If you need a concrete example of that, I can whip one up (I actually do exactly this in some existing code), but: to be honest, you probably want to reconsider the scenario instead.
Edit: something like this:
using System;
using System.Buffers;
using System.Runtime.InteropServices;
class Program
{
static void Main()
{
Memory<byte> bytes = new byte[1024];
Memory<ushort> typed = Utils.Cast<byte, ushort>(bytes);
Console.WriteLine(typed.Length); // 512
// note CPU endianness matters re the layout
typed.Span[0] = 0x5432;
Console.WriteLine(bytes.Span[0]); // 50 = 0x32
Console.WriteLine(bytes.Span[1]); // 84 = 0x54
}
}
static class Utils
{
public static Memory<TTo> Cast<TFrom, TTo>(Memory<TFrom> from)
where TFrom : unmanaged
where TTo : unmanaged
{
// avoid the extra allocation/indirection, at the cost of a gen-0 box
if (typeof(TFrom) == typeof(TTo)) return (Memory<TTo>)(object)from;
return new CastMemoryManager<TFrom, TTo>(from).Memory;
}
private sealed class CastMemoryManager<TFrom, TTo> : MemoryManager<TTo>
where TFrom : unmanaged
where TTo : unmanaged
{
private readonly Memory<TFrom> _from;
public CastMemoryManager(Memory<TFrom> from) => _from = from;
public override Span<TTo> GetSpan()
=> MemoryMarshal.Cast<TFrom, TTo>(_from.Span);
protected override void Dispose(bool disposing) { }
public override MemoryHandle Pin(int elementIndex = 0)
=> throw new NotSupportedException();
public override void Unpin()
=> throw new NotSupportedException();
}
}
If you really want to support pin/unpin, that should be possible - you'll just need to compute the relative ranges and offsets from the competing TFrom/TTo, though - presumably using Unsafe.SizeOf<T> etc, and using MemoryMarshal.TryGetMemoryManager to get the underlying memory manager (if one - note that naked arrays don't have a memory manager). Unless you're going to extensively test that option, throwing is probably safer than getting it wrong.

I don't think you can, however I guess you can return a span from it, though i doubt it will help
Memory.Span Property
Returns a span from the current instance.
var array = new int[4];
var mem = array.AsMemory();
var span = MemoryMarshal.Cast<int, byte>(mem.Span);

You can use Unsafe.As to cast Memory<TFrom> to Memory<TTo> but you would need to take care of the correct length of Memory<TTo>:
using System.Runtime.CompilerServices;
// test data
var floatData = new float[5];
var floatMemory = floatData.AsMemory();
floatMemory.Span.Fill(1.99f);
// cast
var resizeFactor = sizeof(double) / sizeof(float);
var doubleMemory = Unsafe
.As<Memory<float>, Memory<double>>(ref floatMemory)
.Slice(0, floatMemory.Length / resizeFactor);
Edit: this works as long as the byte size of TTo is larger or equal to TFrom

Deserialize Binary Data of Object Array as Elements Are Available

I've been sifting through the posts and forums but could not find a way to achieve this.
I have an array of 10,000,000 Person objects. I'm sending these objects over the network using a Streamed WCF Net.Tcp web service.
The problem is I want to read the first, for example, 5000 Person objects of the array as is it arrives and process only those. Afterwhich I will advance the stream and read another 5000, etc...
I haven't been able to find a way to do this because as far as I can tell there is no explicit size of objects in C#. As in, I can't just read the first 312 Bytes of the stream and say "Yes this is the first Person object. Now read the next 312 Bytes to get the next person.".
I ideally would like to use ProtoBuf-Net to serialize my objects but the .NET BinaryFormatter is fine as well.
I'm also open to sending the data in chunks, such as arrays of 5000. But I want to do so without opening a brand new tcp connection everytime. If only there was a way to tell the code that reads the stream: "Ok, deserialize everything I just sent you (Array of 5000) and then I will continue writing another 5000 to the stream".
Any ideas?
Thanks.

There may not be an explicit size for most objects in .NET but you can find the size of a serialized object. First send the size (in bytes) of the serialized object, then send the serialized object.
// psuedo-code
byte[] serializedObj = DoSerialization(Person); // we see length on an array
using (var writer = new StreamWriter(stream)) {
writer.Write(serializedObj.Length);
stream.Write(serializedObj);
}
You can also do this in bulk by modifying what and how you send your objects. You could create a List<Person>, add N number of Person, serialize the List and send as before.
Although I am not sure if sending the size before sending the data is necessary, it can help when you are reading the stream, to know how many bytes you are expecting.

You can do this with protobuf-net simply by using a ObservableCollection<Person> in your receiving system. When the collection grows larger than 5000 objects during deserialization, remove and processing the items in an ObservableCollection<T>.CollectionChanged callback. Then process any remaining items in an [OnDeserialized] callback.
For instance, consider the following root object:
[ProtoContract]
public class RootObject
{
public RootObject()
{
this.People = new ObservableCollection<Person>();
}
[ProtoMember(1)]
public ObservableCollection<Person> People { get; private set; }
public event EventHandler<EventArgs<StreamingContext>> OnDeserialized;
[OnDeserialized]
internal void OnDeserializedMethod(StreamingContext context)
{
var onDeserialized = OnDeserialized;
if (onDeserialized != null)
onDeserialized(this, new EventArgs<StreamingContext> { Value = context });
}
}
public class EventArgs<T> : EventArgs
{
public T Value { get; set; }
}
Say you have a method you would like to call to process each 5000 Person objects as they are added to the collection, for instance:
const int ProcessIncrement = 5000;
void ProcessItems(ICollection<Person> people, bool force)
{
if (people == null || people.Count == 0)
return;
if (people.Count >= ProcessIncrement || force)
{
// Remove and process the items, possibly on a different thread.
Console.WriteLine(string.Format("Processing {0} people." people.Count));
people.Clear();
}
}
You can pre-allocate your RootObject and add listeners with the necessary logic, and merge contents of the serialization stream into the root:
// Allocate a new RootObject
var newRoot = new RootObject();
// Add listeners to process chunks of Person objects as they are added
newRoot.People.CollectionChanged += (o, e) =>
{
// Process each chunk of 5000.
var collection = (ICollection<Person>)o;
ProcessItems(collection, false);
};
newRoot.OnDeserialized += (o, e) =>
{
// Forcibly process any remaining no matter how many.
ProcessItems(((RootObject)o).People, true);
};
// Deserialize from the stream onto the pre-allocated newRoot
Serializer.Merge(stream, newRoot);
As required, ProcessItems will be called every time an object is added to the collection, processing them in increments of 5000 then processing the remainder unconditionally.
Now, the only question is, does protobuf-net load the entire stream into memory before deserializing the collection, or does it do streaming deserialization? As it turns out, it does the latter, as shown by this sample fiddle that shows the stream position being gradually incremented as the items in the People collection are added, processed and removed.
Here I added the listeners to RootObject manually before deserialization. If you were to add them in the constructor itself, you could use ProtoBuf.Serializer.Deserialize<RootObject>(Stream stream) instead of Serializer.Merge onto a pre-allocated root object, which might be easier to integrate into your current architecture.
Incidentally, this technique should work with XmlSerializer and Json.NET as well.

Implementing IFormatter recursively

I'm trying to implement a custom formatter using the .NET IFormatter interface.
After a couple of hours of search, I just found a very basic sample which unfortunately doesn't include recursion. I also tried with Reflector to look at BinaryFormatter and SoapFormatter, but they are rather complex.
My question is:
Should I implement recursion myself, or there's something I've missed in FormatterServices?
Following my code:
public void Serialize(Stream serializationStream, object graph)
{
// Get fields that are to be serialized.
MemberInfo[] members = FormatterServices.GetSerializableMembers(graph.GetType(), Context);
// Get fields data.
object[] data = FormatterServices.GetObjectData(graph, members);
// Write class name and all fields & values to file
StreamWriter sw = new StreamWriter(serializationStream);
string accumulator = string.Empty;
for (int i = 0; i < data.Length; ++i)
{
// Skip this field if it is marked NonSerialized.
if (Attribute.IsDefined(members[i], typeof(NonSerializedAttribute)))
continue;
FieldInfo field = (FieldInfo)members[i];
if (field.FieldType.IsPrimitive)
{
}
else //TODO: What should I do here?
}
sw.Close();
}

If by recursion you mean traversing through the object tree then yes, it up to you when you implement your own IFormatter.
Simply check if the value of the property is not null and if it is implementing IFormatter interface. If it is then just call it and use the value it returns.
If not then it is up to you again: you may throw an exception saying that IFormatter must be implemented, or just fall-back to some sort of default formatter (XML or Binary one).
The recursion per se is tricky. When, let's say, the object references itself, you need to be smart enough to handle this situation and not to end up with the infinite loop:
public class A {
public object SomeProperty { get; set; }
}
var a = new A();
a.SomeProperty = a;
There are a number of tricky aspects in implementing formatters, like what if two properties are actually reference the same object? Will you serialize/format it twice or just once and keep the information about these references somehow?
You don't probably need this if you want just one-way serialization, but if you want to be able to restore the object it might be important...

C# memcpy equivalent

I have 2 objects from the same type and i would like to shallow copy one state to the other. In C++ i have memcpy which is great. How can i do it in C#? The MemberwiseClone() is not good enough because it creates & returns a new object and i like to copy to an existing object. I thought of using reflection but i'm afraid it will be too slow for production code. I also thought of using one of the .Net serializers but i think they also create object rather than setting an existing one.
My Use Case:
I have a template object (class not struct) which needs to be updated by one of its instances (objects made of this template)
Any ideas?

In C# (and in C++ too), there is no difference between "new object" and "a copy of existing object" as long as all their members equal to each other.
Given:
Int32 a = 5;
, both operations:
Int32 b = 5;
Int32 b = a;
yield the same result.
As stated in MSDN reference:
The MemberwiseClone method creates a shallow copy by creating a new object, and then copying the nonstatic fields of the current object to the new object.
If a field is a value type, a bit-by-bit copy of the field is performed.
If a field is a reference type, the reference is copied but the referred object is not; therefore, the original object and its clone refer to the same object.
, i.e. it does just the same as memcpy() in C++

[edit] regarding your clarification:
As I understand, you have N objects, each has a (direct) reference to the template object. You want to write back to the template so all objects "see" these changes.
Suggestion: imlement a template broker.
class TemplateProvider
{
public MyData Template { get; set; }
}
Instead of passing the template, pass the template provider to the objects.
to simplyfy the syntax in the components, you can add a (private/internal?) property
MyData Template { get { return m_templateProvider.Template; } }
void UpdateTemplate() { m_templateProvider.Template =
(MyData) this.MemberwiseClone(); }
The template provider also simplifies locking in multithreaded scenarios.
In short, no way unless you do it yourself. But why not create a new object if you override all properties anyway?
memcopy and similar low level constructs are not supported since they undermine guarantees made by the environment.
A shallow copy for structs is made by assignment. For classes, MemberwiseClone is the method to do that - but as you say that creates a new object.
There is no built in way for that, and as it potentially breaks encapsulation it should be used with care anyway.
You could build a generic routine using reflection, but whether it works or not depends on the class itself. And yes, ti will be comparedly slow.
What's left is supporting it by a custom interface. You can provide a generic "Shallow Copy" routine that checks for the interface and uses that, and falls back to reflection when it doesn't. This makes the functionality available generally, and you can optimize the classes for which performance matters later.

I guess you could just do something like:
YourObjectType A = new YourObjectType();
YourObjectType B = a.MemberwiseClone();
This will create a new object inside the MemberwiseClone method an make the B object reference it. I guess it serves your purposes.

Assignment of one struct to another, for all intents and purposes, works exactly like memcpy in C++ on POD objects.
If you feel that this doesn't apply in your situation then I can assure you that your C++ code was not standard-conforming (i.e., contained bugs in the form of undefined behaviour). Please specify (in the question) what effect you want to achieve. This will be more useful than talking about replicating undefined behaviour in another language.

namespace WindowsFormsApplication7
{
[Serializable] // just put this in your class
class Mate
{
public string SomeProperty { get; set; }
}
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
var mA = new Mate();
mA.SomeProperty = "Hey";
var vf = new BinaryFormatter();
var ns = new MemoryStream();
vf.Serialize(ns, mA);
byte[] vytes = ns.ToArray();
var vfx = new BinaryFormatter();
var nsx = new MemoryStream();
nsx.Write(vytes, 0, vytes.Length);
nsx.Seek(0, 0);
var mB = (Mate)vfx.Deserialize(nsx);
mA.SomeProperty = "Yo";
MessageBox.Show(mA.SomeProperty); // Yo
MessageBox.Show(mB.SomeProperty); // Hey
}
}
}

C# / .Net memcpy equivalent is Buffer.MemoryCopy .
void MemoryCopy (void* source, void* destination, long destinationSizeInBytes, long sourceBytesToCopy);
https://learn.microsoft.com/en-us/dotnet/api/system.buffer.memorycopy?view=net-5.0

namespace WindowsFormsApplication7
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
var dt = new DataTable();
dt.Columns.Add("lastname", typeof(string));
dt.Columns.Add("firstname", typeof(string));
dt.Rows.Add("lennon", "john");
dt.Rows.Add("mccartney", "paul");
var ms = new MemoryStream();
var bf = new BinaryFormatter();
bf.Serialize(ms, dt);
byte[] bytes = ms.ToArray();
var bfx = new BinaryFormatter();
var msx = new MemoryStream();
msx.Write(bytes, 0, bytes.Length);
msx.Seek(0, 0);
// doesn't just copy reference, copy all contents
var dtx = (DataTable)bfx.Deserialize(msx);
dtx.Rows[0]["lastname"] = "Ono";
// just copy reference
var dty = dt;
dty.Rows[0]["lastname"] = "Winston";
MessageBox.Show(dt.Rows[0]["lastname"].ToString()); // Winston
MessageBox.Show(dtx.Rows[0]["lastname"].ToString()); // Ono
MessageBox.Show(dty.Rows[0]["lastname"].ToString()); // Winston
}
}
}