I am trying to deserialize data from Mongodb to MyClass that has been created by protoc (version 3).
I am using MongoDB.Driver (version 4)
BsonSerializer.Deserialize(result, MyClass.GetType());
Which uses XmlSerializer to deserialize the data.
The problem lies in how protobuf represents its collections.
A List is created as Google.Protobuf.Collections.RepeatedField
with no setter. XmlSerializer cannot deserialize fields with no setters.
How can I solve this problem in a way that it doesn't become an ugly hack?
Options I have come up with:
Create my own Serializer that MongoDB can use to handle properties with no setter.
Create my own class generator that adds a public setter.
Create a new class as a mapper between MongoDB's serialized data and protobufs class.
Option 1 seems daunting, option 2 makes me create a new fork, which is a pain to maintain. Option 3 is the easiest by far, but also messy, I'd prefer not to create another layer of complexity if there are better ways to deal with this.
So my question is:
Is there any other ways to solve this problem? Is there anything I am missing that is already built-in or maybe I am missing something trivial?
Edit:
This is a snippet of what is getting generated by protoc verison 3:
/// <summary>Field number for the "Recipients" field.</summary>
public const int RecipientsFieldNumber = 5;
private static readonly pb::FieldCodec<string> _repeated_recipients_codec =
pb::FieldCodec.ForString(42);
private readonly pbc::RepeatedField<string> recipients_ = new pbc::RepeatedField<string>();
public pbc::RepeatedField<string> Recipients {
get { return recipients_; }
}
It comes from this proto-file:
syntax = "proto3";
package DataModels.Data;
message MailTemplateMessage {
string UUID = 1;
string SubjectLine = 2;
string Body = 3;
string Sender = 4;
repeated string Recipients = 5;
}
Related
I have a proto file that describes a message:
message SharedData
{
string instanceName = 1;
string userName = 2;
bytes data = 3;
uint64 data_size = 4;
}
When serializing using c++, I can use
SharedData data_instance;
std::string string_data = someObject.SerializeAsString();
data_instance.set_data(stringData);
to save the object in the data field.
I'm trying to do the same in C#. The data field's type is ByteString. There is no SerializeAsString method available for objects. I tried using someObject.ToByteString() but the data is not interpreted correctly on the other side.
Is there a C# equivalent to SerializeAsString()?
Thanks
Solved it. The problem was that I kept an instance as a public variable and updated it. The correct way is to create a new object each time and use
someObject.ToByteString()
on the new object.
Now I'm trying to find a way to deserialize it in C#...
When trying to look at all instance attrs of a given class that has been instantiated, I could do this in python:
myObject.__dict__ to see all key/value pairs stored for this instance.
Can this be done in C#?
Not quite a duplicate, so I'm not flagging it as such, but take a look at How to get the list of properties of a class?. There are some good examples of how to use the Reflection library. For example, you can use myObject.GetType().GetProperties(). This only returns the properties that have at least one accessor (get or set). So, an instance with public int num = 0 will not be included in the return, but public int num {get; set;} = 0 will be.
Type.GetFields(), and Type.GetField(string) in the Reflection library may also be close to what you're looking for. For example:
Type t = typeof(myType);
FieldInfo[] arr = t.GetFields(BindingFlags.Public|BindingFligs.NonPublic);
var newInstance = new myType();
foreach (FieldInfo i in arr)
{
Console.WriteLine(i.GetValue(newInstance));
}
I'm not sure of any methods that do that specifically. However, you can JSON encode the object and print that as a somewhat similar concept.
var jsonSerializer = new System.Web.Script.Serialization.JavaScriptSerializer();
string json = jsonSerializer.Serialize(yourDictionary);
//output json
It might also be worthwhile to make it do a "pretty print" so it's easier to read:
How do I get formatted JSON in .NET using C#?
This uses JSON.net, but I like that better anyway:
string json = JsonConvert.SerializeObject(yourDictionary, Formatting.Indented);
Console.WriteLine(json);
I want to save the list of type Myclass in PlayerPrefs.
I saw an example with converting in a String, how can I do that?
public class MyClass{
private int ex;
}
List<MyClass> l = new List<MyClass>();
//How to save it?
Approach using JSON to serialize/deserialize your object into string is the better option in my personal opinion.
You can find a lot of JSON libs for C#, just research for it and find one that suits better for you.
Or take a look at unity wiki ArrayPrefs2 and extends or use it to create your own parser to save your object structures. I, personally, wouldn't recommend this approach to save entire objects but is an option to mention...
When your object is simple enough to use JsonConverter from Newtonsoft as suggested by Povl Eller:
To Save
Character char = new Character();
char.Name = "Apple";
char.Level = 1;
char.Health = 150;
char.Attributes = new int[] { 10, 10, 10 };
PlayerPrefs.SetString( "PlayerCharacter", JsonConvert.SerializeObject(char) );
To Load
Character char = JsonConvert.DeserializeObject<Character>( PlayerPrefs.GetString( "PlayerCharacter" ) );
If you need to control the serialization for more complex objects or something you can use JsonSerializer that is documented in this same link at newtonsoft website.
OBS: I write this code directly here and didn't test it, probably you'll need to adjust it to be useful...
Perhaps something like
for(int i=0;i<list.length;i++)
{
PlayerPrefs.SetString(list(i).name, list(i).value);
}
Try to use newtonsoft.json.
They have extended examples and these two should get you started:
http://www.newtonsoft.com/json/help/html/SerializeObject.htm
http://www.newtonsoft.com/json/help/html/DeserializeObject.htm
Best Regards
I am trying to serialize an array of my dataobjects through WCF with protobuf-net.
If I serialize the Array of my dataobjects manually, it works successfully:
var proto = Serializer.CreateFormatter<DataType[]>();
which is way faster and smaller than the ordinary binary xml DataContractSerializer - thats why I wanna use it!
The 'DataType' class is just an example - I have many of those. When the reponse of my service is just a single object, everything works just fine.
But when my service returns an Array of objects it seems it does not know what to do and uses the ordinary DataContractSerializer.
The ProtoBehavior is applied:
endpoint.Behaviors.Add(new ProtoBuf.ServiceModel.ProtoEndpointBehavior());
My dataobject is more or less like that:
[Serializable]
[DataContract]
[ProtoContract]
public class DataType
{
[DataMember(EmitDefaultValue = false, Name = "K")]
[ProtoMember(1)]
public string Key { get; set; }
// many more to come
}
and that's basically my service:
[ServiceContract(CallbackContract = typeof(IBaseDataObjectUpdate), SessionMode = SessionMode.Required)]
[ServiceKnownType("GetKnownTypes", typeof(KnownTypesProvider))]
public interface IDataTypeService
{
[OperationContract]
DataType[] Load(Filter[] filter, Guid clientGuid);
// some more
}
I could track it down to the TryCreate in the XmlProtoSerializer. The call:
int key = GetKey(model, ref type, out isList);
does not return a valid key, therefore no XmlProtoSerializer is created.
That explains the behavior, but what are my options here?
I found an old answer of Marc Gravell where he suggests the creation of an object which consists of the Array. But as it is from 2011 it might be outdated:
https://stackoverflow.com/a/6270267/2243584
Or can I add the model to protobuf-net somehow manually? As mentioned above the manual serialization is working.
Any comment is appreciated!
OK, so far I have come up with 2 solutions.
Do not use Arrays! It works with any other collection. Which caused me to investigate and leaded to solution:
Support Arrays in protobuf-net
I have adapted the method internal static Type GetListItemType(TypeModel model, Type listType) in the TypeMode class as follows:
if (listType.IsArray) // NEW
{
if (listType.GetElementType() == typeof(byte))
return null;
}
if (listType == model.MapType(typeof(string)) // || listType.IsArray // CHANGED!!!
|| !model.MapType(typeof(IEnumerable)).IsAssignableFrom(listType)) return null;
I think I did figure out why arrays are excluded. Because if you support byte[], you get some problems when finally sending the data to the wire. At least I got some Assert and exception in the encode factory when dealing with byte[].
As I have no idea on the side effects of solution Nr. 2 - I stick with solution Nr. 1.
Nevertheless I am quite keen on a comment from Marc - of course, everybody is welcome!
So I have a small problem:
A message is sent using MQTT, it consists of a series of serialized objects using protobuf-net in C# (I can't modify this code, but I have access to the source). At the other end i receive the serialized objects in Java, the problem is I can't seem to be able to deserialize the objects using protobuf, if anyone ever had this problem and solved it, please help :)
Example of object from C#:
using ProtoBuf;
namespace Concentrator.Services
{
[ProtoContract]
public class MeterID
{
private byte[] _id;
[ProtoMember(1)]
public byte[] ID
{
get { return _id; }
set { _id = value.Length == 16 ? value : null; }
}
[ProtoMember(2)] public string MeterType;
}
}
My attempt at recreating the same object in Java (the .proto file):
syntax = "proto2";
package mqtt.entity;
option java_package = "mqtt.entity";
option java_outer_classname = "ProtoMeter";
message Meter {
optional bytes ID = 1;
optional string MeterType = 2;
}
message MeterID {
repeated Meter mid = 1;
}
A solution to this example would be a huge help, Thanks a lot.
The code where the object is deserialized in C#:
var ms = new MemoryStream(data, 7, data.Length - 9)
var res = Serializer.Deserialize<List<MeterID>>(ms);
this works in C#, I'm trying to achieve the same thing in java
The message in your C# code matches just:
message MeterID {
optional bytes ID = 1;
optional string MeterType = 2;
}
There is no need for a 2-level model (unless you're using *WithLengthPrefix in the C# code). You can also get that output by using:
var proto = Serializer.GetProto<MeterID>();
With your edit, a List<MeterID> could be mapped as
message List_MeterID {
repeated MeterID items = 1;
}
to be used in combination with the previous MeterID fragment. Which is what you have in the question. So it comes down to "what currently happens?".
try regenerate proto-file by GetProto<T>