I am trying to deserialize data from Mongodb to MyClass that has been created by protoc (version 3).
I am using MongoDB.Driver (version 4)
BsonSerializer.Deserialize(result, MyClass.GetType());
Which uses XmlSerializer to deserialize the data.
The problem lies in how protobuf represents its collections.
A List is created as Google.Protobuf.Collections.RepeatedField
with no setter. XmlSerializer cannot deserialize fields with no setters.
How can I solve this problem in a way that it doesn't become an ugly hack?
Options I have come up with:
Create my own Serializer that MongoDB can use to handle properties with no setter.
Create my own class generator that adds a public setter.
Create a new class as a mapper between MongoDB's serialized data and protobufs class.
Option 1 seems daunting, option 2 makes me create a new fork, which is a pain to maintain. Option 3 is the easiest by far, but also messy, I'd prefer not to create another layer of complexity if there are better ways to deal with this.
So my question is:
Is there any other ways to solve this problem? Is there anything I am missing that is already built-in or maybe I am missing something trivial?
Edit:
This is a snippet of what is getting generated by protoc verison 3:
/// <summary>Field number for the "Recipients" field.</summary>
public const int RecipientsFieldNumber = 5;
private static readonly pb::FieldCodec<string> _repeated_recipients_codec =
pb::FieldCodec.ForString(42);
private readonly pbc::RepeatedField<string> recipients_ = new pbc::RepeatedField<string>();
public pbc::RepeatedField<string> Recipients {
get { return recipients_; }
}
It comes from this proto-file:
syntax = "proto3";
package DataModels.Data;
message MailTemplateMessage {
string UUID = 1;
string SubjectLine = 2;
string Body = 3;
string Sender = 4;
repeated string Recipients = 5;
}
I have an internal generic method byte[] catchAllOperation<T>(T genericItem) and another one byte[] arrayOperation<T>(T[] genericArray). Additionally, I have a highly specialized operation byte[] specializedOperation<TKey,TValue>(CustomMap<Tkey,TValue> customMap).
How to write a method like this? (pseudocode):
public byte[] universalOperation<T>(T anything){
if(T is CustomMap<UKey,UValue>){ // UKey/UValue are unknown
return specializedOperation<UKey,UValue>(anything);
} else if(T is U[]){ // U is unknown
return arrayOperation<U>(anything);
}else{
return catchAllOperation<T>(anything);
}
}
How to get U and how to call arrayOperation<U>(anything); with U if I have only T (and the same for CustomMap<>)? Most questions I have seen assume that U is already known. Serializers use reflection to construct a separate method for each concrete type and then cache the method. But here I only want to redirect/dispatch a universal method to special cases when I could detect these cases.
I will have more types similar to CustomMap, so there is time to make any radical changes to code/approach. All special methods leverage the fact that for blittable underlying types some transformations and custom layout significantly boost the compression ratio of custom types. For custom types I could implement an interface, but for generic arrays it is not an option.
Your question isn't entirely clear to me, but it sounds like you basically need to look in typeof(T) with reflection. Fortunately, dynamic typing in C# means you can get the compiler team to do the hard work - admittedly at execution time:
// Note: all names changed to be more conventional
public byte[] UniversalOperation<T>(T value)
{
dynamic d = value;
return DynamicOperation(d);
}
private byte[] DynamicOperation<UKey, UValue>(CustomMap<UKey, UValue> map)
{
// Do stuff with the map here
}
private byte[] DynamicOperation<U>(U[] array)
{
// Do something with the array here
}
private byte[] DynamicOperation(object value)
{
// Fallback
}
Note that your UniversalOperation method doesn't have to be generic now - it will just use the execution-time type of the value. Of course, that means it may not be quite as you originally intended - for example, if the value is null, you're in trouble - whereas you could have potentially worked with typeof(T) despite that. Without knowing what you're trying to achieve, we can't tell whether or not that's a problem.
Like you are able to read comments it is not possible with generics. The only type is T do getting U and other types is not possible without reflection.
Sample solution:
public static void UniversalOperation<T>(T anything)
{
// check for null etc.
var anythingType = anything.GetType();
if (anythingType.IsGenericType &&
anythingType.GetGenericTypeDefinition() == typeof(CustomMap<,>))
{
var genericArgs = anythingType.GetGenericArguments();
var keyType = genericArgs[0];
var valueType = genericArgs[1];
return specializedOperation(keyValue, valueType, anything);
}
else if (anythingType.IsArray)
{
var elemType = anythingType.GetElementType();
return arrayOperation(elemType, anything);
}
else
{
//T is parameter, so you can pass it
catchAllOperation<T>(anything);
}
}
Unfortunately, solution above required non-generic version of specializedOperation. Anyway, most of serializes (did I understand corretly, you serialize it?) share with non-generic overloads.
I've got a WebAPI that returns primitive types, like int, Int64, UInt64 etcetera.
public Int64 Count()
{
return 1337;
}
Returns:
1337
Even though my client requests data with the HTML header
Accept: application/json
The API still just returns the number, so invalid in JSON terms. In XML it will correctly serialize as
<long>1337</long>
Is there any "easy" way to solve this for a lot of methods?
Tried solutions
I tried to use this simple approach:
public Dictionary<string, Int64> Count()
{
return new Dictionary<string, Int64>() {{"Count", 1337}};
}
This has a very ugly XML output though. So now I made a MODEL for a primitive type..
public Number Count()
{
return new Number(1337);
}
Returns:
{ "Value" : 1337 }
I'm looking for something easier, I might end up writing a generic class which does nothing else then wrapping primitives.
I want to see your ideas on a efficient way to check values of a newly serialized object.
Example I have an xml document I have serialized into an object, now I want to do value checks. First and most basic idea I can think of is to use nested if statments and checks each property, could be from one value checking that it has he correct url format, to checking another proprieties value that is a date but making sue it is in the correct range etc.
So my question is how would people do checks on all values in an object? Type checks are not important as this is already taken care of it is more to do with the value itself. It needs to be for quite large objects this is why I did not really want to use nested if statements.
Edit:
I want to achieve complete value validation on all properties in a given object.
I want to check the value it self not that it is null. I want to check the value for specific things if i have, an object with many properties one is of type string and named homepage.
I want to be able to check that the string in the in the correct URL format if not fail. This is just one example in the same object I could check that a date is in a given range if any are not I will return false or some form of fail.
I am using c# .net 4.
Try to use Fluent Validation, it is separation of concerns and configure validation out of your object
public class Validator<T>
{
List<Func<T,bool>> _verifiers = new List<Func<T, bool>>();
public void AddPropertyValidator(Func<T, bool> propValidator)
{
_verifiers.Add(propValidator);
}
public bool IsValid(T objectToValidate)
{
try {
return _verifiers.All(pv => pv(objectToValidate));
} catch(Exception) {
return false;
}
}
}
class ExampleObject {
public string Name {get; set;}
public int BirthYear { get;set;}
}
public static void Main(string[] args)
{
var validator = new Validator<ExampleObject>();
validator.AddPropertyValidator(o => !string.IsNullOrEmpty(o.Name));
validator.AddPropertyValidator(o => o.BirthYear > 1900 && o.BirthYear < DateTime.Now.Year );
validator.AddPropertyValidator(o => o.Name.Length > 3);
validator.Validate(new ExampleObject());
}
I suggest using Automapper with a ValueResolver. You can deserialize the XML into an object in a very elegant way using autommaper and check if the values you get are valid with a ValueResolver.
You can use a base ValueResolver that check for Nulls or invalid casts, and some CustomResolver's that check if the Values you get are correct.
It might not be exacly what you are looking for, but I think it's an elegant way to do it.
Check this out here: http://dannydouglass.com/2010/11/06/simplify-using-xml-data-with-automapper-and-linqtoxml
In functional languages, such as Haskell, your problem could be solved with the Maybe-monad:
The Maybe monad embodies the strategy of combining a chain of
computations that may each return Nothing by ending the chain early if
any step produces Nothing as output. It is useful when a computation
entails a sequence of steps that depend on one another, and in which
some steps may fail to return a value.
Replace Nothing with null, and the same thing applies for C#.
There are several ways to try and solve the problem, none of them are particularly pretty. If you want a runtime-validation that something is not null, you could use an AOP framework to inject null-checking code into your type. Otherwise you would really have to end up doing nested if checks for null, which is not only ugly, it will probably violate the Law of Demeter.
As a compromise, you could use a Maybe-monad like set of extension methods, which would allow you to query the object, and choose what to do in case one of the properties is null.
Have a look at this article by Dmitri Nesteruk: http://www.codeproject.com/Articles/109026/Chained-null-checks-and-the-Maybe-monad
Hope that helps.
I assume your question is: How do I efficiently check whether my object is valid?
If so, it does not matter that your object was just deserialized from some text source. If your question regards checking the object while deserializing to quickly stop deserializing if an error is found, that is another issue and you should update your question.
Validating an object efficiently is not often discussed when it comes to C# and administrative tools. The reason is that it is very quick no matter how you do it. It is more common to discuss how to do the checks in a manner that is easy to read and easily maintained.
Since your question is about efficiency, here are some ideas:
If you have a huge number of objects to be checked and performance is of key importance, you might want to change your objects into arrays of data so that they can be checked in a consistent manner. Example:
Instead of having MyObject[] MyObjects where MyObject has a lot of properties, break out each property and put them into an array like this:
int[] MyFirstProperties
float[] MySecondProperties
This way, the loop that traverses the list and checks the values, can be as quick as possible and you will not have many cache misses in the CPU cache, since you loop forward in the memory. Just be sure to use regular arrays or lists that are not implemented as linked lists, since that is likely to generate a lot of cache misses.
If you do not want to break up your objects into arrays of properties, it seems that top speed is not of interest but almost top speed. Then, your best bet is to keep your objects in a serial array and do:
.
bool wasOk = true;
foreach (MyObject obj in MyObjects)
{
if (obj.MyFirstProperty == someBadValue)
{
wasOk = false;
break;
}
if (obj.MySecondProperty == someOtherBadValue)
{
wasOk = false;
break;
}
}
This checks whether all your objects' properties are ok. I am not sure what your case really is but I think you get the point. Speed is already great when it comes to just checking properties of an object.
If you do string compares, make sure that you use x = y where possible, instead of using more sophisticated string compares, since x = y has a few quick opt outs, like if any of them is null, return, if the memory address is the same, the strings are equal and a few more clever things if I remember correctly. For any Java guy reading this, do not do this in Java!!! It will work sometimes but not always.
If I did not answer your question, you need to improve your question.
I'm not certain I understand the depth of your question but, wouldn't you just do somthing like this,
public SomeClass
{
private const string UrlValidatorRegex = "http://...
private const DateTime MinValidSomeDate = ...
private const DateTime MaxValidSomeDate = ...
public string SomeUrl { get; set; }
public DateTime SomeDate { get; set; }
...
private ValidationResult ValidateProperties()
{
var urlValidator = new RegEx(urlValidatorRegex);
if (!urlValidator.IsMatch(this.Someurl))
{
return new ValidationResult
{
IsValid = false,
Message = "SomeUrl format invalid."
};
}
if (this.SomeDate < MinValidSomeDate
|| this.SomeDate > MinValidSomeDate)
{
return new ValidationResult
{
IsValid = false,
Message = "SomeDate outside permitted bounds."
};
}
...
// Check other fields and properties here, return false on failure.
...
return new ValidationResult
{
IsValid = true,
};
}
...
private struct ValidationResult
{
public bool IsValid;
public string Message;
}
}
The exact valdiation code would vary depending on how you would like your class to work, no? Consider a property of a familar type,
public string SomeString { get; set; }
What are the valid values for this property. Both null and string.Empty may or may not be valid depending on the Class adorned with the property. There may be maximal length that should be allowed but, these details would vary by implementation.
If any suggested answer is more complicated than code above without offering an increase in performance or functionality, can it be more efficient?
Is your question actually, how can I check the values on an object without having to write much code?
BinaryFormatter behaving in weird way in my code. I have code like following
[Serializable]
public class LogEntry
{
private int id;
private List<object> data = new List<object>();
public int Id
{
get { return id; }
}
public IList<object> Data
{
get { return data.AsReadOnly(); }
}
...
}
....
....
private static readonly BinaryFormatter logSerializer = new BinaryFormatter();
....
....
public void SerializeLog(IList<LogEntry> logEntries)
{
using (MemoryStream serializationStream = new MemoryStream())
{
logSerializer.Serialize(serializationStream, logEntries);
this.binarySerializedLog = serializationStream.GetBuffer();
}
}
In some machine (32 or 64 bit machine), it is serializing in binary format - which is expected. But in some machine ( all of them are 64 bit machine and not for debug builds) it is not serializing, binarySerializedLog is showing ToString() value of all individual Data, class name (...LogEntry) and id value. My question is - are there specific reason for this type of behavior or am I doing some mistake? Thanks in advance.
Your question isn't very clear (can you define "not serializing"?), but some thoughts:
You should really use ToArray() to capture the buffer, not GetBuffer() (which is cheaper, but returns the oversized array, and should only be used in conjunction with Length).
Where are you seeing this .ToString()? BinaryFormatter writes the objects type, then either uses reflection to write the fields (for [Serializable]) or uses customer serialization (for ISerializable). It never calls .ToString() (unless that is what your ISerializable does). However, strings etc will be in the output "as is".
Note that BinaryFormatter can be brittle between versions, so be careful if you are keeping this data for any length of time (it is generally fine for transport, though, assuming you update both ends at the same time). If you know in advance what your .Data objects are, there are a range of contract-based serializers that might provide more stability. I can provide more specific help if you think this would be worth investigating.