Background Info
I have a .Net Standard 2.0 class library that uses Protobuf-net.grpc's code first approach to define a gRPC service. In this definition I have classes defining different data structures that we are using to record sensor data and serialize this out using protobuf-net. My programs are ingesting serveral hundred thousand large objects /s (which will soon scale into the millions) and are intended to be used in embedded environments.
Problem
In my class below, I would like to include as a member a System.Numerics.Quaterion. I cannot seem to get this serialized out. Using static constructors, the RuntimeTypeModel throws exceptions as the Quaternion model has somehow already been created by the time the static constructor is executed. As this is a class library, and I desperately want to avoid invoking the RuntimeTypeModel in each different program using the gRPC service. I'm hoping to find a way to serialize the System.Numerics.Quaternion.
I have tried placing this static constructor runtime definition at the highest level of the class hierarchy to no avail. Exceptions still thrown.
[ProtoContract]
public class IMUData : SensorData, ISensorData
{
static IMUData()
{
RuntimeTypeModel.Default.Add(typeof(Quaternion), false)
.Add("W")
.Add("X")
.Add("Y")
.Add("Z");
}
... //Other members
[ProtoMember(8)]
public Quaternion Orientation
{
get; set;
}
... //Other methods and members
}
Question
Is what I would like to do even possible, or should I simply create my own Quaternion class and define implicit operators? (I'd rather avoid this as processing billions of these objects takes long enough)
This is ultimately a timing problem - when the serializer attempts to reflect on SensorData to prepare the serializer, the static constructor in IMUData has not yet executed, so it prepares the serializer with incomplete information, and then later the static constructor tries to reconfigure the model - too late.
If you use C# 9.0, you can fix this by using a module initializer instead of a static constructor (if we assume that SensorData and IMUData are in the same module, which is probably a safe assumption). The following works fine, for example:
[ProtoContract]
public class IMUData : SensorData //, ISensorData
{
[ModuleInitializer]
internal static void Init()
{
RuntimeTypeModel.Default.Add(typeof(Quaternion), false)
.Add("W")
.Add("X")
.Add("Y")
.Add("Z");
}
Note that if you're not using .NET 5 (preview, currently), you can define the necessary attribute yourself:
namespace System.Runtime.CompilerServices
{
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
internal sealed class ModuleInitializerAttribute : Attribute { }
}
If this isn't an option, you can simply put the model configuration code much earlier in your application - ideally during startup, so that it happens long before the serializer attempts to start building models.
Related
I'm pretty new to c#, and I'm using Harmony patches to make a mod for a video game. The method I'm trying to patch is a private method which takes an internal class instance as a parameter. I've been able to use reflection to handle private methods in a few other patches, but when I try to add the internal parameter, I get a build error saying the class is inaccessible due to it's protection level.
I was trying to use the solution from this question, but I think I'm having some scope issues. Right now, I have something like
using System;
...
using System.Reflection;
using HarmonyLib;
using namespacesFromGame; // Including namespace where the internal is declared
...
namespace MyMod
{
[HarmonyPatch(typeof(GameClass))]
class MyPatch
{
Type MyInternal = typeof(GameClass).Assembly.GetType("GameInternal");
public static bool MethodPatch(GameClass__instance,..., MyInternal myInternal, ...)
{
...
}
}
}
When I try to do this, it tells me The type or namespace name 'MyInternal' cannot be found.
Where should I be putting my MyInternal declaration so it can be used as a parameter to MethodPatch, and so I will also be able to use the myInternal instance in the patch?
In C# you cannot declare the type of a property with another variable.
I see two solutions to this problem.
You can either do this :
using System;
...
using HarmonyLib;
using namespacesFromGame; // Including namespace where the internal is
namespace MyMod
{
[HarmonyPatch(typeof(GameClass))]
class MyPatch
{
public static bool MethodPatch(GameClass __instance,..., object myInternal, ...)
{
...
// do reflexion to access the method, the field and prop the object
}
}
}
This should work; but if you are new to C# the reflection may not necessarily be the easiest thing to do and it can quickly make your code unreadable.
Or do this :
You can publicise (make public) the dll you want to use. If you do this, you will have access to all classes, methods and ect... By doing this you will only have to use the desired type. But, you will have to compile your code in unstable.
For publicise, i found two github repo :
https://github.com/rwmt/Publicise
https://github.com/iRebbok/APublicizer
(you can also create your own but I think that to start it would be better to take one already made)
It will require republishing the assembly each time there is an update if your mod is outdated.
I also create modes on unity games. This is the solution I use and some FrameWork for modding uses to.
I don't know if there are performance impacts of using unstable code and calling private methods.
I advise you this solution, you would get cleaner code and you will have access to code more easily. But that is my personal opinion.
Imagine the following scenario in a Xamarin solution:
Assembly A (PCL):
public abstract class MyBaseClass
{
public MyBaseClass()
{
[...]
}
[...]
}
Assembly B (3rd Party Library):
public class SomeLibClass
{
[...]
public void MethodThatCreatesClass(Type classType){
[...]
//I want to allow this to work
var obj = Activator.CreateInstance(classType);
[...]
}
[...]
}
Assembly C (Main project):
public class ClassImplA:MyBaseClass{
[...]
}
public class ClassImplA:MyBaseClass{
[...]
}
public class TheProblem{
public void AnExample(){
[...]
//I want to block these instantiations for this Assembly and any other with subclasses of MyBaseClass
var obj1 = new ClassImplA()
var obj2 = new ClassImplB()
[...]
}
}
How can I prevent the subclasses from being instantiated on their own assembly and allow them only on the super class and the 3rd Party Library (using Activator.CreateInstance)?
Attempt 1
I though I could make the base class with an internal constructor but then, I saw how silly that was because the subclasses wouldn't be able to inherit the constructor and so they wouldn't be able to inherit from the superclass.
Attempt 2
I tried using Assembly.GetCallingAssembly on the base class, but that is not available on PCL projects. The solution I found was to call it through reflection but it also didn't work since the result of that on the base class would be the Assembly C for both cases (and I think that's because who calls the constructor of MyBaseClass is indeed the default constructors of ClassImplA and ClassImplB for both cases).
Any other idea of how to do this? Or am I missing something here?
Update
The idea is to have the the PCL assembly abstract the main project (and some other projects) from offline synchronization.
Given that, my PCL uses its own DB for caching and what I want is to provide only a single instance for each record of the DB (so that when a property changes, all assigned variables will have that value and I can ensure that since no one on the main project will be able to create those classes and they will be provided to the variables by a manager class which will handle the single instantions).
Since I'm using SQLite-net for that and since it requires each instance to have an empty constructor, I need a way to only allow the SQLite and the PCL assemblies to create those subclasses declared on the main project(s) assembly(ies)
Update 2
I have no problem if the solution to this can be bypassed with Reflection because my main focus is to prevent people of doing new ClassImplA on the main project by simple mistake. However if possible I would like to have that so that stuff like JsonConvert.DeserializeObject<ClassImplA> would in fact fail with an exception.
I may be wrong but none of the access modifiers will allow you to express such constraints - they restrict what other entities can see, but once they see it, they can use it.
You may try to use StackTrace class inside the base class's constructor to check who is calling it:
public class Base
{
public Base()
{
Console.WriteLine(
new StackTrace()
.GetFrame(1)
.GetMethod()
.DeclaringType
.Assembly
.FullName);
}
}
public class Derived : Base
{
public Derived() { }
}
With a bit of special cases handling it will probably work with Activator class , but isn't the best solution for obvious reasons (reflection, error-prone string/assembly handling).
Or you may use some dependency that is required to do anything of substance, and that dependency can only be provided by your main assembly:
public interface ICritical
{
// Required to do any real job
IntPtr CriticalHandle { get; }
}
public class Base
{
public Base(ICritical critical)
{
if (!(critical is MyOnlyTrueImplementation))
throw ...
}
}
public class Derived : Base
{
// They can't have a constructor without ICritical and you can check that you are getting you own ICritical implementation.
public Derived(ICritical critical) : base(critical)
{ }
}
Well, other assemblies may provide their implementations of ICritical, but yours is the only one that will do any good.
Don't try to prevent entity creation - make it impossible to use entities created in improper way.
Assuming that you can control all classes that produce and consume such entities, you can make sure that only properly created entities can be used.
It can be a primitive entity tracking mechanism, or even some dynamic proxy wrapping
public class Context : IDisposable
{
private HashSet<Object> _entities;
public TEntity Create<TEntity>()
{
var entity = ThirdPartyLib.Create(typeof(TEntity));
_entities.Add(entity);
return entity;
}
public void Save<TEntity>(TEntity entity)
{
if (!_entities.Contains(entity))
throw new InvalidOperationException();
...;
}
}
It won't help to prevent all errors, but any attempt to persist "illegal" entities will blow up in the face, clearly indicating that one is doing something wrong.
Just document it as a system particularity and leave it as it is.
One can't always create a non-leaky abstraction (actually one basically never can). And in this case it seems that solving this problem is either nontrivial, or bad for performance, or both at the same time.
So instead of brooding on those issues, we can just document that all entities should be created through the special classes. Directly instantiated objects are not guaranteed to work correctly with the rest of the system.
It may look bad, but take, for example, Entity Framework with its gotchas in Lazy-Loading, proxy objects, detached entities and so on. And that is a well-known mature library.
I don't argue that you shouldn't try something better, but that is still an option you can always resort to.
I have a dll library TaskLibrary.dll with a class Execution performing some Operation(s) and a class ParallelizeExecution that takes a single Execution, clones it and executes the Run method of the multiple Execution instances.
The Clone method of the Execution class works by converting the Execution in xml and reverting it back to normal as a new instance
public Execution{
List<AOperation> operations;
public Run(){
foreach(var op in operations){
//...do stuff...
}
}
public Execution Clone(){
MyXmlSerializer.DeserializeObject<Execution>(
MyXmlSerializer.SerializeObject(this));
}
}
public ParallelizeExecution{
List<Execution> toRun;
public RunParallel(Execution e,int numExecutions){
toRun=new List<Execution>();
for(var i=0;i<numExecutions;i++){
toRun.Add(e.Clone());
}
}
}
The Execution class is serializable as is each of the classes implementing Operation. This is obtained by using an abstract class (AOperation) which all the IOperation implementations extend, using the XmlInclude annotation to make MyXmlSerializer work for each IOperation.
[XmlInclude(typeof(Operation1))]
[XmlInclude(typeof(Operation2))]
public abstract class AOperation:IOperation{
...
}
Now I have a new project referencing the TaskLibrary.dll. I need to add a new kind of Operation to a Execution:
public class Operation3: Operation2 {
}
Everything works fine with a single execution, but when I use ParallelizeExecution Operation3 is correctly serialized as Operation2 thus executing an unwanted Run method. How can I add a new type of AOperation to the Execution class and expect it to be correctly serialized and run in the Execution?
OR
How can I avoid the problem without altering the way the Execution class is serialized?
Caveat: I know that it is possible to use Reflection to xmlserialize any tipe extending a given one, but i'd rather learn how to do this using standard OOP (if possible).
EDIT: I could modify TaskLibrary.dll but I'd rather avoid this approach, it would void my efforts in learning new things and aiding the community, moreover it would be quite painful to redistribute the library to those already using it.
You can use the extraTypes argument to the XmlSerializer constructor. Use reflection to find all of the relevant types, and create the XmlSerializer with the complete list. The overall process is laid out here, but for posterity's sake:
// use reflection to get all derived types
List<type> knownTypes = new List<type>();
// Iterate over whichever assembly has your types.
foreach(Type t in Assembly.GetExecutingAssembly().GetTypes())
if (typeof(Car).IsAssignableFrom(t) ||
typeof(Wheel).IsAssignableFrom(t) ||
typeof(Door).IsAssignableFrom(t))
knownTypes.Add(t);
// prepare to serialize a car object
XmlSerializer serializer = new XmlSerializer(typeof(Car), knownTypes.ToArray());
I used the OWLGrinder to create the assembly and imported the library into my project. That works fine. Now I want to write my own set of classes. Therefore I extended these with the equivalent of the assembly. But it just doesn't work.
The ontology holds a class named ManagementObject.
I created another Class (C#) called RealWorldObject:
public class RealWorldObject : ManagementObject
{
public RealWorldObject(string uri) : base(uri) { }
public RealWorldObject(string uri, RdfDocument rdfdocument) : base(uri, rdfdocument) { }
public RealWorldObject(RdfDocument rdfdocument) : base(rdfdocument) { }
public String getClassName()
{
return this.OwlClassName;
}
public static RdfDocument addRealWorldObjectIndividualt(RdfDocument rdfDocument)
{
Vehicle vehicle = new Vehicle("vehicle1", rdfDocument);
FixedEvent fxE1 = new FixedEvent("autoGekauft", rdfDocument);
fxE1.agent = new xmlns.com.foaf._01.Person("robert", rdfDocument);
vehicle.hasFixedEvent = fxE1;
return rdfDocument;
}
Which leads to the error:
ObjectManagement.Object.RealWorldObject does declare one (and only one) OwlClassAttribute. This is an implementation bug of the plugin.
How else should I extend the generated classes by the OWLGrinder.
Thx it is a long time ago that I used C#, so I'm kind of rusty.
The auto-generated classes produced by OwlGrinder.exe have not been designed for inheritance in mind. I am not saying it is wrong, it is just not designed for that. The auto-generated classes contain plenty of metadata defined as class attributes and inheritance hides all of that. The infrastructure counts on the presence of these attributes and if they are hidden, you get these runtime error messages.
Using Visual Studio Object Browser, take a look of the attributes over the auto-generated classes. OwlClassAttribute, SubClassOfAttribute, LightVersionAttribute are certainly mandatory. You may simply copy/paste the class attributes of ManagementObject on the top of your RealWorldObject class. I assume, it will work. But again, you might bump into additional show stoppers, as you do not follow the default routes ROWLEX has been designed for. This is a bit living on the edge :)
Instead of inheritance, you might consider reverse engineering your auto-generated assembly to C# using Reflector or other tools. Having the source code in your hand, you may modify the generated classes directly. You might make your ManagementObject class partial, and implement your additional methods in a separate file.
This might be a little subjective, but I'd like to get your input on my current situation. I have a class that will be used to serialize/deserialize an object.
public class MyClass
{
public static string ToXmlString( MyClass c ) { /*...*/ }
public static MyClass FromXmlString( string xml ) { /*...*/ }
}
I only like this approach because it keeps the two functions at the same level. However, my goal is to avoid using static methods (when feasable). It also feels like I might be vilolating SRP, but the main goal of this object is that it can be seriliazed/deserialized from an xml string.
Any thoughts on the use of static methods in this situation? Should I just make the ToXmlString non-static, but leave the FromXmlString static? Should I create a new class that will only handle serilization of MyClass?
EDIT:
The class that I'm discussion here is a simple transfer object. It is used to save/restore values from a thrid party tool.
Thanks!
FWIW I think that serialization is a problematic that should be separated from the rest of your class, above all if your class is a business type.
The general rule when developing a component is to ensure that it only addresses a few concerns and to separate business concerns from technical ones.
What if later you need to manage serialization from a database or a binary format ?
You might end with more and more technical methods (SaveToDB, LoadFromDB, ToBinaryStream, FromBinaryStream...) that would clutter your class and make it more and more difficult to maintain, hiding its primary purposes (business for example).
The convention in the standard libs for both C# and Java is that To__ methods are instance methods and From__ methods are static (by necessity). For example: ToString() is an instance method.
Elaborating on Benoit's answer, here's an example where the class that is being serialized defines the serializing behavior (I did not write this):
// : c12:SerialCtl.java
// Controlling serialization by adding your own
// writeObject() and readObject() methods.
// From 'Thinking in Java, 3rd ed.' (c) Bruce Eckel 2002
// www.BruceEckel.com. See copyright notice in CopyRight.txt.
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
public class SerialCtl implements Serializable {
private String a;
private transient String b;
public SerialCtl(String aa, String bb) {
a = "Not Transient: " + aa;
b = "Transient: " + bb;
}
public String toString() {
return a + "\n" + b;
}
private void writeObject(ObjectOutputStream stream) throws IOException {
stream.defaultWriteObject();
stream.writeObject(b);
}
private void readObject(ObjectInputStream stream) throws IOException,
ClassNotFoundException {
stream.defaultReadObject();
b = (String) stream.readObject();
}
public static void main(String[] args) throws IOException,
ClassNotFoundException {
SerialCtl sc = new SerialCtl("Test1", "Test2");
System.out.println("Before:\n" + sc);
ByteArrayOutputStream buf = new ByteArrayOutputStream();
ObjectOutputStream o = new ObjectOutputStream(buf);
o.writeObject(sc);
// Now get it back:
ObjectInputStream in = new ObjectInputStream(new ByteArrayInputStream(
buf.toByteArray()));
SerialCtl sc2 = (SerialCtl) in.readObject();
System.out.println("After:\n" + sc2);
}
}
Note the use of transient to describes fields that will not be serialized.
If you want a standard serialization (XML or not), both serialize/deserialize methods should not be static.
In MyClass, you should redefine "writeObject" and "readObject" to replace the default serialization methods by yours. Here is a Sun tutorial about theses methods.
If you don't want a "standard serialization", using static methods looks fine for me. Static util methods are not an heresy.
PS : it is not the question, but if you want WML serialization, you can use the XStream API.
You could define a constructor that takes an XMLReader (or a string if you really insist). The main advantage of this is that it allows you to have stronger invariants in your class, and to be explicit about any immutable members through the use of readonly.
I don't think it's too terrible for complementary methods to be separated with regard to static vs. instance, since the Framework does this occasionally (String.Split / Join, for example).
But having said that, I think the goal of minimizing the use of static methods is not a good idea. The thing to avoid is static mutable state, not static methods. A static method that only operates on its parameters, rather than static variables, is pure awesomeness.
A pure static function can be more maintainable than an instance method, since the instance method does not communicate in an obvious way which instance fields it can mutate. By following the rule that no static state whatsoever is maintained, a static method can be relied upon to only operate on its parameters, and thus the method's role in the application can be better predicted. This is especially important when multi-threading.
Since the ToXmlString method is being applied to an instance of the class in which it is defined, some of these considerations don't apply. It could easily change the state of the object that is being passed to it in underhanded ways, since it can access all the private members of the instance. But I just mean to say that as a general rule static methods are not a problem.