Json serialization and storage strategy [closed] - c#

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Json is often used in web applications. I set a field in the database to text to store json strings. It may not be a good idea to design a database in this way, but my reason is that the composition of the data may be volatile and does not require external connections. Using Json serialization tools such as Newtonsoft can easily de-serialize a json string from a database into a JObject object, but in a static language program, it will definitely require some more concrete objects to express it, and now I'm facing some Optional options:
De-serialize the json string from the database to get JObject, then
use this object directly
continue to resolve JObject, instantiate a more specific object A, and then use the object A
Option 1 is very convenient for storage and initialization of the read, but when used to get the value string, error-prone. Scenario 2 requires an extra layer of conversion for storing and initializing reads, but it is more intuitive and convenient to use. I am very tangled about which scheme to use.

"More intuitive and convenient to use" is the key here. You should always try to write code in a readable and maintainable way, and if creating model objects for your JSON data helps you achieve this goal, and this doesn't impact performance beyond what's acceptable for your project, do it; the extra conversion layer will be worth it.

Unless there is a reason that you cannot, I would have a concrete dto that your JSON can be de-serialised into. This provides you compile time type safety for all usages (at least after the initial instantiating). I would normally go another step and have a business object class that can instantiate itself from that dto, but that obviously depends on your specific requirements.
On top of type safety, you get a whole bunch more benefits when using a concrete object, but I suspect that at this point I'm preaching to the choir.
One reason you may not be able to is where the content itself is dynamic in nature (and by extension your code expects nothing specific about the JSON string other than maybe that it is well formed). Very few problems are like this though.
So the downside is usually the overhead of the time and effort in writing those concrete classes. Once you have them defined, deserialising them is literally 1 line of code. The trick then is to use a tool to reverse engineer the necessary classes from the JSON string. Due to SO policies, I cannot recommend tools, but if you were to use a search engine that rhymes with frugal and search for something like JSON to c#, you are bound to find a quick way to create those dtos.

Related

Is protobuf a good choice of the universal data model? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 11 months ago.
Improve this question
We are trying to find a universal data model, which can be applied everywhere.
I think there is no doubt that protobuf is good in the microservice architecture as a protocol among processes of different platforms (Windows, Linux, etc.) and languages (Java, C#, C++, Python, etc.)
But how about using protobuf within the process?
To make it simple and easy to illustrate the problems, let say I am making a C# gRPC service and GUI (gRPC client).
After some study, it seems that it is not suitable from what I can see.
Stability: It looks like that protobuf is still in a changing phase. For example, removing Optional keyword in proto3, but adding it back again in proto release 3.15.
Data type: The data types in proto are not fully compatible with common data types. For example, decimal. We need to define another complex data type (What's the best way to represent System.Decimal in Protocol Buffers?) and doing the conversion.
Conversion: We need conversion before we can utilize it in the related language. You cannot add the self-defined proto decimal directly in c#.
Repeated conversion: Conversion is not one off, but back and forth. Let say we have this proto object passing through 5 functions and need to have some calculations on the decimal field at each function. That means, we will need to convert the decimal field in the proto object to C# decimal in each function, have the calculation to get the result, convert and assign the result back to the decimal field in the proto object, then pass the proto object to the next function.
GUI control binding: We cannot bind the proto field (for those without an matched type in C#). Even we can specify the conversion to do so in the control somehow, (Indeed, I am not sure if we can/it is good to do so) in simple control, like textbox. It may not be easy for complicated control like datagridview because there may be different built-in editors for different native data types. If we use proto data types, that means we need to write customized editors for them. And also, we may need to define other behaviors, like how to sort them.
Using the auto-generated proto class within the process seems not a good idea to me as the reasons listed above. The above case only focus on C# service and client. When it comes to different languages, the cases should be more complicated.
Although .net soap services are slow and wordy (if you look at the wsdl definition), one thing I appreciate very much is that both the service and the client are using the same object with native data types, which can be used directly without any issue. The conversion is done in the communication directly and automatically.
The way I can think of at the moment is that:
Using proto as a communication protocol only
Write a class (use native/built-in data types) for each proto message type and our own conversion logic. Doing so because I cannot find any framework/lib to do so.
After receiving the proto object, convert it directly to an object with native/built-in types before further processing. (the benefit here is that even there is a major change in proto spec, we only need to change the conversion logic only without affecting other layers).
Am I on the right track? What is common/best practice to resolve the problems listed above?
Any help is highly appreciated!!!

Serializing Events into JSON [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I am working on a serialization system using Json, but I need to save events from Buttons (onClick, onHover, etc.) Is there a way about doing this efficiently? (NOTE: The events are all Actions)
Frankly, it is is terrible idea to try to serialize events.
JSON is usually used to serialize data; events are not data - they are implementation details. Most JSON serializers (or more broadly: most serializers) are not interested in delegates / events, because that isn't relevant to data, so: there's a good chance that anything you'd want to do here will need to be manual. Specifically, the problem here is that an event (or rather, the underlying multicast delegate) is effectively zero, one, or multiple pairs of "instance" (optional) and "method" (required).
The method here is a MethodInfo, and there aren't great ways to serialize a MethodInfo as text (although it is at least theoretically possible, although it would be very brittle vs changes to your code.
The instance, however, is an object - and most serializers hate that; in this case, it would combine object (reference) tracking, possibly of objects not otherwise inside the payload, of indeterminate types (so: possibly needing to store type metadata).
Also, deserializing an object model that allows you to point to arbitrary types and methods is a massive security hole, and is a well-known RCE weakness in serializers that (unwisely, IMO) allow this kind of thing (such as BinaryFormatter; for a longer discussion of this topic, see here).
As for what to do instead: whenever an implementation isn't a great fit for a given serializer, the most pragmatic option is to stop fighting the serializer, and work with it instead of against it. For example, it might be that you can create a model that looks kinda like your domain model, but instead of having events/delegates, it might just have a string[] / List<string> that represents the events you need to apply, and your code would worry about how to map between them (mapping methods to strings, and figuring out what the target instance should be, etc). This avoids all of the pain points above, and additionally means that your data is now platform independent, with the payload and the implementation details (your form layout) separate from each-other.

Avoid unsafe binary serialization when using clipboard [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
There are a few questions that address already how to store custom objects into the clipboard. This requires marking classes as [Serializable] and then using binary serialization to retrieve them from the clipboard. An example is here.
However, Microsoft issues the following warning about binary serialization:
"Binary serialization can be dangerous. Never deserialize data from an untrusted source and never round-trip serialized data to systems not under your control."
(here).
I tried to use the clipboard with [DataContract] instead of [Serializable] to avoid binary serialization, and this doesn't seem to be working, and classes that aren't marked as [Serializable] are retrieved as 'null'.
Thus, summing up:
Is it safe to use binary serialization in the specific scenario of
the clipboard? I cannot assume that I trust the information in the
clipboard -- the user may have copied it from anywhere.
Alternatively, is it possible to avoid binary serialization to store
and retrieve custom objects from the clipboard?
Edit: using "GetText" to store everything in text removes the ability to paste text only in text recipients (e.g. Notepad) versus pasting in other containers that are able to process the additional information.
There is no need for you to take the data from the clipboard and rely on binary serialization/deserializtion. You can take the data from the clipboard as string which is basically the type that contains the information that you need and how it is represented on the clipboard, then you can process it as you want.
Just use the GetText method and then use the data as you want.
string clipboardText = Clipboard.GetText(TextDataFormat.Text);
Then if there is really a type that you want to deserialize the data into, then you could use newtonsoft.json nuget package to deserialize the data.
var myObject = Json.DeserializeObject<MyType>(clipboardText);
Then you can work with your custom object. Just make sure to include the deserialization in a try catch block in order to treat the case when the data is not in the proper format.
If there is no need for you to stick to binary serialization, then I would not suggest you to go for it, especially because it is harder to follow and maintain if bugs occur.

Why is there no inverse to object.ToString()? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
It seems like a good design decision that the System.Object class, and hence all classes, in .NET provide a ToString() method which, unsurprisingly, returns a string representation of the object. Additionally in C# this method is implemented for native types so that they integrate nicely with the type system.
This often comes in handy when user interaction is required. For example, objects can directly be held in GUI widgets like lists and are "automatically" displayed as text.
What is the rationale in the language design to not provide a similarly general object.FromString(string) method?
Other questions and their answers discuss possible objections, but I find them not convincing.
The parse could fail, while a conversion to string is always possible.
Well, that does not keep Parse() methods from existing, does it? If exception handling is considered an undesirable design, one could still define a TryParse() method whose standard implementation for System.Object simply returns false, but which is overridden for concrete types where it makes sense (e.g. the types where this method exists today anyway).
Alternatively, at a minimum it would be nice to have an IParseable interface which declares a ParseMe() or TryParse() method, along the lines of ICloneable.
Comment by Tim Schmelter's "Roll your own": That works of course. But I cannot write general code for native types or, say, IPAddress if I must parse the values; instead I have to resort to type introspection or write wrappers which implement a self-defined interface, which is either maintenance-unfriendly or tedious and error-prone.
Comment by Damien: An interface can only declare non-static functions for reasons discussed here by Eric Lippert. This is a very valid objection. A static TryParse() method cannot be specified in an interface. A virtual ParseMe(string) method though needs a dummy object, which is a kludge at best and impossible at worst (with RAII). I almost suspect that this is the main reason such an interface doesn't exist. Instead there is the elaborate type conversion framework, one of the alternatives mentioned as solutions to the "static interface" oxymoron.
But even given the objections listed, the absence of a general parsing facility in the type system or language appears to me as an awkward asymmetry, given that a general ToString() method exists and is extremely useful.
Was that ever discussed during language/CLR design?
It seems like a good design decision that the System.object class, and hence all classes, in .NET provide a ToString() method
Maybe to you. It's always seemed like a really bad idea to me.
which, unsurprisingly, returns a string representation of the object.
Does it though? For the vast majority of types, ToString returns the name of the type. How is that a string representation of the object?
No, ToString was a bad design in the first place. It has no clear contract. There's no clear guidance on what its semantics should be, aside from having no side effects and producing a string.
Since ToString has no clear contract, there is practically nothing you can safely use it for except for debugger output. I mean really, think about it: when was the last time you called ToString on object in production code? I never have.
The better design therefore would have been methods static string ToString<T>(T) and static string ToString(object) on the Debug class. Those could have then produced "null" if the object is null, or done some reflection on T to determine if there is a debugger visualizer for that object, and so on.
So now let's consider the merits of your actual proposal, which is a general requirement that all objects be deserializable from string. Note that first, obviously this is not the inverse operation of ToString. The vast majority of implementations of ToString do not produce anything that you could use even in theory to reconstitute the object.
So is your proposal that ToString and FromString be inverses? That then requires that every object not just be "represented" as a string, but that it actually be round trip serializable to string.
Let's think of an example. I have an object representing a database table. Does ToString on that table now serialize the entire contents of the table? Does FromString deserialize it? Suppose the object is actually a wrapper around a connection that fetches the table on demand; what do we serialize and deserialize then? If the connection needs my password, does it put my password into the string?
Suppose I have an object that refers to another object, such that I cannot deserialize the first object without also having the second in hand. Is serialization recursive across objects? What about objects where the graph of references contains loops; how do we deal with those?
Serialization is difficult, and that's why there are entire libraries devoted to it. Making it a requirement that all types be serializable and deserializable is onerous.
Even supposing that we wanted to do so, why string of all things? Strings are a terrible serialization data type. They can't easily hold binary data, they have to be entirely present in memory at once, they can't be more than a billion characters tops, they have no structure to them, and so on. What you really want for serialization is a structured binary storage system.
But even given the objections listed, the absence of a general parsing facility in the type system or language appears to me as an awkward asymmetry, given that a general ToString() method exists and is extremely useful.
Those are two completely different things that have nothing to do with each other. One is a super hard problem best solved by libraries devoted to it, and the other is a trivial little debugging aid with no specification constraining its output.
Was that ever discussed during language/CLR design?
Was ToString ever discussed? Obviously it was; it got implemented. Was a generalized serialization library ever discussed? Obviously it was; it got implemented. I'm not sure what you're getting at here.
Why is there no inverse to object.ToString()?
Because object should hold the bare minimum functionality required by every object. Comparing equality and converting to string (for a lot of reasons) are two of them. Converting isn't. The problem is: how should it convert? Using JSON? Binary? XML? Something else? There isn't one uniform way to convert from a string. Hence, this would unnecessarily bloat the object class.
Alternatively, at a minimum it would be nice to have an IParseable interface
There is: IXmlSerializable for example, or one of the many alternatives.

Why anonymous types aren't dynamic as the ExpandoObject? [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
With dynamic we pretty much have a dynamic pointer, but not exactly a dynamic object. The true dynamic object in C# is the ExpandoObject, but that is a really unknown class for most of people. The expando allows creating and removing members at runtime, much like a hash (similar to JavaScript).
Why the ExpandoObject goodness was implemented in a separate class rather than just being, let's say, implemented as a feature of anonymous types?
Maybe that wouldn't be a good move because the lacking of type-safety? Or maybe due the (DLR) overhead involved?
Because anonymous types have other very important feature - they provide you compile time type safety.
And because dynamic and anonymous types are just different concepts. The first one gives you ability to dispatch object members at runtime, the second lets you create statically typed objects with some base functionality (equality, hashcode, etc) without creating corresponding POCO classes. Why should they be implemented in the same way then?
btw. I use them quite a lot and really rarely needed to use dynamic to deal with them. Are you sure you're using these language features correctly?
Update
I think that's very important part of anonymous types tutorial:
If you must store query results or pass them outside the method boundary, consider using an ordinary named struct or class instead of an anonymous type.

Categories