Is there something like XmlRootAttribute that can be used with System.Text.JsonSerializer?
I need to be able to download data from this vendor using both XML an JSON. See sample data here:
{
"categories": [
{
"id": 125,
"name": "Trade Balance",
"parent_id": 13
}
]
}
Note the data elements are wrapped by an array named categories. When downloading using XML I can set the root element to categories and the correct object is returned (see XMLClient below ). When using JSONClient however I cannot (or do not know how) to set the root element. Best workaround I could find is to use JsonDocument which creates a string allocation. I could also create some wrapper classes for the JSON implementation but that is a lot of work that involves not only creating additional DTOs but also requires overwriting many methods on BaseClient. I also don't want to write converters - it pretty much defeats the purpose of using well-known serialization protocols. (Different responses will have a different root wrapper property name. I know the name in runtime, but not necessarily at compile time.)
public class JSONClient : BaseClient
{
protected override async Task<T> Parse<T>(string uri, string root)
{
uri = uri + (uri.Contains("?") ? "&" : "?") + "file_type=json";
var document = JsonDocument.Parse((await Download(uri)), new JsonDocumentOptions { AllowTrailingCommas = true });
string json = document.RootElement.GetProperty(root).GetRawText(); // string allocation
return JsonSerializer.Deserialize<T>(json);
}
}
public class XMLClient : BaseClient
{
protected override async Task<T> Parse<T>(string uri, string root)
{
return (T)new XmlSerializer(typeof(T), new XmlRootAttribute(root)).Deserialize(await Download(uri)); // want to do this - using JsonSerializer
}
}
public abstract class BaseClient
{
protected virtual async Task<Stream> Download(string uri)
{
uri = uri + (uri.Contains("?") ? "&" : "?") + "api_key=" + "xxxxx";
var response = await new HttpClient() { BaseAddress = new Uri(uri) }.GetAsync(uri);
return await response.Content.ReadAsStreamAsync();
}
protected abstract Task<T> Parse<T>(string uri, string root) where T : class, new();
public async Task<Category> GetCategory(string categoryID)
{
string uri = "https://api.stlouisfed.org/fred/category?category_id=" + categoryID;
return (await Parse<List<Category>>(uri, "categories"))?.FirstOrDefault();
}
}
JSON has no concept of a root element (or element names in general), so there's no equivalent to XmlRootAttribute in System.Text.Json (or Json.NET for that matter). Rather, it has the following two types of container, along with several atomic value types:
Objects, which are unordered sets of name/value pairs. An object begins with {left brace and ends with }right brace.
Arrays, which are ordered collections of values. An array begins with [left bracket and ends with ]right bracket.
As System.Text.Json.JsonSerializer is designed to map c# objects to JSON objects and c# collections to JSON arrays in a 1-1 manner, there's no built-in attribute or declarative option to tell the serializer to automatically descend the JSON hierarchy until a property with a specific name is encountered, then deserialize its value to a required type.
If you need to access some JSON data that is consistently embedded in some wrapper object containing a single property whose name is known at runtime but not compile time, i.e.:
{
"someRuntimeKnownWrapperPropertyName" : // The value you actually want
}
Then the easiest way to do that would be to deserialize to a Dictionary<string, T> where the type T corresponds to the type of the expected value, e.g.:
protected override async Task<T> Parse<T>(string uri, string root)
{
uri = uri + (uri.Contains("?") ? "&" : "?") + "file_type=json";
using var stream = await Download(uri); // Dispose here or not? What about disposing of the containing HttpResponseMessage?
var options = new JsonSerializerOptions
{
AllowTrailingCommas = true,
// PropertyNameCaseInsensitive = false, Uncomment if you need case insensitivity.
};
var dictionary = await JsonSerializer.DeserializeAsync<Dictionary<string, T>>(stream, options);
// Throw an exception if the dictionary does not have exactly one entry, with the required name
var pair = dictionary?.Single();
if (pair == null || !pair.Value.Key.Equals(root, StringComparison.Ordinal)) //StringComparison.OrdinalIgnoreCase if required
throw new JsonException();
// And now return the value
return pair.Value.Value;
}
Notes:
By deserializing to a typed dictionary rather than a JsonDocument you avoid the intermediate allocations required for the JsonDocument itself as well as the string returned by GetRawText(). The only memory overhead is for the dictionary itself.
Note also that JsonDocument is disposable. Failure to dispose it will result in the memory not being returned to the pool, which will increase GC impact across various parts of the framework.
Alternatively, you could create a [custom JsonConverter](Failure to properly dispose this object will result in the memory not being returned to the pool, which will increase GC impact across various parts of the framework.) to read through the incoming JSON until a property of the required name is encountered, then deserialize the value to the expected type.
For some examples, see Is there a simple way to manually serialize/deserialize child objects in a custom converter in System.Text.Json? or https://stackoverflow.com/a/62155881/3744182.
Rather than allocating an HttpClient for every call, you should allocate only one and reuse it. See Do HttpClient and HttpClientHandler have to be disposed between requests?.
You are not disposing of the HttpResponseMessage or response content stream. You may want to refactor your code to do that; see When or if to Dispose HttpResponseMessage when calling ReadAsStreamAsync?. You may also want to check response.IsSuccessStatusCode before deserializing.
In .Net 5, HttpClientJsonExtensions can make deserializing JSON returned by HttpClient simpler.
Related
I want to serialize a C# object as JSON into a stream, but to avoid the serialization if the object is not valid according to a schema. How should I proceed with this task using JSON.NET and Json.NET Schema? From what I see there is no method in the JSON.NET library which allows the validation of a C# object against a JSON schema. It seems somewhat weird that there is no direct method to just validate the C# object without encoding it. Do you have any idea why this method is not available?
It seems this API not currently available. At a guess, this might be because recursively generating the JSON values to validate involves most of the work of serializing the object. Or it could just be because no one at Newtonsoft ever designed, specified, implemented, tested, documented and shipped that feature.
If you want, you could file an enhancement request requesting this API, probably as a part of the SchemaExtensions class.
In the meantime, if you do need to test-validate a POCO without generating a complete serialization of it (because e.g. the result would be very large), you could grab NullJsonWriter from Reference to automatically created objects, wrap it in a JSchemaValidatingWriter and test-serialize your object as shown in Validate JSON with JSchemaValidatingWriter. NullJsonWriter doesn't actually write anything, and so using it eliminates the performance and memory overhead of generating a complete serialization (either as a string or as a JToken).
First, add the following static method:
public static class JsonExtensions
{
public static bool TestValidate<T>(T obj, JSchema schema, SchemaValidationEventHandler handler = null, JsonSerializerSettings settings = null)
{
using (var writer = new NullJsonWriter())
using (var validatingWriter = new JSchemaValidatingWriter(writer) { Schema = schema })
{
int count = 0;
if (handler != null)
validatingWriter.ValidationEventHandler += handler;
validatingWriter.ValidationEventHandler += (o, a) => count++;
JsonSerializer.CreateDefault(settings).Serialize(validatingWriter, obj);
return count == 0;
}
}
}
// Used to enable Json.NET to traverse an object hierarchy without actually writing any data.
class NullJsonWriter : JsonWriter
{
public NullJsonWriter()
: base()
{
}
public override void Flush()
{
// Do nothing.
}
}
Then use it like:
// Example adapted from
// https://www.newtonsoft.com/jsonschema/help/html/JsonValidatingWriterAndSerializer.htm
// by James Newton-King
string schemaJson = #"{
'description': 'A person',
'type': 'object',
'properties': {
'name': {'type':'string'},
'hobbies': {
'type': 'array',
'maxItems': 3,
'items': {'type':'string'}
}
}
}";
var schema = JSchema.Parse(schemaJson);
var person = new
{
Name = "James",
Hobbies = new [] { ".Net", "Blogging", "Reading", "XBox", "LOLCATS" },
};
var settings = new JsonSerializerSettings { ContractResolver = new CamelCasePropertyNamesContractResolver() };
var isValid = JsonExtensions.TestValidate(person, schema, (o, a) => Console.WriteLine(a.Message), settings);
// Prints Array item count 5 exceeds maximum count of 3. Path 'hobbies'.
Console.WriteLine("isValid = {0}", isValid);
// Prints isValid = False
Watch out for cases by the way. Json.NET schema is case sensitive so you will need to use an appropriate contract resolver when test-validating.
Sample fiddle.
You cannot do that from the JSON string, you need an object and a schema to compare with first..
public void Validate()
{
//...
JsonSchema schema = JsonSchema.Parse("{'pattern':'lol'}");
JToken stringToken = JToken.FromObject("pie");
stringToken.Validate(schema);
i have a web api 2 project the client will request some data that is in a xml format. That XML will never change and i am wondering how i could keep it in ram so that it doesnt deserialize the xml each time it needs data from that file.
Would deserializing it at launch and then keep it in a static variable be the best way as it will only be use for reading ?
[HttpPost]
[Route("api/dosomething")]
public string DoSomething() {
var myData = XmlSerializer(MyDataStruct).Deserialize(something);
return myDate;
}
Here the xml is only used to communicate values to clients. How can i make it so that i could deserialize it once and then return that directly. Would using static member enable this feature ?
A simple cache-aside approach with a static field could be a fair option:
private static MyDataStruct _myData;
[HttpPost]
[Route("api/dosomething")]
public string DoSomething() {
if(_myData == null)
{
_myData = new XmlSerializer(typeof(MyDataStruct)).Deserialize(something);
}
return _myData;
}
If you want even better performance and completely skip both the deserialization from your XML and the serialization of your response body into JSON/XML, then I strongly suggest you an HTTP output caching approach, using a library like this one: AspNetWebApi-OutputCache.
The context is as follows
I want to refactor code by moving it around to different projects
Some of this code comprises of serializable DTOs that are used to
send and receive data across multiple endpoints
If I move the code around, serialization breaks (therefore it is not
backward compatible with older versions of my application)
A solution to this problem is the SerializationBinder which allows me to "redirect" in a sense from one type to another.
I therefore want to create a SerializationBinder to meet this need. However it must do so by meeting the following requirements
The inputs to the SerializationBinder should be a list of old type
to new type mappings. The mapping should include the old assembly
name (no version, no public key token) and the old full name of the
type (namespace and name) as well as the new assembly name and new
full name of the type
For the types that are in the inputs, version numbers of assemblies should be ignored
It should handle generics if my types happen to be in generics
(List, Dictionary, etc) without needing to include the generics in the inputs
For anything that is not in the inputs (i.e. types that have not
moved or .NET types like dataset for example) it should default to
using the out of the box algorithm of the binary serializer
Is this possible or am I dreaming? Is there something out there that already does this? I would assume this is a common problem.
So far I see no easy way of doing 3 and no way at all of doing 4.
Here is an attempt
public class SmartDeserializationBinder : SerializationBinder
{
/// <summary>
/// Private class to handle storing type mappings
/// </summary>
private class TypeMapping
{
public string OldAssemblyName { get; set; }
public string OldTypeName { get; set; }
public string NewAssemblyName { get; set; }
public string NewTypeName { get; set; }
}
List<TypeMapping> typeMappings;
public SmartDeserializationBinder()
{
typeMappings = new List<TypeMapping>();
}
public void AddTypeMapping(string oldAssemblyName, string oldTypeName, string newAssemblyName, string newTypeName)
{
typeMappings.Add(new TypeMapping()
{
OldAssemblyName = oldAssemblyName,
OldTypeName = oldTypeName,
NewAssemblyName = newAssemblyName,
NewTypeName = newTypeName
});
}
public override Type BindToType(string assemblyName, string typeName)
{
//Need to handle the fact that assemblyName will come in with version while input type mapping may not
//Need to handle the fact that generics come in as mscorlib assembly as opposed to the assembly where the type is defined.
//Need to handle the fact that some types won't even be defined by mapping. In this case we should revert to normal Binding... how do you do that?
string alternateAssembly = null;
string alternateTypeName = null;
bool needToMap = false;
foreach (TypeMapping mapping in typeMappings)
{
if (typeName.Contains(mapping.OldTypeName))
{
alternateAssembly = mapping.NewAssemblyName;
alternateTypeName = mapping.NewTypeName;
needToMap = true;
break;
}
}
if (needToMap)
{
bool isList = false;
if (typeName.Contains("List`1"))
isList = true;
// other generics need to go here
if (isList)
return Type.GetType(String.Format("System.Collections.Generic.List`1[[{0}, {1}]]", alternateTypeName, alternateAssembly));
else
return Type.GetType(String.Format("{0}, {1}", alternateTypeName, alternateAssembly));
}
else
return null; // this seems to do the trick for binary serialization, but i'm not sure if it is supposed to work
}
}
This could work (instead of your override).
public override Type BindToType(string assemblyName, string typeName)
{
var m = Regex.Match(typeName, #"^(?<gen>[^\[]+)\[\[(?<type>[^\]]*)\](,\[(?<type>[^\]]*)\])*\]$");
if (m.Success)
{ // generic type
var gen = GetFlatTypeMapping(m.Groups["gen"].Value);
var genArgs = m.Groups["type"]
.Captures
.Cast<Capture>()
.Select(c =>
{
var m2 = Regex.Match(c.Value, #"^(?<tname>.*)(?<aname>(,[^,]+){4})$");
return BindToType(m2.Groups["aname"].Value.Substring(1).Trim(), m2.Groups["tname"].Value.Trim());
})
.ToArray();
return gen.MakeGenericType(genArgs);
}
return GetFlatTypeMapping(assemblyName,typeName);
}
Then you just have to implement your way the function GetFlatTypeMapping (not worrying of about generic arguments).
What you will have to do is to return typeof(List<>) and typeof(Dictionary<,>) (or any other generic you would like to use) when asked.
nb: I said typeof(List<>) ! not typeof(List<something>) ... that's important.
disclaimer: because of the regex "(?[^]]*)", this snipped does not support nested generic types like List<List<string>> ... you will have to tweak it a bit to support it !
Although it might not answer your question this might solve your problem:
I was able to serialize all but one assembly required for deserialization alongside an object structure. The trick is to use reflection to inspect your object structure for types and the assemblies they are defined in. You can then write the assemblies as binary data into an object on serialization and load them into an AppDomain where you can use them to deserialize the rest of the object structure.
You have to put the AppDomain handling, the root object and some base classes into an assembly that won't change and everything else is defined in assemblies depending on this "anchor".
Upsides:
serialization does not break as long as the anchor assembly does not change
can be used for obfuscation, e.g. some required assembly can be hidden in any file
can be used for online updates, e.g. ship the assembly with any data
as far as I'm concerned no problem with generics
Downsides:
security issues as bootstrapping could be used to inject code (some extra work for assembly SecurityEvidences)
AppDomain borders are some work to cross
blows up your serialized data (ok, for files - bad, for communication)
But hey, communication does not break unless you have different client versions and then you can do the bootstrapping on handshake.
Sorry, I can't provide the code as it's too complex and too specific. But I share some insights in these other answers of mine:
difference between DataContract attribute and Serializable attribute in .net
Need to hookup AssemblyResolve event when DisallowApplicationBaseProbing = true
Is it a hard requirement that you MUST be using the BinaryFormatter for your de/serialization?
If it's not a hard requirement for you to be using the BinaryFormatter to do your serialization, consider alternative serialization methods, such as JSON.Net or ProtoBuf.Net.
Either one of these will create platform- and version-independent serializations of your data.
Alternatively, you can perform binary serialization yourself (which is both faster and smaller than the BinaryFormatter, but generally more code-intensive as you have to be sure to write the serializer and deserializer essentially identically to each other.
If you must use BinaryFormatter, be sure to construct it with FormatterAssemblyStyle.Simple, to get around versioning problems. That tells it to not do the pedantic check of the assembly version.
I am serializing and deserializing an Object in C# for Windows 8 Apps.
I am serializing my Object before passing it to the next View, because passing an object throws out exceptions.
function OnNavigatedTo:
protected override void OnNavigatedTo(NavigationEventArgs e)
{
base.OnNavigatedTo(e);
string XMLString = e.Parameter.ToString();
var thisChannel = XmlDeserializeFromString(XMLString, typeof(Channel));
....}
Deserializing Function:
public static Channel XmlDeserializeFromString<Channel>(string objectData)
{
return (Channel)XmlDeserializeFromString(objectData, typeof(Channel));
}
public static object XmlDeserializeFromString(string objectData, Type type)
{
var serializer = new XmlSerializer(type);
object result;
using (TextReader reader = new StringReader(objectData))
{
result = serializer.Deserialize(reader);
}
return result;
}
I want to access the data in this Object, but something like: thisChannel.Name doesn't work. And I don't know how that I can continue working with this Object.
Start by dropping var in this line:
//var thisChannel = XmlDeserializeFromString(XMLString, typeof(Channel));
Channel thisChannel = XmlDeserializeFromString(XMLString, typeof(Channel));
and then you will at least get an error when the wrong object XmlDeserializeFromString() is selected.
And to be sure you use the right one:
Channel thisChannel = XmlDeserializeFromString<Channel>(XMLString);
Overloading should be used with care and generally not mixed with Type parameters.
XmlDeserializeFromString returns an object, which does not have a Name property. You need to either:
cast it to the type you want to use it as
use the generic method you added which does that:
var thisChannel = XmlDeserializeFromString<Channel>(XMLString);`
use dynamic to resolve the method name at runtime
use reflection to find the Name property at runtime
Yes JSON > XML, though you want to stick to XML, use TCD.Serialization, it offers serialization and deserialization XML and JSON to/from stream & string.
.
Don't do this.
Passing non-primitive types through a navigation parameter will cause your application to crash when it restores from Suspend.
Only ever pass a primitive type as a navigation parameter in a Windows 8 app.
See SuspensionManager Error when application has more than 1 page in windows 8 XAML/C# app
I created a Web-API and i would like to get all routes with parameters BeginAddress (string), EndAddress(string), BegineDate (Datetime). I created a new Class SearchRoute with these properties.
I can do a normal Getwith an id or a string but how to do a Get by giving an object? Is this possible?
Would it be possible to do a post/put with an object and than ask for a return?
using (HttpClient client = new HttpClient())
{
HttpResponseMessage response = await client.GetAsync(url + userid);
if (response.IsSuccessStatusCode)
{
string content = await response.Content.ReadAsStringAsync();
List<Route> list = await SerializeService.Deserialize<List<Route>>(content);
return list;
}
return null;
}
Web API Function
public List<Route> GetAllByCity(SearchRoute sr)
{
return RouteDAO.GetAllByCity(sr);
}
Update:
If i do this, the Post doesn't work but if i create a new controller it works.
[HttpPost]
// POST api/route
public void Post([FromBody]Route route)
{
RouteDAO.Create(route);
}
// POST api/route
[HttpPost]
public List<Route> Post([FromBody]SearchRoute sr)
{
return RouteDAO.GetAllByCity(sr);
}
I prefer sticking with GET even when using a complex object as a parameter. If you are concerned about the length of the URI then remember that:
Prefixing the property names for simple like complex objects is not necessary because the Web API object binding can auto resolve based on property names alone.
The maximum allowed URL length is 2083 characters which is more than sufficient in most cases.
If you we take your example
public class SearchRoute {
public string BeginAddress {get;set;}
public string EndAddress {get;set;}
public DateTime BeginDate {get;set;}
}
[HttpGet]
public List<Route> Get([FromUri]SearchRoute sr)
{
return RouteDAO.GetAllByCity(sr);
}
Uri when searching on
BeginAddress = "Some beginning";
EndAddress = "Some ending"
BeginDate = "2016-01-01T16:40:00"
Resulting query string:
?BeginAddress=Some beginning&EndAddress=Some ending&BeginDate=2016-01-01T16:40:00
Again, the properties will auto resolve even without the object prefix/qualifier and populate the object instance.
Add a domain info to the URL maybe another 50 or so characters
Add a controller name maybe another 30 or so characters
Add the query string = 82 characters
Note that I am not taking into account resolving the special characters like spaces to Url escaped character sequence
Total ≈ 162 characters give or take
Not bad considering that the maximum allowed URL length is 2083 characters, so you have used up only 7% of what is possible in this simple example.
This would probably be the preferred way of doing it because it conforms to the RESTful API standard where GET calls/verbs do not alter data and POST calls/verbs do.
You can pass an object by using a complex type in the URI. You need to help Web API by using the correctly formatted Query String. This would be an example:
?SearchRoute.BeginAddress=TheAddressValue&SearchRoute.EndAddress=TheAddressValue
However, if your Query String starts to become too big, you might be modeling the interaction incorrectly.
Then, in the server you should let Web API know that it should look in the URI for the values:
public List<Route> GetAllByCity([FromUri]SearchRoute sr)
{
return RouteDAO.GetAllByCity(sr);
}