How can I efficiently get full json string in JsonConverter.ReadJson() ?
I can do:
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
var json = JObject.Load(reader).ToString(Formatting.None);
However that seems very inefficient as I serialize and deserialize for no reason
Any better way ?
ReadJson() must fully parse the JSON being read so that the JSON is confirmed to be well-formed and the JsonReader is correctly positioned at the end of the current value upon exit. However, it is not necessary to load the entire JSON into an intermediate JObject hierarchy simply to re-convert it to a JSON string. Instead, you may be able to get better performance by using JRaw.Create():
var json = JRaw.Create(reader).ToString();
As can be seen in the reference source, this method streams directly from the incoming JsonReader to a StringWriter - without loading into an intermediate JToken hierarchy and re-serializing - by using JsonWriter.WriteToken(JsonReader):
public static JRaw Create(JsonReader reader)
{
using (StringWriter sw = new StringWriter(CultureInfo.InvariantCulture))
using (JsonTextWriter jsonWriter = new JsonTextWriter(sw))
{
jsonWriter.WriteToken(reader);
return new JRaw(sw.ToString());
}
}
The resulting JRaw simply encapsulates that string in its Value. (Of course, there is no guarantee that the resulting JSON represents an object, only that it represents well-formed JSON.)
Note that JsonTextReader will automatically recognize and parse dates and times in common formats as DateTime objects, and also parse floating point values as double. If you need the "most literal" JSON string you may want to suppress DateTime recognition and/or parse floating point values as decimal. The following extension method, modeled on JRaw.Create(), does the job:
public static string ReadOuterJson(this JsonReader reader, Formatting formatting = Formatting.None, DateParseHandling? dateParseHandling = null, FloatParseHandling? floatParseHandling = null)
{
// If you would prefer a null JSON value to return an empty string, remove this line:
if (reader.TokenType == JsonToken.Null)
return null;
var oldDateParseHandling = reader.DateParseHandling;
var oldFloatParseHandling = reader.FloatParseHandling;
try
{
if (dateParseHandling != null)
reader.DateParseHandling = dateParseHandling.Value;
if (floatParseHandling != null)
reader.FloatParseHandling = floatParseHandling.Value;
using (var sw = new StringWriter(CultureInfo.InvariantCulture))
using (var jsonWriter = new JsonTextWriter(sw) { Formatting = formatting })
{
jsonWriter.WriteToken(reader);
return sw.ToString();
}
}
finally
{
reader.DateParseHandling = oldDateParseHandling;
reader.FloatParseHandling = oldFloatParseHandling;
}
}
And then call it like, e.g.:
var json = reader.ReadOuterJson(dateParseHandling: DateParseHandling.None);
For details on why this may be necessary, see:
Json.NET interprets and modifies ISO dates when deserializing to JObject #862.
JObject.Parse modifies end of floating point values.
Related
Try to convert JObject to BsonDocument using this example https://www.newtonsoft.com/json/help/html/WriteJTokenToBson.htm (BsonWriter obsolete, so I use BsonDataWriter)
var jObject = JObject.Parse("{\"name\":\"value\"}");
using var writer = new BsonDataWriter(new MemoryStream());
jObject.WriteTo(writer);
var bsonData = writer.ToBsonDocument();
Console.WriteLine(bsonData.ToJson());
output:
{ "CloseOutput" : true, "AutoCompleteOnClose" : true, "Formatting" : 0, "DateFormatHandling" : 0, "DateTimeZoneHandling" : 3, "StringEscapeHandling" : 0, "FloatFormatHandling" : 0, "DateFormatString" : null
, "Culture" : { "Name" : "", "UseUserOverride" : false }, "DateTimeKindHandling" : 1 }
Expected output is:
{"name": "value"}
How can I fix it?
UPD: I have a JObject, and I want to convert it directly to BSONDocument, avoid serialization to string and string parsing
You can write a JObject to a BSON stream using Newtonsoft's BsonDataWriter like so:
var json = "{\"name\":\"value\"}";
var jObject = JObject.Parse(json);
using var stream = new MemoryStream(); // Or open a FileStream if you prefer
using (var writer = new BsonDataWriter(stream) { CloseOutput = false })
{
jObject.WriteTo(writer);
}
// Reset the stream position to 0 if you are going to immediately re-read it.
stream.Position = 0;
Then, you can parse the written stream with the MongoDB .NET Driver like so:
// Parse the BSON using the MongoDB driver
BsonDocument bsonData;
using (var reader = new BsonBinaryReader(stream))
{
var context = BsonDeserializationContext.CreateRoot(reader);
bsonData = BsonDocumentSerializer.Instance.Deserialize(context);
}
And check the validity of the created document like so:
// Verify that the BsonDocument is semantically identical to the original JSON.
// Write it to JSON using the MongoDB driver
var newJson = bsonData.ToJson();
Console.WriteLine(newJson); // Prints { "name" : "value" }
// And assert that the old and new JSON are semantically identical
Assert.IsTrue(JToken.DeepEquals(JToken.Parse(json), JToken.Parse(newJson))); // Passes
Notes:
When you do var bsonData = writer.ToBsonDocument(); you are actually serializing Newtonsoft's BsonDataWriter using the MongoDB extension method BsonExtensionMethods.ToBsonDocument(), which explains the strange contents of the bsonData document in your test code.
Instead, the serialized BSON can be obtained from the stream to which it was just written.
If you are going to immediately re-read the stream, you can keep it open by setting JsonWriter.CloseOutput = false. Set the stream position to 0 after writing.
While your approach avoids the overhead of serializing and parsing a JSON string, you are still serializing and deserializing a BSON binary stream.
Demo fiddle here.
If you have the json string already, you can simply create a bson document out of it
var document= BsonDocument.Parse("{\"name\":\"value\"}");
If you strictly have a JObject, you can Serialize the JObject and then parse the result to a BSONDocument
var document = BsonDocument.Parse(Newtonsoft.Json.JsonConvert.SerializeObject(jObject));
Accepted answer avoids serialization to a string but serializes to binary instead just to then deserialize it right after.
The following maps JObject do BsonDocument by straightforward traversal:
public static BsonDocument ToBsonDocument(this JObject o) =>
new(o.Properties().Select(p => new BsonElement(p.Name, p.Value.ToBsonValue())));
public static BsonValue ToBsonValue(this JToken t) =>
t switch
{
JObject o => o.ToBsonDocument(),
JArray a => new BsonArray(a.Select(ToBsonValue)),
JValue v => BsonValue.Create(v.Value),
_ => throw new NotSupportedException($"ToBsonValue: {t}")
};
Here is the code:
string s = "2012-08-08T01:54:45.3042880+00:00";
JObject j1 = JObject.FromObject(new
{
time=s
});
Object o = j1["time"];
We can check that o is string and equals "2012-08-08T01:54:45.3042880+00:00"
Now we transfer j1.ToString() to another program, which is
{
"time": "2012-08-08T01:54:45.3042880+00:00"
}
then at the other program, try to parse it back to JObject, which is
JObject j2 = JObject.Parse(j1.ToString());
Object o2 = j2["time"];
Now, if we check o2, o2's type is Date, o2.ToString() is 8/7/2012 9:54:45 PM.
My question is:
Is there is way to disable the Date deserialization for JObject.Parse , and just get the raw string?
Thanks in advance
When parsing from an object to JObject you can specify a JsonSerializer which instructs how to handle dates.
JObject.FromObject(new { time = s },
new JsonSerializer {
DateParseHandling = DateParseHandling.None
});
Unfortunately Parse doesn't have this option, although it would make sense to have it. Looking at the source for Parse we can see that all it does is instantiate a JsonReader and then passes that to Load. JsonReader does have parsing options.
You can achieve your desired result like this:
using(JsonReader reader = new JsonTextReader(new StringReader(j1.ToString()))) {
reader.DateParseHandling = DateParseHandling.None;
JObject o = JObject.Load(reader);
}
For background, see Json.NET interprets and modifies ISO dates when deserializing to JObject #862, specifically this comment from JamesNK: I have no plans to change it, and I would do it again if give the chance.
You can accomplish this using JsonConvert.DeserializeObject as well, by using JsonSerializerSettings:
string s = "2012-08-08T01:54:45.3042880+00:00";
string jsonStr = $#"{{""time"":""{s}""}}";
JObject j1 = JsonConvert.DeserializeObject<JObject>(jsonStr, new JsonSerializerSettings {DateParseHandling = DateParseHandling.None});
I've implemented a Json Serializer based on Json.net to accept any object type and serialize it (for placement into my cache)
The cache interface doesn't allow me to speficy the type, so When I retrieve from the cache I need to dynamically create the type from the meta information.
Which works well for objects, the problem I am now facing is that i doesn't work for value types, I will get an exception saying something along the lines of cannot cast JValue to JObject.
My question is how can I cater for value types as well as object types? It would be great if there was a TryParse for a JObject, which I could write myself, but feel like I am going down a rabbit hole?
What is the best way to achieve this?
My code is as follows, settings for Json.net:
_settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
NullValueHandling = NullValueHandling.Ignore,
DateTimeZoneHandling = DateTimeZoneHandling.Utc,
TypeNameHandling = TypeNameHandling.All
};
_settings.Converters.Add(new StringEnumConverter());
The set function (serialize):
public void Put(string cacheKey, object toBeCached, TimeSpan cacheDuration)
{
_cache.Set(cacheKey, JsonConvert.SerializeObject(toBeCached, _settings), cacheDuration);
}
And the get (deserialize):
public object Get(string cacheKey)
{
try
{
var value = _cache.Get(cacheKey);
if (!value.HasValue)
{
return null;
}
var jobject = JsonConvert.DeserializeObject<JObject>(value);
var typeName = jobject?["$type"].ToString();
if (typeName == null)
{
return null;
}
var type = Type.GetType(typeName);
return jobject.ToObject(type);
}
catch (Exception e)
{
// Todo
return null;
}
}
You need to parse to a JToken rather than a JObject, then check to see if the returned type is a JValue containing a JSON primitive:
public static object Get(string value)
{
var jToken = JsonConvert.DeserializeObject<JToken>(value);
if (jToken == null)
return null;
else if (jToken is JValue)
{
return ((JValue)jToken).Value;
}
else
{
if (jToken["$type"] == null)
return null;
// Use the same serializer settings as used during serialization.
// Ideally with a proper SerializationBinder that sanitizes incoming types as suggested
// in https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_TypeNameHandling.htm
var _settings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver(),
NullValueHandling = NullValueHandling.Ignore,
DateTimeZoneHandling = DateTimeZoneHandling.Utc,
TypeNameHandling = TypeNameHandling.All,
Converters = { new StringEnumConverter() },
//SerializationBinder = new SafeSerializationBinder(),
};
// Since the JSON contains a $type parameter and TypeNameHandling is enabled, if we deserialize
// to type object the $type information will be used to determine the actual type, using Json.NET's
// serialization binder: https://www.newtonsoft.com/json/help/html/SerializeSerializationBinder.htm
return jToken.ToObject(typeof(object), JsonSerializer.CreateDefault(_settings));
}
}
Note, however, that type information for primitives will not get precisely round-tripped:
Integers will be returned as a long (or a BigInteger if larger than long.MaxValue).
Floating point values will be returned as a double or a decimal, depending upon the setting of JsonSerializerSettings.FloatParseHandling -- either FloatParseHandling.Double or FloatParseHandling.Decimal.
JSON strings that "look like" dates will get converted to DateTime, DateTimeOffset, or left as strings, depending on the setting JsonSerializerSettings.DateParseHandling.
enum names will be converted to strings.
If you need to round-trip type information for primitives, consider using TypeWrapper<T> from Deserialize specific enum into system.enum in Json.Net to encapsulate your root objects.
Finally, if there is any chance you might be deserializing untrusted JSON (and if you are deserializing from a file or from the internet then you certainly are), please note the following caution from the Json.NET documentation:
TypeNameHandling should be used with caution when your application deserializes JSON from an external source. Incoming types should be validated with a custom SerializationBinder when deserializing with a value other than None.
For a discussion of why this may be necessary, see TypeNameHandling caution in Newtonsoft Json, How to configure Json.NET to create a vulnerable web API, and Alvaro Muñoz & Oleksandr Mirosh's blackhat paper https://www.blackhat.com/docs/us-17/thursday/us-17-Munoz-Friday-The-13th-JSON-Attacks-wp.pdf
StreamReader qryTmpltStream = new StreamReader(tmpltPath + "templates.json");
JsonTextReader qryTmpltReader = new JsonTextReader(qryTmpltStream);
JsonSerializer qryTmpltSrlzr = new JsonSerializer();
object jsonObject = qryTmpltSrlzr.Deserialize(qryTmpltReader);
var tplts = JsonConvert.DeserializeObject<JSONRepClass>(jsonObject);
In above code I'm trying to read in a json file then deserialize it into a class. The problem is, this: JsonConvert.DeserializeObject wants a string, but the Deserailize method call before it returns an object.
I tried casting to string and ToString(), but no go.
Anyone see what I'm missing here?
Try this, just read the json file contents into a string and deserialize it using Json.Net
var jSonString = File.ReadAllText(tmpltPath + "templates.json");
var tplts = JsonConvert.DeserializeObject<JSONRepClass>(jsonString);
This is the simplest way to use JSON.net to turn a json string into a strongly typed class.
YourClass myclass = new YourClass();
JsonConvert.PopulateObject(yourstring,myclass);
Here is the code:
string s = "2012-08-08T01:54:45.3042880+00:00";
JObject j1 = JObject.FromObject(new
{
time=s
});
Object o = j1["time"];
We can check that o is string and equals "2012-08-08T01:54:45.3042880+00:00"
Now we transfer j1.ToString() to another program, which is
{
"time": "2012-08-08T01:54:45.3042880+00:00"
}
then at the other program, try to parse it back to JObject, which is
JObject j2 = JObject.Parse(j1.ToString());
Object o2 = j2["time"];
Now, if we check o2, o2's type is Date, o2.ToString() is 8/7/2012 9:54:45 PM.
My question is:
Is there is way to disable the Date deserialization for JObject.Parse , and just get the raw string?
Thanks in advance
When parsing from an object to JObject you can specify a JsonSerializer which instructs how to handle dates.
JObject.FromObject(new { time = s },
new JsonSerializer {
DateParseHandling = DateParseHandling.None
});
Unfortunately Parse doesn't have this option, although it would make sense to have it. Looking at the source for Parse we can see that all it does is instantiate a JsonReader and then passes that to Load. JsonReader does have parsing options.
You can achieve your desired result like this:
using(JsonReader reader = new JsonTextReader(new StringReader(j1.ToString()))) {
reader.DateParseHandling = DateParseHandling.None;
JObject o = JObject.Load(reader);
}
For background, see Json.NET interprets and modifies ISO dates when deserializing to JObject #862, specifically this comment from JamesNK: I have no plans to change it, and I would do it again if give the chance.
You can accomplish this using JsonConvert.DeserializeObject as well, by using JsonSerializerSettings:
string s = "2012-08-08T01:54:45.3042880+00:00";
string jsonStr = $#"{{""time"":""{s}""}}";
JObject j1 = JsonConvert.DeserializeObject<JObject>(jsonStr, new JsonSerializerSettings {DateParseHandling = DateParseHandling.None});