I have the following test code:
[TestClass]
public class TestJsonDeserialize
{
public class MyClass
{
[JsonProperty("myint")]
public int MyInt { get; set; }
[JsonProperty("Mybool")]
public bool Mybool { get; set; }
}
[TestMethod]
public void Test1()
{
var errors = new List<string>();
var json1 = "{\"myint\":1554860000,\"Mybool\":false}";
var json2 = "{\"myint\":3554860000,\"Mybool\":false}";
var i = JsonConvert.DeserializeObject<MyClass>(json2, new JsonSerializerSettings
{
Error = delegate (object sender, Newtonsoft.Json.Serialization.ErrorEventArgs args)
{
Debug.WriteLine(args.ErrorContext.Error.Message);
errors.Add(args.ErrorContext.Error.Message);
args.ErrorContext.Handled = true;
}
});
Assert.IsTrue(errors.Count <= 1);
}
}
The call to JsonConvert.DeserializeObject produces 2 errors. One of them is expected, but the other not.
The errors are:
JSON integer 3554860000 is too large or small for an Int32. Path 'myint', line 1, position 19.
Unexpected token when deserializing object: Boolean. Path 'Mybool', line 1, position 34.
Why is there a 2nd error although the 1st error is marked as handled.
I already updated from Newtonsoft.Json 8.0.2 to 9.0.1 but it remains.
When passing the first string (json1 instead of json2), then no errors at all occur.
Update
Reported as Issue 1194: JsonTextReader.ParseNumber leads to error after ThrowReaderError and closed by Newtonsoft as not reproducible in then-current build which subsequently was released as Json.NET 10.0.1.
Original Answer
This may be a bug in JsonTextReader.
In JsonTextReader.ParseNumber(ReadType readType, char firstChar, int initialPosition) there is the following logic, somewhat simplified:
else if (readType == ReadType.ReadAsInt32)
{
// Snip
int value;
ParseResult parseResult = ConvertUtils.Int32TryParse(_stringReference.Chars, _stringReference.StartIndex, _stringReference.Length, out value);
if (parseResult == ParseResult.Success)
{
numberValue = value;
}
else if (parseResult == ParseResult.Overflow)
{
throw ThrowReaderError("JSON integer {0} is too large or small for an Int32.".FormatWith(CultureInfo.InvariantCulture, _stringReference.ToString()));
}
else
{
throw ThrowReaderError("Input string '{0}' is not a valid integer.".FormatWith(CultureInfo.InvariantCulture, _stringReference.ToString()));
}
}
numberType = JsonToken.Integer;
}
// Snip
// Finally, after successfully parsing the number
ClearRecentString();
// index has already been updated
SetToken(numberType, numberValue, false);
At the point the exception is thrown by ThrowReadError(), the stream position has been advanced past the too-large integer. However, the value for JsonReader.TokenType has not been updated and still returns JsonToken.PropertyName for the last token that was successfully parsed, namely the "myint" name. Later, after the exception is swallowed and ignored, the inconsistency between the stream position and current token value causes the "Mybool" property name to be skipped, leading to the second error.
If, in the debugger, when the exception is thrown I manually call
SetToken(JsonToken.Undefined);
ClearRecentString();
Then the remainder of the file can be successfully parsed. (I'm not sure JsonToken.Undefined is the right choice here.)
You might want to report an issue to Newtonsoft.
Since the JsonReader is not passed in to the error handler, the only workaround I could find is to subclass JsonTextReader as follows:
public class FixedJsonTextReader : JsonTextReader
{
public FixedJsonTextReader(TextReader reader) : base(reader) { }
public override int? ReadAsInt32()
{
try
{
return base.ReadAsInt32();
}
catch (JsonReaderException)
{
if (TokenType == JsonToken.PropertyName)
SetToken(JsonToken.None);
throw;
}
}
}
And then do:
var errors = new List<string>();
var json2 = "{\"myint\":3554860000,\"Mybool\":false}";
using (var reader = new FixedJsonTextReader(new StringReader(json2)))
{
var settings = new JsonSerializerSettings
{
Error = delegate(object sender, Newtonsoft.Json.Serialization.ErrorEventArgs args)
{
Debug.WriteLine(args.ErrorContext.Error.Message);
errors.Add(args.ErrorContext.Error.Message);
args.ErrorContext.Handled = true;
}
};
var i = JsonSerializer.CreateDefault(settings).Deserialize<MyClass>(reader);
}
Assert.IsTrue(errors.Count <= 1); // Passes
Related
What does this error mean on my code i see while debugging some output are negative which triggers this error while also in the other hand i saw a Dictionary with two parameters int and decimal but then to be stored to a list of int declare could this be also triggering the crash? what are the possible remedy for this crash error?
Picture of the Error Message Crash
function is declared on a class called PriceTierModel
private Dictionary<int, decimal> _unitPrices = new Dictionary<int, decimal>(); // key == TurnTime
public void AddPrice(int turnTimeValue, decimal price)
{
_unitPrices[turnTimeValue] = price;
}
public List<int> TurnTimes
{
get
{
List<int> turnTimes = _unitPrices.Keys.ToList();
turnTimes.Sort();
return turnTimes;
}
}
which is used here in this other class
public override string Error
{
get
{
PriceTierModel priceTier = GetPriceTier();
try
{
if (priceTier != null && _desiredTurnTime < Math.Abs(priceTier.TurnTimes[0]))
{
return String.Format(CadFramework.Rm.GetString("TurnTimeTooSoon"), priceTier.TurnTimes[0]);
}
}
catch (Exception ei) { return String.Format(CadFramework.Rm.GetString("TurnTimeTooSoon"), Math.Abs(priceTier.TurnTimes[0])); }
return "";
}
}
Added Code for Showing implementation use of the function.
foreach (XElement tier in priceTiers)
{
// possible that the tier element is invalid
//<price_tier>
// <turn_time></turn_time>
// <unit_price>N/A</unit_price>
// <turn_time_days>None days</turn_time_days>
//</price_tier>
int turnTime;
// ReSharper disable once PossibleNullReferenceException
if (int.TryParse(tier.Element("turn_time").Value, out turnTime))
{
decimal unitPrice = decimal.Parse(!string.IsNullOrEmpty(tier.Element("unit_price").Value) ? tier.Element("unit_price").Value : "0", CultureInfo.InvariantCulture);
priceTier.AddPrice(turnTime, unitPrice); <-- Here
}
}
This error means, that the line:
try
{
/* --> */ if (priceTier != null && _desiredTurnTime < Math.Abs(priceTier.TurnTimes[0]))
{
return String.Format(CadFramework.Rm.GetString("TurnTimeTooSoon"), priceTier.TurnTimes[0]);
}
or the line:
public void AddPrice(int turnTimeValue, decimal price)
{
/* --> */ _unitPrices[turnTimeValue] = price;
}
is entering or accessing (thanks to commentator) an item that is not existing in one of the both list.
From MSDN (IndexOutOfRangeException):
The exception that is thrown when an attempt is made to access an
element of an array or collection with an index that is outside its
bounds.
Using .NET Core, I am trying to save and retrieve a JSON Array of the object from Redis using IDistributedCache. Below is my code for storing and reading from Redis cache:
public void Save(string key, object content, int duration)
{
string s;
if (content is string)
{
s = (string)content;
}
else
{
s = JsonConvert.SerializeObject(content);
}
duration = duration <= 0 ? DefaultCacheDuration : duration;
Cache.Set(key, Encoding.UTF8.GetBytes(s), new DistributedCacheEntryOptions()
{
AbsoluteExpiration = DateTime.Now + TimeSpan.FromSeconds(duration)
});
}
public T Get<T>(string key) where T : class
{
var c = Cache.Get(key);
if (c == null)
{
return null;
}
var str = Encoding.UTF8.GetString(c);
if (typeof(T) == typeof(string))
{
return str as T;
}
return JsonConvert.DeserializeObject<T>(str);
}
the object that I want to store is
public class RuleLoadCollection_Result
{
[Key]
public int RuleId { get; set; }
public string RuleName { get; set; }
}
In my biz logic, am saving the object like this
public IQueryable<RuleLoadCollection_Result> GetRuleLibrary()
{
var result = _dbClient.GetRuleLibrary();
_cache.Save("TestKey", result);
return result;
}
the output here is an Array of Object.
[{"ruleId":1,"ruleName":"a1"}]
What code should I write to return the same array of objects from cache? I tried a few options, most of them gave compile or runtime errors. After the bit of browsing, I tried below, it worked, but it is giving only the first element of the array.
public RuleLoadCollection_Result GetRuleLibraryFromCache()
{
return (_cache.Get<List<RuleLoadCollection_Result>>("TestKey").First());
}
output for this is
{"ruleId":1,"ruleName":"a1"}
which I understand why, but what c# should I write to JSON array back which I saved?
below code gives the runtime error
public IQueryable<RuleLoadCollection_Result> GetRuleLibraryFromCache()
{
return (_cache.Get<IQueryable<RuleLoadCollection_Result>>("TestKey"));
}
the runtime error is:
Cannot create and populate list type System.Linq.IQueryable`1[RuleLoadCollection_Result]. Path '', line 1, position 1.
This worked.
public IQueryable<RuleLoadCollection_Result> GetRuleLibraryFromCache()
{
var result = _cache.Get<IEnumerable<RuleLoadCollection_Result>>("TestKey").AsQueryable();
return result;
}
I have a couple validators that is validating an IDeliveryObject, which conceptually can be described as a file with several rows. That part is working fine.
IEnumerable<IDeliveryValidator> _validators; // Populated in ctor. Usually around 20 different validators.
private IEnumerable<IValidationResult> Validate(IDeliveryObject deliveryObject)
{
var validationErrors = new List<IValidationResult>();
int maxNumberOfErrors = 10;
foreach (IDeliveryValidator deliveryValidator in _validators)
{
IEnumerable<IValidationResult> results = deliveryValidator.Validate(deliveryObject).Take(maxNumberOfErrors);
validationErrors.AddRange(results);
if (validationErrors.Count >= maxNumberOfErrors )
{
return validationErrors.Take(maxNumberOfErrors).ToList();
}
}
return validationErrors;
}
The logic iterates through a couple of validators, which all validates the file for different things.
And a validator can look something like this:
public IEnumerable<IValidationResult> Validate(IDeliveryObject deliveryObject)
{
using (var reader = File.OpenText(deliveryObject.FilePath))
{
int expectedLength = 10; // Or some other value.
string line;
while ((line = reader.ReadLine()) != null)
{
var lineLength = line.Length;
if (lineLength != expectedLength)
{
// yield an error for each incorrect row.
yield return new DeliveryValidationResult("Wrong length...");
}
}
}
}
The ValidationResult looks like this:
public class DeliveryValidationResult : ValidationResult, IValidationResult
{
public DeliveryValidationResult(bool isSoftError, string errorMessage) : base(errorMessage)
{
IsSoftError = isSoftError;
}
public DeliveryValidationResult(string errorMessage) : base(errorMessage)
{
}
public DeliveryValidationResult(string errorMessage, IEnumerable<string> memberNames) : base(errorMessage, memberNames)
{
}
public DeliveryValidationResult(ValidationResult validationResult) : base(validationResult)
{
}
public bool IsSoftError { get; set; }
}
public interface IValidationResult
{
string ErrorMessage { get; set; }
bool IsSoftError { get; set; }
}
Thanks to Take(maxNumberOfErrors) and yield each validator will only return 10 validationresults, which used to be fine. But now I need to handle "soft validation result", which is the same kind of validation result, but it should not be included in the number of results yielded. It's a kind of warning, which is defined by setting IsSoftError in IValidationResult. A validator can yield both "soft validation result" and "regular validation result".
What I want is to take x validation results + unlimited soft validation results, so that all IValidationResults with IsSoftError == true will be included in the collection, but not in the count. I know that it sounds weird, but the concept is that there's no need to keep validating the file after x errors, but the validation can return unlimited "warnings".
It's very important that the Enumeration isn't enumerated more than one time, because it's CPU-heavy. Below is the code I want to change.
private IEnumerable<IValidationResult> Validate(IDeliveryObject deliveryObject)
{
var validationErrors = new List<IValidationResult>();
int maxNumberOfErrors = 10;
foreach (IDeliveryValidator deliveryValidator in _validators)
{
// Here I want results to contain MAX 10 regular validation results, but unlimited soft validation results
IEnumerable<IValidationResult> results = deliveryValidator.Validate(deliveryObject).Take(maxNumberOfErrors);
validationErrors.AddRange(results);
if (validationErrors.Count(x => !x.IsSoftError) >= maxNumberOfErrors)
{
return validationErrors.Take(maxNumberOfErrors).ToList();
}
}
return validationErrors;
}
EDIT:
When I got 10 'hard' errors I want to stop the cycle completely. The main issue here is that the cycle doesn't stop when 10 'soft' errors occured.
In case you want to completely stop after 10 'hard' errors, you could try this:
int count = 0;
IEnumerable<IValidationResult> results = deliveryValidator.Validate(deliveryObject)
.TakeWhile(error => error.IsSoftError || count++ < maxNumberOfErrors);
this would stop when the 11th hard error is encountered.
//Go through all the items and sort them into Soft and NotSoft
//But ultimately these are in memory constructs...so this is fast.
var foo = Validate(delivery).ToLookup(x => x.IsSoftError);
var soft = foo[true];
var hard = foo[false].Take(10);
var result = Enumerable.Concat(soft, hard);
This problem is affecting my ASP.Net WebApi Patch method which looks a lot like this:
public MyModel Patch(int id, [FromBody]Delta<MyModel> newRecord){/*stuff here*/}
But it's not WebApi that's the problem - the failure is between Json.Net and OData.Delta.
The problem is JsonConvert.DeserializeObject does not see integers of OData.Delta objects and I'm wondering if there's a workaround or fix I can apply.
UPDATE: Have written code (see right down below) in the Json.Net library that will fix this. Just need it to be included in the next update (if James Newton-King allows it)
UPDATE 2: After further testing, I've decided the best course of action is to stop using OData.Delta and write my own (see answer)
Unit tests to prove the problem exists (using statements moved below for clarity)
Test 1: Fails with an int (Int32):
class TestObjWithInt
{
public int Int { get; set; }
}
[TestMethod]
public void IsApplied_When_IntIsDeserializedToDelta()
{
string testData = "{\"Int\":1}";
var deserializedDelta = JsonConvert.DeserializeObject<Delta<TestObjWithInt>>(testData);
var result = deserializedDelta.GetChangedPropertyNames().Contains("Int");
Assert.IsTrue(result);
}
Test 2: Succeeds with a long (Int64)
class TestObjWithLong
{
public long Long { get; set; }
}
[TestMethod]
public void IsApplied_When_LongIsDeserializedToDelta()
{
string testData = "{\"Long\":1}";
var deserializedDelta = JsonConvert.DeserializeObject<Delta<TestObjWithLong>>(testData);
var result = deserializedDelta.GetChangedPropertyNames().Contains("Long");
Assert.IsTrue(result);
}
And just to be sure that deserialization works to begin with, these two tests both pass.
[TestMethod]
public void IsApplied_When_LongIsDeserializedToTestObject()
{
string testData = "{\"Long\":1}";
var deserializedObject = JsonConvert.DeserializeObject<TestObjWithLong>(testData);
var result = deserializedObject.Long == 1;
Assert.IsTrue(result);
}
[TestMethod]
public void IsApplied_When_IntIsDeserializedToTestObject()
{
string testData = "{\"Int\":1}";
var deserializedObject = JsonConvert.DeserializeObject<TestObjWithInt>(testData);
var result = deserializedObject.Int == 1;
Assert.IsTrue(result);
}
I found this OData bug report which sounds like a similar issue but its old and closed so probably not.
Any help would be great.
Using statements (from the top of the test file):
using System;
using System.Linq;
using System.Web.Http.OData;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Newtonsoft.Json;
Solution if accepted by James Newton-King - change to release 6.0.6.
Replace JsonSerializerInternalReader.cs line 1581:
contract.TrySetMember(newObject, memberName, value);
with:
bool done = false;
while (!(done = done || contract.TrySetMember(newObject, memberName, value)))
{
switch (reader.TokenType)
{
case JsonToken.Integer:
if (value is long && ((long)value) <= Int32.MaxValue && ((long)value) >= Int32.MinValue)
value = Convert.ToInt32(value);
//Add else if (...) to cast to other data types here (none additional required to date).
else
done = true;
break;
default:
done = true;
break;
}
}
OData.Delta<T> does not work with Json.Net for any number Types other than Int64. The easiest approach is to write a replacement for OData.Delta<T> (which I've done on company time so I can't post it in its entirety sorry) containing methods like this:
private bool TrySetInt32(object value, PropertyInfo propertyInfo, bool isNullable)
{
var done = false;
if (value is Int32)
{
propertyInfo.SetValue(_obj, value);
done = true;
}
else if (value == null)
{
if (isNullable)
{
propertyInfo.SetValue(_obj, value);
done = true;
}
}
else if (value is Int64) //Json.Net - fallback for numbers is an Int64
{
var val = (Int64)value;
if (val <= Int32.MaxValue && val >= Int32.MinValue)
{
done = true;
propertyInfo.SetValue(_obj, Convert.ToInt32(val));
}
}
else
{
Int32 val;
done = Int32.TryParse(value.ToString(), out val);
if (done)
propertyInfo.SetValue(_obj, val);
}
return done;
}
The class can be a dynamic generic like this:
public sealed class Patchable<T> : DynamicObject where T : class, new()
With a working variable like this:
T _obj = new T();
In the overridden TrySetMember method, we need to check the underlying type of the property using reflection and call the appropriate TrySet... method like this:
if (underlyingType == typeof(Int16))
done = TrySetInt16(value, propertyInfo, isNullable);
else if (underlyingType == typeof(Int32))
done = TrySetInt32(value, propertyInfo, isNullable);
If the value is set successfully we can add the property name to a list that we can then use for patching the original record like this:
if (done)
_changedPropertyNames.Add(propertyInfo.Name);
public void Patch(T objectToPatch)
{
foreach (var propertyName in _changedPropertyNames)
{
var propertyInfo = _obj.GetType().GetProperty(propertyName);
propertyInfo.SetValue(objectToPatch, propertyInfo.GetValue(_obj));
}
}
68 unit tests later, it all seems to work pretty well. Here's an example:
class TestObjWithInt32
{
public Int32 Int32 { get; set; }
public Int32? SetNullable { get; set; }
public Int32? UnsetNullable { get; set; }
}
[TestMethod]
public void IsApplied_When_Int32IsDeserializedToPatchable()
{
string testData = "{\"Int32\":1,\"SetNullable\":1}";
var deserializedPatchable = JsonConvert.DeserializeObject<Patchable<TestObjWithInt32>>(testData);
var result = deserializedPatchable.ChangedPropertyNames.Contains("Int32");
Assert.IsTrue(result);
var patchedObject = new TestObjWithInt32();
Assert.AreEqual<Int32>(0, patchedObject.Int32);
deserializedPatchable.Patch(patchedObject);
Assert.AreEqual<Int32>(1, patchedObject.Int32);
Assert.IsNull(patchedObject.UnsetNullable);
Assert.IsNotNull(patchedObject.SetNullable);
}
This is my implementation for this issue based on Rob solution:
public sealed class Patchable<T> : DynamicObject where T : class {
private readonly IDictionary<PropertyInfo, object> changedProperties = new Dictionary<PropertyInfo, object>();
public override bool TrySetMember(SetMemberBinder binder, object value) {
var pro = typeof (T).GetProperty(binder.Name);
if (pro != null)
changedProperties.Add(pro, value);
return base.TrySetMember(binder, value);
}
public void Patch(T delta) {
foreach (var t in changedProperties)
t.Key.SetValue(
delta,
t.Key.PropertyType.IsEnum ? Enum.Parse(t.Key.PropertyType, t.Value.ToString()) : Convert.ChangeType(t.Value, t.Key.PropertyType));
}
}
I removed the requisite of an empty constructor in generic type parameter using the dictionary instead of a temporal object.
Thanks Rob ;)
I'm maintaining a legacy WebForms application and one of the pages just serves GET requests and works with many query string parameters. This work is done in the code-behind and does a lot of this type of check and casting.
protected override void OnLoad(EventArgs e)
{
string error = string.Empty;
string stringParam = Request.Params["stringParam"];
if (!String.IsNullOrEmpty(stringParam))
{
error = "No parameter";
goto LoadError;
}
Guid? someId = null;
try
{
someId = new Guid(Request.Params["guidParam"]);
}
catch (Exception){}
if (!someId.HasValue)
{
error = "No valid id";
goto LoadError;
}
// parameter checks continue on
LoadError:
log.ErrorFormat("Error loading page: {0}", error);
// display error page
}
I'd like to create a testable class that encapsulates this parsing and validation and moves it out of the code-behind. Can anyone recommend some approaches to this and/or examples?
As a first big step, I'd probably create some form of mapper/translator object, like this:
class SpecificPageRequestMapper
{
public SpecificPageRequest Map(NameValueCollection parameters)
{
var request = new SpecificPageRequest();
string stringParam = parameters["stringParam"];
if (String.IsNullOrEmpty(stringParam))
{
throw new SpecificPageRequestMappingException("No parameter");
}
request.StringParam = stringParam;
// more parameters
...
return request;
}
}
class SpecificPageRequest
{
public string StringParam { get; set; }
// more parameters...
}
Then your OnLoad could look like this:
protected override void OnLoad(EventArgs e)
{
try
{
var requestObject = requestMapper.Map(Request.Params);
stringParam = requestObject.StringParam;
// so on, so forth. Unpack them to the class variables first.
// Eventually, just use the request object everywhere, maybe.
}
catch(SpecificPageRequestMappingException ex)
{
log.ErrorFormat("Error loading page: {0}", ex.Message);
// display error page
}
}
I've omitted the code for the specific exception I created, and assumed you instantiate a mapper somewhere in the page behind.
Testing this new object should be trivial; you set the parameter on the collection passed into Map, then assert that the correct parameter on the request object has the value you expect. You can even test the log messages by checking that it throws exceptions in the right cases.
Assuming that you may have many such pages using such parameter parsing, first create a simple static class having extension methods on NamedValueCollection. For example,
static class Parser
{
public static int? ParseInt(this NamedValueCollection params, string name)
{
var textVal = params[name];
int result = 0;
if (string.IsNullOrEmpty(textVal) || !int.TryParse(textVal, out result))
{
return null;
}
return result;
}
public static bool TryParseInt(this NamedValueCollection params, string name, out int result)
{
result = 0;
var textVal = params[name];
if (string.IsNullOrEmpty(textVal))
return false;
return int.TryParse(textVal, out result);
}
// ...
}
Use it as follows
int someId = -1;
if (!Request.Params.TryParseInt("SomeId", out someId))
{
// error
}
Next step would be writing page specific parser class. For example,
public class MyPageParser
{
public int? SomeId { get; private set; }
/// ...
public IEnumerable<string> Parse(NamedValueCollection params)
{
var errors = new List<string>();
int someId = -1;
if (!params.TryParseInt("SomeId", out someId))
{
errors.Add("Some id not present");
this.SomeId = null;
}
this.SomeId = someId;
// ...
}
}