We Keep Coding

Newtonsoft Json converter for json with filters

I am writing converter for json like this:
{
"datatable": {
"data": [
[
"A85002072C",
"1994-11-15",
678.9
]
],
"columns": [
{
"name": "series_id",
"type": "String"
},
{
"name": "date",
"type": "Date"
},
{
"name": "value",
"type": "double"
}
]
},
"meta": {
"next_cursor_id": null
}
}
At the moment my converter looks like this:
public class AbsToModelConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType.Name.Equals("AbsFseModel");
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JArray array = JArray.Load(reader);
return new QuandlAbsModel
{
SeriesId = array[0].ToString(),
Date = array[1].ToObject<DateTime>(),
Value = array[2].ToObject<decimal?>()
};
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var orderItem = value as QuandlAbsModel;
JArray arra = new JArray();
arra.Add(orderItem.SeriesId);
arra.Add(orderItem.Date);
arra.Add(orderItem.Value);
arra.WriteTo(writer);
}
}
It works at the moment, but when i am using filters my json can contain not full data, for example:
"data":[["1994-11-15",678.9]]
And my JsonConverter stops working, because there is no element array[2] and it throws error. Problem is that elements in data array don't have names (i get JSON from web API, so i can't change json at all). Is there any way to make my converter deserialize json with filters?
I have column names in my json after the data table, maybe this will help. But i don't understand how i can use them atm. Any advices?

You don't need a JsonConverter for this.
Define classes to represent the parts of the JSON you need:
class APIResponse
{
public DataTable DataTable { get; set; }
}
class DataTable
{
public object[][] Data { get; set; }
}
Use JsonConvert.DeserializeObject<T>() to deserialize the JSON:
var parsed = JsonConvert.DeserializeObject<APIResponse>(json);
Then get your values:
var rows = parsed.DataTable.Data.Select(r => new QuandLabsModel
{
SeriesId = Convert.ToString(r[0]),
Date = Convert.ToDateTime(r[1]),
Value = Convert.ToDecimal(r[2])
});

JLRishe is correct that your problem is solvable without a custom converter. That's a good approach in many cases. If you're able to insert a translation over the JSON serializer/deserializer, it might be simpler to write, understand, and maintain than a custom JsonConverter. It's similar in spirit to the "serialization proxy pattern" used in the Java world. In essence, you're copying your data to a new serialization-specific object before serializing, and then doing the reverse to re-serialize.
This problem is solvable with a custom converter, and I've written an example to show that it can be done, but do consider using a translation proxy/layer first.
This example is a proof-of-concept; not production-ready code. I made very little effort to defend against malformed input or other errors. Its handling of the different fields/types is also very rudimentary--any changes to the fields/types will require changes to the converter. That sort of brittleness is likely to cause bugs and maintenance headaches over time.
To narrow down the problem a bit, I reduced the original question's sample JSON to its bare minimum:
{
"datatable": {
"data": [
"A85002072C",
"1994-11-15",
678.9
],
"columns": [
{
"name": "series_id"
},
{
"name": "date"
},
{
"name": "value"
}
]
}
}
For reference, here's the C# class definition I'm deserializing to:
public class Model
{
public string SeriesId { get; set; }
public DateTime Date { get; set; }
public Decimal? Value { get; set; }
}
And here's the proof-of-concept converter:
public sealed class ModelConverter : JsonConverter
{
public static readonly ModelConverter Instance = new ModelConverter();
private ModelConverter() {}
public override bool CanConvert(Type objectType) => objectType == typeof(Model);
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var obj = JObject.Load(reader);
var data = (JArray)obj["datatable"]["data"];
var columns = (JArray)obj["datatable"]["columns"];
if (data.Count != columns.Count)
throw new InvalidOperationException("data and columns must contain same number of elements");
var model = new Model();
for (int i = 0; i < data.Count; i++)
{
// A "switch" works well enough so long as the number of fields is finite and small.
// There are smarter approaches, but I've kept the implementation basic
// in order to focus on the core problem that was presented.
switch (columns[i]["name"].ToString())
{
case "series_id":
model.SeriesId = data[i].ToString();
break;
case "date":
model.Date = data[i].ToObject<DateTime>();
break;
case "value":
model.Value = data[i].ToObject<decimal?>();
break;
}
}
return model;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var data = new JArray();
var columns = new JArray();
var model = (Model)value;
// Like the "switch" used in deserialization, these "if" blocks are
// pretty rudimentary. There are better ways, but I wanted to keep
// this proof-of-concept implementation simple.
if (model.SeriesId != default(string))
{
data.Add(model.SeriesId);
columns.Add(new JObject(new JProperty("name", "series_id")));
}
if (model.Date != default(DateTime))
{
data.Add(model.Date.ToString("yyyy-MM-dd"));
columns.Add(new JObject(new JProperty("name", "date")));
}
if (model.Value != default(Decimal?))
{
data.Add(model.Value);
columns.Add(new JObject(new JProperty("name", "value")));
}
var completeObj = new JObject();
completeObj["datatable"] = new JObject();
completeObj["datatable"]["data"] = data;
completeObj["datatable"]["columns"] = columns;
completeObj.WriteTo(writer);
}
}
I wrote a few unit tests to verify the serializer. The tests are based on xUnit.Net:
[Fact]
public void TestDeserializeSampleInputWithAllFields()
{
var json = File.ReadAllText(BasePath + "sampleinput.json");
var obj = JsonConvert.DeserializeObject<Model>(json, ModelConverter.Instance);
Assert.Equal("A85002072C", obj.SeriesId);
Assert.Equal(new DateTime(1994, 11, 15), obj.Date);
Assert.Equal(678.9M, obj.Value);
}
[Fact]
public void TestSerializeSampleInputWithAllFields()
{
var model = new Model
{
SeriesId = "A85002072C",
Date = new DateTime(1994, 11, 15),
Value = 678.9M,
};
var expectedJson = File.ReadAllText(BasePath + "sampleinput.json");
Assert.Equal(expectedJson, JsonConvert.SerializeObject(model, Formatting.Indented, ModelConverter.Instance));
}
And to prove that the serializer works without all fields present:
{
"datatable": {
"data": [
"B72008039G",
543.2
],
"columns": [
{
"name": "series_id"
},
{
"name": "value"
}
]
}
}
[Fact]
public void TestDeserializeSampleInputWithNoDate()
{
var json = File.ReadAllText(BasePath + "sampleinput_NoDate.json");
var obj = JsonConvert.DeserializeObject<Model>(json, ModelConverter.Instance);
Assert.Equal("B72008039G", obj.SeriesId);
Assert.Equal(default(DateTime), obj.Date);
Assert.Equal(543.2M, obj.Value);
}
[Fact]
public void TestSerializeSampleInputWithNoDate()
{
var model = new Model
{
SeriesId = "B72008039G",
Value = 543.2M,
};
var expectedJson = File.ReadAllText(BasePath + "sampleinput_NoDate.json");
Assert.Equal(expectedJson, JsonConvert.SerializeObject(model, Formatting.Indented, ModelConverter.Instance));
}

Json combine two files for a large value

Here are two json samples.
I want to combine this json into a file.
If a key exists in a value that is combined without thinking, it is difficult to replace only the value that is high in value.
First Sample.
{
"employees": [
{
"firstName": "Tom",
"HighValue": "3"
},
{
"firstName": "Maria",
"HighValue": "4"
},
{
"firstName": "Robert",
"HighValue": "45"
}
]
}
Second Sample.
{
"employees": [
{
"firstName": "Tom",
"HighValue": "6"
},
{
"firstName": "Maria",
"HighValue": "4"
},
{
"firstName": "Robert",
"HighValue": "45"
},
{
"firstName": "John",
"HighValue": "1"
}
]
}
I want Result:
{
"employees": [
{
"firstName": "Tom",
"HighValue": "6"
},
{
"firstName": "Maria",
"HighValue": "4"
},
{
"firstName": "Robert",
"HighValue": "45"
},
{
"firstName": "John",
"HighValue": "1"
}
]
}
The goal is to combine two samples, Json, into one result json. What's the best way?

One simple approach would be by making use of a json framework like Json.NET which handles serialization/deserialization for you.
First create a data model into which your json data can be deserialized to. I used the online tool json2csharp for this. This will give you the following model:
public class Employee
{
public string firstName { get; set; }
public int HighValue { get; set; }
}
public class RootObject
{
public List<Employee> employees { get; set; }
}
Now you can simply deserialize your json strings into objects like this:
string json1 = "";
string json2 = "";
var obj1 = JsonConvert.DeserializeObject<RootObject>(json1);
var obj2 = JsonConvert.DeserializeObject<RootObject>(json2);
After this step you just have to iterate over your employees, check if they exist in both lists and add/update them accordingly:
foreach(var emp in obj2.employees)
{
Employee existing = null;
try
{
existing = obj1.employees.SingleOrDefault(e => e.firstName == emp.firstName);
}
catch(Exception ex)
{
// The same employee exists multiple times in the first list
}
if(existing != null)
{
// obj1 already contains an employee with the given name, check which value is bigger
if(existing.HighValue < emp.HighValue)
{
// The value of the existing employee is smaller
// -> Update the value with the value from the second object
existing.HighValue = emp.HighValue;
}
}
else
{
// obj1 does not already contain an employee with the given name, add the whole employee
obj1.employees.Add(emp);
}
}
Now obj1 contains the combined list of employees.
To serialize the combined list back to json do the following:
var json = JsonConvert.SerializeObject(obj1);

(For the record JSON isn't one of my strongest points but I thought answering this for you would be a good challenge)
The following code takes both JSON samples as JObjects using Newtonsoft.Json, then merges the two samples together. I then use LINQ to group and select the people with the highest values and output a new array.
The advantage to this solution is if you only want to use C# to merge, not use it for any other logic, you don't need to deserialize to an object.
Give it a try:
var firstSample = JObject.Parse("Your first sample goes here");
var secondSample = JObject.Parse("Your second sample goes here");
var newJsonArray = new JArray(); //Makes new array for output
firstSample.Merge(secondSample); //Merges the firstSample JObject with secondSample JObject
var filteredJson = firstSample["employees"] //Makes new JObject with filtered values
.GroupBy(x => x["firstName"]) // Groups by the first name
.Select(x => x.Where(y => y["HighValue"] == x.Max(z => z["HighValue"]))) // Select the employee in each group with the highest value
.ToList(); // Converts to a list out
foreach (var token in filteredJson)
{
newJsonArray.Add(token); // For each selected employee, add it to the new array
}
var outputJson = new JObject(new JProperty("employees", newJsonArray)); // Composes the new JSon string

Change status value from int to string

I have a status property with int values in array of objects. Below is the output of array of objects.
[
{
"enterpriseServiceId": 1,
"enterpriseServices": {},
"status": 3,
"id": 1,
"createdOn": "2017-12-29T17:58:15.4855946",
"createdBy": "System",
"modifiedOn": "2017-12-29T17:58:15.4855946",
"modifiedBy": "System"
},
{
"enterpriseServiceId": 2,
"enterpriseServices": {},
"status": 1,
"id": 2,
"createdOn": "2017-12-29T17:58:15.4855946",
"createdBy": "System",
"modifiedOn": "2017-12-29T17:58:15.4855946",
"modifiedBy": "System"
}
]
The status property is enum value type. I am trying to convert that to string value so it is easier to read when it is returned in output.
Below is my code that gets the data from sql server database.
ENUM
namespace CoreLibrary.Models.Enums
{
public static class Common
{
public enum Status
{
Active = 1,
InActive,
Completed,
Failed,
InProgress,
Pause,
Resume,
Skip,
Running
}
}
}
Entity Model:
namespace CoreLibrary.Models.Entities
{
public class Deployments : BaseProperties
{
public int EnterpriseServiceId { get; set; }
[ForeignKey("EnterpriseServiceId")]
public virtual EnterpriseServices EnterpriseServices { get; set; }
public Common.Status Status { get; set; }
}
}
Method that retrieves data from database:
[HttpGet("~/api/Deployments/GetWithJoins")]
public JsonResult GetWithJoins()
{
try
{
// includeList for including data from external tables (JOIN Query)
var includeList = new List<Expression<Func<Deployments, object>>>();
includeList.Add(d => d.EnterpriseServices);
IEnumerable<Deployments> result = _repository.GetByIdWithJoins(queryable: includeList).ToList();
return Json(result);
}
catch (Exception ex)
{
var innerException = ex.InnerException == null ? null : ex.InnerException.Message.ToString();
Response.StatusCode = (int)HttpStatusCode.BadRequest;
return Json(new { status = "failed", message = ex.Message.ToString(), innerException = innerException });
}
}
The result variable returns the output I shared at very beginning of this post. I am trying to convert status to return string value instead of int.
I am not sure if thats even possible or it is the right way to output of status int to string value?
I tried online, but I havent found a solution to my requirement. I would appreciate any help you guys can give. :)

When we call JSON(someObject), ASP.NET uses its JavaScriptSerializer to create a JsonResult. The default configuration of the JavaScriptSerializer converts enums to their integer representation.
We can configure the serializer to use a string representation of enums instead. There are details on how to do that here: JSON serialization of enum as string
If that approach does not appeal, then you can use LINQ to map to an anonymous object using Enum.ToString(). Here is an example:
var result = deployments.Select(x =>
{
return new
{
Status = x.Status.ToString(),
x.EnterpriseServiceId
};
});
While the LINQ approach will work, it might lead to maintenance problems in the long-term.

How to update a nested document in a mongodb with c#

I hope someone can help me to understand how I can update a document in a mongodb.
My problem is that i have a document which contains an array. And in this array there are objects with a specific ID(like you would find in sql database). Now I want to update the data inside those objects if they have the searched id.
A document looks like this
{
"_id": "63dafa72f21d48312d8ca405",
"tasks": [{
"_ref": "63d8d8d01beb0b606314e322",
"data": {
"values": [{
"key": "Deadline",
"value": "2014-10-13"
}]
}
}, {
"_ref": "84dd046c6695e32322d842f5",
"data": {
"values": []
}
}]
}
I did write the method updateProject
public bool updateProject(Project pro, Project dbPro)
{
var collection = db.GetCollection<BsonDocument>("projects");
var filter = Builders<BsonDocument>.Filter.Eq("_id", dbPro.Id);
var update = Builders<BsonDocument>.Update.Set("tasks", pro.Tasks);
var result = collection.UpdateOne(filter, update);
if (result.IsModifiedCountAvailable)
{
if (result.ModifiedCount == 1)
{
return true;
}
}
return false;
}
And thats how a project does look like in c#.
class Project{
public string id;
public List<Task> Tasks;
}
class Task{
public string id;
public List<Value> Values;
}
class Value{
public String key;
public String value;
}
but i cant figure out how i can go deeper to find the searched id.

C# DataContractJsonSerializer fails when value can be an array or a single item

I use the DataContractJsonSerializer to parse a json string into a object hierarchie.
The json string looks like this:
{
"groups": [
{
"attributes": [
{
"sortOrder": "1",
"value": "A"
},
{
"sortOrder": "2",
"value": "B"
}
]
},
{
"attributes": {
"sortOrder": "1",
"value": "C"
}
}
]
}
As you can see the sub value of "attributes" can be an array or a single item.
I found the code part where the problem occures:
[DataContract]
public class ItemGroup
{
[DataMember(Name="attributes")]
public List<DetailItem> Items { get; set; }
}
This works for the first one but fails on the second one.
Has anyone an answer for this?
Thx

If you control how the JSON is created then make sure that attributes is an array even if it only contains one element. Then the second element will look like this and parse fine.
{
"attributes": [{
"sortOrder": "1",
"value": "C"
}]
}

As Daniel said, if you can control the creation of Json, it is better to continue that way.
But if you can't, then you can use Json.Net library & the JsonObject class from
this link to write some code like:
JObject o = (JObject)JsonConvert.DeserializeObject(input);
dynamic json = new JsonObject(o);
foreach (var x in json.groups)
{
var attrs = x.attributes;
if (attrs is JArray)
{
foreach (var y in attrs)
{
Console.WriteLine(y.value);
}
}
else
{
Console.WriteLine(attrs.value);
}
}

I tried to get this working with DataContractJsonSerializer, JavaScriptSerializer, and JSON.Net and none would deserialize directly to an object successfully in all cases. I used a similar approach as L.B, using JSON.Net, although without the use of dynamics and the extra JsonObject class. Adapting my code to your scenario would look something like the following:
private List<ItemGroup> ParseItemGroupList(string input)
{
JObject json = JObject.Parse(input);
List<ItemGroup> groups = new List<ItemGroup>();
JArray gArray = json["groups"] as JArray;
foreach (var gToken in gArray)
{
ItemGroup newGroup = new ItemGroup();
JToken attrToken = gToken["attributes"] as JToken;
if (attrToken is JArray)
{
newGroup.Items = attrToken.Children().Select(MapDetailItem()).ToList();
}
else
{
newGroup.Items = new List<DetailItem>() { MapDetailItem().Invoke(attrToken) };
}
groups.Add(newGroup);
}
return groups;
}
private static Func<JToken, DetailItem> MapDetailItem()
{
return json => new DetailItem
{
SortOrder = (string)json["sortOrder"],
Value = (string)json["value"]
};
}
Hopefully, someone will add a setting for JSON.Net to allow it to force deserialization to a collection with a single item rather than throwing an exception. It's a shame that you have to do all of the parsing manually when there is only one small portion of the JSON that doesn't parse correctly automatically.