We Keep Coding

Find item(s) in a nested dictionary by secondary key

This is my problem: I have a nested dictionary (primary keys: orderIds, secondary keys: productIds).
My scheme:
Dictionary<string, Dictionary<string, List<Task>>>
I need to look for the productId and return that value (Task object).
It's usually not the case that a product key appears in several orders. So it's unique.
Here is a json example:
{
"O1": {
"P1": [
{
"Field": "V1"
},
{
"Field": "V7"
}
],
"P2": [
{
"Field": "V2"
},
{
"Field": "V8"
}
]
},
"O2": {
"P1": [
{
"Field": "V3"
},
{
"Field": "V5"
}
],
"P2": [
{
"Field": "V4"
},
{
"Field": "V6"
}
]
}
}
If I look for productId "P5" I want to get...
[{"Field":"V5"}]
The only way I found that runs is...
return base.Values.Single(x => x.ContainsKey(productId))[productId];
//base is the nested Dictionary
But I don't like what I'm doing here. Because I detach the correct dictionary from the collection (values) where the key exists and finally I fetch only the value via key filtering (collection[key]).
That are basically two steps, but I suspect there is an easier way with just one step. - I just can't find this one.
Maybe you can help me out. :)

For performance tuning, use HashSet instead of List.
I also can recommend ISBN: 0321637003 (LINQ to Objects), maybe there is an updated version. But anyway its still very good content.
If you have millions of entries in your dict, you can try PLINQ.
return base.Values.Single(x => x.ContainsKey(productId))[productId];
The Sample json throws an exception, because your productId (P1, P2) is not unique and P5 does not exist.
If I look for productId "P5" I want to get...
[{"Field":"V5"}]
There Is no P5
I am not sure its helping but, here is my sample code. I have put your json into a file and deserialize it.
var lFile = new FileInfo(#"C:\_test\data.json");
using var lReader = lFile.OpenText();
var lJsonStr = lReader.ReadToEnd();
var lDicDic = JsonConvert.DeserializeObject<Dictionary<string, Dictionary<string, List<ProductId>>>>(lJsonStr);
//var lTest1 = lDicDic.Values.Single(x => x.ContainsKey("P1"))["P1"]; //Not working, P1 is not unique!
var lTest2 = lDicDic["O2"]["P1"];
//contains => "Field": "V3" + "Field": "V5"
var lTest3 = lDicDic
.SelectMany(p => p.Value.Values.SelectMany(qItem => qItem))
.FirstOrDefault(qProducts => qProducts.Field == "V5");
//contains => "Field": "V5"
var lTest4 = lDicDic["O2"]["P1"].Last();
//contains => "Field": "V5"
EDIT:
After your Fiddler code I was able to test it.
public List<MyTask> GetByProductId(string productId)
{
var lProductDict = Tasks.Values.SingleOrDefault(x => x.ContainsKey(productId));
return lProductDict?.GetValueOrDefault(productId);
}
My stomach tells my, your current approach is hard to make any better ;)
If you have a lot of calls on this nasted dict, and its changing not very often it could make sense to refactor the dict from TopDown to BottomUp at the beginning for the processing.
Anyway, If my comments are helpful plz upvote my answere :)
If I were you I would make some testcases with real data.
Dont forget: If you are in DEBUG mode, to enable "Optimize Code" at the Project. In my testcase, I had an AVG of 47ms without and 42ms with optimized code:
You can give PLINQ a try:
https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/introduction-to-plinq
Here us my test scenario:
var lStopWatch = new Stopwatch();
lStopWatch.Restart();
var lJobs = new Jobs("F1");
const int TestOrderCount = 1000000;
const int TestAvgCount = 1000;
for (var lIndex = 1; lIndex < TestOrderCount; lIndex++)
{
var lNoStr = lIndex.ToString("D6");
lJobs.Add($"O{lNoStr}", $"P{lNoStr}", $"V{lNoStr}");
}
var GetTimes = new List<long>();
var lRandom = new Random();
var lTestCases = Enumerable
.Range(1, TestAvgCount - 1)
.Select(r => $"P{lRandom.Next(1, TestOrderCount - 1):D6}")
.ToList();
var lSetupTimeMs = lStopWatch.ElapsedMilliseconds;
Debug.WriteLine($"SetupTimeMs: {lSetupTimeMs}");
foreach (var lTestCase in lTestCases)
{
lStopWatch.Restart();
var lTest = lJobs.GetByProductId(lTestCase);
GetTimes.Add(lStopWatch.ElapsedMilliseconds);
}
var lAvg = GetTimes.Sum() / TestAvgCount; //AVG Ms per get
Debug.WriteLine($"AVG: {lAvg}");

C# find JSON value based only on Key name through multiple levels of array

I have a variety of input JSON formatted data which all contain a particular key-name terminalSize. This is the only piece I know. The total number of JSON trees or the exact depth of terminalSize inside the JSON tree will forever be an unknown and subject to change.
I'm looking for a C# solution to loop through every child of the JSON string and find terminalSize then fetch the value.
I've tried this with success but it will only work if terminalSize is in the first level of the JSON:
var list = JsonConvert.DeserializeObject<List<Dictionary<string, string>>>(jsonString);
var dict = list.SelectMany(d => d).ToDictionary(p => p.Key, p => p.Value);
var terminal = dict["terminalSize"];
Example 1.
{
"status": "success",
"data": {
"terminalSize": 3766505.46,
"totalTerminalSize": 3766505.46
},
"message": null
}
Example 2.
{
"lastUpdated": 1588020678,
"terminalData": {
"terminalSize": "451679852",
"totalTerminalSize": "2100000000"
},
"terminalValueSeries": {
"8x7": 2.33,
"8x6": 3.73,
"8x5": 4.49,
"8x4": 3.68,
"8x3": 13998,
"8x2": 274936,
"8x1": 5.09
}
}
Example 3.
{
"terminalSize": "492612346.17",
"terminalStatus": "online"
}

You can parse your JSON to a JToken, then use SelectToken with the recursive descent JsonPath operator .. to get the terminalSize anywhere in the JSON:
var terminalSize = (double?) JToken.Parse(json).SelectToken("$..terminalSize");
Fiddle: https://dotnetfiddle.net/5ziYbP
If there might be multiple terminalSize keys in the JSON, for example if you had an array of terminals, you can use SelectTokens instead and put the terminal sizes into a Dictionary keyed by path:
var sizes = JToken.Parse(json4)
.SelectTokens("$..terminalSize")
.ToDictionary(t => t.Path, t => (double)t);
Fiddle: https://dotnetfiddle.net/ivSM88

You could also use linq and filter the JProperty collection based on JProperty.Name. For example
var result = JObject.Parse(jsonString)
.DescendantsAndSelf()
.OfType<JProperty>()
.Single(x=>x.Name.Equals("terminalSize"))
.Value;

You may parse your JSON into JObject, then recursively go through all properties and sub objects to find a terminalSize value. There is no need to deserialize the entire JSON into specific object
var json = JObject.Parse(jsonString);
var result = GetTerminalSize(json);
double GetTerminalSize(JObject input)
{
foreach (var property in input.Properties())
{
if (property.Name == "terminalSize")
return property.Value.Value<double>();
if (property.Value.Type == JTokenType.Object)
return GetTerminalSize((JObject) property.Value);
//not sure, if the is a need to handle an array
if (property.Value.Type == JTokenType.Array)
foreach (var item in (JArray) property.Value)
return GetTerminalSize((JObject) item);
}
return 0;
}
It returns a correct value for all 3 examples

C# REST dictionary DTO

What is the best way to design a DTO for a REST API to get a resource that is semantically a dictionary?
Example: Get the number of employees by age -> should return a set of couples (age, number) of type (int, int).
An easy way should be an array of structures, like:
[
{
age: 30,
number: 3
},
{
age: 31,
number: 4
},
{
age: 32,
number: 5
}
]
but this does not enforces that the key should be unique.
Semantically, I'd rather prefer a dictionary, like:
{
"30": 3,
"31": 4,
"32": 5
}
Is there a way to define such a DTO in C#, so that it can be easily serialized/deserialized?

Your second Json is not valid, it should look like:
{
"30": 3,
"31": 4,
"32": 5
}
And in c# it's just a Dictionary<string, int>
You could easiliy deserialize/serialize it with Json.Net
var result =JsonConvert.DeserializeObject<Dictionary<string, int>>(text);
var serialized = JsonConvert.SerializeObject(
new Dictionary<string, int> { { "1", 2 }, {"3",4}});

As suggested by maksim-simkin in his answer, dictionaries are natively supported by Json.Net.
Here's an example:
var result = JsonConvert.DeserializeObject<Dictionary<int, int>>(text);
var serialized = JsonConvert.SerializeObject(new Dictionary<int, int> { { 30, 3 }, { 31, 4}});
// here serialized == "{\"30\":3,\"31\":4,\"32\":5}"

If it can be useful, this serialization is supported in Nancy through the DynamicDictionary class (in namespace Nancy). In a NancyModule, you can create a model like this:
DynamicDictionary dynamicDictionary = ToDynamicDictionary(dictionary);
return this
.Negotiate
.WithModel(dynamicDictionary)
.WithStatusCode(HttpStatusCode.OK);
Then, you have to convert your keys to a string, using this helper method:
private static DynamicDictionary ToDynamicDictionary(IDictionary<int, int> dictionary)
{
DynamicDictionary result = new DynamicDictionary();
foreach (var pair in dictionary)
{
result.Add(pair.Key.ToString(CultureInfo.InvariantCulture), pair.Value);
}
return result;
}

C# - Adding objects dynamically (adding dynamic property names)

I'm trying to create some dynamic ExpandoObject. I've encountered a certain problem.
As I don't know what the name of these different properties in my objects should be, I can't do like this:
var list = new ArrayList();
var obj = new ExpandoObject();
obj.ID = 1,
obj.Product = "Pie",
obj.Days = 1,
obj.QTY = 65
list.Add(obj);
Let me explain my situation: I wish to get data from a random DB (I don't know which, but building a connection string from the information I get from the UI), therefore I don't know what data I need to get. This could be an example of a DB table
TABLE Sale
ID: int,
Product: nvarchar(100),
Days: int,
QTY: bigint
This could be another exmaple:
TABLE Foobar
Id: int,
Days: int
QTY: bigint
Product_Id: int
Department_Id: int
As you see, I don't know what the DB looks like (this is 100% anonymous, therefore it needs to be 100% dynamic), and the data I want to return should look like a well constructed JSON, like so:
[
{
"ID": 1,
"Product": "Pie"
"Days": 1,
"QTY": 65
},
{
"ID": 2,
"Product": "Melons"
"Days": 5,
"QTY": 12
}
]
Or, with the other example:
[
{
"ID": 1,
"Days": 2,
"QTY": 56,
"Product_Id": 5,
"Department_Id": 2
}
{
"ID": 2,
"Days": 6,
"QTY": 12,
"Product_Id": 2,
"Department_Id": 5
}
]
I've tried working with these ExpandoObjects, but can't seem to make it work, as I can't do what's illustrated in the top of this question (I don't know the names of the properties). Is there a way for me to say something like:
var obj = new ExpandoObject();
var propName = "Product";
var obj.propName = "Pie"
Console.WriteLine("Let's print!: " + obj.Product);
//OUTPUT
Let's print!: Pie
Does anyone have a solution, og simply guidance to a structure, that might solve this situation?

Rather than creating an ExpandoObject or some other dynamic type, you could create a List<Dictionary<string, object>> where each Dictionary<string, object> contains the name/value pairs you want to serialize. Then serialize to JSON using Json.NET (or JavaScriptSerializer, though that is less flexible):
var list = new List<Dictionary<string, object>>();
// Build a dictionary entry using a dictionary initializer: https://msdn.microsoft.com/en-us/library/bb531208.aspx
list.Add(new Dictionary<string, object> { { "ID", 1 }, {"Product", "Pie"}, {"Days", 1}, {"QTY", 65} });
// Build a dictionary entry incrementally
// See https://msdn.microsoft.com/en-us/library/xfhwa508%28v=vs.110%29.aspx
var dict = new Dictionary<string, object>();
dict["ID"] = 2;
dict["Product"] = "Melons";
dict["Days"] = 5;
dict["QTY"] = 12;
list.Add(dict);
Console.WriteLine(JsonConvert.SerializeObject(list, Formatting.Indented));
Console.WriteLine(new JavaScriptSerializer().Serialize(list));
The first outputs:
[
{
"ID": 1,
"Product": "Pie",
"Days": 1,
"QTY": 65
},
{
"ID": 2,
"Product": "Melons",
"Days": 5,
"QTY": 12
}
]
The second outputs the same without the indentation:
[{"ID":1,"Product":"Pie","Days":1,"QTY":65},{"ID":2,"Product":"Melons","Days":5,"QTY":12}]

Use dynamic, then cast to IDictionary<string, object> to loop through your properties:
dynamic obj = new ExpandoObject();
obj.Product = "Pie";
obj.Quantity = 2;
// Loop through all added properties
foreach(var prop in (IDictionary<string, object>)obj)
{
Console.WriteLine(prop.Key + " : " + prop.Value);
}
I've made a fiddle: https://dotnetfiddle.net/yFLy2u
Now this is a solution to your question... other answers like #dbc's might be better suited to the problem (which is not the question, really)

As you can see here ExpandoObject Class, the ExpandoObject is implementing IDictionary<string, object>, so you can use that fact like
IDictionary<string, object> obj = new ExpandoObject();
var propName = "Product";
obj[propName] = "Pie"
Console.WriteLine("Let's print!: " + obj[propName]);
// Verify it's working
Console.WriteLine("Let's print again!: " + ((dynamic)obj).Product);

While I was writing the answer, I see you already got proper answer. You can use a Dictionary<string, onject> or even Tuple.
But as per your original question, you wanted to add properties dynamically. For that you can refer to other answer using ExpandoObject. This is just the same solution (using ExpandoObject to dynamically add properties) with classes similar to your code.
//example classes
public class DictKey
{
public string DisplayName { get; set; }
public DictKey(string name) { DisplayName = name; }
}
public class DictValue
{
public int ColumnIndex { get; set; }
public DictValue(int idx) { ColumnIndex = idx; }
}
//utility method
public static IDictionary<string, object> GetExpando(KeyValuePair<DictKey, List<DictValue>> dictPair)
{
IDictionary<string, object> dynamicObject = new ExpandoObject();
dynamicObject["Station"] = dictPair.Key.DisplayName;
foreach (var item in dictPair.Value)
{
dynamicObject["Month" + (item.ColumnIndex + 1)] = item;
}
return dynamicObject;
}
Ans usage example:
var dictionaryByMonth = new Dictionary<DictKey, List<DictValue>>();
dictionaryByMonth.Add(new DictKey("Set1"), new List<DictValue> { new DictValue(0), new DictValue(2), new DictValue(4), new DictValue(6), new DictValue(8) });
dictionaryByMonth.Add(new DictKey("Set2"), new List<DictValue> { new DictValue(1), new DictValue(2), new DictValue(5), new DictValue(6), new DictValue(11) });
var rowsByMonth = dictionaryByMonth.Select(item => GetExpando(item));

First part, read this blog post by C# team thoroughly.
Lets see your code
var obj = new ExpandoObject();
var propName = "Product";
var obj.propName = "Pie"
Console.WriteLine("Let's print!: " + obj.Product);
//OUTPUT
Let's print!: Pie
In your code you are using var obj = new ExpandoObject();, so you are creating a statically typed object of type ExpandableObject. In the blog they specifically call out
I didn’t write ExpandoObject contact = new ExpandoObject(), because if I did contact would be a statically-typed object of the ExpandoObject type. And of course, statically-typed variables cannot add members at run time. So I used the new dynamic keyword instead of a type declaration, and since ExpandoObject supports dynamic operations, the code works
So if you rewrite your code to use dynamic obj, and add the dynamic properties as properties it should work!
But for your particular use case you better use Dictionaries as suggested above by #dbc
dynamic obj = new ExpandoObject();
obj.Product= "Pie"
Console.WriteLine("Let's print!: " + obj.Product);
//OUTPUT
Let's print!: Pie

Convert JSON to DataTable

I have JSON in the following format:
[
{"id":"10","name":"User","add":false,"edit":true,"authorize":true,"view":true},
{"id":"11","name":"Group","add":true,"edit":false,"authorize":false,"view":true},
{"id":"12","name":"Permission","add":true,"edit":true,"authorize":true,"view":true}
]
How can I convert that into a C# DataTable object as follows?
---------------------------------------------------------------------
ID | Name | Add | Edit | View | Authorize
---------------------------------------------------------------------
10 | User | true | true | true | true
11 | Group | true | true | true | true
12 | Permission| true | true | true | true

There is an easier method than the other answers here, which require first deserializing into a c# class, and then turning it into a datatable.
It is possible to go directly to a datatable, with JSON.NET and code like this:
DataTable dt = (DataTable)JsonConvert.DeserializeObject(json, (typeof(DataTable)));

Deserialize your jsonstring to some class
List<User> UserList = JsonConvert.DeserializeObject<List<User>>(jsonString);
Write following extension method to your project
using System.ComponentModel;
public static DataTable ToDataTable<T>(this IList<T> data)
{
PropertyDescriptorCollection props =
TypeDescriptor.GetProperties(typeof(T));
DataTable table = new DataTable();
for(int i = 0 ; i < props.Count ; i++)
{
PropertyDescriptor prop = props[i];
table.Columns.Add(prop.Name, prop.PropertyType);
}
object[] values = new object[props.Count];
foreach (T item in data)
{
for (int i = 0; i < values.Length; i++)
{
values[i] = props[i].GetValue(item);
}
table.Rows.Add(values);
}
return table;
}
Call extension method like
UserList.ToDataTable<User>();

One doesn't always know the type into which to deserialize. So it would be handy to be able to take any JSON (that contains some array) and dynamically produce a table from that.
An issue can arise however, where the deserializer doesn't know where to look for the array to tabulate. When this happens, we get an error message similar to the following:
Unexpected JSON token when reading DataTable. Expected StartArray, got StartObject. Path '', line 1, position 1.
Even if we give it come encouragement or prepare our json accordingly, then "object" types within the array can still prevent tabulation from occurring, where the deserializer doesn't know how to represent the objects in terms of rows, etc. In this case, errors similar to the following occur:
Unexpected JSON token when reading DataTable: StartObject. Path '[0].__metadata', line 3, position 19.
The below example JSON includes both of these problematic features:
{
"results":
[
{
"Enabled": true,
"Id": 106,
"Name": "item 1",
},
{
"Enabled": false,
"Id": 107,
"Name": "item 2",
"__metadata": { "Id": 4013 }
}
]
}
So how can we resolve this, and still maintain the flexibility of not knowing the type into which to derialize?
Well here is a simple approach I came up with (assuming you are happy to ignore the object-type properties, such as __metadata in the above example):
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Data;
using System.Linq;
...
public static DataTable Tabulate(string json)
{
var jsonLinq = JObject.Parse(json);
// Find the first array using Linq
var srcArray = jsonLinq.Descendants().Where(d => d is JArray).First();
var trgArray = new JArray();
foreach (JObject row in srcArray.Children<JObject>())
{
var cleanRow = new JObject();
foreach (JProperty column in row.Properties())
{
// Only include JValue types
if (column.Value is JValue)
{
cleanRow.Add(column.Name, column.Value);
}
}
trgArray.Add(cleanRow);
}
return JsonConvert.DeserializeObject<DataTable>(trgArray.ToString());
}
I know this could be more "LINQy" and has absolutely zero exception handling, but hopefully the concept is conveyed.
We're starting to use more and more services at my work that spit back JSON, so freeing ourselves of strongly-typing everything, is my obvious preference because I'm lazy!

It can also be achieved using below code.
DataSet data = JsonConvert.DeserializeObject<DataSet>(json);

You can make use of JSON.Net here. Take a look at JsonConvert.DeserializeObject method.

I recommend you to use JSON.NET. it is an open source library to serialize and deserialize your c# objects into json and Json objects into .net objects ...
Serialization Example:
Product product = new Product();
product.Name = "Apple";
product.Expiry = new DateTime(2008, 12, 28);
product.Price = 3.99M;
product.Sizes = new string[] { "Small", "Medium", "Large" };
string json = JsonConvert.SerializeObject(product);
//{
// "Name": "Apple",
// "Expiry": new Date(1230422400000),
// "Price": 3.99,
// "Sizes": [
// "Small",
// "Medium",
// "Large"
// ]
//}
Product deserializedProduct = JsonConvert.DeserializeObject<Product>(json);

json = File.ReadAllText(System.AppDomain.CurrentDomain.BaseDirectory + "App_Data\\" +download_file[0]);
DataTable dt = (DataTable)JsonConvert.DeserializeObject(json, (typeof(DataTable)));

Here is another seamless approach to convert JSON to Datatable using Cinchoo ETL - an open source library
Sample below shows how to convert
string json = #"[
{""id"":""10"",""name"":""User"",""add"":false,""edit"":true,""authorize"":true,""view"":true},
{ ""id"":""11"",""name"":""Group"",""add"":true,""edit"":false,""authorize"":false,""view"":true},
{ ""id"":""12"",""name"":""Permission"",""add"":true,""edit"":true,""authorize"":true,""view"":true}
]";
using (var r = ChoJSONReader.LoadText(json))
{
var dt = r.AsDataTable();
}
Sample fiddle: https://dotnetfiddle.net/y0siCi

I solved it by the following - Used Pravin Pawar's answer below - for converting the JSON object to dt.
https://stackoverflow.com/a/11982180/504351
1)Send the JSON from Angular to C# controller method.
2)In C# Controller method -
convert the string data (JSON object) to a datatable.
DataTable dt = (DataTable)JsonConvert.DeserializeObject(data, (typeof(DataTable)));
3)Create a table in SQL Server database by parsing through rows and columns of the datatable dt
int dt_length = dt.Columns.Count;
Random rand = new Random();
string tablename = "Test" + rand.Next().ToString();
string query = "Create table " + tablename + " ( ";
for (int i = 0; i < dt_length; i ++)
{
if(i == dt_length -1)
{
query = query + dt.Columns[i].ColumnName + " varchar(max) ) ";
}
else
{
query = query + dt.Columns[i].ColumnName + " varchar(max), ";
}
}
4)Similarly insert each of the row data in the table created.
5)This helps to dynamically create a database table from JSON.