We Keep Coding

How can I insert JSON efficiently without looping through all items in the array?

I have 3 JSON files. Each contain 10,000 objects each with 2 properties.
I have been trying to take a key value from each file, and use it to create a new JSON file, under a slightly different structure. My code works, but it takes a really long time to complete via my current code.
The issue is caused by a for each loop inside of a for each loop. I am looking for the "TokenId" of all items to check I am inserting in the correct location of the JSON. So, to begin with token 0, starts looping through the new JSON, finds it straight away. token 1 then checks token 0, it's not a match, so then checks token 1 which is. As I started to get to the higher numbers, for example 4000, it is checking every item 0-3999, which just seems unnecessary and incredibly time and resource inefficient. How else can I do this?
Here's the code:
dynamic objFinal = JObject.Parse("{ 'Tokens':[] }");
JArray finalArray = (JArray)objFinal["Tokens"];
DirectoryInfo dir = new DirectoryInfo(#"C:\Users\s\Desktop\JsonTesting\");
foreach (var file in dir.GetFiles("*.json"))
{
dynamic objJson = JObject.Parse(File.ReadAllText(file.FullName));
JArray tokens = (JArray)objJson["Tokens"];
foreach (JToken token in tokens)
{
JToken searchResult = finalArray.Where(t => Convert.ToString(t["TokenId"]) == Convert.ToString(token["TokenId"])).FirstOrDefault();
if (searchResult == null)
{
//push new tokenid
JObject newFileRanks = new JObject();
newFileRanks[file.Name.Remove(file.Name.Length - 5)] = token["rank"];
dynamic newToken = new JObject();
newToken["TokenId"] = token["TokenId"];
newToken["Ranks"] = newFileRanks;
finalArray.Add(newToken);
Console.WriteLine("Added new token");
}
else
{
//append to fileRanks
JObject newFileRanks = new JObject();
newFileRanks[file.Name.Remove(file.Name.Length - 5)] = token["rank"];
foreach (var item in finalArray)
{
Console.WriteLine("Current token TokenId: " + token["TokenId"]);
Console.WriteLine("Current Item TokenId: " + item["TokenId"]);
if ((string)item["TokenId"] == (string)token["TokenId"])
{
Console.WriteLine("Adding Rank");
dynamic ranksObject = item["Ranks"];
ranksObject[file.Name.Remove(file.Name.Length - 5)] = token["rank"];
break;
}
}
}
}
}
Console.Write(JsonConvert.SerializeObject(objFinal));
File.AppendAllText(#"C:\Users\samcl\Desktop\JsonTesting\cryptohippos_Rankings.json", JsonConvert.SerializeObject(objFinal));`
Here is the raw JSON structure for each file:
{"Tokens":[{"TokenId":0,"rank":2804},{"TokenId":1,"rank":977},
And here is the desired result JSON structure:
{"Tokens":[{"TokenId":0,"ranks":{"File1rank":977},{"File2rank":987},{"File3rank":967}},{"TokenId":1,"ranks":{"File1rank":977},{"File2rank":987},{"File3rank":967}}}

You should use a proper object model to do this, rather than JObject and dynamic etc, all of which can be slow.
There are other efficiencies, for example you shouldn't loop through your existing list each time. Instead store your intermediate Token objects in a dictionary so you can quickly access it.
class Root
{
public List<Token> Tokens = new List<Token>();
}
class Token
{
public int TokenId;
public int? rank;
public Dictionary<string, int> ranks;
}
var rootObj = JsonConvert.DeserializeObject<Root>(File.ReadAllText(file.FullName));
var tokensDict = new Dictionary<int, Token>();
DirectoryInfo dir = new DirectoryInfo(#"C:\Users\s\Desktop\JsonTesting\");
foreach (var file in dir.EnumerateFiles("*.json"))
{
foreach (var token in rootObj.Tokens)
{
if(!tokensDict.TryGetValue(token.TokenId, out var dictToken)
{
tokensDict[token.TokenId] = dictToken = token;
token.ranks = new Dictionary<string, int>();
token.ranks.Add(file.Name.Remove(file.Name.Length - 5), token.rank);
token.rank = null;
}
else
{
dictToken.ranks.Add(file.Name.Remove(file.Name.Length - 5), token.rank);
}
}
}
var objFinal = new Root();
objFinal.Tokens.AddRange(tokensDict.Value);
File.AppendAllText(#"C:\Users\samcl\Desktop\JsonTesting\cryptohippos_Rankings.json", JsonConvert.SerializeObject(objFinal));`

Deserialize the JSON to an object and try manipulating the object using Linq functions.
Serialize back to JSON when done.
JSON objects are meant for data transfer and should ideally be used only for that purpose.

CSVHelper - Possible to Write List of Dictionary

I have a list of Dictionary with same list of keys, but different value. Is there a way to write that to CSV file using the CSVHelper? I have the sample code below, but obviously it didn't work.
static void Main(string[] args)
{
List<Dictionary<String, String>> records = new List<Dictionary<string, string>>();
Dictionary<String, String> data1 = new Dictionary<String, String>();
data1.Add("Name1", "Value1");
data1.Add("Name2", "Value2");
records.Add(data1);
Dictionary<String, String> data2 = new Dictionary<String, String>();
data2.Add("Name1", "Value1");
data2.Add("Name2", "Value2");
records.Add(data2);
using (var writer = new StreamWriter("e:\\temp\\test.csv"))
using (var csv = new CsvWriter(writer))
{
csv.WriteRecords(records);
//GEtting exception here
//CsvHelper.Configuration.CsvConfigurationException: 'Types that inherit IEnumerable cannot be auto mapped. Did you accidentally call GetRecord or WriteRecord which acts on a single record instead of calling GetRecords or WriteRecords which acts on a list of records?'
}
}
Is there any way around that?
Thanks!

I believe the only way to do it will be to write them out by hand.
using (var writer = new StreamWriter("e:\\temp\\test.csv"))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
var headings = new List<string>(records.First().Keys);
foreach (var heading in headings)
{
csv.WriteField(heading);
}
csv.NextRecord();
foreach (var item in records)
{
foreach (var heading in headings)
{
csv.WriteField(item[heading]);
}
csv.NextRecord();
}
}

dictionary to json with key names

I have a dictionary Dictionary<int, string> of ints and strings, where ints are ids and strings are usernames, and when I convert it to JSON using Json.NET I get something like the following:
{"3":"jack","2":"john"}
I convert it like so:
Dictionary<int, string> dictFriends = new Dictionary<int, string>();
foreach (var id in resultList)
{
var user = db.Users.Find(id);
string friend = user.Username;
dictFriends.Add(id, friend);
}
string json = JsonConvert.SerializeObject(dictFriends);
But I am hoping to get something like so:
[
{ "id": "3", "user": "jack"},
{ "id": "2", "user": "john"},
]
Is it possible?

As far as I know you'd have to transform the dictionary into something JSON.NET would recognise as being an IEnumerable:
// YOUR DICTIONARY
var dictFriends = new Dictionary<int, string>() {
{1,"Jack"},
{2,"John"},
{3,"Jeff"}
};
// TRANSFORM INTO IENUMERABLE
var transformed = from key in dictFriends.Keys
select new { id = key, user = dictFriends[key] };
// SERIALIZE
var json = JsonConvert.SerializeObject(transformed);
Output:
[
{"id":1, "user":"Jack"},
{"id":2, "user":"John"},
{"id":3, "user":"Jeff"}
]

You're trying to use a Dictionary as an Array/List, writing to an existing key will overwrite it. Also your current key type is int therefore you would have JSON output such as
{1: "jack", 2: "john"}
Instead set your object type to List<Dictionary<string, Object>>
List<Dictionary<string, object>> friends = new List<Dictionary<string, Object>>();
foreach (var id in resultList)
{
var user = db.Users.Find(id);
string friend = user.Username;
Dictionary<string, object> dictFriend = new Dictionary<string, Object>();
dictFriend.Add("id", id);
dictFriend.Add("name" , friend);
friends.Add(dictFriend);
}
string json = JsonConvert.SerializeObject(friends);

You could use the DataContractJsonSerializer: https://msdn.microsoft.com/en-us/library/system.runtime.serialization.json.datacontractjsonserializer(v=vs.110).aspx
The below will produce output in the form you're after; only instead of id and user your fields would be named key and value. The reason being those are the property names on the dictionary.
If you needed to change those names also (i.e. it's not just the structure you're interested in), you'd need to override the dictionary with a custom class, where you could add attributes such as [JsonProperty(PropertyName = "User")] to the properties to change how they're parsed... See http://www.newtonsoft.com/json/help/html/SerializationAttributes.htm for more.
Dictionary<int, string> dictFriends = new Dictionary<int, string>();
dictFriends.Add(1, "Alice");
dictFriends.Add(2, "Bob");
string jsonString;
using (MemoryStream ms = new MemoryStream()) {
//NB: DataContractJsonSerializer is in assembly System.Runtime.Serialization.dll - and others; http://stackoverflow.com/a/2682197/361842
DataContractJsonSerializer dcjs = new DataContractJsonSerializer(dictFriends.GetType());
dcjs.WriteObject(ms, dictFriends);
ms.Position = 0;
using(StreamReader sr = new StreamReader(ms)) {
jsonString = sr.ReadToEnd();
}
}
Debug.WriteLine(jsonString);
Sample output:
[{"Key":1,"Value":"Alice"},{"Key":2,"Value":"Bob"}]

C# Method is not executing sequentially

I have the below code:
public Dictionary<int, Ticket> GetNewTickets()
{
Dictionary<int, Ticket> output = new Dictionary<int, Ticket>();
foreach (KeyValuePair<int, Ticket> item in ticketStore)
{
if (!ticketStoreNew.ContainsKey(item.Key))
{
output.Add(item.Key, item.Value);
}
}
ticketStoreNew = ticketStore;
return output;
}`
It takes a dictionary, ticketStore, checks to see if it has any new elements not in ticketStoreNew and puts them in the output dictionary. Then, ticketStoreNew is set to ticketStore until ticketStore is updated with another method and this method is ran again.
However, when I include the line ticketStoreNew = ticketStore, the program returns an empty dictionary. It looks like the method is not executing sequentially and this is running prior to the for loop.
I really just need to return any new items added to the ticketStore dictionary.
EDIT
Below is the code for getting ticketStore:
public void UpdateTickets(string inputXml)
{
// If no new tickets exit
if (inputXml.Trim() == "") { return; }
//xmlString = inputXml;
// Load XML into an enumerable
XElement xelement = XElement.Parse(inputXml);
IEnumerable<XElement> xml = xelement.Elements();
foreach (var item in xml)
{
if (item.Name == "incident")
{
int id;
// If ID can be converted to INT
if (Int32.TryParse(item.Element("id").Value, out id))
{
// If ticket is not already in store create ticket and populate data
if (!ticketStore.ContainsKey(id))
{
Ticket ticket = new Ticket();
ticket.id = id;
ticket.number = Int32.Parse(item.Element("number").Value);
ticket.title = item.Element("name").Value;
ticket.description = item.Element("description").Value;
ticketStore.Add(id, ticket);
}
}
}
}
}
}
The tickets are all based on getting XML from the Samanage API.

If another method updates ticketStore then the assignment is the problem. It doesn't copy the contents of ticketStore to ticketStoreNew it sets the reference ticketStoreNew to point to the same instance as ticketStore. Thus they are the same object and always have the same contents. Try creating a new Dictionary to copy the items:
ticketStoreNew = new Dictionary<int, Ticket>(ticketStore);

Try this code:
private Dictionary<int, Ticket> ticketStoreNew =
new Dictionary<int, Ticket>(); // add this line
public Dictionary<int, Ticket> GetNewTickets()
{
Dictionary<int, Ticket> output = new Dictionary<int, Ticket>();
foreach (KeyValuePair<int, Ticket> item in ticketStore)
{
if (!ticketStoreNew.ContainsKey(item.Key))
{
output.Add(item.Key, item.Value);
ticketStoreNew.Add(item.Key, item.Value); // add this line
}
}
//ticketStoreNew = ticketStore; remove this line
return output;
}

Neo4j + bolt + c#; how do you create multiple nodes via one query by passing a map as a parameter

can anyone advise how I should change my code (this is based on section 3.5.1.4.2 from the 3.0 developer manual). I am trying to create multiple nodes via one query in bolt.
using (var driver = GraphDatabase.Driver(Neo4jCredentials.Instance, AuthTokens.Basic(Neo4jCredentials.Username, Neo4jCredentials.Password)))
using (var session = driver.Session())
{
string query = "UNWIND { props } AS map CREATE(n) SET n = map";
Dictionary<string, object> myParameter = new Dictionary<string, object>();
myParameter.Add("props", "{\"props\":[{\"name\":\"Andres\",\"position\":\"Developer\"},{\"name\":\"Michael\",\"position\":\"Developer\"}]}");
return session.Run(query, myParameter);
}
The error I am getting is:
{"Expected map to be a map, but it was :`{\"props\":[{\"name\":\"Andres\",\"position\":\"Developer\"},{\"name\":\"Michael\",\"position\":\"Developer\"}]}`"}
Thanks in advance my learned friends...

Try forming your dictionary of params using an array of dictionaries:
Dictionary<string, object> myParameter = new Dictionary<string, object>();
Dictionary<string, object>[] props =
{
new Dictionary<string, object> {{"name", "Andres"}, {"position", "Developer"}},
new Dictionary<string, object> {{"name", "Michael"}, {"position", "Developer"}}
};
myParameter.Add("props",props);
or with a few less characters:
var myParameter = new Dictionary<string, object>
{
{
"props", new[]
{
new Dictionary<string, string> {{"name", "Andres"}, {"position", "Developer"}},
new Dictionary<string, string> {{"name", "Michael"}, {"position", "Developer"}}
}
}
};