I would like to convert a csv file into json with nested objects and nested array. The sample csv looks like below. My csv structure can be dynamic, but I am ok to keep it static to get the required format below. I know there are so many answers to similar questions, but none addressing my specific issue of nested objects.
Id,name,nestedobject/id,nestedarray/0/name
1,name,2,namelist
2,name1,3,namelist1
I want the json like below, specifically having trouble with column 3 & 4 (nested objects)
[{"Id": 1,
"name": "name",
"nestedobject": {
"id": 2
},
"nestedarray": [{
"name": "namelist"
}
]
},
{"Id": 2,
"name": "name1",
"nestedobject": {
"id": 3
},
"nestedarray": [
{"name": "namelist1"}]
}
]
Please note to keep the csv file structure dynamic I pass the datatype in the column name - for e.g.. "System.Int32:Id" will be my first column instead of "Id". How can I modify my code to account for the nested objects?
public static DataTable ReadCsvFile(string fileName)
{
DataTable oDataTable = new DataTable();
StreamReader oStreamReader = new StreamReader(fileName);
bool hasColumns = false;
List<string> ColumnNames = new List<string>();
string[] oStreamDataValues = null;
while (!oStreamReader.EndOfStream)
{
String oStreamRowData = oStreamReader.ReadLine();
if(oStreamRowData.Length > 0)
{
oStreamDataValues = oStreamRowData.Split(',');
if(!hasColumns)
{
int i = 0;
foreach(string csvcolumn in oStreamRowData.Split(','))
{
string[] nameWithType = csvcolumn.Split(':');
DataColumn oDataColumn = new DataColumn(nameWithType[1], typeof(string));
ColumnNames.Add(nameWithType[1]);
var type = System.Type.GetType(nameWithType[0]);
oDataColumn.DataType = type;
oDataTable.Columns.Add(oDataColumn);
i = i + 1;
}
hasColumns = true;
}
else
{
DataRow oDataRow = oDataTable.NewRow();
for(int i = 0; i < ColumnNames.Count; i++)
{
oDataRow[ColumnNames[i]] = oStreamDataValues[i] == null ? string.Empty : oStreamDataValues[i].ToString();
}
oDataTable.Rows.Add(oDataRow);
}
}
}
oStreamReader.Close();
oStreamReader.Dispose();
return oDataTable;
}
I serialize the DataTable in the end
DataTable dt = ReadCsvFile(filePath);
dt.CaseSensitive = true;
var jsonData = JsonConvert.SerializeObject(dt);
With Cinchoo ETL, an open source library, you can do it as follows
string csv = #"Id,name,nestedobject/id,nestedarray/0/name, nestedarray/0/city, nestedarray/1/name, nestedarray/200/city
1,name,2,namelist10, citylist10,namelist11, citylist11
2,name1,3,namelist20, citylist20,namelist21, citylist21";
StringBuilder json = new StringBuilder();
using (var w = new ChoJSONWriter(json))
{
using (var r = ChoCSVReader.LoadText(csv).WithFirstLineHeader()
.Configure(c => c.NestedColumnSeparator = '/')
)
w.Write(r);
}
Console.WriteLine(json.ToString());
Output:
[
{
"Id": 1,
"name": "name",
"nestedobject": {
"id": 2
},
"nestedarray": [
{
"name": "namelist10",
"city": "citylist10"
},
{
"name": "namelist11"
}
]
},
{
"Id": 2,
"name": "name1",
"nestedobject": {
"id": 3
},
"nestedarray": [
{
"name": "namelist20",
"city": "citylist20"
},
{
"name": "namelist21"
}
]
}
]
Related
For example we have following document in elastic:
{
"name": "Bob",
"age": "22",
"phrase": "ohohoho",
"date": "2022-10-20T00:00:00Z"
}
string phrase ;
DateTime? date;
Then we want put following:
{
"name": "not Bob",
"age": "22",
"phrase": null,
"date": null
}
in c#:
var updateRequest = new UpdateRequest<T, T>(entity)
{
ScriptedUpsert = true,
Script = new InlineScript(
$"if (someCondition) {{ctx._source.putAll(params.entity);}} else {{ctx.op = \"noop\";}}")
{
Lang = "painless",
Params = new Dictionary<string, object>() { { "entity", entity } },
},
Upsert = Activator.CreateInstance<T>()
};
but in the end it will not update phrase and date.
It makes following request:
POST /myIndex/_update/b90278fd-1a66-40bf-b775-d076122c6c02
{
"script": {
"source": ""if (someCondition) {{ctx._source.putAll(params.entity);}} else {{ctx.op = \"noop\";}}"",
"lang": "painless",
"params": {
"entity": {
"name": "",
"age": 22
}
}
},
"upsert": {
"age": 0
}
}
Idk why but it skips all fields with null.
How to update nullable fields to null?
NEST does not support sending null values by default.
You can have a check in script such that if a value is not passed then you can remove it from document.
var updateRequest = new UpdateRequest<T, T(entity)
{
ScriptedUpsert = true,
Script = new InlineScript($"if (params.entity.phrase==null)ctx._source.remove('phrase');")
{
Lang = "painless",
Params = new Dictionary<string, object>() { { "entity", entity } },
},
Upsert = Activator.CreateInstance<T>()
};
You can check for more details here
I'm reading my json file from and trying to replace the property values. JSON file is below.
{
"fields": {
"summary": "summaryValue",
"project": {
"key": "projectValue"
},
"priority": {
"name": "priorityValue"
},
"Requestor": {
"name": "RequestorValue"
},
"issue": {
"name": "issueValue"
},
"labels": "LabelValue",
"customfield_xyz": "customfield_xyzValue"
}
}
How can I replace the value for each item inside the fields property ?
for ex:
{"fields": {
"summary": "NewsummaryValue",
"project": {
"key": "NewprojectValue"
},
"priority": {
"name": "NewpriorityValue"
}
}
}
Below is the code to parse my json file,
StreamReader r = new StreamReader(filepath);
var jsondata = r.ReadToEnd();
var jobj = JObject.Parse(jsondata);
foreach (var item in jobj.Properties())
{
\\replace code
}
I do not know exactly what you want. But I changed the json information in the code snippet as you wanted.
dynamic dataCollection = JsonConvert.DeserializeObject<dynamic>(jsonData);
string summary = dataCollection["fields"]["summary"];
string project = dataCollection["fields"]["project"]["key"];
string priority = dataCollection["fields"]["priority"]["name"];
dynamic json = new JObject();
json.summary = summary;
json.project = project;
json.priority = priority;
dynamic jsonRoot = new JObject();
jsonRoot.fields = json;
Console.WriteLine(jsonRoot.ToString());
output:
I want to map JSON path properties from key-value pairs to generate the JSON object in C# where the path contains nested array index path
Input:
Dictionary<string, string> properties = new Dictionary<string, string>();
properties.put("id", "1");
properties.put("name", "sample_name");
properties.put("category.id", "1");
properties.put("category.name", "sample");
properties.put("tags[0].id", "1");
properties.put("tags[0].name", "tag1");
properties.put("tags[1].id", "2");
properties.put("tags[1].name", "tag2");
properties.put("status", "available");
Output:
{
"id": 1,
"name": "sample_name",
"category": {
"id": 1,
"name": "sample"
},
"tags": [
{
"id": 1,
"name": "tag1"
},
{
"id": 2,
"name": "tag2"
}
],
"status": "available"
}
Using Jackson's JavaPropsMapper it can easily be achieved like:
JavaPropsMapper javaPropsMapper = new JavaPropsMapper();
JsonNode json = javaPropsMapper.readMapAs(properties, JsonNode.class);
How to implement this idea in C# so that I am able to generate the JSON object from the given JSON path node.
you can create anonymous object an serialize
var values = new {
id = "id",
name = "name",
category = new { id = 1, name = "sample"},
tags = new { id = 0, name = "sample" },
status = "available"
};
string json = JsonConvert.SerializeObject(values);
This is my Data Structure and I'm trying to access it with simpleJason, through unity/c#, I have accidentally gotten the right data here and there, and otherwise gotten completely empty arrays, I'd like to know if my JSON file is improperly setup for my data structure, or if the parser is somehow falling though, or not properly matching what I'm looking for.
JSON File:
{
"categories": [
{
"name" : "entertainment",
"projects": [
{
"name": "Awards",
"description": "Awards Shows",
"credits": [
"Lead Engineer - Dave Jones",
"VFX Supervisor - John Adrian",
"CG Supervisor - Evan Klein"
],
"meta": [
"awards",
"show",
"stars",
"red carpet"
],
"assets": [
{
"name": "Screen Actors Guild Awards",
"filename": "SAG_Awards.mp4",
"icon": "sag.png",
"stereo": false,
"meta": [
"Screen",
"Actors",
"Guild"
]
},
{
"name": "No Awards",
"filename": "No_SAG_Awards.mp4",
"icon": "no_sag.png",
"stereo": false,
"meta": [
"Screen",
"Actors",
"Guild"
]
},
{
"name": "None Awds",
"filename": "None_SAG_Awards.mp4",
"icon": "none_sag.png",
"stereo": false,
"meta": [
"Screen",
"Actors",
"Guild"
]
}
]
}
]
}
]
}
This is the struct :
private struct jsonAsset {
public string category;
public string project;
public string description;
public string[] credits;
public string[] meta;
public string asset;
public FileInfo file;
public FileInfo icon;
public bool stereo;
public bool overUnder;
};
This is the function:
jsonAsset LoadSceneDataFromJSON(FileInfo jsonFile)
{
jsonAsset asset = new jsonAsset();
Debug.Log("Processing : " + jsonFile);
// Parse File for Data
var N = JSON.Parse(File.ReadAllText(jsonFile.FullName));
var cat_arr = N["categories"].AsArray;
asset.category = N["categories"]["name"].Value;
Debug.Log(N["categories"]["projects"]["assets"]["filename"].Value);
foreach (JSONNode n in cat_arr)
{
asset.project = n["name"].Value;
// Credits
var proj_credits = n["credits"].AsArray;
foreach (JSONNode pc in proj_credits)
{
asset.credits[asset.credits.Length] = pc["credits"].Value;
}
// Project Meta
var proj_meta = n["meta"].AsArray;
foreach (JSONNode pm in proj_meta)
{
asset.meta[asset.meta.Length] = pm["meta"].Value;
}
// Project Array
var proj_arr = n["projects"].AsArray;
foreach (JSONNode nn in proj_arr)
{
var asset_arr = nn["assets"].AsArray;
asset.asset = nn["assets"]["name"].Value;
foreach (JSONNode nnn in asset_arr)
{
asset.asset = nnn["name"].Value;
asset.file = new FileInfo(m_dir + nnn["filename"].Value);
asset.icon = new FileInfo(m_dir + nnn["icon"].Value);
var asset_meta = nnn["meta"].AsArray;
foreach (JSONNode am in asset_meta)
{
asset.meta[asset.meta.Length] = am.Value;
}
}
}
}
return asset;
}
You are treating projects as an object instead of an array.
I dont know what JSON library you are using to give you guidance, but something like this will probably work:
N["categories"]["projects"][0]["assets"]["filename"].Value
I don't understand how can I output id properly when displaying objects. It's either ObjectId() that I can't parse back to object or some object that has id in it that I can parse but it looks weird.
Note that mapping it to class is not an option, it needs to be fully dynamic because different users have different fields.
Code
public List<object> Get()
{
var client = new MongoClient("mongodb://localhost");
var server = client.GetServer();
var database = server.GetDatabase("api_test");
var collection = database.GetCollection("users");
var json = collection.FindAllAs<BsonDocument>().ToJson(new JsonWriterSettings { OutputMode = JsonOutputMode.Strict });
var obj = JsonConvert.DeserializeObject<List<object>>(json);
return obj;
}
Example
[
{
"_id": {
"$oid": "528e7f9bb1fece903aa9b246"
},
"Name": "Steve",
"Age": 60
},
{
"_id": {
"$oid": "528e7fabb1fece903aa9b247"
},
"Name": "Alice",
"Age": 44
}
]
What I would like
[
{
"_id": "528e7f9bb1fece903aa9b246",
"Name": "Steve",
"Age": 60
},
{
"_id": "528e7fabb1fece903aa9b247",
"Name": "Alice",
"Age": 44
}
]
You can do this by explicitly updating your query result to convert _id to a string before it's serialized to JSON:
var docs = test.FindAll().ToList();
foreach (var doc in docs)
{
doc["_id"] = doc["_id"].ToString();
}
var json = docs.ToJson(
new JsonWriterSettings { OutputMode = JsonOutputMode.Strict });