We Keep Coding

C# serializing JSON data from Database for API

My goal is to get the data from the database, serializing them into JSON format and send it to the API. The problem is that I don't know how to get right JSON format for the API.
C# Worker service collecting data from database.
from database i got:
1|John|Wick|Action|101
my API needs this JSON:
{
"Name":"John",
"Surname":"Wick",
"Type":"Action",
"Length":"101"
}
when i use in C# serializing to JSON:
var jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(values);
i got:
[John,Wick,Action,101]
is there any way how to add name of values to JSON ?

First split the database result based on the delimiter
string dbResult = ...; //1|John|Wick|Action|101
string[] dbResults = dbResult.Split("|");
Second create an anonymous object (if you don't want to introduce a data model class/struct/record)
var result = new
{
Name = dbResults[0],
Surname = dbResults[1],
Type = dbResults[2],
Length = dbResults[3],
};
Third serialize the anonymous object
var jsonString = Newtonsoft.Json.JsonConvert.SerializeObject(result);

Convert Nested JSON to CSV in C# via ChoETL

Does anyone know how to convert the below nested JSON to CSV via CHOETL (An ETL framework for .NET)? Thank you!
I'm using this code but it will only return the first equipment record.
CODE:
{
using (var json = new ChoJSONReader("./test.json"))
{
csv.Write(json.Cast<dynamic>().Select(i => new
{
EquipmentId = i.GpsLocation.Equipment[0].EquipmentId,
InquiryValue = i.GpsLocation.Equipment[0].InquiryValue,
Timestamp = i.GpsLocation.Equipment[0].Timestamp
}));
}
}
JSON:
"GpsLocation": {
"Equipment": [
{
"EquipmentId": "EQ00001",
"InquiryValue": [
"IV00001"
],
"Timestamp": "2020-01-01 01:01:01.01",
},
{
"EquipmentId": "EQ00002",
"InquiryValue": [
"IV00002"
],
"Timestamp": "2020-01-01 01:01:01.01"
}
]
}
}````

As others suggest, the issue is you are only looking at the first element of the array.
It appears that the easiest way to control what you serialise into CSV is by correctly defining your source objects from JSON. JSON Path expressions come in pretty handy.
What I ended up doing here is query all JSON to return an array of Equipment objects regardless of where they are in the hierarchy (which means you may need to filter it a bit better depending on your full JSON).
Then it's pretty easy to define each field based on JSON path and just pass the result to CSVWriter.
Also check out some gotchas that I outlined in the respective comment lines.
void Main()
{
var jsonString = "{\"GpsLocation\":{\"Equipment\":[{\"EquipmentId\":\"EQ00001\",\"InquiryValue\":[\"IV00001\"],\"Timestamp\":\"2020-01-01 01:01:01.01\"},{\"EquipmentId\":\"EQ00002\",\"InquiryValue\":[\"IV00002\"],\"Timestamp\":\"2020-01-01 01:01:01.01\"}]}}";
var jsonReader = new StringReader(jsonString);
var csvWriter = new StringWriter(); // outputs to string, comment out if you want file output
//var csvWriter = new StreamWriter(".\\your_output.csv"); // writes to a file of your choice
using (var csv = new ChoCSVWriter(csvWriter))
using (var json = new ChoJSONReader(jsonReader)
.WithJSONPath("$..Equipment[*]", true) // firstly you scope the reader to all Equipment objects. take note of the second parameter. Apparently you need to pass true here as otherwise it just won't return anythig
.WithField("EquipmentId", jsonPath: "$.EquipmentId", isArray: false) // then you scope each field in the array to what you want it to be. Since you want scalar values, pass `isArray: false` for better predictability
.WithField("InquiryValue", jsonPath: "$.InquiryValue[0]", isArray: false) // since your InquiryValue is actually an array, you want to obtain first element here. if you don't do this, fields names and values would go askew
.WithField("Timestamp", jsonPath: "$.Timestamp", fieldType: typeof(DateTime), isArray: false)) // you can also supply field type, otherwise it seems to default to `string`
{
csv.WithFirstLineHeader().Write(json);
}
Console.WriteLine(csvWriter.GetStringBuilder().ToString()); // comment this out if writing to file - you won't need it
}
Update summary:
Pivoted to update the code to rely on JSON Path scoping - this seems to allow for field name manipulation with pretty low effort
Looking at your comment, you could probably simplify your file writer a little bit - use StreamWriter instead of StringWriter - see updated code for example

Here is the working sample of producing CSV from your JSON
string json = #"{
""GpsLocation"": {
""Equipment"": [
{
""EquipmentId"": ""EQ00001"",
""InquiryValue"": [
""IV00001""
],
""Timestamp"": ""2020-02-01 01:01:01.01"",
},
{
""EquipmentId"": ""EQ00002"",
""InquiryValue"": [
""IV00002""
],
""Timestamp"": ""2020-01-01 01:01:01.01""
}
]
}
}";
StringBuilder csv = new StringBuilder();
using (var r = ChoJSONReader.LoadText(json)
.WithJSONPath("$.GpsLocation.Equipment")
.WithField("EquipmentId")
.WithField("InquiryValue", jsonPath: "InquiryValue[0]", fieldType: typeof(string))
.WithField("Timestamp", fieldType: typeof(DateTime))
)
{
using (var w = new ChoCSVWriter(csv)
.WithFirstLineHeader())
w.Write(r);
}
Console.WriteLine(csv.ToString());
Output:
EquipmentId,InquiryValue,Timestamp
EQ00001,IV00001,2/1/2020 1:01:01 AM
EQ00002,IV00002,1/1/2020 1:01:01 AM
Sample fiddle: https://dotnetfiddle.net/hJWtqH

Your code is sound, but the issue is that you're only writing the first variable in the array by using i.GpsLocation.Equipment[0]. Instead, try looping over everything by putting it into a for loop, and changing the [0] to your iterating variable inside of said loop.

Newtonsoft Object serialized to String. JObject instance expected

Hi so am trying to parse this JSON line but i got some others that are like this in files thats why i want to automate this so i can remove the invalid lines to make the file a valid JSON for reading, The problem is that the JSON contains multiple JSON in 1 line
Example:
{"item":"value"}{"anotheritem":"value"}
Is there anyway to remove
{"anotheritem":"value"}
So it turns in to a valid JSON that is readable to start parsing the files
I tried doing using StreamReader cause there in a file i have multiple files that contain these invalid JSON
So i got it to be able to detect the Invalid JSON but for some reason i can't get it to read the JSON so i can use .remove to remove the invalid line
using (StreamReader r = new StreamReader(itemDir))
{
string json = r.ReadToEnd();
if (json.Contains("anotheritem"))
{
JObject NoGood = JObject.FromObject(json);
MessageBox.Show(NoGood.ToString());
}
}
The Error:
Object serialized to String. JObject instance expected.
Thank you all for your time and help.

If each object are side by side without space or any other character, you can convert your string to an json array.
string value = "{\"item\":\"value\"}{\"anotheritem\":\"value\"}";
string arrayValue = "[" + value.Replace("}{", "},{") + "]";
var array = JArray.Parse(arrayValue);
var goopArray = array.OfType<JObject>().Where(o => o.Property("anotheritem") == null);
Edit : see my second answer. More robust solution. More modern. And support dotnet core builtin json serializer.

Json.Net
Even better solution, Json.NET have a builtin feature for this exact scenario. See Read Multiple Fragments With JsonReader
The JsonTextReader have a property SupportMultipleContent that allow to read consecutive items when set to true
string value = "{\"item\":\"value\"}{\"anotheritem\":\"value\"}";
var reader = new JsonTextReader(new System.IO.StringReader(value));
reader.SupportMultipleContent = true;
var list = new List<JObject>();
while (reader.Read())
{
var item = JObject.Load(reader);
list.Add(item);
}
System.Text.Json
If you want to use System.Text.Json, it's also acheivable. They are no SupportMultipleContent property but Utf8JsonReader will do the job for you.
string value = "{\"item\":\"value\"}{\"anotheritem\":\"value\"}";
var bytes = Encoding.UTF8.GetBytes(value).AsSpan();
var list = new List<JsonDocument>();
while (bytes.Length != 0)
{
var reader = new Utf8JsonReader(bytes);
var item = JsonDocument.ParseValue(ref reader);
list.Add(item);
bytes = bytes.Slice((int) reader.BytesConsumed);
}

How to parse json value that is stored in an array and store selected value in array in c#?

I read json values from a text and store it in array using this code.
string[] allLines = System.IO.File.ReadAllLines("D:\\tweets.txt");
I need to extract certain fields from this array containing Json.
My Json is of this type:
{"Name":"John","Id":"45","Time":"11 pm"}
{"Name":"Pear","Id":"34","Time":"3 pm"}
I want to extract each "Name" in one array and each "Id" in one array, something like this.
string[] Name= null;
string[] Id= null;
for (var i = 0; i < allLines[i].length; i++)
{
Name = allLines[i].Name;
Id = allLines[i].Id;
}
I tried another way using json parsing as well. I can obtain one row at one time json deserialized this way. But then confused how to obtain the selected fields.
StreamReader streamReader = System.IO.File.OpenText("D:\\tweets.txt");
string lineContent = streamReader.ReadLine();
do
{
if (lineContent != null)
{
var a = JsonConvert.DeserializeObject(lineContent);
}
lineContent = streamReader.ReadLine();
}
while (streamReader.Peek() != -1);
streamReader.Close();
Please help.

I recommend using Json.NET to parse JSON, you can download it as a NuGet package.
It has some great documentation about querying JSON here
Querying your JSON with LINQ would look something like:
JObject json = JObject.Parse(json);
var name = from p in json
select (string)p["Name"];

This example uses the NewtonSoft Json library to deserialize your Json into an object. Then, linq is used to pull out the lists that you are interested in.
I have written this example as a Console Application in Visual Studio. You will need to create a new Console Application, then copy this code into it.
Also, to use the NewtonSoft library in your new Console Application, you will need to load it from NuGet. To do this in VisualStudio, you will need to
Right-click on the project name
Click on Manage NuGet Packages...
In the search box on the top right, enter "newtonsoft" (without the quotes)
Newtonsoft.Json should show up in the list. Click it and press the Install button. This will load the binary and set up the references in your project. After that, you can use the sample code shown in this example.
static void Main(string[] args)
{
TestParseJson();
Console.WriteLine();
Console.WriteLine("Press Any Key to End");
Console.ReadLine(); // Wait for input so we can see our output text
}
// 1 - Construct an object used for deserialization. You will need to make this class match the format of the records in your json text file.
public class User
{
public string Name { get; set; }
public int Id { get; set; }
public DateTime Time { get; set; }
}
// 2 - Simulate Json creation and then use the NewtonSoft library to deserialize. You will need to just extract from the DeserializeObject line down
public static void TestParseJson()
{
// Read list of json objects from file, one line at a time. The json in the test file users.json is in this format:
// {"Name":"John","Id":45,"Time":"2015-11-05T19:18:02.0324468Z"}
// {"Name":"Pear","Id":34,"Time":"2015-11-06T19:18:02.0329474Z"}
var jsonLines = File.ReadLines(#"c:\temp\users.json");
var deserializedUsers = jsonLines.Select(JsonConvert.DeserializeObject<User>).ToList();
// Use Linq to project the list of deserializedUsers into the lists that you want
var names = deserializedUsers.Select(user => user.Name);
var ids = deserializedUsers.Select(user => user.Id);
// Output list of names and ids for debugging purposes
Console.WriteLine("");
Console.WriteLine(" Names:");
foreach (var name in names)
{
Console.WriteLine(" " + name);
}
Console.WriteLine(" Ids:");
foreach (var id in ids)
{
Console.WriteLine(" " + id);
}
}

Your JSON does not actually represent an array, but rather a series of individual objects back-to-back. To be considered a valid JSON array, the objects would need to be enclosed in square brackets and separated by commas (see JSON.org). Regardless, it is still possible to read and parse this non-standard JSON using Json.Net. The JsonTextReader class has a special SupportMultipleContent setting that is designed to cope with this situation. You can process your file using the following code:
List<string> names = new List<string>();
List<string> ids = new List<string>();
JsonSerializer serializer = new JsonSerializer();
using (StreamReader sr = new StreamReader("D:\\tweets.txt"))
using (JsonTextReader reader = new JsonTextReader(sr))
{
reader.SupportMultipleContent = true;
while (reader.Read())
{
if (reader.TokenType == JsonToken.StartObject)
{
JObject jo = JObject.Load(reader);
names.Add((string)jo["Name"]);
ids.Add((string)jo["Id"]);
}
}
}
Console.WriteLine("Names: " + string.Join(", ", names));
Console.WriteLine("Ids: " + string.Join(", ", ids));
Fiddle: https://dotnetfiddle.net/tYVLLr

Change SSIS variable with reference in C#?

I access a read write a "SSIS variable" with C# in script task (Inside SSIS, an ETL tool. Don't worry about the SSIS part. Lets look at the C# only). Its Dts.Variables["strRope"].Value = "onFire"; Is there any way I could refer
to a SSIS variable without using this big name ? I was thinking of -
Object var = Dts.Variables["strRope"].REFERENCE_TO_VARIABLE;
Now, if I want assign a new value to Dts.Variables["strRope"], can I simply say var = (String) "Ten thousand thundering typhoons"; .
Is there any way I can do such a thing ?
EDIT Code example -
public void Main()
{
object var = Dts.Variables["strRope"].Value;
MessageBox.Show("original value = " + Dts.Variables["strRope"].Value.ToString());//original value = "Hello World"
//Try to change the value of Dts.Variables["strRope"].Value using var ???
var = (object)"Hello cruel world !";
MessageBox.Show("new value = " + Dts.Variables["strRope"].Value.ToString());//new value = ???
Dts.TaskResult = (int)ScriptResults.Success;
}

Note that var is a C# 3.0 (and above) keyword, and you can use it instead of object to save on typing and to avoid explicitly casting the type. Try using it along the following lines,
var strRope = Dts.Variables["strRope"];
if (strRope.Value == "onFire") { ... }
strRope.Value = "Ten thousand thundering typhoons";