We Keep Coding

C# how to insert JSON into Google BigQuery table schema

I have an array of complex JSON objects with embedded arrays of JSON objects. By way of an example, one JSON object might look like...
{
"shopID": "12194",
"name": "Shop Name",
"timeZone": "US\/Eastern",
"timeStamp": "2020-11-30T13:25:13+00:00",
"Contact": {
"contactID": "90",
"Addresses": [
"ContactAddress": {
"address1": "123 Some Street",
"address2": "",
"city": "A city",
"state": "A state",
"zip": "12345",
"country": "United States",
"countryCode": "US"
}
],
"Emails": "",
"Websites": {
"ContactWebsite": {
"url": "www.companyname.com"
}
}
},
"TaxCategory": {
"taxCategoryID": "2",
"isTaxInclusive": "false",
"tax1Name": "Tax region name",
"tax1Rate": "0.07"
}
}
I cannot modify the above format... I am calling a public API that is returning this as a response.
Assuming that the corresponding table schema has been created in Google BigQuery, how do I simply insert this JSON as a row in to the table?
My code looks like this...
bqc = BigQueryClient.Create("a-project-id");
var trShops = new TableReference()
{
ProjectId = "a-project-id",
DatasetId = "a-dataset-name",
TableId = "Shops"
};
# Call to my API data source that returns a JSON array of Shops...
content = await _lsrApiHelper.GetRawDataAsync(uri, currentOffset, _apiLimit);
# After call to API I deserialize the JSON response...
response = JsonSerializer.Deserialize<ShopResponseRoot>(content);
# Then I want loop through Shops in the List<ShopResponse> and insert them into BigQuery...
foreach (ShopResponse shop in response.Shop)
{
var newRow = new BigQueryInsertRow();
newRow.InsertId = "shopID";
newRow.Add(shop); # This is where I am struggling
bqc.InsertRow(trShops, newRow);
}
I am struggling with working out how to use a JSON or C# object to populate the data that I want inserting into BigQuery.
Any and all guidance greatly appreciated.

How do you search a JSON file in C# to find a specific value?

I have a json file that looks like this:
{
"id": 1,
"FirstName": "jen",
"LastName": "may",
"UserName": "jenmay",
"Password": "august",
"securityQ1": "leeds",
"securityQ2": "smith"
}{
"id": 2,
"FirstName": "lucy",
"LastName": "reed",
"UserName": "lucyreed1",
"Password": "bucket",
"securityQ1": "manchester",
"securityQ2": "bow"
}
I want to read the file to check that a username and password match a users input. I have the input as uname and pword.
I first attempt to deserialize the JSON.
public List<User> DeserializedUsersList(string json)
{
List<User> user = JsonConvert.DeserializeObject<List<User>>(json);
return user;
}
it keeps throwing me a 'Newtonsoft.Json.JsonSerializationException' exception.
I'm really new to C# and I'm not sure how to find username in the file. Then check that password matches the inputted password. How should I go about this?
Thank you!

Your JSON is a bit off, meaning, you're missing commas between objects and square brackets around it. If it is an array of objects, you can use Linq to query specific user.
[{
"id": 1,
"FirstName": "jen",
"LastName": "may",
"UserName": "jenmay",
"Password": "august",
"securityQ1": "leeds",
"securityQ2": "smith"
},{
"id": 2,
"FirstName": "lucy",
"LastName": "reed",
"UserName": "lucyreed1",
"Password": "bucket",
"securityQ1": "manchester",
"securityQ2": "bow"
}
]
and your code would look something like this,
var user = JsonConvert.DeserializeObject<List<User>>(json);
var selectedUser = user.Where(x => x.UserName.Equals("jenmay")).FirstOrDefault();

Json.Net support multiple content deserialization when using underlying JsonTextReader class.
var json = #"
{""id"": 1,""FirstName"": ""jen"",""LastName"": ""may"",""UserName"": ""jenmay"",""Password"": ""august"",""securityQ1"": ""leeds"",""securityQ2"": ""smith""}
{""id"": 2,""FirstName"": ""lucy"",""LastName"": ""reed"",""UserName"": ""lucyreed1"",""Password"": ""bucket"",""securityQ1"": ""manchester"",""securityQ2"": ""bow""}";
var jsonSerializer = new JsonSerializer();
using var reader = new JsonTextReader(new System.IO.StringReader(json));
reader.SupportMultipleContent = true;
var users = new List<User>();
while (reader.Read())
{
var item = (User)jsonSerializer.Deserialize(reader, typeof(User));
users.Add(item);
}

Is it possible to obtain a parameter's value from JSON response

This is my first post.
I am trying to integrate Shippo to our application and wish to obtain a specific value out of a Json response.
I am getting the following Json response:
{
"count": 3,
"next": null,
"previous": null,
"results": [
{
"object_created": "2019-08-28T12:58:57.064Z",
"object_id": "16b602e0ajdsk87c4313920bc5e3174XYZ",
"object_owner": "some#email.com",
"shipment": "bd62234e151244dab2b2152fdcd15e76",
"attributes": [
"FASTEST"
],
"amount": "31.30",
"currency": "USD",
"amount_local": "31.30",
"currency_local": "USD",
"provider": "USPS",
"provider_image_75": "https://shippo-static.s3.amazonaws.com/providers/75/USPS.png",
"provider_image_200": "https://shippo-static.s3.amazonaws.com/providers/200/USPS.png",
"servicelevel": {
"name": "Priority Mail Express",
"token": "usps_priority_express",
"terms": ""
},
"estimated_days": 1,
"arrives_by": null,
"duration_terms": "Overnight delivery to most U.S. locations.",
"messages": [],
"carrier_account": "4e1506b8b7f7449e90620967e45aa1e9",
"test": false,
"zone": "4"
},
{
"object_created": "2019-08-28T12:58:57.063Z",
"object_id": "ebdee42047aa49a3b7e08b1903ea02ea",
"object_owner": "some#email.com",
"shipment": "bd62234e151244dab2b2152fdcd15e76",
"attributes": [
"BESTVALUE",
"CHEAPEST"
],
"amount": "7.49",
"currency": "USD",
"amount_local": "7.49",
"currency_local": "USD",
"provider": "USPS",
"provider_image_75": "https://shippo-static.s3.amazonaws.com/providers/75/USPS.png",
"provider_image_200": "https://shippo-static.s3.amazonaws.com/providers/200/USPS.png",
"servicelevel": {
"name": "Priority Mail",
"token": "usps_priority",
"terms": ""
},
"estimated_days": 2,
"arrives_by": null,
"duration_terms": "Delivery within 1, 2, or 3 days based on where your package started and where it’s being sent.",
"messages": [],
"carrier_account": "4e1506b8b7f7449e90620967e45aa1e9",
"test": false,
"zone": "4"
},
{
"object_created": "2019-08-28T12:58:57.062Z",
"object_id": "ad410a41c84940ee80eb30c41c507613",
"object_owner": "some#email.com",
"shipment": "bd62234e151244dab2b2152fdcd15e76",
"attributes": [],
"amount": "7.78",
"currency": "USD",
"amount_local": "7.78",
"currency_local": "USD",
"provider": "USPS",
"provider_image_75": "https://shippo-static.s3.amazonaws.com/providers/75/USPS.png",
"provider_image_200": "https://shippo-static.s3.amazonaws.com/providers/200/USPS.png",
"servicelevel": {
"name": "Parcel Select",
"token": "usps_parcel_select",
"terms": ""
},
"estimated_days": 7,
"arrives_by": null,
"duration_terms": "Delivery in 2 to 8 days.",
"messages": [],
"carrier_account": "4e1506b8b7f7449e90620967e45aa1e9",
"test": false,
"zone": "4"
}
]
}
I am using this call:
WebClient webClient = new WebClient();
webClient.Headers.Add("Authorization: ShippoToken " + authToken);
var result = JsonValue.Parse(webClient.DownloadString("https://api.goshippo.com/shipments/"+ theObject.ObjectId + "/rates/USD"));
My question is how could I get just the "amount" value from the response?
Thank you for any help!

Instead of recreating the json layout as a class model you can work with Newtonsofts JToken. Which will leave you with less boilerplate code to create:
var json = "your json data";
// Parse the whole json string
var obj = JObject.Parse(json);
// Extract the results data as JArray
var results = (JArray)obj["results"];
// Iterate over all array entries and get the amount property
foreach(var result in results)
{
var amount = result["amount"].ToString();
// Do something with the amount
}

You could create some classes based on json then deserialize using JsonConvert
e.g https://www.newtonsoft.com/json/help/html/DeserializeObject.htm
tip:
Copy the json and put in json viewer like https://jsonformatter.curiousconcept.com/

Use https://www.newtonsoft.com/json to deserialize the json response. If you didn't want that you could use a series of string.Split()'s to get the amount. But using newtonsoft (via the nuget package manager) would be the easiest.

How can I use a JSONPath expression as a filter inside another JSONPath expression?

I want to select objects from a JSON string by filtering using a JSONPath expression with another expression embedded in the filter. In other words, I want to filter for a value that is present elsewhere in the JSON data.
For example:
In the following JSON data there is a value in $.Item.State.stepId (currently "QG2.0"). I need to have a JSONPath expression that selects values based on this value, like this:
$..Step[?(#.stepId==$Item.State.stepId)].actionDate
But this will not return any results. If I use the string ("QG2.0") directly like this:
$..Step[?(#.stepId=='QG2.0')].actionDate
it will return the required data.
What's wrong, or is it not even possible? My JSON is below:
{
"Item": {
"Common": {
"folio": "PSH-000016020",
"setName": "123-XZ200-1",
"wfId": "Kat1_002",
"wfIssue": "002",
"wfIdIssue": "Kat1_002.002"
},
"State": {
"status": "IN WORK",
"stepId": "QG2.0",
"stepDescription": "Validation"
},
"Participants": {
"Participant": [
{
"role": "PR",
"roleDescription": "Product Responsible",
"loginName": "marc102",
"email": "mark#abc.de"
}, {
"role": "CR",
"roleDescription": "Chapter Responsible",
"loginName": "uli26819",
"email": "uli#abc.de"
}
]
},
"Steps": {
"Step": [
{
"stepId": "QG1.0",
"stepTitle": "Preparation",
"actionDate": "2016-06-28T10:28:09",
"actionDueDate": "",
"actionBy_Name": "Marc",
"actionBy_Account": "marc102",
"action": "complete",
"Comment": ""
}, {
"stepId": "QG2.0",
"stepTitle": "Check Requirements",
"actionDate": "2016-08-08T14:17:04",
"actionDueDate": "",
"actionBy_Name": "Uli",
"actionBy_Account": "uli26819",
"action": "complete",
"Comment": ""
}
]
}
}
}

I don't think Json.Net's implementation of JSONPath supports this concept.
However, you can still get the information you want if you break the query into two steps:
JObject obj = JObject.Parse(json);
JToken stepId = obj.SelectToken("Item.State.stepId");
JToken actionDate = obj.SelectToken(string.Format("$..Step[?(#.stepId=='{0}')].actionDate", stepId));
Console.WriteLine(actionDate.ToString());
Fiddle: https://dotnetfiddle.net/KunYTf

JSON.NET reader problem

i got some problem when i generate .json file from spring .json ,and i got this format
{ "models": [
{
"id":1,
"modelName":"dfdsf",
"created":{
"userFullname":"demo",
"time":"150301",
"date":"20110208",
"userId":"123"
},
"remark":"dsfdf",
"updated":"~unique-id~1"
},
{
"id":2,
"modelName":"test",
"created":{
"userFullname":"test",
"time":"150301",
"date":"20110210",
"userId":"124"
},
"remark":"test",
"updated":{
"userFullname":"test",
"time":"150301",
"date":"20110209",
"userId":"test"
}
}
]}
first time i used JObject Parse for convert
JObject job = JObject.Parse(fullJson);
and the other hand i used jtoken to focus "models"
JToken jDetail = job["models"];
but the problem is {[{ xxx }]} it look like jarray i don't have any idea to convert it
i ever use JArray, JsonTextReader but it doesn't work.
could suggestion some? because if i pass this ll i set some value to object.
thank you for every idea.

string fullJson = File.ReadAllText("TextFile1.txt"); // for brevity
var job = JObject.Parse(fullJson);
var models = job.Value<JArray>("models");
Console.WriteLine(models[0]);
result:
{
"id": 1,
"modelName": "dfdsf",
"created": {
"userFullname": "demo",
"time": "150301",
"date": "20110208",
"userId": "123"
},
"remark": "dsfdf",
"updated": "~unique-id~1"
}