We Keep Coding

AWS API Gateway/Lambda using an external API how to retrieve and store data into dynamoDb

Using an open API (im using New York Times book api) create an API Gateway endpoint that does the following:
Retrieves data from an API.
Stores the data in a database.
im able to retrieve the data and store it in a string but im have difficulties extracting that data from the string and storing it into a dynamoDb. Any help is appreciated thank you.
using System;
using System.Collections.Generic;
using System.Dynamic;
using System.Linq;
using System.Net.Http;
using System.Threading.Tasks;
using Newtonsoft.Json;
using Amazon.Lambda.Core;
using Newtonsoft.Json.Linq;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.DocumentModel;
using Amazon.Lambda.APIGatewayEvents;
using Amazon.DynamoDBv2.Model;
// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace last
{
public class Result
{
public string url { get; set; }
public string publication_dt { get; set; }
public string byline { get; set; }
public string book_title { get; set; }
public string book_author { get; set; }
public string summary { get; set; }
public List<string> isbn13 { get; set; }
}
public class Root
{
public string status { get; set; }
public string copyright { get; set; }
public int num_results { get; set; }
public List<Result> results { get; set; }
}
public class Function
{
private static AmazonDynamoDBClient client1 = new AmazonDynamoDBClient();
public static readonly HttpClient client = new HttpClient();
public async Task<ExpandoObject> FunctionHandler(string input, ILambdaContext context)
{
string tblName = "last";
string url = "https://api.nytimes.com/svc/books/v3/reviews.json?title=Becoming&api-key=myKey";
string message = await client.GetStringAsync(url);
Result myDeserializedClass = JsonConvert.DeserializeObject<Result>(message);
var request = new PutItemRequest
{
TableName = tblName,
Item = new Dictionary<string, AttributeValue>()
{
{ "bookId", new AttributeValue { S = "202" }},
{ "publication_dt", new AttributeValue { S = myDeserializedClass.publication_dt.ToString() }},
{ "byline", new AttributeValue { S = myDeserializedClass.byline.ToString() }},
{ "book_title", new AttributeValue { S = myDeserializedClass.book_title.ToString()}},
{ "book_author", new AttributeValue { S = myDeserializedClass.book_author.ToString() }},
{ "summary", new AttributeValue { S = myDeserializedClass.summary.ToString() }},
{ "isbn13", new AttributeValue { S = myDeserializedClass.isbn13.ToString()}},
}
};
await client1.PutItemAsync(request);
//Root myDeserializedClass = JsonConvert.DeserializeObject<Root>(message);
//await tblName.PutItemAsync((Document)myDeserializedClass.ToString());
return JsonConvert.DeserializeObject<ExpandoObject>(message);
}
}
}

How can I parse some JSON dynamically without knowing JSON values?

So I am using TDAmeritrade API to receive stock data with a C# Winforms program on Visual Studio. It takes the user input stock symbol and searches for the info. I am using HttpClient and Newtonsoft.Json and have been able to successfully perform the GET request and receive a JSON string back, but I do not know how to get all of the information I need out of it.
Here is the JSON:
https://drive.google.com/file/d/1TpAUwjyqrHArEXGXMof_K1eQe0hFoaw5/view?usp=sharing
Above is the JSON string sent back to me then formatted. My goal is to record information for each price in "callExpDateMap.2021-02-19:11" and "callExpDateMap.2021-03-19:39". The problem is that for each different stock, the dates that show up in "callExpDateMap" are going to be different.
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
var response = await client.GetAsync(url);
var info = await response.Content.ReadAsStringAsync();
dynamic config = JsonConvert.DeserializeObject<dynamic>(info, new ExpandoObjectConverter());
return config;
This is the code I have right now. I know the last for statement is not correct. How can I parse to the specific sections I want (callExpDateMap.expirationdate.StrikePrice) and get the information needed from each without knowing the dates and Strike prices beforehand? Is there a way to innumerate it and search through the JSON as if it were all a bunch of arrays?

The code below is perhaps not the most elegant nor complete, but I think it will get you going. I would start by using the JObject.Parse() from the Newtonsoft.Json.Linq namespace and take it from there.
JObject root = JObject.Parse(info);
string symbol = root["symbol"].ToObject<string>();
foreach (JToken toplevel in root["callExpDateMap"].Children())
{
foreach (JToken nextlevel in toplevel.Children())
{
foreach (JToken bottomlevel in nextlevel.Children())
{
foreach (JToken jToken in bottomlevel.Children())
{
JArray jArray = jToken as JArray;
foreach (var arrayElement in jArray)
{
InfoObject infoObject = arrayElement.ToObject<InfoObject>();
Console.WriteLine(infoObject.putCall);
Console.WriteLine(infoObject.exchangeName);
Console.WriteLine(infoObject.multiplier);
}
}
}
}
}
public class InfoObject
{
public string putCall { get; set; }
public string symbol { get; set; }
public string description { get; set; }
public string exchangeName { get; set; }
// ...
public int multiplier { get; set; }
// ...
}

This is official documentation of Newtonsoft method you are trying to use.
https://www.newtonsoft.com/json/help/html/Overload_Newtonsoft_Json_JsonConvert_DeserializeObject.htm
If an API's method returns different json propeties and you cannot trust it's property names all the times, then you can try using a deserialize method that returns .Net object, for example: JsonConvert.DeserializeObject Method (String)
https://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_JsonConvert_DeserializeObject.htm
That method's signature is this:
public static Object DeserializeObject(string value)
Parameter is: value of type json string.
Return Value is: Object of type object.
If you do not want an Object, then you can of course use a .Net type you have. Such as this method:
JsonConvert.DeserializeObject Method (String)
Any property that you have in both (the .net type and json object) will get populated. If .net type has properties that do not exist in json object, then those will be ignored. If json object has properties that do not exist in.net, then those will be ignored too.
Here's an example of a .Net type
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Serialization;
using System.Collections.Generic;
using System.IO;
using System.Text;
namespace MyNameSpace
{
public class TDAmeritradeStockData
{
[JsonProperty("symbol")]
public string Symbol { get; set; }
[JsonProperty("status")]
public string Status { get; set; }
[JsonProperty("callExpDateMap")]
public object CallExpDateMap { get; set; }
//...
//...
public CallExpDateMapType[] CallExpDateMapList { get; set; }
}
public class CallExpDateMapType
{
[JsonProperty("expirationdate")]
public string Expirationdate { get; set; }
[JsonProperty("StrikePrice")]
public List<StrikePriceType> StrikePriceList { get; set; }
}
public class StrikePriceType
{
public string StrikePrice { get; set; }
public List<StrikePricePropertiesType> StrikePricePropertiesList { get; set; }
}
public class StrikePricePropertiesType
{
[JsonProperty("putCall")]
public string PutCall { get; set; }
[JsonProperty("symbol")]
public string Symbol { get; set; }
[JsonProperty("description")]
public string Description { get; set; }
[JsonProperty("exchangeName")]
public string ExchangeName { get; set; }
[JsonProperty("bid")]
public double Bid { get; set; }
[JsonProperty("ask")]
public double Ask { get; set; }
//...
//...
}
[TestClass]
public class TestTestTest
{
[TestMethod]
public void JsonTest()
{
var jsondata = ReadFile("data.json");
var model = JsonConvert.DeserializeObject<TDAmeritradeStockData>(jsondata);
JObject jObject = (JObject)model.CallExpDateMap;
var count = ((JObject)model.CallExpDateMap).Count;
model.CallExpDateMapList = new CallExpDateMapType[count];
var jToken = (JToken)jObject.First;
for (var i = 0; i < count; i++)
{
model.CallExpDateMapList[i] = new CallExpDateMapType
{
Expirationdate = jToken.Path,
StrikePriceList = new List<StrikePriceType>()
};
var nextStrikePrice = jToken.First.First;
while (nextStrikePrice != null)
{
var nextStrikePriceProperties = nextStrikePrice;
var srikePriceList = new StrikePriceType
{
StrikePrice = nextStrikePriceProperties.Path,
StrikePricePropertiesList = JsonConvert.DeserializeObject<List<StrikePricePropertiesType>>(nextStrikePrice.First.ToString())
};
model.CallExpDateMapList[i].StrikePriceList.Add(srikePriceList);
nextStrikePrice = nextStrikePrice.Next;
}
jToken = jToken.Next;
}
Assert.IsNotNull(model);
}
private string ReadFile(string fileName)
{
using (var fileStream = new FileStream(fileName, FileMode.Open, FileAccess.Read))
{
var data = new StringBuilder();
using (var streamReader = new StreamReader(fileStream))
{
while (!streamReader.EndOfStream) data.Append(streamReader.ReadLine());
streamReader.Close();
}
fileStream.Close();
return data.ToString();
}
}
}
}

How to assign value to variable from a google API Json Response c#

I am trying to assign the value of a key from an async JSON response to a variable, the JSON key in question is the "threatType" Key. I am querying google's safebrowsing v4 API and I get a good response, the problem is when I try to assign a key to a variable nothing is assigned. Here's my code:
public static async Task<string> CheckUrl( string Api_Key, string MyUrl)
{
safeBrowsing_panel i = new safeBrowsing_panel();
var service = new SafebrowsingService(new BaseClientService.Initializer
{
ApplicationName = "Link-checker",
ApiKey = Api_Key
});
var request = service.ThreatMatches.Find(new FindThreatMatchesRequest()
{
Client = new ClientInfo
{
ClientId = "Link-checker",
ClientVersion = "1.5.2"
},
ThreatInfo = new ThreatInfo()
{
ThreatTypes = new List<string> { "SOCIAL_ENGINEERING", "MALWARE" },
PlatformTypes = new List<string> { "ANY_PLATFORM" },
ThreatEntryTypes = new List<string> { "URL" },
ThreatEntries = new List<ThreatEntry>
{
new ThreatEntry
{
Url = MyUrl
}
}
}
});
var response = await request.ExecuteAsync();
string json = JsonConvert.SerializeObject(await request.ExecuteAsync());
string jsonFormatted = JToken.Parse(json).ToString(Formatting.Indented);
Console.WriteLine(jsonFormatted);
return jsonFormatted;
}
I created classes to parse the json:
public class ThreatRes
{
public string threatType;
}
public class RootObjSB
{
public List<ThreatRes> matches;
}
And I call it with:
string SB_Result = await CheckUrl("MyAPIKey", "Bad_URL");
RootObjSB obj = JsonConvert.DeserializeObject<RootObjSB>(SB_Result);
The JSON response from google:
{
"matches": [
{
"cacheDuration": "300s",
"platformType": "ANY_PLATFORM",
"threat": {
"digest": null,
"hash": null,
"url": "http://badurl.com/",
"ETag": null
},
"threatEntryMetadata": null,
"threatEntryType": "URL",
"threatType": "SOCIAL_ENGINEERING",
"ETag": null
}
],
"ETag": null
}
I need help please.

So I tried using JavaScriptSerializer and it seemed to work just fine, I also reconstructed my classes to parse all the properties on the JSON response.
Reconstructed Classes:
public class Threat
{
public object digest { get; set; }
public object hash { get; set; }
public string url { get; set; }
public object ETag { get; set; }
}
public class Match
{
public string cacheDuration { get; set; }
public string platformType { get; set; }
public Threat threat { get; set; }
public object threatEntryMetadata { get; set; }
public string threatEntryType { get; set; }
public string threatType { get; set; }
public object ETag { get; set; }
}
public class RootObjSB
{
public List<Match> matches { get; set; }
public object ETag { get; set; }
}
and i deserialize it like this:
string SB_Result = await CheckUrl("MyAPIKey", "Bad_URL");
var obj = new JavaScriptSerializer().Deserialize<RootObjSB>(SB_Result);
string threat = obj.matches[0].threatType;
Console.WriteLine(threat);
I really don't know why the first option I tried didn't parse the data correctly but this was how I overcame that problem. Hope it helps anyone going through the same thing

Reading JSON file with Array

I'm currently working on JSON string extraction using C#. My JSON string consist of an array with repetitive keys. Not sure if I'm describing it right since I'm new to this.
This is my JSON string
{"Index":
{ "LibraryName":"JGKing"
, "FormName":"AccountsPayable"
, "User":null
, "FilingPriority":null
, "FileDescription":null
, "Fields":
{"Field":
[
{ "Name":"invItemID"
, "Value":"6276"
}
,{ "Name":"invEntityCode"
, "Value":"16"
}
,{ "Name":"invVendorCode"
, "Value":"MIRUB01"
}
,{ "Name":"invNumber"
, "Value":"PWD5"
}
,{ "Name":"invDate"
, "Value":"2017-08-21"
}
,{ "Name":"invStatus"
, "Value":""
}
,{ "Name":"invCurrencyCode"
, "Value":"AU"
}
,{ "Name":"invCurrencyRate"
, "Value":"1"
}
,{ "Name":"invTax"
, "Value":"454.3"
}
, {"Name":"invTotal"
, "Value":"4997.27"
}
, {"Name":"invReceivedDate"
, "Value":"2017-09-06"
}
,{ "Name":"InvoiceLine1"
, "Value":"{\r\n \"invLineNumber\": \"1\",\r\n \"invPONumber\": \"\",\r\n \"invPOLineNo\": \"1\",\r\n \"invPOJobID\": \"\",\r\n \"invCostCode\": \"\",\r\n \"invCategory\": \"\",\r\n \"invGLCode\": \"61-01-49-6862.517\",\r\n \"invDescription\": \"\",\r\n \"invEntryType\": \"\",\r\n \"invAmount\": \"4542.97\",\r\n \"invTaxAmount\": \"454.3\",\r\n \"invTaxCode\": \"GST\",\r\n \"invAmountIncTax\": \"4997.27\"\r\n}"}]}}}
I need to extract the value of invItemID key which is inside the array.
I tried to serialize my json string from a class but it returns null in the List<>
Here's my code
public void CFExport(string jsonFile)
{
string ItemIDField;
string ItemIDValue;
using (StreamReader r = new StreamReader(jsonFile))
{
JsonSerializer s = new JsonSerializer();
var Idx = (JSONMain)s.Deserialize(r, typeof(JSONMain));
var flds = (Fields)s.Deserialize(r, typeof(Fields));
if (flds != null)
{
foreach (var _field in flds.Field)
{
ItemIDField = _field.Name;
ItemIDValue = _field.Value;
}
}
}
}
public class JSONMain
{
public Index Index { get; set; }
}
public class Index
{
public string LibraryName { get; set; }
public string FormName { get; set; }
public string User { get; set; }
public string FilingPriority { get; set; }
public string FileDescription { get; set; }
}
public class Fields
{
public List<Field> Field { get; set; }
}
public class Field
{
public string Name { get; set; }
public string Value { get; set; }
}
I hope you can help me.
Thanks in advance

Try to reflect the JSON file with your classes like this:
public class Index
{
public string LibraryName { get; set; }
public string FormName { get; set; }
public string User { get; set; }
public string FilingPriority { get; set; }
public string FileDescription { get; set; }
public Fields Fields { get; set; } //this line makes the difference
}
If you deserialize now, the fields should be populated automatically. I also advice you to use JsonConvert.Deserialze<>() since it is a bit easier (see documentation) and you are new to this topic.
Getting the value of invItemID could look like this:
public void CFExport(string jsonFile)
{
string ItemIDField = "invItemID";
string ItemIDValue;
using (StreamReader r = new StreamReader(jsonFile))
{
var Idx = JsonConvert.DeserializeObject<JSONMain>(r);
foreach(var field in Idx.Index.Fields.Field)
{
if(field.Name == ItemIDField)
{
ItemIDValue = field.Value;
}
}
}
}
Whoohoo. My first answer on Stackoverflow! I hope this helps you.

I need to extract the value of invItemID key which is inside the array.
You can retrieve value for your specified key invItemID from Field array by using JObject.
So you have no more need to manage classes for your json.
Here i created a console app for your demonstration purpose.
class Program
{
static void Main(string[] args)
{
//Get your json from file
string json = File.ReadAllText(#"Your path to json file");
//Parse your json
JObject jObject = JObject.Parse(json);
//Get your "Field" array to List of NameValuePair
var fieldArray = jObject["Index"]["Fields"]["Field"].ToObject<List<NameValuePair>>();
//Retrieve Value for key "invItemID"
string value = fieldArray.Where(x => x.Name == "invItemID").Select(x => x.Value).FirstOrDefault();
//Print this value on console
Console.WriteLine("Value: " + value);
Console.ReadLine();
}
}
class NameValuePair
{
public string Name { get; set; }
public string Value { get; set; }
}
Output:

Getting variable data from string in c#

I have got data downloaded from url it is as follows.
//[
{
"id": "2932675",
"t": "GNK",
"e": "LON",
"l": "915.00",
"l_fix": "915.00",
"l_cur": "GBX915.00",
"s": "0",
"ltt": "5:08PM GMT",
"lt": "Dec 11,
"5": 08PM
"GMT",
"
"lt_dts": "2015-12-11T17:08:26Z",
"c": "-7.50",
"c_fix": "-7.50",
"cp": "-0.81",
"cp_fix": "-0.81",
"ccol": "chr",
"pcls_fix": "922.5"
}
]
and want following variable t : GNK and l:915 from above string and done following
void method1()
{
string scrip = textBox1.Text;
string s;
WebClient wc = new WebClient();
string url = ("http://finance.google.com/finance/infoclient=ig&q=NSE:" + scrip);
s = wc.DownloadString(url);
textBox2.Text = s.Substring(58, 6);
textBox3.Text = s;
}
public class LatestPrice
{
public string id { get; set; }
public string Name { get; set; }
public string type { get; set; }
public string l { get; set; }
public string l_fix { get; set; }
public string l_cur { get; set; }
public string s { get; set; }
public string lt { get; set; }
public string lt_dts { get; set; }
public string c { get; set; }
public string c_fix { get; set; }
public string cp { get; set; }
public string cp_fix { get; set; }
public string ccol { get; set; }
public string pcls_fix { get; set; }
}
public string[][] convert_string()
{
string[][] stockprice = null;
string stockprice1 = null;
string getdownloadstr = getstring();
stockprice1 = getdownloadstr.Replace("//", "").Trim();
var v = JsonConvert.DeserializeObject<List<LatestPrice>>(stockprice1);
}
i have made changes to program -- but how to access the t : gnk value or l = 915 value

You can convert to JArray to JObject and can get the value directly through JOject parameter key.
string jsonStr = "[{ \"id\":\"2932675\", \"t\" : \"GNK\" , \"e\" : \"LON\" , \"l\" : \"915.00\" , \"l_fix\" : \"915.00\" , \"l_cur\" : \"GBX915.00\" , \"s\": \"0\" , \"ltt\":\"5:08PM GMT\" , \"lt\" : \"Dec 11 5:08PM GMT\"}]";
var obj = JsonConvert.DeserializeObject<JArray>(jsonStr).ToObject<List<JObject>>().FirstOrDefault();
Console.WriteLine("t = " + obj["t"]);
Console.WriteLine("l = " + obj["l"]);
The above print the output as
t = GNK
l = 915.00
You can refer to this fiddle created for your question.

Parse it either split at , and then (split on : ) add each line to dictionary using the t as the key and the other as the value.
Then you could simply access by t and also by l .
Split the entire string by commas you have a list of item : value, then split on colon and add to dictionary. Then look up info in dictionary getvalue = Dictionary[key] ;

You could use Regex to match the data you need:
string t = Regex.Match(str, "\"t\" : \"[A-Z]{3}\"").Value;
string l = Regex.Match(str, "\"l\" : \"\\d{3}.\\d{2}\"").Value;
Where str is the data string you have downloaded.
String t matches a substring that is in the format "t" : "XXX", where XXX can contain any uppercase characters.
String l matches a substring that is in the format "l" : "XXX.XX", where XXX.XX can contain any digit.

1 . Add NewtonSoft.Json in your Reference in Solution Explorer. Steps(Reference [rightClick]-> Manage NuGetPackage -> Search for "Json.Net" -> Install them)
Add using Newtonsoft.Json;
code :
public class CurrentValue
{
public string id { get; set; }
public string Name { get; set; }
public string type { get; set; }
public string l { get; set; }
public string l_fix { get; set; }
public string l_cur { get; set; }
public string s { get; set; }
public string lt { get; set; }
public string lt_dts { get; set; }
public string c { get; set; }
public string c_fix { get; set; }
public string cp { get; set; }
public string cp_fix { get; set; }
public string ccol { get; set; }
public string pcls_fix { get; set; }
}
Create a method and use following code
Uri url = new Uri("http://www.google.com/finance/info?q=NSE%3A" + NameofCompany);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "application/json; charset=utf-8";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string Responsecontent = new StreamReader(response.GetResponseStream()).ReadToEnd();
string CurrentContent = Responsecontent.Replace("//", "").Trim();
var v = JsonConvert.DeserializeObject<List<CurrentValue>>(CurrentContent);
Now "var v" have all the data of the class "CurrentValue".
You can load a xml file containing all " NameofCompany " and Load it on FormLoad. Use a for loop to get data of all " NameofCompany "

i have done in follwing way
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Web;
using System.Timers;
using System.IO;
using System.Net;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System.Text.RegularExpressions;
namespace google_downloader
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
WebClient wc = new WebClient();
string s= wc.DownloadString("http://finance.google.com/finance/info?client=ig&q=NSE:sbin");
//index for t
int index7= s.IndexOf('t');
int index8 = s.IndexOf('e');
textBox1.Text = ("frist index is" + index7 + "second indes is " + index8);
textBox1.Text = s.Substring(index7+6,(index8-index7)-10);
}
}
}