So I.m storing data in dynamoDb as a list of maps. I can easily store the data just by using SaveAsync(ClassObject). But I cant fetched the data. This is the data I'm trying to fetch. But in the database it is Storing with AttributeValue such as [m:{s:{subservice : writer}}]
{
"Column" : "Writing",
"Requirements" : [
{
"subService" : "writer",
"description" : "info why writer"
},
{
"subService" : "blogger",
"description" : "info why blogger"
}
]
}
JArray req = new JArray();
string columnValue = data["Column"].ToObject<string>();
ScanRequest request = FilterRequest(tableName, "Column", columnValue );
//FilterRequest is just a scanrequest
var response = await dbClient.ScanAsync(request);
var serviceRequirements = response.Items[0];
foreach (var requirement in serviceRequirements["Requirements"])
{
JObject subCat = new JObject();
subCat.Add("subService", requirement["subService"].S.ToString());
subCat.Add("description", requirement["description"].S.ToString());
req.Add(requirement);
}
return req;
I'm stuck at this for two days. And I cannot find any reference to my problem. Im just simply want to fetch.
it took lot of hit and trial...but it works now
just want to share the answer, hope it helps someone
JArray req = new JArray();
string columnValue = data["Column"].ToObject<string>();
ScanRequest request = FilterRequest(tableName, "Column", columnValue );
//FilterRequest is just a scanrequest
var response = await dbClient.ScanAsync(request);
var serviceRequirements = response.Items[0]["Requirements"].L.ToArray();
var out1 = serviceRequirements[0].M.Values.ToArray();
foreach(var element in serviceRequirements)
{
JObject subCat = new JObject();
subCat.Add("subService",element.M.Values.ToArray()[0].S.ToString());
subCat.Add("description",element.M.Values.ToArray()[1].S.ToString());
req.Add(subCat);
}
return req;
this returns
[
{
"subService" : "writer",
"description" : "info why writer"
},
{
"subService" : "blogger",
"description" : "info why blogger"
}
]
Related
Problem description
I need to receive JSON array of objects (with almost any shape) and store them to ES database using function IndexMany (or some similar bulk indexing function). I found some clumsy solution with one drawback - it doesn't set _id property in ES correctly (according to the id property in JSON object).
And also I would like to know if there is any more elegant way how to achieve my goal without casting every JArray item to string and back to ExpandoObject.
Additional info
Elasticsearch DB 7.5.1
NEST (7.6.1)
Newtonsoft.Json (12.0.3)
TLDR
Is there any elegant solution of following code:
var settings = new ConnectionSettings(new Uri("http://localhost:9200")).DefaultIndex("people");
var client = new ElasticClient(settings);
// this string represents incoming JSON message
string json = #"[
{
""id"": ""1"",
""name"": ""Andrej"",
""surname"": ""Burak"",
""dob"": ""1921-11-10T00:00:00+00:00""
},
{
""id"": ""2"",
""name"": ""Franta"",
""surname"": ""Dobrota"",
""dob"": ""1933-10-05T00:00:00+00:00""
},
{
""id"": ""3"",
""name"": ""Milos"",
""surname"": ""Ovcacek"",
""dob"": ""1988-05-05T00:00:00+00:00""
}
]";
JArray jArray = JArray.Parse(json);
foreach (var jtoken in jArray)
{
var jobj = (JObject)jtoken;
jobj.Add("DbCreated", JToken.FromObject(DateTime.UtcNow));
jobj.Add("DbCreatedBy", JToken.FromObject("authors name"));
}
//working, but seems to me a bit too clumsy to convert every item to string and then back to dynamic object
var converter = new ExpandoObjectConverter();
dynamic[] dlst = jArray.Select(t => (dynamic)JsonConvert.DeserializeObject<ExpandoObject>(t.ToString(), converter)).ToArray();
//not working cast
dynamic[] dlstNW = jArray.ToObject<dynamic[]>();
var indexManyResponse = client.IndexMany(dlst); //working partially (but not using ID as index)
var indexManyResponseNotWorking = client.IndexMany(jArray); //not working
var indexManyResponseNotWorking2 = client.IndexMany(dlstNW); //not working
// expected behavior
dynamic[] jsondyn = new dynamic[]
{
new { Id = "1", Name = "foo" },
new { Id = "2", Name = "bar" },
new { Id = "3", Name = "baz" },
};
var indexManyResponseWithIndex = client.IndexMany(jsondyn); //working perfectly, but don't know how to acieve this
After #gnud pointed me to low-level client I think I have found my answer. Important thing is to build request manually like it was explained here.
var settings = new ConnectionSettings(new Uri("http://localhost:9200")).DefaultIndex("people").DisableDirectStreaming();
var client = new ElasticClient(settings);
// this string represents incoming JSON message
string json = #"[
{
""id"": ""1"",
""name"": ""Andrej"",
""surname"": ""Burak"",
""dob"": ""1921-11-10T00:00:00+00:00""
},
{
""Id"": ""2"",
""name"": ""Franta"",
""surname"": ""Dobrota"",
""dob"": ""1933-10-05T00:00:00+00:00""
},
{
""Id"": ""3"",
""name"": ""Milos"",
""surname"": ""Ovcacek"",
""dob"": ""1988-05-05T00:00:00+00:00""
}
]";
JArray jArray = JArray.Parse(json);
// I have to build my request manually
List<string> esRequest = new List<string>();
foreach (var jtoken in jArray)
{
var jobj = (JObject)jtoken;
jobj.Add("DbCreated", JToken.FromObject(DateTime.UtcNow));
jobj.Add("DbCreatedBy", JToken.FromObject("authors name"));
string id;
// Ensure that Id is set even if we receive it in lowercase
if( jobj["Id"] == null)
{
if( jobj["id"] == null)
{
throw new Exception("missing Id");
}
id = jobj["id"].Value<string>();
}
else
{
id = jobj["Id"].Value<string>();
}
string indexName = "people";
esRequest.Add($"{{\"index\":{{\"_index\":\"{indexName}\",\"_id\":\"{id}\"}}}}");
esRequest.Add(jobj.ToString(Formatting.None));
}
PostData pd = PostData.MultiJson(esRequest);
var llres = client.LowLevel.Bulk<BulkResponse>(pd);
Hope it helps someone.
I am using Json.Net to parse my JSON
This is my JSON:
"OptionType": {
"C": [
"C",
"Call"
],
"P": [
"P",
"Put"
]
}
Before this step, when processed, as a result, I would get a value from any of this.
For example Option Type: Call
Whatever value I get, I need it to be transcodified according to the above JSON.
Example: Option Type: C
First of all your sample data is not a valid JSON. You need to wrap it to the curvy braces.
If your sample is a part of the JSON object, you can map OptionType to the Dictionary<string, List<string>>.
To find valid option you will need to iterate this dictionary like in the sample below:
var valueToCheck = "Call";
string type = null;
foreach (var kvp in optionTypes)
{
if (kvp.Value.Contains(valueToCheck))
{
type = kvp.Key;
break;
}
}
Same using JObject with fixed JSON data:
var json = #"{
""OptionType"":{
""C"": [
""C"",
""Call""
],
""P"": [
""P"",
""Put""
]
}
}";
var valueToCheck = "Call";
string type = null;
var ot = JObject.Parse(json);
var objectType = ot["OptionType"];
foreach (var token in objectType)
{
var prop = token.ToObject<JProperty>();
var key = prop.Name;
var values = prop.Value.ToObject<List<string>>();
if (values.Contains(valueToCheck))
{
type = key;
break;
}
}
Code is not perfect but it is just an idea what to do.
You need to iterate over properties of JObject and then search your option type and then replace your search option with its parent key.
This is custom function can do above task.
public static JObject GetJObject(JObject jObject, List<string> optionType)
{
foreach (var type in jObject["OptionType"])
{
var key = type.ToObject<JProperty>().Name;
var values = type.ToObject<JProperty>().Value.ToObject<List<string>>();
foreach (var option in optionType)
{
if (values.Contains(option))
{
int index = values.IndexOf(option);
values.RemoveAt(index);
values.Insert(index, key);
}
}
JToken jToken = JToken.FromObject(values);
jObject.SelectToken("OptionType." + key).Replace(jToken);
}
return jObject;
}
And you can use above custom function this like
string json = File.ReadAllText(#"Path to your json file");
JObject jObject = JObject.Parse(json);
List<string> optionType = new List<string> { "Call", "Put" };
JObject result = GetJObject(jObject, optionType);
Output:
I need to add one more node to Json string.
Following is the code from where I am reading the data.
var url = "https://xyz_12232_abc/0908978978979.json";
var sys = new WebClient();
var content = sys.DownloadString(url);
I received following output from above code:
{
"2312312312313":
{
"emailId":"abc#gmail.com",
"model":"XYZ001",
"phone":"+654784512547",
"userName":"User1"
},
"23456464512313":
{
"emailId":"abcd#gmail.com",
"model":"XYZ002",
"phone":"+98745114474",
"userName":"User2"
},
"45114512312313":
{
"emailId":"abcde#gmail.com",
"model":"XYZ3",
"phone":"+214784558741",
"userName":"User3"
}
}
But, I want this output like below:
{
"Records": [
{
"UID":"2312312312313":,
"emailId":"abc#gmail.com",
"model":"XYZ001",
"phone":"+654784512547",
"userName":"User1"
},
{
"UID":"23456464512313":,
"emailId":"abcd#gmail.com",
"model":"XYZ002",
"phone":"+98745114474",
"userName":"User2"
},
{
"UID":"45114512312313":,
"emailId":"abcde#gmail.com",
"model":"XYZ3",
"phone":"+214784558741",
"userName":"User3"
}
]
}
Now, how can it be achieved ?
You can use Json.NET to massage the data into your desired output:
var jsonStr = #"..."; // your JSON here
var obj = JsonConvert.DeserializeObject<Dictionary<string, JObject>>(jsonStr);
var formattedObj = new
{
Records = obj.Select(x =>
{
x.Value.AddFirst(new JProperty("UID", x.Key));
return x.Value;
})
};
// serialize back to JSON
var formattedJson = JsonConvert.SerializeObject(formattedObj);
I have a JArray which looks as follows:
[
{
"key": "S8710 Server",
"value": "M"
},
{
"key": "Java",
"value": "M"
}
]
This JArray needs to be converted into a JObject by taking the key and value such that the output object looks like this:
{
"S8710 Server": "M",
"Java": "M"
}
Is this conversion Possible? Any help would be greatly appreciated.
What I tried is extracting the keys and values from a DataTable and serializing the result of that. Then I tried JObject.Parse.
var skillList = from skill in ds.Tables[4].AsEnumerable()
where skill.Field<Int64>("ResumeID") == applicantValue.ResumeID
select new clsResume.ExtractRatingInfo
{
key = skill.Field<string>("Skill"),
value = skill.Field<string>("SkillRating")
};
string skillResult = JsonConvert.SerializeObject(skillList, Newtonsoft.Json.Formatting.None);
JObject obj = JObject.Parse(skillResult);
Yes, you can transform your JSON like this:
JObject result = new JObject(
jArray.Select(jt => new JProperty((string)jt["key"], jt["value"]))
);
Fiddle: https://dotnetfiddle.net/CcLj3D
Note: the keys must be distinct across all objects in the array for this to work properly.
Create JToken by passing the content string into JArray`s static method Parse.
get the inner childs from JToken and then look for the properties and their values.
like this:
JArray jArray = JArray.Parse(YOUR_CONTENT_GOES_HERE);
foreach (var token in jArray.Children<JToken>())
{
var firstProp = token.Children<JProperty>().First(param => param.Name == "S8710 Server");
var firstValue = firstProp.Value<string>();
var secondProp = token.Children<JProperty>().First(param => param.Name == "Java");
var secondValue = firstProp.Value<string>();
}
I'm working on a project in C# in which I want to enter a search term, hit the search button and then retrieve parts of the response from Google to an array so I can iterate through them.
Searching Google using their JSON-based API is pretty easy
var client = new HttpClient();
var address = new Uri("https://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=" + term);
HttpResponseMessage response = await client.GetAsync(address);
String stream = await response.Content.ReadAsStringAsync();
This returns a JSON string like the one below (Results for the term "Test search")
{
"responseData":{
"results":[
{
"GsearchResultClass":"GwebSearch",
"unescapedUrl":"http://en.wikipedia.org/wiki/Wikipedia:Search_engine_test",
"url":"http://en.wikipedia.org/wiki/Wikipedia:Search_engine_test",
"visibleUrl":"en.wikipedia.org",
"cacheUrl":"http://www.google.com/search?q\u003dcache:g6KEStELS_MJ:en.wikipedia.org",
"title":"Wikipedia:\u003cb\u003eSearch\u003c/b\u003eengine\u003cb\u003etest\u003c/b\u003e-Wikipedia,thefreeencyclopedia",
"titleNoFormatting":"Wikipedia:Searchenginetest-Wikipedia,thefreeencyclopedia",
"content":"A\u003cb\u003esearch\u003c/b\u003eengine\u003cb\u003etest\u003c/b\u003ecannothelpyouavoidtheworkofinterpretingyourresultsanddecidingwhattheyreallyshow.Appearanceinanindexaloneisnotusually\u003cb\u003e...\u003c/b\u003e"
},
{
"GsearchResultClass":"GwebSearch",
"unescapedUrl":"http://techcrunch.com/2008/07/16/google-continues-to-test-a-search-interface-that-looks-more-like-digg-every-day/",
"url":"http://techcrunch.com/2008/07/16/google-continues-to-test-a-search-interface-that-looks-more-like-digg-every-day/",
"visibleUrl":"techcrunch.com",
"cacheUrl":"http://www.google.com/search?q\u003dcache:r2laSUVQw8kJ:techcrunch.com",
"title":"GoogleContinuesTo\u003cb\u003eTest\u003c/b\u003eA\u003cb\u003eSearch\u003c/b\u003eInterfaceThatLooksMoreLike\u003cb\u003e...\u003c/b\u003e",
"titleNoFormatting":"GoogleContinuesToTestASearchInterfaceThatLooksMoreLike...",
"content":"Jul16,2008\u003cb\u003e...\u003c/b\u003eAcoupleofdaysagowepostedscreenshotsofanew\u003cb\u003esearch\u003c/b\u003einterfacebeingbucket\u003cb\u003etested\u003c/b\u003ebyGooglethatletsusersvoteupordownon\u003cb\u003e...\u003c/b\u003e"
},
{
"GsearchResultClass":"GwebSearch",
"unescapedUrl":"http://googleblog.blogspot.com/2006/04/this-is-test-this-is-only-test.html",
"url":"http://googleblog.blogspot.com/2006/04/this-is-test-this-is-only-test.html",
"visibleUrl":"googleblog.blogspot.com",
"cacheUrl":"http://www.google.com/search?q\u003dcache:Ozl1cQzRT0IJ:googleblog.blogspot.com",
"title":"Thisisa\u003cb\u003etest\u003c/b\u003e.Thisisonlya\u003cb\u003etest\u003c/b\u003e.|OfficialGoogleBlog",
"titleNoFormatting":"Thisisatest.Thisisonlyatest.|OfficialGoogleBlog",
"content":"Apr24,2006\u003cb\u003e...\u003c/b\u003eFromtimetotime,werunliveexperimentsonGoogle—\u003cb\u003etests\u003c/b\u003evisibletoarelativelyfewpeople--todiscoverbetterwaysto\u003cb\u003esearch\u003c/b\u003e.Wedothis\u003cb\u003e...\u003c/b\u003e"
},
{
"GsearchResultClass":"GwebSearch",
"unescapedUrl":"http://alistapart.com/article/testing-search-for-relevancy-and-precision",
"url":"http://alistapart.com/article/testing-search-for-relevancy-and-precision",
"visibleUrl":"alistapart.com",
"cacheUrl":"http://www.google.com/search?q\u003dcache:02Sjrd5mb0YJ:alistapart.com",
"title":"\u003cb\u003eTestingSearch\u003c/b\u003eforRelevancyandPrecision·AnAListApartArticle",
"titleNoFormatting":"TestingSearchforRelevancyandPrecision·AnAListApartArticle",
"content":"Sep22,2009\u003cb\u003e...\u003c/b\u003eDespitethefactthatsite\u003cb\u003esearch\u003c/b\u003eoftenreceivesthemosttraffic,it\u0026#39;salsotheplacewheretheuserexperiencedesignerbearstheleastinfluence."
}
],
"cursor":{
"resultCount":"1,010,000,000",
"pages":[
{
"start":"0",
"label":1
},
{
"start":"4",
"label":2
},
{
"start":"8",
"label":3
},
{
"start":"12",
"label":4
},
{
"start":"16",
"label":5
},
{
"start":"20",
"label":6
},
{
"start":"24",
"label":7
},
{
"start":"28",
"label":8
}
],
"estimatedResultCount":"1010000000",
"currentPageIndex":0,
"moreResultsUrl":"http://www.google.com/search?oe\u003dutf8\u0026ie\u003dutf8\u0026source\u003duds\u0026start\u003d0\u0026hl\u003den\u0026q\u003dTest+search",
"searchResultTime":"0.23"
}
},
"responseDetails":null,
"responseStatus":200
}
How do I get the value of url in each node pushed into an array so I can iterate through it?
You can use dynamic keyword with Json.Net
dynamic jObj = JsonConvert.DeserializeObject(json);
foreach (var res in jObj.responseData.results)
{
Console.WriteLine("{0} => {1}\n",res.title,res.url);
}
You can use Linq too
var jObj = (JObject)JsonConvert.DeserializeObject(json);
string[] urls = jObj["responseData"]["results"]
.Select(x => (string)x["url"])
.ToArray();