We Keep Coding

How to set field to null in Elasticsearch using C# NEST + script?

For example we have following document in elastic:
{
"name": "Bob",
"age": "22",
"phrase": "ohohoho",
"date": "2022-10-20T00:00:00Z"
}
string phrase ;
DateTime? date;
Then we want put following:
{
"name": "not Bob",
"age": "22",
"phrase": null,
"date": null
}
in c#:
var updateRequest = new UpdateRequest<T, T>(entity)
{
ScriptedUpsert = true,
Script = new InlineScript(
$"if (someCondition) {{ctx._source.putAll(params.entity);}} else {{ctx.op = \"noop\";}}")
{
Lang = "painless",
Params = new Dictionary<string, object>() { { "entity", entity } },
},
Upsert = Activator.CreateInstance<T>()
};
but in the end it will not update phrase and date.
It makes following request:
POST /myIndex/_update/b90278fd-1a66-40bf-b775-d076122c6c02
{
"script": {
"source": ""if (someCondition) {{ctx._source.putAll(params.entity);}} else {{ctx.op = \"noop\";}}"",
"lang": "painless",
"params": {
"entity": {
"name": "",
"age": 22
}
}
},
"upsert": {
"age": 0
}
}
Idk why but it skips all fields with null.
How to update nullable fields to null?

NEST does not support sending null values by default.
You can have a check in script such that if a value is not passed then you can remove it from document.
var updateRequest = new UpdateRequest<T, T(entity)
{
ScriptedUpsert = true,
Script = new InlineScript($"if (params.entity.phrase==null)ctx._source.remove('phrase');")
{
Lang = "painless",
Params = new Dictionary<string, object>() { { "entity", entity } },
},
Upsert = Activator.CreateInstance<T>()
};
You can check for more details here

C# MVC Return List in Formatted Json

Have a list that returns in Json but want to format it as below. Can't figure out for the life of me why I can't seem to get it formatted right.
[{
"AppFormName": "TFB Test Application",
"Questions": [{
"QuestionName": "How old are you?",
"QuestionType": 1
},
{
"QuestionName": "Where are you from?",
"QuestionType": 1
}]
},
{
"AppFormName": "HLL",
"Questions": [{
"QuestionName": "How old are you?",
"QuestionType": 1
},
{
"QuestionName": "Where are you from?",
"QuestionType": 1
},
{
"QuestionName": "What Game are you applying for?",
"QuestionType": 2
},
{
"QuestionName": "Do you agree to the clan rules",
"QuestionType": 3
}]
}]
This is what my code produces currently:
[
{
"AppFormName": "TFB Test Application",
"QuestionName": "How old are you?",
"QuestionType": 1
},
{
"AppFormName": "TFB Test Application",
"QuestionName": "Where are you from?",
"QuestionType": 1
},
{
"AppFormName": "HLL",
"QuestionName": "How old are you?",
"QuestionType": 1
},
{
"AppFormName": "HLL",
"QuestionName": "Where are you from?",
"QuestionType": 1
},
{
"AppFormName": "HLL",
"QuestionName": "What Game are you applying for?",
"QuestionType": 2
},
{
"AppFormName": "HLL",
"QuestionName": "Do you agree to the clan rules",
"QuestionType": 3
}
]
This is my controller where I'm trying to serialize the results;
public ActionResult AccessToken(string authorizationCode)
{
UserFunctions.AccessToken(authorizationCode);
var results = UserFunctions.userApplications;
return Json(results , JsonRequestBehavior.AllowGet);
}
Model:
public class UserApplications
{
public string AppFormName { get; set; }
public string QuestionName { get; set; }
public int QuestionType { get; set; }
}
Function to get data;
public static List<Models.UserApplications> GetUserApplications(string ClientId)
{
userApplications.Clear();
var getUserApplications = getUserApplicationsSQL;
using (var conn = new MySqlConnection(dataConn))
{
using (var cmd = new MySqlCommand(getUserApplications, conn))
{
cmd.CommandType = CommandType.Text;
cmd.Parameters.AddWithValue("#DiscordClientID", ClientId);
cmd.Connection.Open();
using (var dr = cmd.ExecuteReader(CommandBehavior.CloseConnection))
{
while (dr.Read())
{
var u = new Models.UserApplications
{
AppFormName = dr["AppFormName"].ToString(),
QuestionName = dr["QuestionName"].ToString(),
QuestionType = dr.GetInt32(dr.GetOrdinal("TypeID")),
};
userApplications.Add(u);
}
}
}
}
return userApplications;
}

You need code to translate from your existing data structure into one which will produced the desired JSON structure. This can be accomplished relatively easily using Linq. And then you need to return that list in your action method:
public ActionResult AccessToken(string authorizationCode)
{
UserFunctions.AccessToken(authorizationCode);
var results = UserFunctions.userApplications;
var appFormList = results.GroupBy(s => s.AppFormName).Select(g => new
{
AppFormName = g.Key, Questions = g.Select(a => new
{
a.QuestionName,
a.QuestionType
})
});
return Json(appFormList, JsonRequestBehavior.AllowGet);
}
Live demo: https://dotnetfiddle.net/sRIy72
(Credit to this post for the idea to use Linq.)

This can be simply achieved by Grouping:
public ActionResult AccessToken(string authorizationCode)
{
UserFunctions.AccessToken(authorizationCode);
List<UserApplications> results = UserFunctions.userApplications;
var data = from p in results
group p by p.AppFormName into g
select new { AppFormName = g.Key, Questions = g.Select(x=> new { QuestionName = x.QuestionName, QuestionType = x.QuestionType}).ToList() };
return Json(data , JsonRequestBehavior.AllowGet);
}

How to create json string from C# list of object with specific properties?

Consider I have below values in the drop-down (dropDownList variable) and user selected values are in selectedDropDownValues list.
And DB api returns the list of customers (customerDBList variable).
Now the requirement is to build JSON from selected values as mentioned below -
var dropDownList = new[] { "Customer.Id", "Customer.FirstName", "Customer.LastName", "Customer.Address.AddressLine1", "Customer.Address.AddressLine2" };
var selectedDropDownValues = new[] { "Customer.Id", "Customer.FirstName", "Customer.Address.AddressLine1" };
var customerDBList = new List<Customer>(){
new Customer {
Id=1,
FirstName="John",
LastName="Desouza",
Address=new Address{
AddressLine1="1 Street",
AddressLine2="Linking Road"
}},
new Customer {
Id=2,
FirstName="Sam",
LastName="Lewis",
Address=new Address{
AddressLine1="Fedral Highway",
AddressLine2="Louisville"
}
}};
Expected JSON output as -
[
{
"Customer": {
"Id": 1,
"FirstName": "John",
"Address": {
"AddressLine1": "1 Street"
}
}
},
{
"Customer": {
"Id": 2,
"FirstName": "Sam",
"Address": {
"AddressLine1": "Fedral Highway"
}
}
}
]

As it happens, your selectedDropDownValues strings "Customer.Address.AddressLine1" are in JSONPath syntax, so you could convert your Customer objects to an intermediate JObject then prune unwanted values using JsonExtensions.RemoveAllExcept(this JObject obj, IEnumerable<string> paths) from this answer to How to perform partial object serialization providing "paths" using Newtonsoft JSON.NET.
Note that your desired JSON has an extra level of nesting { "Customer": { ... } } not present in your data model, so it will need to be manually inserted before filtering:
var rootName = "Customer";
var query = customerDBList
// Convert to JObject
.Select(c => JObject.FromObject(c))
// Add additional level of object nesting { "Customer": { ... } }
.Select(o => new JObject( new JProperty(rootName, o)))
// Remove all but selected properties.
.Select(o => o.RemoveAllExcept(selectedDropDownValues));
var json = JsonConvert.SerializeObject(query, Formatting.Indented);
Working sample .Net fiddle here which shows the generated JSON to be, as required,
[
{
"Customer": {
"Id": 1,
"FirstName": "John",
"Address": {
"AddressLine1": "1 Street"
}
}
},
{
"Customer": {
"Id": 2,
"FirstName": "Sam",
"Address": {
"AddressLine1": "Fedral Highway"
}
}
}
]

linq query where some from values might be missing

I'm pulling contacts from JSON, there are contacts inside Data that might contain emails and phone numbers.
If they do the following query works perfectly:
var results = from d in json.Data
from c in d.Contacts
from e in c.Emails
from p in c.Phones
select new IoModel {Email = e.EmailValue, Id = d.Id, FullName = string.IsNullOrEmpty(c.Name) ? c.Name : d.DisplayName, Phone = p.PhoneNumber, DOB = d.CustomDOB};
If the Data doesn't have Emails or Phones then it runs with no error but it doesn't return the Contact Name.
Is there a way to do this without using if and and's to keep the code shorter and cleaner?
Here is an example of the JSON with a missing email:
{
"data": [
{
"addresses": [],
"contacts": [
{
"created_by": "user_ltUL3eXGPRWb5ghDeGTfOe9qjW0LeE2e4ouopLcSSWj",
"date_created": "2017-12-28T17:13:00.392000+00:00",
"date_updated": "2017-12-28T17:13:58.453000+00:00",
"emails": [],
"id": "cont_6hxxhz51ctlTnHfo8gA8cce0rthS1dJy1kguKAj148s",
"integration_links": [
{
"name": "LinkedIn Search",
"url": "https://www.linkedin.com/search/results/index/?keywords=John%20Doe"
}
],
"lead_id": "lead_3UzfxCgQHmw4BlGwElitHhlP6E7q9Tg3sdSkTl1CIXp",
"name": "John Doe",
"organization_id": "orga_PvgGx1opSZDsCHl73P8OoSFZUlJ3qsNGI2kwgoObM17",
"phones": [
{
"phone": "+15555555555",
"phone_formatted": "+1 555-555-5555",
"type": "mobile"
}
],
"title": "Main Service",
"updated_by": "user_ltUL3eXXPRWb5shDeGTfOe9qjP0LeE2e4ouopLcSSWj",
"urls": []
}
],
"created_by": "user_ltUL3eXXPRWb5shDeGTfOe9qjW0LeE2e4ouopLcSSWP",
"created_by_name": "John",
"custom": {
"Date Of Birth": "1901-08-01",
"Department": "Main Service",
"Initial Service": "Main Service"
},
"custom.lcf_ufMH5ZhlR99zcdvJxKMxdgxcIbV4wtgTb3EdWDEkL8g": "1971-08-17",
"date_created": "2017-12-28T17:13:00.388000+00:00",
"date_updated": "2017-12-29T21:41:53.840000+00:00",
"description": "",
"display_name": "John Doe",
"html_url": "https://app.close.io/lead/lead_3UzfxCgQHmw4BlGwElitHhlP6E7q9Tg3sdSkTl1CIXp/",
"id": "lead_3UzfxCgQHmw4BlGwElitHhlP5E7q9Tg3sdSkTl1CIXp",
"integration_links": [
{
"name": "Google Search",
"url": "http://google.com/search?q=John%20Doe"
}
],
"name": "",
"opportunities": [],
"organization_id": "orga_AvcGx4opSZDsCHl73H8OogDDUlJ3qsNGI2kwgoObA17",
"status_id": "stat_LpJu8FO72WgIob4qDDVnS4GEieoU41zmQ8xBquTvusm",
"status_label": "Established Patient",
"tasks": [],
"updated_by": "user_0JFRnl8QRvRhMhAlLz4JJxgmrzPeLs3xboxYyj5Pm80",
"updated_by_name": "BT",
"url": null
}
],
"has_more": false,
"total_results": 1
}

Since I don't know what you are deserializing your JSON into, I can only guess at a solution here, but I think if you use DefaultIfEmpty() on your Emails and Phones collections and then use the null-conditional operator on the EmailValue and PhoneNumber, you should get the result you want:
var results = from d in json.Data
from c in d.Contacts
from e in c.Emails.DefaultIfEmpty()
from p in c.Phones.DefaultIfEmpty()
select new IoModel
{
Email = e?.EmailValue,
Id = d.Id,
FullName = string.IsNullOrEmpty(c.Name) ? c.Name : d.DisplayName,
Phone = p?.PhoneNumber,
DOB = d.CustomDOB
};

Let me explain the issue you are running into. Imagine the following two lists:
Issue
var ids = new List<int> { 1, 2, 3, 4 };
var names = new List<string>();
Let's write a Linq query to get all the ids along with names:
var idsAndNames = from id in ids
from name in names
select new { Id = id, Name = name };
The above query will return no results. Why? This is more obvious if you were to use a foreach instead of Linq. The foreach version will look roughly like below:
foreach (var id in ids)
{
foreach (var name in names)
{
//select new { Id = id, Name = name };
}
}
It is very clear and obvious why no results will be returned in the foreach version above (the code within the inner loop will not be executed because the names list is empty).
Fix
We need to tell our query: If there are no items in names collection, we will accept the default. We can do that using DefaultIfEmpty method.
var idsAndNames = from id in ids
from name in names.DefaultIfEmpty()
select new { Id = id, Name = name };
The above will return 4 results. It is sort of like doing the following:
foreach (var id in ids)
{
if (names.Any())
{
//select new { Id = id, Name = name };
}
else
{
//select new { Id = id, Name = "" }; // <-- Notice empty string
}
}
Answer to your question
#BrianRogers has answered your question but I elaborated on what the issue is.

No Data Shown On JQGrid

I'm fairly new to MVC and Jquery. for last couple of days I was trying to use Jqgrid http://www.trirand.com/blog/ to show data in my database. I use EF Code first to create my only class 'Author'
public class Author
{
public int AuthorID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string FullName
{
get
{
return FirstName+ " "+LastName ;
}
}
}
and this is my 'AuthorController' which create Json data:
public ActionResult LinqGridData(string sidx, string sord, int page, int rows)
{
var jsonData = new
{
total = 5,
page = 1,
records = db.Authors.Count(),
rows = db.Authors.Select(a => new
{
id = a.AuthorID,
cell = new { a.AuthorID, a.FirstName, a.LastName }
}
)
};
return Json(jsonData, JsonRequestBehavior.AllowGet);
}
I also tried different method to get my Json data:
public ActionResult LinqGridData (string sidx, string sord, int page, int rows)
{
var jsonData = new {
total = 5,
page=1,
records = db.Authors.Count(),
rows = (from a in db.Authors
select new
{
id = a.AuthorID,
cell = new { a.AuthorID, a.FirstName, a.LastName }
}
)
};
return Json(jsonData,JsonRequestBehavior.AllowGet);
}
and here is my JavaScript, which I use in my view:
$(function () {
$("#list").jqGrid({
url: '/Author/LinqGridData',
datatype:'json',
mtype: 'GET',
colNames:['Author ID', 'First Name', 'Last Name'],
colModel:[
{name:'AuthorID',index:'AuthorID',width:55 },
{name:'FirstName',index:'FirstName',width:90 },
{name:'LastName',index:'LastName',width:80,align:'right' }
],
pager:'#pager',
rowNum: 10,
rowList:[5, 10, 20, 30],
sortname:'AuthorID',
sortorder:'desc',
viewrecords:true,
gridview:true,
caption:'Author List'
});
});
jQuery("#datagrid").jqGrid('navGrid', '#navGrid', { edit: true, add: true, del: true });
I can show the grid with dummy data. with this action method:
public ActionResult GridData(string sidx, string sord, int page, int rows)
{
var jsonData = new
{
total = 1, // we'll implement later
page = 1,
records = 3, // implement later
rows = new[]
{
new {id = 1, cell = new[] {"1", "-7", "Is this a good question?"}},
new {id = 2, cell = new[] {"2", "15", "Is that a good question?"}},
new {id = 3, cell = new[] {"3", "23", "Why is the sky in the sky?"}}
}
};
return Json(jsonData,JsonRequestBehavior.AllowGet);
}
the problem is whenever I want to show the data from my database, I only can show the grid itself not the data.
I tried to convert the json data toList() or toArary() before sending to the view, same result. I hope I made myself clear.
any help would be much appreciated.

Have you tried jsonReader with "repeatitems: false" ?
For example:
$("#list").jqGrid({
url: '/Author/LinqGridData',
datatype:'json',
jsonReader: {
repeatitems: false
},
.....
UPDATE:
If you look at the response body returned from your LinqGridData method, it looks like this:
{"total":5,"page":1,"records":1,"rows":
[
{"id":"1","cell":{"AuthorID":"1","FirstName":"Mike","LastName":"Lee"}}, .....
]
}
But I think it should be like this:
{"total":5,"page":1,"records":1,"rows":
[
{"id":"1","cell":{"1", "Mike", "Lee"}}, .....
]
}
Actually this is your "dummy data" version's response body.
I'll post my working example below. In this example, I did not use 'cell' attribute.
At server side:
var myQuery = from t in db.Customers
select t;
var jsonData = new
{
total = totalPages,
page = pageNum,
records = totalRecords,
rows = myQuery.ToList()
};
return Json(jsonData, JsonRequestBehavior.AllowGet);
At client side:
{
url: '#Url.Action("GetList")',
datatype: 'json',
jsonReader: {
repeatitems: false
},
mtype: 'GET',
colModel: [
{
name: 'CustomerID', label: 'ID', hidden: false, width: 40, sortable: true
},
{
name: 'CompanyName', label: 'Company', width: 100, align: 'center'
},
]
}
Hope this helps.

I finally managed to show data from my db. the problem was in my query. i use this method for my action method:
public JsonResult LinqGridData (string sidx, string sord, int page, object rows)
{
var authors = (from a in db.Authors select a).ToList();
var jsonData = new {
total = 5,
page=1,
records = db.Authors.Count(),
rows = authors.Select(a => new
{
id = a.AuthorID,
cell = new[] { a.AuthorID.ToString(), a.FirstName, a.LastName }
}
)
};
return Json(jsonData,JsonRequestBehavior.AllowGet);
}
and i used this in my Javascript:
$(function () {
$("#list").jqGrid({
url: "/Author/LinqGridData",
datatype: "json",
jsonReader: {
root: "rows",
page: "page",
total: "total",
records: "records",
cell: "cell",
id:"id"
},
mtype: "GET",
colNames: ["Author ID", "First Name", "Last Name"],
colModel: [
{ name: "AuthorID", key: true, width: 55 },
{ name: "FirstName", width: 80 },
{ name: "LastName", width: 100, align: "right" }
],
pager: "#pager",
rowNum: 10,
rowList: [5, 10, 20],
sortname: "AuthorID",
sortorder: "desc",
viewrecords: true,
gridview: true,
width: 610,
height: 250,
caption: "Author List"
});
});
jQuery("#list").jqGrid("navGrid", "#pager", { edit: true, add: true, del: true });
yes , you can omit the 'cell' attribute and reduce your json data. you can also put 'id':0 which normally means treat the first item as the id. i also used 'key' attribute, but it's redundant. you can also pass your query result as Array too. if anyone can tell me is there any benefit of using List over others would much appreciated.
Thanks for your help
good luck