We Keep Coding

Converting Kusto client response to list of objects returns empty objects

I am trying to convert the response received from running a Kusto query in which I am retrieving the schema of a table. My Kusto query looks like this:
tableName | getschema
The response for such a query, as seen in the Kusto Explorer looks something like this (incomplete)
Back in my C# code I have defined the following class:
public class DatasetColumn
{
/// <summary>
/// Name of the column
/// </summary>
public string? ColumnName { get; set; }
/// <summary>
/// Position of this column within the schema
/// </summary>
public int ColumnOrdinal { get; set; }
/// <summary>
/// Type of data contained in this column
/// </summary>
public string? DataType { get; set; }
/// <summary>
/// Type of data contained in this column
/// </summary>
public string? ColumnType { get; set; }
}
And I am attempting to just retrieve a list of DatasetColumn objects from the IDataReader using a method defined in Kusto.Cloud.Data.ExtendedDataReader named ToEnumerable:
using (var client = storageClientFactory.CreateCslQueryProvider(new Kusto.Data.KustoConnectionStringBuilder(connectionDetails!.ClusterUrl)))
{
var schemaQuery = KustoSchemaQueryBuilder.GetKustoSchemaQuery(connectionDetails!.EntityName);
var clientRequestProperties = new ClientRequestProperties() { ClientRequestId = Guid.NewGuid().ToString() };
var queryTask = client.ExecuteQueryAsync(connectionDetails.DatabaseName, schemaQuery, clientRequestProperties);
using (var reader = await queryTask.ConfigureAwait(false))
{
using (reader)
{
return reader.ToEnumerable<DatasetColumn>().ToArray();
}
}
}
Hoping that I will get an array of DatasetColumn objects. I do in fact get a list of DatasetObjects, the number of objects in the list corresponds to the number of results that I am expecting but all the objects have all the fields set to the default values (nulls for strings and 0 for ints).
What am I doing wrong and how do I get this method to return properly initialized objects? Am I misunderstanding the point of this extension method?
Later edit: I am adding a minimal program to reproduce the issue
using Kusto.Cloud.Platform.Data;
using Kusto.Data;
using Kusto.Data.Net.Client;
using Newtonsoft.Json;
namespace ConsoleApp1
{
class DatasetColumn
{
public string? ColumnName { get; set; }
public int ColumnOrdinal { get; set; }
public string? DataType { get; set; }
public string? ColumnType { get; set; }
}
internal class Program
{
static void Main(string[] args)
{
var sb = new KustoConnectionStringBuilder("https://help.kusto.windows.net/Samples; Fed=true; Accept=true");
using (var client = KustoClientFactory.CreateCslQueryProvider(sb))
{
using (var result = client.ExecuteQuery("StormEvents | getschema"))
{
var cols = result.ToEnumerable<DatasetColumn>().ToArray();
Console.WriteLine(JsonConvert.SerializeObject(cols));
}
}
}
}
}

the underlying implementation for the extension method of the DataReader that your code is using, and that is provided as part of the client library, doesn't currently support properties or private fields.
You could either change the properties in the class DatasetColumn to fields, or write your own implementation.
e.g., replace this:
public int ColumnOrdinal { get; set; }
with this:
public int ColumnOrdinal;

Incorrect mapping with Dapper, .NET Core and Postgres

Thanks in advance. I'm seeing an issue with my mapping using Postgres and Dapper. I'm trying to map an Organization object, which has a one-to-many relationship with the Location object. The following is my mapping code:
public Organization GetOrganizationById(long id)
{
var query = #"SELECT
o.organization_id,
o.guid,
o.name,
o.is_active,
o.created,
l.location_id,
l.org_id,
l.guid,
l.internal_identifier,
l.name,
l.address,
l.city,
l.state,
l.zip,
l.phonenumber,
l.is_active,
l.opendate,
l.closedate,
l.created
FROM organization AS o
INNER JOIN location as l
ON o.organization_id = l.org_id
WHERE o.organization_id = #Id";
using (var con = new NpgsqlConnection(_connectionString))
{
var orgResult = con.Query<Organization, List<Location>, Organization>(
query,
(org, locations) =>
{
org.Locations = locations;
return org;
},
new {Id = id},
splitOn: "location_id").FirstOrDefault();
return orgResult;
}
}
I have the following objects created:
public class Organization
{
public long OrganizationId { get; set; }
public Guid Guid { get;set; }
public string Name { get; set; }
public bool IsActive { get; set; }
public DateTime Created { get; set; }
//[JsonIgnore]
public List<Location> Locations { get; set; }
}
and
public class Location
{
public long LocationId { get; set; }
public long OrgId { get; set; }
public Guid Guid { get; set; }
public string InternalIdentifier { get; set; }
public string Name { get; set; }
public string Address { get; set; }
public string City { get; set; }
public string State { get; set; }
public string Zip { get; set; }
public string PhoneNumber { get; set; }
public bool IsActive { get; set; }
public DateTime OpenDate { get; set; }
public DateTime? CloseDate { get; set; }
public DateTime Created { get; set; }
[JsonIgnore]
public Organization Organization { get; set; }
}
Now the issue is, when I get the results it's not even close to accurate. Though the query when copied into a SQL client it's returning the correct results.
Here's what I see in the JSON response along with what's incorrect:
{
"organizationId": 0, // This is incorrect. The organization ID is one (this appears to be the default int/long)
"guid": "4fc55437-8497-4388-be48-c6b8c5dfee93", // This is correct
"name": "TestOrg", // This is correct
"isActive": false, // This is incorrect. Both locations are active, as is the organization
"created": "2021-01-27T05:20:42.287925", // This is correct
"locations": [] // Incorrect, there should be two locations present
}
Anyway, is there anything you think I'm missing with the mapping that would prevent these records from not mapping to their POCOs correctly?

Dapper requires your model property names to mirror your table (not case sensitive).
You can try using the [ColumnName()] attribute above your properties. I've heard this works for some but it didn't for me. I ended up using AutoMapper and created a mapper object.
[ColumnName("SomeFieldName")]
public string SomeProperty { get; set; }
Before you look into that (cause it's a pain) try using Dapper's QueryMultiple() method.
Dapper's Query Multiple
You may have to play around with it a bit. It's been awhile since I've written the full QueryMultiple() method the Dapper way. I made a wrapper class to containerize my dapper functions (I just containerize all 3rd party packages incase of exceptions or something). Here's one of my methods using QueryMultiple(). to return 2 data sets using out parameters. You can adjust this to return as may sets as you need.
/// <summary>
/// Executes a query with multiple results and stores the result sets in the out parameters.
/// </summary>
/// <typeparam name="T1">The type of the first result set.</typeparam>
/// <typeparam name="T2">The type of the second result set.</typeparam>
/// <typeparam name="P">The parameter type. Generally an autonomous/dynamic object, a <see cref="DynamicParameters"/>, or an <see cref="ExpandoObject"/>.</typeparam>
/// <param name="sql">The SQL query string or stored procedure.</param>
/// <param name="parameters">The parameter(s) for the stored procedure.</param>
/// <param name="results1">The first result set.</param>
/// <param name="results2">The second result set.</param>
/// <param name="queryType">
/// <para>The query's command type.</para>
/// Defaults to <strong><see cref="CommandType.StoredProcedure"/></strong>.
/// </param>
public void ExecuteQueryMultiple<T1, T2, P>(string sql, P parameters,
ConnStringKey connectionKey, // This is a personal app setting enum...
out List<T1> results1, out List<T2> results2,
CommandType queryType = CommandType.StoredProcedure)
{
using (IDbConnection connection = new
SqlConnection(configurations.GetConnectionString(connectionKey)))
{
using (SqlMapper.GridReader sqlReader = connection.QueryMultiple(sql,
parameters, commandType: queryType, commandTimeout: ConnectionTimeout))
{
results1 = sqlReader.Read<T1>().AsList();
results2 = sqlReader.Read<T2>().AsList();
}
}
}
You can use something similar if your SQL was like this:
SELECT
o.organization_id,
o.guid,
o.name,
o.is_active,
o.created,
FROM organization AS o
WHERE o.organization_id = #Id
SELECT
l.location_id,
l.org_id,
l.guid,
l.internal_identifier,
l.name,
l.address,
l.city,
l.state,
l.zip,
l.phonenumber,
l.is_active,
l.opendate,
l.closedate,
l.created
FROM location AS l
WHERE l.org_id = #Id
Of course, then you'd have to aggregate the two result sets in the code.
Usage:
var someParams = new { Id = someId };
sql = "your sql";
_sql.ExecuteQueryMultiple<Organization, Location, dynamic>(
sql, someParams,
// this is an enum of mine that ties back to my appsettings.json or app.config file to get the connection string
MyConfigurationConnectionKey,
out List<Organization> organizations,
out List<Location> locations,
CommandType.Text);
// Aggregate the two data sets...
organizations.Locations = locations; // or however you need to do it

Deserializing JSON using C# to return items

I have the following:
{"documents":
[{"keyPhrases":
[
"search results","Azure Search","fast search indexing","sophisticated search capabilities","Build great search experiences","time-sensitive search scenarios","service availability","managed service","service updates","index corruption","near-instantaneous responses","multiple languages","integrated Microsoft natural language stack","multiple indexes","application changes","ranking models","great relevance","years of development","primary interaction pattern","storage","Bing","data volume","rich","suggestions","hassle of dealing","Reliable throughput","website","incremental cost","complexity","faceting","traffic","mobile apps","business goals","users","applications","user expectations","Office"
],
"id":"1"}],
"errors":[]
}
I need to extract the items within the keyPhrases, but have absolutely no idea how to do it.
I have tried the following :
KeyPhraseResult keyPhraseResult = new KeyPhraseResult();
/// <summary>
/// Class to hold result of Key Phrases call
/// </summary>
public class KeyPhraseResult
{
public List<string> keyPhrases { get; set; }
}
keyPhraseResult = JsonConvert.DeserializeObject<KeyPhraseResult>(content);
content contains the JSON string above.
However the keyPhraseResult returns a null value.
Could any body help me in the right direction ?
Thank you.

public class Document
{
public List<string> keyPhrases { get; set; }
public string id { get; set; }
}
public class RootObject
{
public List<Document> documents { get; set; }
public List<object> errors { get; set; }
}
You should have this structure:
var result = JsonConvert.DeserializeObject<RootObject>(content);

Sending data to the server with fnServerParams and aoData for jquery DataTable does not work in MVC4

I want to send extra data to serverside (ASP.Net MVC4) for my jquery datatable. There are many examples on how to this client side, but I can't get it to work on the serverside.
Here's the code:
javascript:
$(document).ready(function () {
var oTable = $('#myDataTable').dataTable({
"bServerSide": true,
"sAjaxSource": "SearchPatient/DataHandler",
"fnServerParams": function (aoData) {
alert('in fnServerParams');
aoData.push( { "name": "more_data", "value": "my_value" } );
}
});
});
Note: the alert goes off, so the function itself is working.
My model class:
/// <summary>
/// Class that encapsulates most common parameters sent by DataTables plugin
/// </summary>
public class JQueryDataTableParamModel
{
/// <summary>
/// fnServerparams, this should be an array of objects?
/// </summary>
public object[] aoData { get; set; }
/// <summary>
/// Request sequence number sent by DataTable, same value must be returned in response
/// </summary>
public string sEcho { get; set; }
/// <summary>
/// Text used for filtering
/// </summary>
public string sSearch { get; set; }
/// <summary>
/// Number of records that should be shown in table
/// </summary>
public int iDisplayLength { get; set; }
/// <summary>
/// First record that should be shown(used for paging)
/// </summary>
public int iDisplayStart { get; set; }
/// <summary>
/// Number of columns in table
/// </summary>
public int iColumns { get; set; }
/// <summary>
/// Number of columns that are used in sorting
/// </summary>
public int iSortingCols { get; set; }
/// <summary>
/// Comma separated list of column names
/// </summary>
public string sColumns { get; set; }
/// <summary>
/// Text used for filtering
/// </summary>
public string oSearch { get; set; }
}
and finally my Controller:
public ActionResult DataHandler(JQueryDataTableParamModel param)
{
if (param.aoData != null)
{
// Get first element of aoData. NOT working, always null
string lSearchValue = param.aoData[0].ToString();
// Process search value
// ....
}
return Json(new
{
sEcho = param.sEcho,
iTotalRecords = 97,
iTotalDisplayRecords = 3,
aaData = new List<string[]>() {
new string[] {"1", "IE", "Redmond", "USA", "NL"},
new string[] {"2", "Google", "Mountain View", "USA", "NL"},
new string[] {"3", "Gowi", "Pancevo", "Serbia", "NL"}
}
},
JsonRequestBehavior.AllowGet);
}
Note: the action handler gets hit, so the ajax call to get data is also working and my datatable gets filled with 3 rows..
The problem is: aoData is always null. I expect the first element to hold "my_value".
Any help is much appreciated!

After searching for hours to find the answer finally posted it here. Only to come up with the answer in minutes:
This does the trick:
Add this line serverside in the DataHandler:
var wantedValue = Request["more_data"];
So the value is in the request and not in the model.
Thanks.

The value(s) are indeed in the model, but they are passed as individual fields, not as elements of aoData:
public class JQueryDataTableParamModel {
/// The "more_data" field specified in aoData
public string more_data { get; set; }
public string sEcho { get; set; }
public string sSearch { get; set; }
public int iDisplayLength { get; set; }
public int iDisplayStart { get; set; }
public int iColumns { get; set; }
public int iSortingCols { get; set; }
public string sColumns { get; set; }
public string oSearch { get; set; }
}
Usage:
public ActionResult DataHandler(JQueryDataTableParamModel param) {
/// Reference the field by name, not as a member of aoData
string lSearchValue = param.more_data;
return Json(
new {
sEcho = param.sEcho,
iTotalRecords = 97,
iTotalDisplayRecords = 3,
aaData = new List<string[]>() {
new string[] {"1", "IE", "Redmond", "USA", "NL"},
new string[] {"2", "Google", "Mountain View", "USA", "NL"},
new string[] {"3", "Gowi", "Pancevo", "Serbia", "NL"}
}
},
JsonRequestBehavior.AllowGet
);
}

I will post another answer just to show a way to avoid code duplication.
You can also make your JQueryDataTableParamModel as a base class for others. Datatables will send the custom data in the same object, so you can't make the model bind it directly, unless your C# View Model matches exactly the DataTables sent object.
This can be achieved as #SetFreeByTruth answered, but you could have some code duplication if you want it to be used in a whole project. This table has more_data, what if you have another table with only a property called custom_data? You would need to fill your object with multiple fields or create several datatables view models, each with your custom data.
For your scenario, you could use inheritance. Create a new class like this:
//Inheritance from the model will avoid duplicate code
public class SpecificNewParamModel: JQueryDataTableParamModel
{
public string more_data { get; set; }
}
Using it like this in a controller:
public JsonResult ReportJson(SpecificNewParamModel Model)
{
//code omitted code for clarity
return Json(Return);
}
If you send the DataTables request and inspect the Model, you can see that it's filled with your custom data (more_data) and the usual DataTables data.

JSON to C# object using Data Contracts - What am I missing here?

I am experiencing an error converting JSON to a strongly-typed class.
My JSON: {"listBoxID":"ctl00_ctl00_MainContentRoot_MainContent_lstBxSettings","sourceItemText":"Horizontal Bar","sourceItemValue":"Horizontal"}
DroppedItem droppedItem = JsonConvert.DeserializeObject<DroppedItem>(json);
/// <summary>
/// Outlines an object which is useful in simplifying how a CormantRadDock is created.
/// Instead of passing in lots of parameters, would rather just pass in an object that the
/// CormantRadDock knows how to interpret.
/// </summary>
[DataContract]
public class DroppedItem
{
private static readonly ILog Logger = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);
[DataMember(Name = "sourceItemText")]
public string Text { get; set; }
[DataMember(Name = "sourceItemValue")]
public string Value { get; set; }
[DataMember(Name = "listBoxID")]
public Reports ReportType { get; set; }
public DroppedItem() { }
public DroppedItem(string text, string value, string listBoxID)
{
Logger.DebugFormat("Text: {0}, Value: {1}, slidingPaneTitle: {2}", text, value, listBoxID);
Text = text;
Value = value;
ReportType = DetermineReportType(listBoxID);
}
private Reports DetermineReportType(string listBoxID)
{
if (listBoxID.Contains("lstBxHistorical"))
{
return Reports.HistoricalReport;
}
else if (listBoxID.Contains("lstBxCustom"))
{
return Reports.CustomReport;
}
else
{
return Reports.None;
}
}
}
The issue is with converting listBoxID to ReportType.
Uncaught Sys.WebForms.PageRequestManagerServerErrorException: Sys.WebForms.PageRequestManagerServerErrorException: Error converting value "ctl00_ctl00_MainContentRoot_MainContent_lstBxSettings" to type 'CableSolve.Web.Reports'
It occurs regardless of whether the if statement finds a hit or defaults to the else block. It does not occur if I do not attempt to pass the listBoxID parameter.
I'm missing something here. Are my DataMember names not doing anything? I thought they would map the listBoxID to the correct property.

Change to something like this:
public Reports ReportType { get; set; }
[DataMember(Name = "listBoxID")]
public string listBoxID
{
set
{
ReportType = DetermineReportType(value);
}
}
Because basically, you can convert that string to a Report without your helper method. The constructor is not being called on deserialization