We Keep Coding

Multiple where clause using C# and SqlKata query engine

I am working on a small Web API which will use SqlKata in the back end as the query engine to talk to SQL Server. In the documentation, SqlKata says the following in relation to multiple where clauses:
Multiple fields
If you want to filter your query against multiple fields, pass an object that represents col/values.
var query = new Query("Posts").Where(new {
Year = 2017 ,
CategoryId = 198 ,
IsPublished = true,
});
My intention is to use query strings to create the WHERE clauses in the back end, but I'm a bit stuck as to how to convert the Key/Value pairs of the query strings into properties of an object to use in the SqlKata query. The request will be highly dynamic in nature, so I can't really use a static object. I am wondering if anyone has any tips on how to construct an object to satisfy these requirements, being that the properties - both the property name and/or value - can be dynamic, as could the number of properties within the object. In my head I could imagine somehow converting the key / value pairs of the query strings into an object at run-time, but I'm stuck on how to achieve that.
I did try the following, but it seems each iteration through the loop the last key/value pair is replaced, so you end up with only the most recent WHERE clause being considered:
if (request.QueryStringParameters != null)
{
foreach (var element in request.QueryStringParameters)
{
CreateCloudWatchLog($"query string {element.Key} value {element.Value}", context, LogLevel.Trace, environmentLogLevel);
if (element.Key != "limit")
{
query = query.Where(element.Key, element.Value);
}
if (element.Key == "limit")
{
query = query.Limit(Convert.ToInt32(element.Value));
}
}
}
I have also tried this approach (query strings => json => dynamic object) but I get the error 'parameter count mismatch' (I am testing with one parameter passed in called 'storenumber' with a value of 399)
var json = JsonConvert.SerializeObject(request.QueryStringParameters, Formatting.Indented);
CreateCloudWatchLog($"Serialised query strings = {json}", context, LogLevel.Trace, environmentLogLevel);
var myobject = JsonConvert.DeserializeObject<dynamic>(json);
query = query.Where(myobject);
Debug logs:
[Trace] Serialised query strings =
{
"storenumber": "399"
}
[Trace] Finished converting JSON to object
[Error] Parameter count mismatch.

Looking at the documentation, it looks like you can use the basic where method which takes a property and a value. For example, assuming you have a dictionary of key value pairs, you could do this:
var filters = new Dictionary<string, object>
{
{ "Year", 2017 },
{ "CategoryId", 198 },
{ "IsPublished", true },
}
var query = new Query("Posts");
foreach(var filter in filters)
{
query = query.Where(filter.Key, filter.Value);
}

How to work with a DynamicJsonObject's property values?

Let's say that in my WebAPI I'm receiving a json like this:
{
prop1: "sometext",
prop2: 123,
prop3: true,
prop4: [1,2,3,4,5],
prop5: ["text1", "text2", "text3"]
}
To work with this data, I transform it into this: dynamic data = System.Web.Helpers.Json.Decode(json);. In most cases it's really easy to work with it, but I'm struggling with a situation where I need to repass this data to a database. Most conversions are pretty straightforward, making it easy to iterate over the DynamicJsonObject and send the information to the DB.
My big problem is dealing with properties like prop4 and prop5. This array is being converted to a DynamicJsonArray. To convert it to a primitive array, I'm currently using this:
Array arr = (Array) data["prop4"];
data["prop4"] = arr.Cast<Int32>().ToArray();
The code above works fine and it successfully converts the property value to an integer array that can be sent to the DB. However, when I do it with the string array:
Array arr = (Array) data["prop5"];
data["prop5"] = arr.Cast<string>().ToArray();
The data["prop5"] value isn't changed, and when I inspect it's value, it still points to the type System.Web.Helpers.DynamicJsonArray. Why does this happen?
Also, the thing is that I'm not experienced on working with dynamic objects in C# and I use that because in my situation I do not always know what properties the JSON I'm receiving is going to have. Is there a cleaner way to make this kind of conversion? Actually, is there any need to use dynamic objects to deal with JSONs?
EDIT:
Sorry for being extremely unclear. To clear things up, that's what I'm actually doing:
I'm using a generic function to call a DB procedure with whatever data that comes from the parameter data:
public static IEnumerable<Dictionary<string, object>> CallDBProcedure(string proc, dynamic data)
{
using (NpgsqlConnection conn = new NpgsqlConnection(ConnStr))
{
try
{
conn.Open();
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = proc;
cmd.CommandType = System.Data.CommandType.StoredProcedure;
foreach (string property in data.GetDynamicMemberNames())
{
object propertyValue = data[property];
// The method convertNullValue returns the own value if it's not null
// and returns DBNull.Value if it's null. Every value must be properly
// converted, but I can't directly convert a DynamicJsonArray.
cmd.Parameters.Add(new NpgsqlParameter(property, convertNullValue(propertyValue)));
}
var reader = cmd.ExecuteReader();
return new DrToDictionary().Serialize(reader);
}
}
catch (Exception e)
{
throw e;
}
finally
{
conn.Close();
}
}
}
And then, in order to avoid exceptions, I'm processing data before
I pass it to CallDBProcedure, iterating over the DynamicJsonObject
and converting every DynamicJsonArray inside it to a primitive array:
public static void ProcessData(dynamic data)
{
// Directly using data.GetDynamicMemberNames() and modifiying data
// inside the loop causes an exception. Using a string list instead (hacky code?).
List<string> properties = ((Dictionary<string, object>.KeyCollection) data.GetDynamicMemberNames()).ToList();
foreach (string property in properties)
{
object propertyValue = data[property];
if (propertyValue != null && propertyValue.GetType() == typeof(DynamicJsonArray))
{
Array arr = (Array)data[property];
if (arr.Length == 0)
{
// This works.
data[property] = null;
}
else
{
// Discover the type of my array.
if (arr.GetValue(0) is int)
{
// This works.
data[property] = arr.Cast<Int32>().ToArray();
}
else
{
// This doesn't.
data[property] = arr.Cast<string>().ToArray();
}
}
}
}
}
For now I'm iterating over data twice. I know that there must be an obvious and easy way to make this ugly code cleaner, but I'm just not seeing it.
EDIT2:
A solution that I found was to take only the string arrays and put them inside a different variable. Then I can use them normally. I really don't understand why dynamic objects can change their types in some cases and in other cases they can't.
EDIT3:
Forgot to mention that I'm receiving my json like this: dynamic data = System.Web.Helpers.Json.Decode(json);

It is just reading the json object which is returned from the API.
E.g
You can use
dynamic stuff = JsonConvert.DeserializeObject("{ 'prop1' : 'sometext', 'prop2': '123',
'prop3': 'True'}");
string Prop1= stuff.First.Prop1;
string Prop2= stuff.First.Prop2;
or
// For that you will need to add reference to System.Web.Helpers
dynamic json = System.Web.Helpers.Json.Decode(#""{ 'prop1' : 'sometext', 'prop2': '123',
'prop3': 'True'}"");
Console.WriteLine(json.prop1);
another similar quetion in Stackoverflow as below
How can I parse JSON with C#?

C# / JSON - Invoke a method with a List<object> as parameters

I am having a lot of trouble on my code, I am trying to make some sort of parser. Basically I am trying to get data from a JSON file, and use that data to call methods from my code.
Here's a basic example of my JSON file
{ "story": { "Talk": [ "me", 1, 1 ] } }
Now I have a class called DialogueSystem, it contains a function called Talk with three parameters, string, int, int.
I am using SimpleJSON to get my JSON data, but I am guessing it's similar to other JSON parsers.
Also, I have other functions which have different parameters which is why I am forced to use reflection
Anyways, here's the code that gets the json data, and tries to use Reflection to call the Talk method.
// Gets the JSON and parses it
JSONNode Node = JSONNode.Parse(jsonFile());
var method = sys.GetType().GetMethod(""); // Reflection stuff
foreach (JSONNode item in Node["story"].Keys) // the Keys just gives me every key that's in the story node/key
{
List<object> parameters = new List<object>(); // List containing the parameters, to be used when invoking the method
for (int i = 0; i < Node["story"][item].Count; i++)
{
//This part tests if it's a string or int and adds it as such
string data = Node["story"][item][i];
int n;
bool isNum = int.TryParse(data, out n);
if (isNum)
{
parameters.Add(n);
}
else
{
parameters.Add(data);
}
}
// Invoke the method using it's method name and the method parameters
method.Invoke(item, parameters.ToArray());
}
Btw, my Talk class just prints a text based on the given input.
For some reason, I get this error
NullReferenceException: Object reference not set to an instance of an
object Dialogue.StoryTeller.ReadStory (Dialogue.DialogueSystem sys)
(at Assets/Dialogue System/Scripts/StoryTeller.cs:57)
If you have any idea how to fix it, or maybe make it better, that would be awesome!
Thanks!

Your .getmethod call is using an empty string. Therefore your methodinfo object is null. Hence the exception. https://msdn.microsoft.com/en-us/library/8zz808e6(v=vs.110).aspx

Ok, after searching everywhere on the Microsoft docs and stack overflow, I found out that I needed to change two things.
First of all I had to change var method = sys.GetType().GetMethod("");
to this var method = sys.GetType().GetMethod(item.ToString().Replace("\"", ""));
(Which I also moved inside the for loop in order to use item)
Also, I needed to change the invoke method.Invoke(item, parameters.ToArray()); to this method.Invoke(sys, parameters.ToArray());
So the whole code is now this
Node = JSONNode.Parse(jsonFile());
foreach (JSONNode item in Node["story"].Keys)
{
List<object> parameters = new List<object>();
var method = sys.GetType().GetMethod(item.ToString().Replace("\"", ""));
for (int i = 0; i < Node["story"][item].Count; i++)
{
string data = Node["story"][item][i];
int n;
bool isNum = int.TryParse(data, out n);
if (isNum)
{
parameters.Add(n);
}
else
{
parameters.Add(data);
}
}
method.Invoke(sys, parameters.ToArray());
}

C# function to return generic objects / entities

In my application I need to display a list of records returned by different stored procedures. Each store procedure returns different types of records (ie the number of columns and the column type are different).
My original thought was to create a class for each type of records and create a function which would execute the corresponding stored procedure and return List< MyCustomClass>. Something like this:
public class MyCustomClass1
{
public int Col1 { get; set; } //In reality the columns are NOT called Col1 and Col1 but have proper names
public int Col2 { get; set; }
}
public static List<MyCustomClass1> GetDataForReport1(int Param1)
{
List<MyCustomClass1> output = new List<MyCustomClass1>();
using (SqlConnection cn = new SqlConnection("MyConnectionString"))
using (SqlCommand cmd = new SqlCommand("MyProcNameForReport1", cn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#Param1", SqlDbType.Int).Value = Param1;
SqlDataReader rdr=cmd.ExecuteReader();
int Col1_Ordinal = rdr.GetOrdinal("Col1");
int Col2_Ordinal = rdr.GetOrdinal("Col2");
while (rdr.Read())
{
output.Add(new MyCustomClass1
{
Col1 = rdr.GetSqlInt32(Col1_Ordinal).Value,
Col2 = rdr.GetSqlInt32(Col2_Ordinal).Value
});
}
rdr.Close();
}
return output;
}
This works fine but as I don't need to manipulate those records in my client code (I just need to bind them to a graphical control in my application layer) it doesn't really make sense to do it this way as I would end up with plenty of custom classes that I wouldn't actually use. I found this which does the trick:
public static DataTable GetDataForReport1(int Param1)
{
DataTable output = new DataTable();
using (SqlConnection cn = new SqlConnection("MyConnectionString"))
using (SqlCommand cmd = new SqlCommand("MyProcNameForReport1", cn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#Param1", SqlDbType.Int).Value = Param1;
output.Load(cmd.ExecuteReader());
}
return output;
}
This returns a DataTable which I can bind to whatever control I use in my application layer. I'm wondering if using a DataTable is really needed.
Couldn't I return a list of objects created using anonymous classes:
public static List<object> GetDataForReport1(int Param1)
{
List<object> output = new List<object>();
using (SqlConnection cn = new SqlConnection("MyConnectionString"))
using (SqlCommand cmd = new SqlCommand("MyProcNameForReport1", cn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#Param1", SqlDbType.Int).Value = Param1;
SqlDataReader rdr=cmd.ExecuteReader();
int Col1_Ordinal = rdr.GetOrdinal("Col1");
int Col2_Ordinal = rdr.GetOrdinal("Col2");
while (rdr.Read())
{
output.Add(new
{
Col1 = rdr.GetSqlInt32(Col1_Ordinal).Value,
Col2 = rdr.GetSqlInt32(Col2_Ordinal).Value
});
}
rdr.Close();
}
return output;
}
Any other ideas? Basically I just want the function to return 'something' which I can bind to a graphical control and I prefer not to have to create custom classes as they wouldn’t really be used. What would be the best approach?

The good news is: this isn't the first time the entities vs datasets issue has come up. It's a debate much older than my own programming experience. If you don't want to write DTO's or custom entities then your options are DataTables/DataSets or you could reinvent this wheel again. You wouldn't be the first and you won't be the last. The argument for DTO's/Entities is that you don't have the performance overheads that you get with DataSets. DataSets have to store a lot of extra information about the datatypes of the various columns, etc...
One thing, you might be happy to hear is that if you go the custom entity route and you are happy for your object property names to match the column names returned by your sprocs, you can skip writing all those GetDataForReport() mapping functions where you are using GetOrdinal to map columns to properties. Luckily, some smart monkeys have properly sussed that issue here.
EDIT: I was researching an entirely different problem today (binding datasets to silverlight datagrids) and came across this article by Vladimir Bodurov, which shows how to transform an IEnumerable of IDictionary to an IEnumerable of dynamically created objects (using IL). It occured to me that you could easily modify the extension method to accept a datareader rather than the IEnumerable of IDictionary to solve your issue of dynamic collections. It's pretty cool. I think it would accomplish exactly what you were after in that you no longer need either the dataset or the custom entities. In effect you end up with a custom entity collection but you lose the overhead of writing the actual classes.
If you are lazy, here's a method that turns a datareader into Vladimir's dictionary collection (it's less efficient than actually converting his extension method):
public static IEnumerable<IDictionary> ToEnumerableDictionary(this IDataReader dataReader)
{
var list = new List<Dictionary<string, object>>();
Dictionary<int, string> keys = null;
while (dataReader.Read())
{
if(keys == null)
{
keys = new Dictionary<int, string>();
for (var i = 0; i < dataReader.FieldCount; i++)
keys.Add(i, dataReader.GetName(i));
}
var dictionary = keys.ToDictionary(ordinalKey => ordinalKey.Value, ordinalKey => dataReader[ordinalKey.Key]);
list.Add(dictionary);
}
return list.ToArray();
}

If you're going to XmlSerialize them and want that to be efficient, they can't be anonymous. Are you going to send back the changes? Anonymous classes wont be much use for that.
If you're really not going to do anything with the data other than gridify it, a DataTable may well be a good answer for your context.
In general, if there's any sort of interesting business login going on, you should be using Data Transfer Objects etc. rather than just ferrying around grids.

If you use anonymous classes, you are still defining each type. You just get to use Type Inference to reduce the amount of typing, and the class type will not clutter any namespaces.
The downside of using them here is that you want to return them out of the method. As you noted, the only way to do this is to cast them to objects, which makes all of their data inaccessable without reflection! Are you never going to do any processing or computation with this data? Display one of the columns as a picture instead of a value, or hide it based on some condition? Even if you are only ever going to have a single conditional statement, doing it with the anonymous class version will make you wish you hadn't.

Generic method to recive any type of object and convert the object to a List<string>

EDIT AGAIN: the solution was probably different from my original question. Thanks everyone very much your great ideas. I wish I could vote for more than one answer.
EDIT: I am populating a Jquery table plugin from datatables/.net and it requires the data (Json) to be in a certain format like this;
"sEcho": 1,
"iTotalRecords": 57,
"iTotalDisplayRecords": 57,
"aaData": [
[
"Gecko",
"Firefox 1.0",
"Win 98+ / OSX.2+",
"1.7",
"A"
],
[
"Gecko",
"Firefox 1.5",
"Win 98+ / OSX.2+",
"1.8",
"A"
],
...
]
}
I am recieving data from a service that is returning a collection of object. I would like one method which I can pass these collections into and it will return the appropriate string
thanks
END EDIT
I would like to build a method that can receive and object that we build and will return an array List each containing the value of each object passed in. For example;
I have a collection of 'Car' objects
What I would like to do is
public object[] Something<T>(_cars)
{
object[] objArray = new object[_cars.Count];
foreach(Car _car in _cars ){
IList objList = new List<string>;
objList.Add(_car.Color);
objList.Add(_car.EngineSize);
//etc etc
objArray[i] = objList;//i'll have to use a for loop to get the i counter but you get my idea
}
return objArray
}
my problem is how can I access the properties of the object without knowing what type of object it is?
thanks for any help

Update: To answer your revised question - produce a JSON result of a data structure - use the built-in JavaScriptSerializer class:
JavaScriptSerializer serializer = new JavaScriptSerializer();
string json = seriaizer.Serialize(myObjectOrArray);
Below is the previous answer.
how can I access the properties of the object without knowing what type of object it is
Using Reflection, and grabbing the properties which are strings. Note this is not necessarily a good idea. The fact that you have to use reflection to get what you want is usually a HUGE WARNING FLAG that your design is wrong.
However, in the hopes of learning something useful, here's how it could be done:
public object[] Something<T>(T[] items)
{
IList objList = new List<object>();
//get the properties on which are strings
PropertyInfo[] properties = typeof(T).GetProperties().Where(p => p.PropertyType == typeof(string));
foreach(T item in items)
{
IList stringList = new List<string>;
foreach(PropertyInfo property in properties)
{
objList.Add(property.GetValue(item, null) as string);
}
objList.Add(stringList);
}
}
return objList.ToArray();
}
A far, far better solution would be to require all the objects coming into this method to conform to some interface that requires them to provide their own string-formatted data. Or maybe take two steps back and ask for help on the underlying problem. This approach is fraught with problems. It's a rabbit hole you don't want to go down.

Use the System.Web.Script.Serialization.JavaScriptSerializer class. It was specifically provided for JSON serialization.
using System.Web.Script.Serialization;
public string ToJson(object o)
{
JavaScriptSerializer serializer = new JavaScriptSerializer();
return serializer.Serialize(o);
}
EDIT: Oops, I missed that your plugin doesn't want a true JSON representation of the objects; it just wants arrays of values. You could use reflection to iterate over the properties of the objects as others have suggested, but then you have no control over which properties end up in which columns. It is not clear from your question whether that is a problem for you.
If you need a strict mapping between properties and columns, then you will have to define that somehow in C#. To do this you could implement IEnumerable as Ed demonstrates or create a custom interface:
public interface ITableDataSource
{
IList<string> GetTableData();
}
Then implement this on any objects that might need to be data sources for the jQuery table plugin:
public class Car : ITableDataSource
{
//...class implementation details...
public IList<string> GetTableData()
{
return new List<string>()
{
this.Color,
this.EngineSize,
this.NumberOfSeats.ToString()
};
}
}
Finally, in your method that is returning the data to the jQuery plugin, use the interface to construct your response object, then pass it to my ToJson() method to serialize it:
public string DoSomething(IList<ITableDataSource> objects)
{
var result = new
{
sEcho = 1,
iTotalRecords = 1,
iTotalDisplayRecords = 1,
aaData = new List<IList<string>>()
};
foreach (ITableDataSource ds in objects)
result.aaData.Add(ds.GetTableData());
return ToJson(result);
}

While it would be relatively straightforward to use reflection to loop through all of your objects and all the properties on those objects to build that string, it's already been written.
I would highly recommend looking at the Json.NET project found here. Add this DLL to your project and converting a list of objects into a Json formatted string is as easy as:
string json = Newtonsoft.Json.JsonConvert.SerializeObject( listOfCars );

IList objList = new List<string>();
foreach ( PropertyInfo prop in _car.GetType().GetProperties() )
{
var value = prop.GetValue( _car, null );
objList.Add( value != null ? value.ToString() : null );
}
objArray[i] = objList;

The code to get the Property values of an object is
foreach (PropertyInfo info in myObject.GetType().GetProperties())
{
if (info.CanRead)
{
object o = propertyInfo.GetValue(myObject, null);
}
}

my problem is how can I access the properties of the object without knowing what type of object it is?
Well, in the general sense you can't. Generics is all about treating objects generically. However, you can impose type constraints by using a where clause. Based on the foreach loop I would say constrain your types to types that implement IEnumerable, but then you go on to use properties like "Color" and "EngineSize", which are very specific. I don't even know why you would have properties named "Color" and "EngineSize" that are strings, but that is another problem altogether...
It seems like the best approach for you would be to define an interface or an abstract base class that each of these objects inherits from. Then you can use the 'where' clause to constrain to objects of that interface/base class only So...
public object[] Something<T>( T _cars) where T : IEnumerable<MyInterface>
However, if we are going to go down this road I don't see why the method should be generic at all. It could simply take an IEnumerable<T> as an input. When we only want to use one type in a method generics is not needed.