We Keep Coding

Understanding details on Dapper objects and splits and maps in C# and .NET Core

I am looking at a new project and have a bug that needs fixing but have never used Dapper every (only read about it as I believe this site uses it). I am trying to understand a couple of things in the following code block below. The entire code block is listed initially for readability and then the questions are broken down below that with snippets from the code.
Code block:
_sql = "SELECT p.*, " +
"s.SupplierId AS 'SplitOnSupplierId', s.*, " +
"pd.ProductId AS 'SplitOnProductId', pd.* " +
"FROM Parts p " +
"JOIN Supplier s ON s.SupplierId = p.SupplierId " +
"JOIN Productss pd ON pd.ProductId = " + productId + " " +
"WHERE p.SupplierId = '" + supplierId + "' " +
"AND p.PartModel = '" + partModel + "'";
var result = new List<Parts>();
try
{
using (_myDb = DapperConnect.GetOpenConnection())
{
result = _myDb.Query<Parts, Supplier, Product, Parts>(_sql,
commandType: CommandType.Text,
map: (p, s, pd) =>
{
p.Supplier = s;
p.Product = pd;
return p;
},
splitOn: "SplitOnSupplierId,SplitOnProductId").ToList();
return result.Count == 0 ? null : result[0];
};
}
catch
{
return null;
}
Code with fragments and questions:
The _spl query just gets all the data needed to make the objects in the mapping section below.
I need to go through this an understand the nature of what is needed, so that question can be ignored.
_sql = "SELECT p.*, " +
"s.SupplierId AS 'SplitOnSupplierId', s.*, " +
"pd.ProductId AS 'SplitOnProductId', pd.* " +
"FROM Parts p " +
"JOIN Supplier s ON s.SupplierId = p.SupplierId " +
"JOIN Productss pd ON pd.ProductId = " + productId + " " +
"WHERE p.SupplierId = '" + supplierId + "' " +
"AND p.PartModel = '" + partModel + "'";
var result = new List<Parts>();
try
{
using (_myDb = DapperConnect.GetOpenConnection())
{
I do not understand why Parts is in this list twice and in general terms what does this line do?
result = _myDb.Query<Parts, Supplier, Product, Parts>(
_sql,
commandType: CommandType.Text,
I think I see that the query above is putting all the results from _sql into a list of Parts objects
with their relationships to Supplier and Product?
map: (p, s, pd) =>
{
p.Supplier = s;
p.Product = pd;
return p;
},
I am not quite sure what or how this line below works.
splitOn: "SplitOnSupplierId,SplitOnProductId").ToList();
return result.Count == 0 ? null : result[0];
};
}
catch
{
return null;
}
As I said before I have never used Dapper in any sort of way and this is the first day I am truly reading about it. Any help is
greatly appreciated.

Dapper doesn't know about your database relations, so you need to tell it how to map the data.
The data comes as a set of rows and the mapping is saying what to do with each row. The generic query (Q2) needs to know what parts to split the row into and which type is should return. In your case it says "split into Parts, Supplier and Product and return Parts". That's why Parts is there twice.
The mapping part in Q3 gets your three objects p, s and d and establishes the relations. This should return something of type Parts, and so it does. The last bit (Q4) is about where to cut the data row into the three objects and it mentions two data columns that should be used as separators. So everything before SplitOnSupplierId is Parts, the next bit is Supplier until you reach SplitOnProductId and the rest is Product.
Your code is vulnerable to SQL Injection. You should use parameters instead of building the SQL query as text. It could also increase performance. Your code would look something like this:
_sql = #"SELECT p.*, s.SupplierId AS 'SplitOnSupplierId', s.*,
pd.ProductId AS 'SplitOnProductId', pd.*
FROM Parts p JOIN Supplier s ON s.SupplierId = p.SupplierId
JOIN Productss pd ON pd.ProductId = p.ProductId
WHERE p.SupplierId = #SupplierId
AND p.ProductId = #ProductId
AND p.PartModel = #PartModel";
try
{
using (_myDb = DapperConnect.GetOpenConnection())
{
var result = _myDb.Query<Parts, Supplier, Product, Parts>(_sql,
commandType: CommandType.Text,
map: (p, s, pd) =>
{
p.Supplier = s;
p.Product = pd;
return p;
},
new {SupplierId = supplierId , ProductId = productId , PartModel = partModel },
splitOn: "SplitOnSupplierId,SplitOnProductId").ToList();
return result.Count == 0 ? null : result[0];
};
}
catch
{
return null;
}
You could also use QueryFirstOrDefault or QuerySingleOrDefault to have only one result returned.

SQL data projection with nested query in ef-core

there is a table of messages. scheme:
CREATE TABLE [dbo].[Messages] (
[Id] INT IDENTITY (1, 1) NOT NULL,
[DateCreate] DATETIME2 (7) DEFAULT (getdate()) NOT NULL,
[SenderId] INT NOT NULL,
[RecipientId] INT NOT NULL,
[TextMessage] NVARCHAR (MAX) NULL,
[IsReaded] BIT NOT NULL,
CONSTRAINT [PK_Messages] PRIMARY KEY CLUSTERED ([Id] ASC)
);
I want to get a table:
sender_id [int] - sender id key
all_count_messages [int] - number of messages from the sender
count_unreaded_messages [int] - number of unread messages from the sender
most_unreaded [DateTime] - the date of the most old unread messages
I also need group by [sender_id] and sort the output. Good. I make a simple request:
SELECT
message.SenderId AS sender_id,
COUNT(*) AS all_count_messages,
SUM(CAST(
CASE
WHEN IsReaded = 0
THEN 1
ELSE 0
END AS int)) as count_unreaded_messages,
MIN(message.DateCreate) AS most_unreaded
FROM
Messages AS message
GROUP BY
message.SenderId
ORDER BY
most_unreaded
demo result:
sender_id all_count_messages unreaded_messages most_unreaded
2 3 2 2019-08-15 20:03:59.0000000
1 9 8 2019-08-15 20:04:59.0000000
the answer suits me. how to describe it on EFCore?
try it
var chats = from my_messages in db.Messages
group my_messages by my_messages.SenderId into g
select
new
{
sender_id = g.Key,
all_count_messages = g.Count(),
unreaded_messages = from sub_messages in db.Messages where sub_messages.SenderId == g.Key && !sub_messages.IsReaded group sub_messages by sub_messages.SenderId into sub_g select sub_g.Count(),
most_unreaded = from sub_messages in db.Messages where sub_messages.SenderId == g.Key && !sub_messages.IsReaded group sub_messages by sub_messages.SenderId into sub_g select sub_g.Min(x => x.DateCreate)
};
foreach (var chat in chats) // so, too, has tried: chats.Include(x=>x.unreaded_messages).Include(x => x.most_unreaded)
{
}
get error in foreach (var chat in chats)
An unhandled exception occurred while processing the request.
ArgumentException: must be reducible node
System.Linq.Expressions.Expression.ReduceAndCheck()
I tried otherwise:
var chats = db.Messages.AsNoTracking().FromSql(
"SELECT" +
" message.SenderId AS sender_id," +
" COUNT(*) AS all_count_messages," +
" SUM(CAST(" +
" CASE" +
" WHEN IsReaded = 0" +
" THEN 1" +
" ELSE 0" +
" END AS int)) as count_unreaded_messages," +
" MIN(message.DateCreate) AS most_unreaded " +
"FROM " +
" Messages AS message " +
"GROUP BY " +
" message.SenderId " +
"ORDER BY" +
" most_unreaded ");
foreach (var chat in chats)
{
}
get error in foreach (var chat in chats)
InvalidOperationException: The required column 'Id' was not present in the results of a 'FromSql' operation.
Microsoft.EntityFrameworkCore.Query.Sql.Internal.FromSqlNonComposedQuerySqlGenerator.CreateValueBufferFactory(IRelationalValueBufferFactoryFactory relationalValueBufferFactoryFactory, DbDataReader dataReader)

You need to add Id column in the select statement to your query. It should be like
"SELECT" +
" message.Id AS Id," +
" message.SenderId AS sender_id," +
...

thank #ilkerkaran for the link https://stackoverflow.com/a/28627991/2630427 there I found and slightly altered method of dynamic request
a little finished realisation .net asp core 2.2:
public class AppDbContext : DbContext
{
public DbSet<Message> Messages { get; set; }
IOptions<AppConfig> app_config;
public AppDbContext(DbContextOptions<AppDbContext> options, IOptions<AppConfig> _app_config)
: base(options)
{
app_config = _app_config;
Database.EnsureCreated();
}
public IEnumerable<dynamic> DynamicListFromSql(string Sql, Dictionary<string, object> Params = null)
{
using (var cmd = Database.GetDbConnection().CreateCommand())
{
cmd.CommandText = Sql;
if (cmd.Connection.State != ConnectionState.Open) { cmd.Connection.Open(); }
if (Params != null)
foreach (KeyValuePair<string, object> p in Params)
{
DbParameter dbParameter = cmd.CreateParameter();
dbParameter.ParameterName = p.Key;
dbParameter.Value = p.Value;
cmd.Parameters.Add(dbParameter);
}
using (var dataReader = cmd.ExecuteReader())
{
while (dataReader.Read())
{
var row = new ExpandoObject() as IDictionary<string, object>;
for (var fieldCount = 0; fieldCount < dataReader.FieldCount; fieldCount++)
{
row.Add(dataReader.GetName(fieldCount), dataReader[fieldCount]);
}
yield return row;
}
}
}
}
}
string query =
"SELECT" +
" message.SenderId AS sender_id," +
" COUNT(*) AS all_count_messages," +
" (SELECT COUNT(*) FROM Messages AS sub_message WHERE sub_message.SenderId = message.SenderId AND sub_message.IsReaded = 0) AS unreaded_messages," +
" (SELECT MIN(sub_message.DateCreate) FROM Messages AS sub_message WHERE sub_message.SenderId = message.SenderId) AS most_unreaded " +
"FROM " +
" Messages AS message " +
"GROUP BY " +
" message.SenderId " +
"ORDER BY" +
" most_unreaded ";
//"OFFSET 0 ROWS FETCH NEXT 10 ROWS ONLY", offset, fetch);
foreach (var chat in db.DynamicListFromSql(query))
{
}

how to assign integer value + string value in c# linq query for same column

String + Int + String is not working, when i assign the values to string column
-> I Need the Name column value like :
AS Project Employee ID: 1878 Employee Name: Kevin dominic..
select new
{
ID = AS_ProjectHeader.ID,
PartyID = AS_ProjectHeader.PartyID,
Name = AS_ProjectHeader.Name + " Employee ID: " + AS_ProjectHeader.EmployeeID + " Employee Name:" + AS_ProjectHeader.PAYE_Employee.Forenames + " " + AS_ProjectHeader.PAYE_Employee.Surname,
ClientName = AS_ProjectHeader.AS_PartyList.Name,
StartDate = AS_ProjectHeader.StartDate,
EndDate = AS_ProjectHeader.EndDate,
});
EDIT: Additional information:
If i use convert to string for the integer value i get this error :
LINQ to Entities does not recognize the method 'System.String ToString()' method, and this method cannot be translated into a store expression.

Pleae Use SqlFunctions.StringConvert ,it will work
Name = PSC_ProjectHeader.Name + " Employee ID: " + SqlFunctions.StringConvert((decimal)PSC_ProjectHeader.EmployeeID)
+ " Employee Name:" + PSC_ProjectHeader.PAYE_Employee.Forenames + " " + PSC_ProjectHeader.PAYE_Employee.Surname,

You can easily use .ToString()
but be-careful you could't use control like textBox1.Text inside Linq query

This following is relevent only if using EF version older than 6.1
Linq-To-Entities really doesn't support ToString.
You have to materialize the query first (ToList()) and then select what you want.
So let say that query is what you have until the select.
query.ToList().Select(AS_ProjectHeader =>
{
ID = AS_ProjectHeader.ID,
PartyID = AS_ProjectHeader.PartyID,
Name = AS_ProjectHeader.Name
+ " Employee ID: " + AS_ProjectHeader.EmployeeID
+ " Employee Name:" + AS_ProjectHeader.PAYE_Employee.Forenames + " "
+ AS_ProjectHeader.PAYE_Employee.Surname,
ClientName = AS_ProjectHeader.AS_PartyList.Name,
StartDate = AS_ProjectHeader.StartDate,
EndDate = AS_ProjectHeader.EndDate,
}

dynamic type can't return from web api always different set of columns

My issue is this.
I have an ever changing set of columns due to pivoting on a list of items in a table. Basically each column that you see in the above picture is a line in the database table. I want to display a group of lines as one line on the front end. How do I return this dynamic type from a controller or api?
I have been attempting to use code such as this.
List<dynamic> items = repository.GetAll();
foreach(var item in items){
var properties = item.GetType().GetProperties(BindingFlags.Public);
foreach (var property in properties) {
var PropertyName = property.Name;
var PropertyValue = item.GetType().GetProperty(property.Name).GetValue(item, null);
model.Add(PropertyName, PropertyValue);
}
}
But GetProperties() returns nothing from the dynamic "item". I've also tried using item.GetType().GetFields() to no avail.
How can i go about getting the column name and value from a dynamic data type in c# when i don't know the name of that column? Or just returning the dynamic object from the api? Thanks.
Here's the repository code.
public IEnumerable<dynamic> GetAll()
{
var items = context.Database.SqlQuery<dynamic>(
"DECLARE #cols NVARCHAR(MAX), #sql NVARCHAR(MAX)" +
"SET #cols = STUFF((SELECT DISTINCT ',' + QUOTENAME(FormDetails.Name)" +
" from Value" +
" left join ValueDetails" +
" on Value.Id = ValueDetails.ValueId" +
" left" +
" join FormDetails" +
" on ValueDetails.FormDetailsId = FormDetails.Id" +
" ORDER BY 1" +
" FOR XML PATH(''), TYPE" +
" ).value('.', 'NVARCHAR(MAX)'), 1, 1, '')" +
" SET #sql = 'SELECT Id, FormId, CreatedDateTime, ' + #cols + '" +
" FROM" +
" (" +
" select Value.Id, Value.FormId, FormDetails.Name, ValueDetails.Value, Value.CreatedDateTime" +
" from Value" +
" left join ValueDetails" +
" on Value.Id = ValueDetails.ValueId" +
" left" +
" join FormDetails" +
" on ValueDetails.FormDetailsId = FormDetails.Id" +
" ) s" +
" PIVOT" +
"(" +
"MAX(Value) FOR Name IN(' + #cols + ')" +
") p order by CreatedDateTime desc '" +
" EXECUTE(#sql)").ToList();
return items;
}
Here's a bit more info. Basically my repository returns the data from the first image at the top of this page. But when i return that data to the front end this is what it looks like. The object is there but no properties...

The problem with your code is not in getting an empty list of properties, because they are genuinely not there. SqlQuery<T> will not populate a property if it's not there on type T; you are passing dynamic, which translates to System.Object.
One trick to solving this is to retrieve column names dynamically, and build an ExpandoObject out of it. The technique is described here.
static IList<dynamic> Read(DbDataReader reader) {
var res= new List<Dictionary<string,object>>();
foreach (var item in reader) {
IDictionary<string,object> expando = new ExpandoObject();
foreach (PropertyDescriptor propertyDescriptor in TypeDescriptor.GetProperties(item)) {
var obj = propertyDescriptor.GetValue(item);
expando.Add(propertyDescriptor.Name, obj);
}
res.Add(expando);
}
return res;
}
With this method in place, call
using (var cmd = ctx.Database.Connection.CreateCommand()) {
ctx.Database.Connection.Open();
cmd.CommandText = ...; // Your big SQL goes here
using (var reader = cmd.ExecuteReader()) {
return Read(reader).ToList();
}
}
Once your repository start returning ExpandoObjects, your code for retrieving properties can be replaced with code querying IDictionary<string,object>, an interface implemented by ExpandoObject:
IDictionary<string,object> properties = item as IDictionary<string,object>;
if (properties != null) {
foreach (var p in properties) {
model.Add(p.Key, p.Value);
}
}

How do I create a lambda expression with unknown fields?

I want to dynamically build the lambda expression so that I can build a query with an unknown number of fields...how do I accomplish this?
I am looping through an object that contains all of the fields and values adding to the Where clause for each field...
searcher = searcher.Where(f => f.fieldName.Contains(fieldValue));
i.e. pseudo-code:
foreach(var field in fields){
searcher = searcher.Where(f => field.name.Contains(field.value));
}
If I were living in the stone-age I'd pseudo-code it like this:
var first = true;
string query = " SELECT * FROM TABLE WHERE ";
foreach(var field in fields){
if(first){
query += field.name + " LIKE '%" + field.value + "%' ";
}else{
query += " AND " + field.name + " LIKE '%" + field.value + "%' ";
}
first = false;
}
Please tell me the stone-age isn't more powerful than current technology! ;-)

I hope this will be helpful.
searcher = fields.Aggregate(searcher, (current, field) => current.Where(f => f.Name.Contains(f.Value)));