We Keep Coding

C# Mysql populate list<T>

I'm trying to populate list from mysql with the below code. My model looks like this https://pastebin.com/iQEVV42C
My goal is to populate the lists then take the sub lists and put them into the root (tickets) how can I do this?
public static async Task<IEnumerable<Models.Zelkon.Tickets.Root>> Tickets(int ticketId = 0, int clientId = 0)
{
DataSet ds = new DataSet();
List<Models.Zelkon.Tickets.Root> tickets = new List<Models.Zelkon.Tickets.Root>();
List<Models.Zelkon.Tickets.Incidents> ticketIncidents = new List<Models.Zelkon.Tickets.Incidents>();
List<Models.Zelkon.Tickets.Files> ticketFiles = new List<Models.Zelkon.Tickets.Files>();
List<Models.Zelkon.Tickets.Comments> ticketComments = new List<Models.Zelkon.Tickets.Comments>();
using (MySqlConnection con = new MySqlConnection(Variables.conString))
{
await con.OpenAsync();
string query = "SELECT * FROM tickets;" +
"SELECT * FROM ticket_incidents;" +
"SELECT * FROM ticket_files;" +
"SELECT * FROM ticket_comments;";
using (MySqlCommand cmd = new MySqlCommand(query, con))
{
using (MySqlDataReader reader = (MySqlDataReader)await cmd.ExecuteReaderAsync())
{
// tickets
while (await reader.ReadAsync())
{
tickets.Add(new Models.Zelkon.Tickets.Root()
{
header_text = reader.GetString("name"),
description = reader.GetString("description"),
status = reader.GetString("status"),
source = reader.GetString("source"),
owner = reader.GetString("owner"),
temp = reader.GetString("temp"),
priority = reader.GetString("priority"),
category = reader.GetString("category"),
creation_team = reader.GetString("creation_team"),
last_updated = reader.GetDateTime("last_updated"),
creation_date = reader.GetDateTime("creation_date"),
client_id = reader.GetInt32("client_id"),
ticket_id = reader.GetInt32("ticket_id"),
unique_id = reader.GetInt32("unique_id")
});
}
await reader.NextResultAsync();
// ticket_incidents
while (await reader.ReadAsync())
{
ticketIncidents.Add(new Models.Zelkon.Tickets.Incidents()
{
header_text = reader.GetString("header_text"),
description = reader.GetString("description"),
type = reader.GetString("type"),
creation_team = reader.GetString("creation_team"),
created_by = reader.GetString("created_by"),
notify_team = reader.GetInt32("notify_team"),
notification_seen = reader.GetInt32("notification_seen"),
is_pinned = reader.GetInt32("is_pinned"),
creation_date = reader.GetDateTime("creation_date"),
incident_id = reader.GetInt32("incident_id"),
ticket_id = reader.GetInt32("ticket_id"),
unique_id = reader.GetInt32("unique_id")
});
}
await reader.NextResultAsync();
// ticket_files
while (await reader.ReadAsync())
{
// Not filled yet
}
await reader.NextResultAsync();
// ticket_comments
while (await reader.ReadAsync())
{
// Not filled yet
}
await reader.NextResultAsync();
}
}
}
System.Diagnostics.Debug.WriteLine(tickets.First().owner);
return tickets;
}
My issue is how I can fill the sub classes (ticket_incidents, ticket_files, ticket_comments) and then put it into the root class?

Firstly, you might save a lot of time by using an ORM to query your database; it will turn a set of related tables into an object graph.
Secondly, if you're going to avoid a full ORM and read the data directly via ADO.NET, I would recommend Dapper, which will remove a lot of boilerplate code.
Here's how the main body of your method would look with Dapper:
string query = "SELECT * FROM tickets;" +
"SELECT * FROM ticket_incidents;" +
"SELECT * FROM ticket_files;" +
"SELECT * FROM ticket_comments;";
using var gridReader = await con.QueryMultipleAsync(query);
// NOTE: Assumes database columns are named exactly the same as the object properties
var tickets = (await gridReader.ReadAsync<Models.Zelkon.Tickets.Root>()).ToList();
var ticketIncidents = (await gridReader.ReadAsync<Models.Zelkon.Tickets.Incidents>()).ToList();
var ticketFiles = (await gridReader.ReadAsync<Models.Zelkon.Tickets.Files>()).ToList();
var ticketComments = (await gridReader.ReadAsync<Models.Zelkon.Tickets.Comments>()).ToList();
Thirdly, if you're using async with MySQL, uninstall Oracle's MySQL Connector/NET (i.e., MySql.Data) and use MySqlConnector instead. It's a longstanding bug in Connector/NET that async operations actually operate completely synchronously.
Finally, to link your objects together, iterate over each collection and attach the object to the right Root object. One way to do this would be to put your Root objects in a Dictionary keyed on their ID:
var ticketsById = tickets.ToDictionary(x => x.ticket_id, x => x);
foreach (var incident in ticketIncidents)
ticketsById[incident.ticket_id].incidents.Add(incident);
foreach (var file in ticketFiles)
ticketsById[file.ticket_id].files.Add(file);
foreach (var comment in ticketComments)
ticketsById[comment.ticket_id].comments.Add(comment);
This assumes that the Root object has an incidents property (files, comments) that's initialized to an empty List<Incidents> (List<Files>, List<Comments>). You will need to write that code if it doesn't exist.
Again, any decent ORM (Entity Framework, NHibernate, etc.) will provide all of this code for you "out of the box", so you wouldn't even have to write this method at all.

Entity Framework Core Database First not render Model of Stored Procedure?

Database
I have Stored Procedure name GetAllCustomer
I have Stored Procedure in the EntityFramework 6x and in Project ASP.NET MVC5.
I'm using by calling db.GetAllCustomer.ToList();
But in the EntityFramework Core it's not rendering DbSet and I need to create it manually. It can't use EF 6x.
Here is the image of my works:
Is there a way to call Stored Procedure as simple as EF 6x??

EF Core Power Tools can map stored procedures for you, it is not a built in feature of EF Core.
Sample user code:
using (var db = new NorthwindContext())
{
var procedures = new NorthwindContextProcedures(db);
var orders = await procedures.CustOrderHist("ALFKI");
foreach (var order in orders)
Console.WriteLine($"{order.ProductName}: {order.Total}");
var outOverallCount = new OutputParameter<int?>();
var customers = await procedures.SP_GET_TOP_IDS(10, outOverallCount);
Console.WriteLine($"Db contains {outOverallCount.Value} Customers.");
foreach (var customer in customers)
Console.WriteLine(customer.CustomerId);
}
Read more here: https://github.com/ErikEJ/EFCorePowerTools/wiki/Reverse-Engineering#sql-server-stored-procedures

You can use custom ExecuteQuery in your context for any use including stored procedure.
public List<T> ExecuteQuery<T>(string query) where T : class, new()
{
using (var command = Context.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
Context.Database.OpenConnection();
List<T> result;
using (var reader = command.ExecuteReader())
{
result = new List<T>();
var columns = new T().GetType().GetProperties().ToList();
while (reader.Read())
{
var obj = new T();
for (var i = 0; i < reader.FieldCount; i++)
{
var name = reader.GetName(i);
var prop = columns.FirstOrDefault(a => a.Name.ToLower().Equals(name.ToLower()));
if (prop == null)
continue;
var val = reader.IsDBNull(i) ? null : reader[i];
prop.SetValue(obj, val, null);
}
result.Add(obj);
}
return result;
}
}
}
Usage:
db.ExecuteQuery<YOUR_MODEL_DEPEND_ON_RETURN_RESULT>("SELECT FIELDS FROM YOUR_TABLE_NAME")
db.ExecuteQuery<YOUR_MODEL_DEPEND_ON_RETURN_RESULT>("EXEC YOUR_SP_NAME")
db.ExecuteQuery<YOUR_MODEL_DEPEND_ON_RETURN_RESULT>("EXEC YOUR_SP_NAME #Id = 10")
To reduce errors, create query string easier and do faster, I use several other methods, and I put stored procedures names in a static class.
For example, I have something like this to get customer list:
/// <param name="parameters">The model contains all SP parameters</param>
public List<customerGetDto> Get(CustomerSpGetParameters parameters = null)
{
//StoredProcedures.Customer.Get Is "sp_GetAllCustomers"
//CreateSqlQueryForSp creates a string with stored procedure name and parameters
var query = _publicMethods.CreateSqlQueryForSp(StoredProcedures.Request.Get, parameters);
//For example, query= "Exec sp_GetAllCustomers #active = 1,#level = 3,...."
return _unitOfWork.ExecuteQuery<RequestGetDto>(query);
}

Using two dimensional array as Dapper parameter list

In Dapper it is possible to query a list with where in using this code:
var sql = "SELECT * FROM Invoice WHERE Kind IN #Kind;";
using (var connection = My.ConnectionFactory())
{
connection.Open();
var invoices = connection.Query<Invoice>(sql, new {Kind = new[] {InvoiceKind.StoreInvoice, InvoiceKind.WebInvoice}}).ToList();
}
But I'd like to query a list of products by multiple suppliers and their product code. So to do this, I tried creating an array of arrays. This is my method in my repository:
public Dictionary<int, Dictionary<string, Product>> GetGroupedListByRelationIdAndProductCode(Dictionary<int, List<string>> productKeysByRelationId)
{
Dictionary<int, Dictionary<string, Product>> list = new Dictionary<int, Dictionary<string, Product>>();
string sql = "SELECT * FROM Products WHERE 1=1 ";
int i = 0;
foreach (KeyValuePair<int, List<string>> productKeys in productKeysByRelationId)
{
sql = sql + " AND (ManufacturerRelationId = " + productKeys.Key + " AND ManufacturerProductCode in #ProductCodeList[" + i + "] )";
++i;
}
using (var conn = _connectionFactory.CreateConnection())
{
conn.Open();
var param = new { ProductCodeList = productKeysByRelationId.Select(x => x.Value.ToArray()).ToArray() };
var productsList = conn.Query<Product>(sql, param).ToList();
if (productsList.Count > 0)
{
foreach (var product in productsList)
{
list[product.ManufacturerRelationId][product.ManufacturerProductCode] = product;
}
}
}
return list;
}
This gives me this error though: System.ArgumentException: 'No mapping exists from object type System.String[] to a known managed provider native type.'
Any suggestions on how to do this?

You have several problems with your code.
If you have more than one item in your productKeysByRelationId you will end up getting SQL like:
WHERE 1=1 AND ManufacturerRelationId = 1 ... AND ManufacturerRelationId = 2
That's not likely to return any results, you need to sprinkle some OR's in there.
The error you are getting is because you have something like:
AND ManufacturerProductCode in #ProductCodeList[0]
Dapper cannot handle that. It mostly expects the query parameters to be an object with aptly named members of a few simple types or array.
Thankfully Dapper has a solution to that, DynamicParameters to the rescue!
You can construct your query like this:
var queryParams = new DynamicParameters();
foreach (var productKeys in productKeysByRelationId)
{
sql = sql + $" ... AND ManufacturerProductCode in #ProductCodeList{i} )";
queryParams.Add($"ProductCodeList{i}", productKeys.Value);
i++;
}
Now you have your query parameters in the right format, so you can just do this:
var productsList = conn.Query<Product>(sql, queryParams).ToList();
That ought to fix it, but you really should try to parameterize the ManufacturerRelationId also. It's not just about SQL injection, there might be some SQL cache performance related things.
You might also gain some clarity in code by using the SqlBuilder from Dapper.Contrib.

Dapper query with dynamic list of filters

I have a c# mvc app using Dapper. There is a list table page which has several optional filters (as well as paging). A user can select (or not) any of several (about 8 right now but could grow) filters, each with a drop down for a from value and to value. So, for example, a user could select category "price" and filter from value "$100" to value "$200". However, I don't know how many categories the user is filtering on before hand and not all of the filter categories are the same type (some int, some decimal/double, some DateTime, though they all come in as string on FilterRange).
I'm trying to build a (relatively) simple yet sustainable Dapper query for this. So far I have this:
public List<PropertySale> GetSales(List<FilterRange> filterRanges, int skip = 0, int take = 0)
{
var skipTake = " order by 1 ASC OFFSET #skip ROWS";
if (take > 0)
skipTake += " FETCH NEXT #take";
var ranges = " WHERE 1 = 1 ";
for(var i = 0; i < filterRanges.Count; i++)
{
ranges += " AND #filterRanges[i].columnName BETWEEN #filterRanges[i].fromValue AND #filterRanges[i].toValue ";
}
using (var conn = OpenConnection())
{
string query = #"Select * from Sales "
+ ranges
+ skipTake;
return conn.Query<Sale>(query, new { filterRanges, skip, take }).AsList();
}
}
I Keep getting an error saying "... filterRanges cannot be used as a parameter value"
Is it possible to even do this in Dapper? All of the IEnumerable examples I see are where in _ which doesn't fit this situation. Any help is appreciated.

You can use DynamicParameters class for generic fields.
Dictionary<string, object> Filters = new Dictionary<string, object>();
Filters.Add("UserName", "admin");
Filters.Add("Email", "admin#admin.com");
var builder = new SqlBuilder();
var select = builder.AddTemplate("select * from SomeTable /**where**/");
var parameter = new DynamicParameters();
foreach (var filter in Filters)
{
parameter.Add(filter.Key, filter.Value);
builder.Where($"{filter.Key} = #{filter.Key}");
}
var searchResult = appCon.Query<ApplicationUser>(select.RawSql, parameter);

You can use a list of dynamic column values but you cannot do this also for the column name other than using string format which can cause a SQL injection.
You have to validate the column names from the list in order to be sure that they really exist before using them in a SQL query.
This is how you can use the list of filterRanges dynamically :
const string sqlTemplate = "SELECT /**select**/ FROM Sale /**where**/ /**orderby**/";
var sqlBuilder = new SqlBuilder();
var template = sqlBuilder.AddTemplate(sqlTemplate);
sqlBuilder.Select("*");
for (var i = 0; i < filterRanges.Count; i++)
{
sqlBuilder.Where($"{filterRanges[i].ColumnName} = #columnValue", new { columnValue = filterRanges[i].FromValue });
}
using (var conn = OpenConnection())
{
return conn.Query<Sale>(template.RawSql, template.Parameters).AsList();
}

You can easily create that dynamic condition using DapperQueryBuilder:
using (var conn = OpenConnection())
{
var query = conn.QueryBuilder($#"
SELECT *
FROM Sales
/**where**/
order by 1 ASC
OFFSET {skip} ROWS FETCH NEXT {take}
");
foreach (var filter in filterRanges)
query.Where($#"{filter.ColumnName:raw} BETWEEN
{filter.FromValue.Value} AND {filter.ToValue.Value}");
return conn.Query<Sale>(query, new { filterRanges, skip, take }).AsList();
}
Or without the magic word /**where**/:
using (var conn = OpenConnection())
{
var query = conn.QueryBuilder($#"
SELECT *
FROM Sales
WHERE 1=1
");
foreach (var filter in filterRanges)
query.Append($#"{filter.ColumnName:raw} BETWEEN
{filter.FromValue.Value} AND {filter.ToValue.Value}");
query.Append($"order by 1 ASC OFFSET {skip} ROWS FETCH NEXT {take}");
return conn.Query<Sale>(query, new { filterRanges, skip, take }).AsList();
}
The output is fully parametrized SQL, even though it looks like we're doing plain string concatenation.
Disclaimer: I'm one of the authors of this library

I was able to find a solution for this. The key was to convert the List to a Dictionary. I created a private method:
private Dictionary<string, object> CreateParametersDictionary(List<FilterRange> filters, int skip = 0, int take = 0)
{
var dict = new Dictionary<string, object>()
{
{ "#skip", skip },
{ "#take", take },
};
for (var i = 0; i < filters.Count; i++)
{
dict.Add($"column_{i}", filters[i].Filter.Description);
// some logic here which determines how you parse
// I used a switch, not shown here for brevity
dict.Add($"#fromVal_{i}", int.Parse(filters[i].FromValue.Value));
dict.Add($"#toVal_{i}", int.Parse(filters[i].ToValue.Value));
}
return dict;
}
Then to build my query,
var ranges = " WHERE 1 = 1 ";
for(var i = 0; i < filterRanges.Count; i++)
ranges += $" AND {filter[$"column_{i}"]} BETWEEN #fromVal_{i} AND #toVal_{i} ";
Special note: Be very careful here as the column name is not a parameter and you could open your self up to injection attacks (as #Popa noted in his answer). In my case those values come from an enum class and not from user in put so I am safe.
The rest is pretty straight forwared:
using (var conn = OpenConnection())
{
string query = #"Select * from Sales "
+ ranges
+ skipTake;
return conn.Query<Sale>(query, filter).AsList();
}

Convert Function Based on TSQL with Function Based with Entity Framework

I have two functions that each return the same list of objects. But, the one that uses TSQL is much faster than the one using Entity Framework and I do not understand why one would be faster than the other. Is it possible to modify my EF function to work as fast as the TSQL one?
Any help will be appreciated. My code is below:
TSQL:
public static List<ChartHist> ListHistory_PureSQL()
{
List<DataRow> listDataRow = null;
string srtQry = #"Select LoginHistoryID,
LoginDuration as LoginDuration_Pass,
0 as LoginDuration_Fail,
LoginDateTime,
LoginLocationID,
LoginUserEmailID,
LoginApplicationID,
LoginEnvironmentID,
ScriptFrequency,
LoginStatus,
Reason
From LoginHistory
Where LoginStatus = 'Pass'
UNION
Select LoginHistoryID,
0 as LoginDuration_Pass,
LoginDuration as LoginDuration_Fail,
LoginDateTime,
LoginLocationID,
LoginUserEmailID,
LoginApplicationID,
LoginEnvironmentID,
ScriptFrequency,
LoginStatus,
Reason
From LoginHistory
Where LoginStatus = 'Fail'";
using (SqlConnection conn = new SqlConnection(Settings.ConnectionString))
{
using (SqlCommand objCommand = new SqlCommand(srtQry, conn))
{
objCommand.CommandType = CommandType.Text;
DataTable dt = new DataTable();
SqlDataAdapter adp = new SqlDataAdapter(objCommand);
conn.Open();
adp.Fill(dt);
if (dt != null)
{
listDataRow = dt.AsEnumerable().ToList();
}
}
}
var listChartHist = (from p in listDataRow
select new ChartHist
{
LoginHistoryID = p.Field<Int32>("LoginHistoryID"),
LoginDuration_Pass = p.Field<Int32>("LoginDuration_Pass"),
LoginDuration_Fail = p.Field<Int32>("LoginDuration_Fail"),
LoginDateTime = p.Field<DateTime>("LoginDateTime"),
LoginLocationID = p.Field<Int32>("LoginLocationID"),
LoginUserEmailID = p.Field<Int32>("LoginUserEmailID"),
LoginApplicationID = p.Field<Int32>("LoginApplicationID"),
LoginEnvironmentID = p.Field<Int32>("LoginEnvironmentID"),
ScriptFrequency = p.Field<Int32>("ScriptFrequency"),
LoginStatus = p.Field<String>("LoginStatus"),
Reason = p.Field<String>("Reason")
}).ToList();
return listChartHist;
}
EF:
public static List<ChartHist> ListHistory()
{
using (var db = new LatencyDBContext())
{
var loginHist = (from hist in db.LoginHistories
select new { LoginHistory = hist }).ToList();
//PUT LOGIN HISTORY RECORDS INTO A LOCAL LIST
var listHistory = new List<ChartHist>();
foreach (var item in loginHist)
{
var localHistData = new ChartHist();
localHistData.LoginHistoryID = item.LoginHistory.LoginHistoryID;
//split up the duration for pass and fail values
if (item.LoginHistory.LoginStatus.ToUpper() == "PASS")
{
localHistData.LoginDuration_Pass = Convert.ToDouble(item.LoginHistory.LoginDuration);
localHistData.LoginDuration_Fail = 0;
}
else if (item.LoginHistory.LoginStatus.ToUpper() == "FAIL")
{
localHistData.LoginDuration_Pass = 0;
localHistData.LoginDuration_Fail = Convert.ToDouble(item.LoginHistory.LoginDuration);
}
localHistData.LoginDateTime = item.LoginHistory.LoginDateTime;
localHistData.LoginLocationID = item.LoginHistory.LoginLocationID;
localHistData.LoginUserEmailID = item.LoginHistory.LoginUserEmailID;
localHistData.LoginApplicationID = item.LoginHistory.LoginApplicationID;
localHistData.LoginEnvironmentID = item.LoginHistory.LoginEnvironmentID;
localHistData.LoginStatus = item.LoginHistory.LoginStatus;
localHistData.Reason = item.LoginHistory.Reason;
localHistData.ScriptFrequency = item.LoginHistory.ScriptFrequency;
listHistory.Add(localHistData);
}
return listHistory;
}
}

Of course EF will take longer to execute than a plain old SQL query, and there's very little that you can do about it (except write the most optimal LINQ queries that you can).
There's a very simple reason why this is so. Running a direct SQL command will just send back the data, with no muss and no fuss attached to it, waiting for you to do the data manipulations to get it to the point where it fits nicely into whatever data structure you want it in. Running EF, on the other hand, means that not only does it run the SQL command, but it massages the data for you into objects that you can manipulate right away. That extra action of going through ADO.NET and converting the data into the objects automatically means that it will take longer than just doing the plain SQL query.
On the flip side of that coin, however, EF does provide a very nice and simple way to debug and solve whatever problems you might have from a specific query/function (like by any exceptions thrown).

I can't performance test this, but try this solution instead before you remove EF entirely:
var loginHist = db.LoginHistories.Where(item => item.LoginStatus.ToUpper() == "PASS" || item.LoginStatus.ToUpper() == "FAIL")
.Select(item => new ChartHist()
{
LoginHistoryID = item.LoginHistoryID,
LoginDuration_Pass = item.LoginStatus.ToUpper() == "PASS" ? Convert.ToDouble(item.LoginDuration) : 0,
LoginDuration_Fail = item.LoginStatus.ToUpper() == "FAIL" ? Convert.ToDouble(item.LoginDuration) : 0,
LoginDateTime = item.LoginDateTime,
LoginLocationID = item.LoginLocationID,
LoginUserEmailID = item.LoginUserEmailID,
LoginApplicationID = item.LoginApplicationID,
LoginEnvironmentID = item.LoginEnvironmentID,
LoginStatus = item.LoginStatus,
Reason = item.Reason,
ScriptFrequency = item.ScriptFrequency,
});
return loginHist.ToList();
This is the "correct" way to populate a new object from a select. It will only retrieve the data you care about, and will put it directly into the object, rather than converting it into an object and then converting it again, from one object to another.
Note: I prefer the functional calls to the from / select form, but it'd be correct either way.