We Keep Coding

DapperExtensions: Add "insert ... update on duplicate key"

Now I'm using Dapper + Dapper.Extensions. And yes, it's easy and awesome. But I faced with a problem: Dapper.Extensions has only Insert command and not InsertUpdateOnDUplicateKey. I want to add such method but I don't see good way to do it:
I want to make this method generic like Insert
I can't get cached list of properties for particular type because I don't want to use reflection directly to build raw sql
Possible way here to fork it on github but I want to make it in my project only. Does anybody know how to extend it? I understand this feature ("insert ... update on duplicate key") is supported only in MySQL. But I can't find extension points in DapperExtensions to add this functionality outside.
Update: this is my fork https://github.com/MaximTkachenko/Dapper-Extensions/commits/master

This piece of code has helped me enormously in MySQL -related projects, I definitely owe you one.
I do a lot of database-related development on both MySQL and MS SQL. I also try to share as much code as possible between my projects.
MS SQL has no direct equivalent for "ON DUPLICATE KEY UPDATE", so I was previously unable to use this extension when working with MS SQL.
While migrating a web application (that leans heavily on this Dapper.Extensions tweak) from MySQL to MS SQL, I finally decided to do something about it.
This code uses the "IF EXISTS => UPDATE ELSE INSERT" approach that basically does the same as "ON DUPLICATE KEY UPDATE" on MySQL.
Please note: the snippet assumes that you are taking care of transactions outside this method. Alternatively you could append "BEGIN TRAN" to the beginning and "COMMIT" to the end of the generated sql string.
public static class SqlGeneratorExt
{
public static string InsertUpdateOnDuplicateKey(this ISqlGenerator generator, IClassMapper classMap, bool hasIdentityKeyWithValue = false)
{
var columns = classMap.Properties.Where(p => !(p.Ignored || p.IsReadOnly || (p.KeyType == KeyType.Identity && !hasIdentityKeyWithValue))).ToList();
var keys = columns.Where(c => c.KeyType != KeyType.NotAKey).Select(p => $"{generator.GetColumnName(classMap, p, false)}=#{p.Name}");
var nonkeycolumns = classMap.Properties.Where(p => !(p.Ignored || p.IsReadOnly) && p.KeyType == KeyType.NotAKey).ToList();
if (!columns.Any())
{
throw new ArgumentException("No columns were mapped.");
}
var tablename = generator.GetTableName(classMap);
var columnNames = columns.Select(p => generator.GetColumnName(classMap, p, false));
var parameters = columns.Select(p => generator.Configuration.Dialect.ParameterPrefix + p.Name);
var valuesSetters = nonkeycolumns.Select(p => $"{generator.GetColumnName(classMap, p, false)}=#{p.Name}").ToList();
var where = keys.AppendStrings(seperator: " and ");
var sqlbuilder = new StringBuilder();
sqlbuilder.AppendLine($"IF EXISTS (select * from {tablename} WITH (UPDLOCK, HOLDLOCK) WHERE ({where})) ");
sqlbuilder.AppendLine(valuesSetters.Any() ? $"UPDATE {tablename} SET {valuesSetters.AppendStrings()} WHERE ({where}) " : "SELECT 0 ");
sqlbuilder.AppendLine($"ELSE INSERT INTO {tablename} ({columnNames.AppendStrings()}) VALUES ({parameters.AppendStrings()}) ");
return sqlbuilder.ToString();
}
}

Actually I closed my pull request and remove my fork because:
I see some open pull requests created in 2014
I found a way "inject" my code in Dapper.Extensions.
I remind my problem: I want to create more generic queries for Dapper.Extensions. It means I need to have access to mapping cache for entities, SqlGenerator etc. So here is my way. I want to add ability to make INSERT .. UPDATE ON DUPLICATE KEY for MySQL. I created extension method for ISqlGenerator
public static class SqlGeneratorExt
{
public static string InsertUpdateOnDuplicateKey(this ISqlGenerator generator, IClassMapper classMap)
{
var columns = classMap.Properties.Where(p => !(p.Ignored || p.IsReadOnly || p.KeyType == KeyType.Identity));
if (!columns.Any())
{
throw new ArgumentException("No columns were mapped.");
}
var columnNames = columns.Select(p => generator.GetColumnName(classMap, p, false));
var parameters = columns.Select(p => generator.Configuration.Dialect.ParameterPrefix + p.Name);
var valuesSetters = columns.Select(p => string.Format("{0}=VALUES({1})", generator.GetColumnName(classMap, p, false), p.Name));
string sql = string.Format("INSERT INTO {0} ({1}) VALUES ({2}) ON DUPLICATE KEY UPDATE {3}",
generator.GetTableName(classMap),
columnNames.AppendStrings(),
parameters.AppendStrings(),
valuesSetters.AppendStrings());
return sql;
}
}
One more extension method for IDapperImplementor
public static class DapperImplementorExt
{
public static void InsertUpdateOnDuplicateKey<T>(this IDapperImplementor implementor, IDbConnection connection, IEnumerable<T> entities, int? commandTimeout = null) where T : class
{
IClassMapper classMap = implementor.SqlGenerator.Configuration.GetMap<T>();
var properties = classMap.Properties.Where(p => p.KeyType != KeyType.NotAKey);
string emptyGuidString = Guid.Empty.ToString();
foreach (var e in entities)
{
foreach (var column in properties)
{
if (column.KeyType == KeyType.Guid)
{
object value = column.PropertyInfo.GetValue(e, null);
string stringValue = value.ToString();
if (!string.IsNullOrEmpty(stringValue) && stringValue != emptyGuidString)
{
continue;
}
Guid comb = implementor.SqlGenerator.Configuration.GetNextGuid();
column.PropertyInfo.SetValue(e, comb, null);
}
}
}
string sql = implementor.SqlGenerator.InsertUpdateOnDuplicateKey(classMap);
connection.Execute(sql, entities, null, commandTimeout, CommandType.Text);
}
}
Now I can create new class derived from Database class to use my own sql
public class Db : Database
{
private readonly IDapperImplementor _dapperIml;
public Db(IDbConnection connection, ISqlGenerator sqlGenerator) : base(connection, sqlGenerator)
{
_dapperIml = new DapperImplementor(sqlGenerator);
}
public void InsertUpdateOnDuplicateKey<T>(IEnumerable<T> entities, int? commandTimeout) where T : class
{
_dapperIml.InsertUpdateOnDuplicateKey(Connection, entities, commandTimeout);
}
}
Yeah, it's required to create another DapperImplementor instance because DapperImplementor instance from base class is private :(. So now I can use my Db class to call my own generic sql queries and native queries from Dapper.Extension. Examples of usage Database class instead of IDbConnection extensions can be found here.

Building a Dynamic Linq Query using Equals and an Array

I have been trying to solve the syntax for a dynamic linq query that is needed in my application.
I have a dynamic query that the where clause needs to be specified to either
GuidPrimaryKey is contained in a list of Guid OR
GuidPrimaryKey is equal to an item in a list of Guid (using some type of for-loop)
I have a Guid[] populated with over 5,000 keys. My Query is set up as
If I do this (as a test) it is successful
data = data.where("GuidPrimaryKey.Equals(#0)",array[0]);
as well as
data = data.where("GuidPrimaryKey.Equals(#0) OR GuidPrimaryKey.Equals(#1)",array[0], array[1]);
I have tried:data = data.where("GuidPrimaryKey.Contains(#0)",array); but that gives an error: No applicable method 'Contains' exists in type 'Guid'.
I also tried setting a loop to go through the elements in the array and set the where clause as a giant string, but that did not work either.
string s = "";
string p = ""
int counter = 0;
foreach(Guid g in Array)
{
s+= "GuidPrimaryKey.Equals(#" counter.ToString() + ") OR";
p += "Array[" counter.ToString() + "],";
counter++;
}
s = s.remove(s.length - 3, 3);
p = p.remove(p.length - 1, 1);
data = data.Where(s,p);
This gives me the error message: No Property or field '1' exists in type 'DynamicClass1'
Any ideas? I need to have the where clause build the query to check to see if the primary key (GuidPrimaryKey) exists in the list of keys (Guid[]).

I'm not sure if this works in the standard Dynamic Linq library, but I just tried this is my open-source version, and it works well:
var data = data.Where("GuidPrimaryKey in #0", array);
This also works:
var data = data.Where("#0.Contains(GuidPrimaryKey)", array);
Here is a full unit test I wrote to confirm this:
[TestMethod]
public void ExpressionTests_ContainsGuid()
{
//Arrange
//Generate some users with Id fields of type Guid
var userList = User.GenerateSampleModels(5, false);
var userQry = userList.AsQueryable();
//Generate a list of values that will fail.
var failValues = new List<Guid>() {
new Guid("{22222222-7651-4045-962A-3D44DEE71398}"),
new Guid("{33333333-8F80-4497-9125-C96DEE23037D}"),
new Guid("{44444444-E32D-4DE1-8F1C-A144C2B0424D}")
};
//Add a valid Guid so that this list will succeed.
var successValues = failValues.Concat(new[] { userList[0].Id }).ToArray();
//Act
var found1 = userQry.Where("Id in #0", successValues);
var found2 = userQry.Where("#0.Contains(Id)", successValues);
var notFound1 = userQry.Where("Id in #0", failValues);
var notFound2 = userQry.Where("#0.Contains(Id)", failValues);
//Assert
Assert.AreEqual(userList[0].Id, found1.Single().Id);
Assert.AreEqual(userList[0].Id, found2.Single().Id);
Assert.IsFalse(notFound1.Any());
Assert.IsFalse(notFound2.Any());
}

Dapper parameters not working

I'm trying to use the Dapper orm with the following simple query:
var sqlString = new StringBuilder();
sqlString.Append("select a.acct AccountNumber,");
sqlString.Append(" b.first_name FirstName,");
sqlString.Append(" b.last_name LastName,");
sqlString.Append(" a.rr RrNumber,");
sqlString.Append(" c.addr1 AddressLine1,");
sqlString.Append(" c.addr2 AddressLine2,");
sqlString.Append(" c.addr3 AddressLine3,");
sqlString.Append(" c.addr4 AddressLine4,");
sqlString.Append(" c.addr5 AddressLine5,");
sqlString.Append(" c.addr6 AddressLine6,");
sqlString.Append(" c.addr7 AddressLine7,");
sqlString.Append(" c.addr8 AddressLine8 ");
sqlString.Append("from (pub.mfclac as a left join pub.mfcl as b on a.client=b.client) ");
sqlString.Append("left join pub.mfclad as c on a.client=c.client ");
sqlString.Append("where a.acct = '#ZYX'");
var connection = new OdbcConnection(_connectionString);
var result = connection.Query(sqlString.ToString(),
new
{
ZYX = accountNumber
});
However when I execute this with an accountNumber known to exist, dapper returns nothing. So I tried to remove the quotes to verify that the parameter is in fact being replaced with the account number, however the error being returned from the server indicates a syntax error around "#ZYX". Which means dapper is not replacing the parameter with it's given value. Any ideas why this is happening? From the limited documentation out there, this should 'just work'.
Edit1
Couldn't get this to work. Using string.format to insert the parameter as a work around.

There are two issues here; firstly (although you note this in your question) where a.acct = '#ZYX', under SQL rules, does not make use of any parameter - it looks to match the literal string that happens to include an # sign. For SQL-Server (see note below), the correct usage would be where a.acct = #ZYX.
However! Since you are use OdbcConnection, named parameters do not apply. If you are actually connecting to something like SQL-Server, I would strongly recommend using the pure ADO.NET clients, which have better features and performance than ODBC. However, if ODBC is your only option: it does not use named parameters. Until a few days ago, this would have represented a major problem, but as per Passing query parameters in Dapper using OleDb, the code (but not yet the NuGet package) now supports ODBC. If you build from source (or wait for the next release), you should be able to use:
...
where a.acct = ?
in your command, and:
var result = connection.Query(sqlString.ToString(),
new {
anythingYouLike = accountNumber
});
Note that the name (anythingYouLike) is not used by ODBC, so can be... anything you like. In a more complex scenario, for example:
.Execute(sql, new { id = 123, name = "abc", when = DateTime.Now });
dapper uses some knowledge of how anonymous types are implemented to understand the original order of the values, so that they are added to the command in the correct sequence (id, name, when).
One final observation:
Which means dapper is not replacing the parameter with it's given value.
Dapper never replaces parameters with their given value. That is simply not the correct way to parameterize sql: the parameters are usually sent separately, ensuring:
there is no SQL injection risk
maximum query plan re-use
no issues of formatting
Note that some ADO.NET / ODBC providers could theoretically choose to implement things internally via replacement - but that is separate to dapper.

I landed here from dublicate question: Dapper must declare the scalar variable
Error: Must declare the scalar variable "#Name".
I created queries dynamically with this piece of code:
public static bool Insert<T>(T entity)
{
var tableName = entity.GetType().CustomAttributes.FirstOrDefault(x => x.AttributeType.Name == nameof(TableAttribute))?.ConstructorArguments?.FirstOrDefault().Value as string;
if (string.IsNullOrEmpty(tableName))
throw new Exception($"Cannot save {entity.GetType().Name}. Database models should have [Table(\"tablename\")] attribute.");
DBSchema.TryGetValue(tableName.ToLower(), out var fields);
using (var con = new SqlConnection(ConnectionString))
{
con.Open();
var sql = $"INSERT INTO [{tableName}] (";
foreach (var field in fields.Where(x => x != "id"))
{
sql += $"[{field}]"+",";
}
sql = sql.TrimEnd(',');
sql += ")";
sql += " VALUES (";
foreach (var field in fields.Where(x => x != "id"))
{
sql += "#"+field + ",";
}
sql = sql.TrimEnd(',');
sql += ")";
var affectedRows = con.Execute(sql, entity);
return affectedRows > 0;
}
}
And I got the same error when my models was like this:
[Table("Users")]
public class User
{
public string Name;
public string Age;
}
I changed them to this:
[Table("Users")]
public class User
{
public string Name { get; set; }
public string Age { get; set; }
}
And it solved the problem for me.

How to get the TableName a particular entity property is mapped to

Entity properties can be split across different tables, meaning that a single Entity can have it's columns mapped to different tables. How does one then, in code, retrieve the info on the particular table an entity property is mapped to.
foreach(PropertyInfo pi in typeof(DbContext).GetProperties())
{
if(pi.PropertyType.IsGenericType && pi.PropertyType.Name.Contains("DbSet"))
{
var t = pi.PropertyType.GetGenericArguments().FirstOrDefault();
var tables = t.GetCustomAttributes(true).OfType<TableAttribute>();
foreach (var entityProperty in t.GetProperties())
{
if (entityProperty.GetCustomAttributes(true).OfType<RequiredAttribute>().Any<RequiredAttribute>())
{
var fieldname = entity.Name;
//I need to match this column with the table it belongs to here
}
}
}
}
So far, I have the code below to get at the entity property, from the object itself, how do I determine the particular table the current property is mapped to, in my database? Thanks in advance.

Consider using the OR/M for the actual generation of the SQL you actually need.
var dbContext = new DbContext();
var objectContext = ((IObjectContextAdapter)dbContext).ObjectContext;
var set = objectContext.Set<Foo>();
var query = from x in set
where {stuff}
select new { x.Bar, x.Baz ...};
var objectQuery = (ObjectQuery)query;
var command = objectQuery.CommandText;
var parameters = objectQuery.Parameters;

How do I order a sql datasource of uniqueidentifiers in Linq by an array of uniqueindentifiers

I have a string list(A) of individualProfileId's (GUID) that can be in any order(used for displaying personal profiles in a specific order based on user input) which is stored as a string due to it being part of the cms functionality.
I also have an asp c# Repeater that uses a LinqDataSource to query against the individual table. This repeater needs to use the ordered list(A) to display the results in the order specified.
Which is what i am having problems with. Does anyone have any ideas?
list(A)
'CD44D9F9-DE88-4BBD-B7A2-41F7A9904DAC',
'7FF2D867-DE88-4549-B5C1-D3C321F8DB9B',
'3FC3DE3F-7ADE-44F1-B17D-23E037130907'
Datasource example
IndividualProfileId Name JobTitle EmailAddress IsEmployee
3FC3DE3F-7ADE-44F1-B17D-23E037130907 Joe Blo Director dsd#ad.com 1
CD44D9F9-DE88-4BBD-B7A2-41F7A9904DAC Maxy Dosh The Boss 1
98AB3AFD-4D4E-4BAF-91CE-A778EB29D959 some one a job 322#wewd.ocm 1
7FF2D867-DE88-4549-B5C1-D3C321F8DB9B Max Walsh CEO 1

There is a very simple (single-line) way of doing this, given that you get the employee results from the database first (so resultSetFromDatabase is just example data, you should have some LINQ query here that gets your results).
var a = new[] { "GUID1", "GUID2", "GUID3"};
var resultSetFromDatabase = new[]
{
new { IndividualProfileId = "GUID3", Name = "Joe Blo" },
new { IndividualProfileId = "GUID1", Name = "Maxy Dosh" },
new { IndividualProfileId = "GUID4", Name = "some one" },
new { IndividualProfileId = "GUID2", Name = "Max Walsh" }
};
var sortedResults = a.Join(res, s => s, e => e.IndividualProfileId, (s, e) => e);
It's impossible to have the datasource get the results directly in the right order, unless you're willing to write some dedicated SQL stored procedure. The problem is that you'd have to tell the database the contents of a. Using LINQ this can only be done via Contains. And that doesn't guarantee any order in the result set.

Turn the list(A), which you stated is a string, into an actual list. For example, you could use listAsString.Split(",") and then remove the 's from each element. I’ll assume the finished list is called list.
Query the database to retrieve the rows that you need, for example:
var data = db.Table.Where(row => list.Contains(row.IndividualProfileId));
From the data returned, create a dictionary keyed by the IndividualProfileId, for example:
var dic = data.ToDictionary(e => e.IndividualProfileId);
Iterate through the list and retrieve the dictionary entry for each item:
var results = list.Select(item => dic[item]).ToList();
Now results will have the records in the same order that the IDs were in list.