We Keep Coding

Dynamic LINQ accent insensitive string queries

I am developing a C# .Net MVC application and trying to implement a generic search method for entity fields. As our pages are growing i don't want to code a search method each time a new page is added.
For that, i am using Dynamic.Core LINQ Queries : Check it at : https://dynamic-linq.net/basic-simple-query
the way i implemented it works the following way : at user input in the view, the app sends an ajax request to the controller telling it to search that specific value and then display the new list where the previous one.
The problem is : i could make it case insensitive but not accent insensitive and was wondering if anyone could help me with that.
Here is my code :
public static List<T> SearchEntityList<T>(this IQueryable<T> entityList, string searchBy, List<string> fieldsToCheck)
{
if (searchBy == null)
return entityList.ToList();
searchBy = searchBy.ToLower().RemoveDiacriticsUtil();
// Dynamic LINQ Library
string query = "";
foreach (string str in fieldsToCheck)
{
query += str + ".ToString().ToLower().Contains(#0) ||";
}
if (query != null)
{
// Removes the last "OR" inserted on the foreach here on top
query = query.Substring(0,query.Length - 3);
try
{
entityList = entityList.Where(query, searchBy);
}
catch (Exception e)
{
// query is wrong, list wont be filtered.
return entityList.ToList();
}
}
List<T> filteredList = entityList.ToList(); ;
return filteredList;
}
the method receives a list of string representing the fields to check, for example : "Username"
then a string query is built and checked with the database.
This code works as expected and is case insensitive, now i want to add accent insensitive to it.
i modify this line
query += str + ".ToString().ToLower().Contains(#0) ||";
with this one
query += str + "Collate(" + str + ".toString(), \"SQL_Latin1_General_CP1_CI_AI\").Contains(#0) ||";
and now i cannot make it work.
Got this error :
"No applicable method 'Collate' exists in type '...'"
I tested a lot of other stuff such as RemoveDiacritics, etc.. but they dont work with dynamic string linq queries...
Was wondering if anyone already had the same problem. Thanks !

You're on the right track.
You need to register your custom extension methods for dynamic linq before us use it:
// Registration for the default custom type handler
[DynamicLinqType]
public static class MyExtensions
{
// taken from: https://stackoverflow.com/questions/359827/ignoring-accented-letters-in-string-comparison/368850
public static string RemoveDiacritics(this string text)
{
return string.Concat(
text.Normalize(NormalizationForm.FormD)
.Where(ch => CharUnicodeInfo.GetUnicodeCategory(ch) !=
UnicodeCategory.NonSpacingMark)
).Normalize(NormalizationForm.FormC);
}
}
public static class Entry
{
public static void Main(string[] args)
{
// your input
var query = "éè";
// clean it using your extension method
var cleanQuery = query.RemoveDiacritics();
// a test set for this demo
IQueryable<string> testSet = new EnumerableQuery<string>(new List<string>()
{
"soméè", "tèèst", "séét", "BBeeBB", "NoMatchHere"
});
var results = testSet.Where("it.RemoveDiacritics().Contains(#0)", cleanQuery);
Debug.Assert(results.Count() == 4);
}
}

unable to get data from database by passing string value in entity frame work asp.net MVC

i am trying to get data from database passing a string value. but get null value instead of the data.
i have tried the following code
order getCustomerOrder(string or_n)
{
using (foodorderingEntities db = new foodorderingEntities ())
{
var result = db.orders.Where(or => or.order_no == or_n).FirstOrDefault();
return result;
}
}
please some one guide me to solve this problem.

Please paste your order object, so we know if its reference object/primitive etc, you can use this to understand the two ways to retrieve orders.
I would recommend you read this answer & this MSDN string compare, if your default culture is causing an issue in the comparison, it will help you understand whats going on
Options 1:
// Query syntax
IEnumerable<CustomerOrder> queryResultsCustomerOrder =
from order in orders
where order.number == myOrderNumber
select order;
Options 2:
// Method-based syntax
IEnumerable<CustomerOrder> queryResultsCustomerOrder2 = orders.Where(order => order.Number == myOrderNumber);
using the above, now you can get however many orders the customer has. I am assuming your order is a number, but you can change it to whatever like a string.
Int based order comparison sample
IEnumerable<CustomerOrder> getCustomerOrder(int myOrderNumber)
{
if(myOrderNumber <1) return null;
using (foodorderingEntities dbContextOrderSet = new foodorderingEntities())
{
IEnumerable<CustomerOrder> resultsOneOrManyOrders = orders.Where(order => order.Number == myOrderNumber);
return resultsOneOrManyOrders ;
}
}
string based order comparison
IEnumerable<CustomerOrder> getCustomerOrder(string myOrderNumber)
{
//no orders
if(String.IsNullOrEmpty(myOrderNumber)) return null;
using (foodorderingEntities dbContextOrderSet = new foodorderingEntities())
{
IEnumerable<CustomerOrder> resultsOneOrManyOrders = orders.Where(order => order.Number == myOrderNumber);
// you can replace the *** comparison with .string.Compare and try inside the block
return resultsOneOrManyOrders ;
}
}

DapperExtensions: Add "insert ... update on duplicate key"

Now I'm using Dapper + Dapper.Extensions. And yes, it's easy and awesome. But I faced with a problem: Dapper.Extensions has only Insert command and not InsertUpdateOnDUplicateKey. I want to add such method but I don't see good way to do it:
I want to make this method generic like Insert
I can't get cached list of properties for particular type because I don't want to use reflection directly to build raw sql
Possible way here to fork it on github but I want to make it in my project only. Does anybody know how to extend it? I understand this feature ("insert ... update on duplicate key") is supported only in MySQL. But I can't find extension points in DapperExtensions to add this functionality outside.
Update: this is my fork https://github.com/MaximTkachenko/Dapper-Extensions/commits/master

This piece of code has helped me enormously in MySQL -related projects, I definitely owe you one.
I do a lot of database-related development on both MySQL and MS SQL. I also try to share as much code as possible between my projects.
MS SQL has no direct equivalent for "ON DUPLICATE KEY UPDATE", so I was previously unable to use this extension when working with MS SQL.
While migrating a web application (that leans heavily on this Dapper.Extensions tweak) from MySQL to MS SQL, I finally decided to do something about it.
This code uses the "IF EXISTS => UPDATE ELSE INSERT" approach that basically does the same as "ON DUPLICATE KEY UPDATE" on MySQL.
Please note: the snippet assumes that you are taking care of transactions outside this method. Alternatively you could append "BEGIN TRAN" to the beginning and "COMMIT" to the end of the generated sql string.
public static class SqlGeneratorExt
{
public static string InsertUpdateOnDuplicateKey(this ISqlGenerator generator, IClassMapper classMap, bool hasIdentityKeyWithValue = false)
{
var columns = classMap.Properties.Where(p => !(p.Ignored || p.IsReadOnly || (p.KeyType == KeyType.Identity && !hasIdentityKeyWithValue))).ToList();
var keys = columns.Where(c => c.KeyType != KeyType.NotAKey).Select(p => $"{generator.GetColumnName(classMap, p, false)}=#{p.Name}");
var nonkeycolumns = classMap.Properties.Where(p => !(p.Ignored || p.IsReadOnly) && p.KeyType == KeyType.NotAKey).ToList();
if (!columns.Any())
{
throw new ArgumentException("No columns were mapped.");
}
var tablename = generator.GetTableName(classMap);
var columnNames = columns.Select(p => generator.GetColumnName(classMap, p, false));
var parameters = columns.Select(p => generator.Configuration.Dialect.ParameterPrefix + p.Name);
var valuesSetters = nonkeycolumns.Select(p => $"{generator.GetColumnName(classMap, p, false)}=#{p.Name}").ToList();
var where = keys.AppendStrings(seperator: " and ");
var sqlbuilder = new StringBuilder();
sqlbuilder.AppendLine($"IF EXISTS (select * from {tablename} WITH (UPDLOCK, HOLDLOCK) WHERE ({where})) ");
sqlbuilder.AppendLine(valuesSetters.Any() ? $"UPDATE {tablename} SET {valuesSetters.AppendStrings()} WHERE ({where}) " : "SELECT 0 ");
sqlbuilder.AppendLine($"ELSE INSERT INTO {tablename} ({columnNames.AppendStrings()}) VALUES ({parameters.AppendStrings()}) ");
return sqlbuilder.ToString();
}
}

Actually I closed my pull request and remove my fork because:
I see some open pull requests created in 2014
I found a way "inject" my code in Dapper.Extensions.
I remind my problem: I want to create more generic queries for Dapper.Extensions. It means I need to have access to mapping cache for entities, SqlGenerator etc. So here is my way. I want to add ability to make INSERT .. UPDATE ON DUPLICATE KEY for MySQL. I created extension method for ISqlGenerator
public static class SqlGeneratorExt
{
public static string InsertUpdateOnDuplicateKey(this ISqlGenerator generator, IClassMapper classMap)
{
var columns = classMap.Properties.Where(p => !(p.Ignored || p.IsReadOnly || p.KeyType == KeyType.Identity));
if (!columns.Any())
{
throw new ArgumentException("No columns were mapped.");
}
var columnNames = columns.Select(p => generator.GetColumnName(classMap, p, false));
var parameters = columns.Select(p => generator.Configuration.Dialect.ParameterPrefix + p.Name);
var valuesSetters = columns.Select(p => string.Format("{0}=VALUES({1})", generator.GetColumnName(classMap, p, false), p.Name));
string sql = string.Format("INSERT INTO {0} ({1}) VALUES ({2}) ON DUPLICATE KEY UPDATE {3}",
generator.GetTableName(classMap),
columnNames.AppendStrings(),
parameters.AppendStrings(),
valuesSetters.AppendStrings());
return sql;
}
}
One more extension method for IDapperImplementor
public static class DapperImplementorExt
{
public static void InsertUpdateOnDuplicateKey<T>(this IDapperImplementor implementor, IDbConnection connection, IEnumerable<T> entities, int? commandTimeout = null) where T : class
{
IClassMapper classMap = implementor.SqlGenerator.Configuration.GetMap<T>();
var properties = classMap.Properties.Where(p => p.KeyType != KeyType.NotAKey);
string emptyGuidString = Guid.Empty.ToString();
foreach (var e in entities)
{
foreach (var column in properties)
{
if (column.KeyType == KeyType.Guid)
{
object value = column.PropertyInfo.GetValue(e, null);
string stringValue = value.ToString();
if (!string.IsNullOrEmpty(stringValue) && stringValue != emptyGuidString)
{
continue;
}
Guid comb = implementor.SqlGenerator.Configuration.GetNextGuid();
column.PropertyInfo.SetValue(e, comb, null);
}
}
}
string sql = implementor.SqlGenerator.InsertUpdateOnDuplicateKey(classMap);
connection.Execute(sql, entities, null, commandTimeout, CommandType.Text);
}
}
Now I can create new class derived from Database class to use my own sql
public class Db : Database
{
private readonly IDapperImplementor _dapperIml;
public Db(IDbConnection connection, ISqlGenerator sqlGenerator) : base(connection, sqlGenerator)
{
_dapperIml = new DapperImplementor(sqlGenerator);
}
public void InsertUpdateOnDuplicateKey<T>(IEnumerable<T> entities, int? commandTimeout) where T : class
{
_dapperIml.InsertUpdateOnDuplicateKey(Connection, entities, commandTimeout);
}
}
Yeah, it's required to create another DapperImplementor instance because DapperImplementor instance from base class is private :(. So now I can use my Db class to call my own generic sql queries and native queries from Dapper.Extension. Examples of usage Database class instead of IDbConnection extensions can be found here.

MongoDB: How to filter strings only containing specific characters?

I have a mongo database in which are all english words stored.
I want to get a collection of words which only contain specific characters.
e.g.: in database [ash, clash, bash, has]
asking for [ash] i want to get [has, ash] as an output.
perhaps there is a fast way, that i dont have to build some special datastructures.
i will code in c#.

If you are using the .Net driver for mongo then its LINQ implementation should be straight forward to use. More on that here MONGO LINQThen it would be (some exceptions) just as targeting an ordinary IEnumerable.
Your problem is probably solvable using regex which could be used in a Where clause using linq. Regex have a tendency of being hard to debug so until you find the correct regex here is an alternative example (it can be further optimized).
private void Test()
{
// fetching from database
// var samples = database.GetCollection<T>"collectionname").AsQueryable<Employee>();
string[] samples = new[] {"nnn", "shhhh", "has", "s", "ash"};
string chars = "ash";
var matches = samples.Where(x => IsMatch(x, chars));
}
private bool IsMatch(string sample, string matchChars)
{
if (sample.Length != matchChars.Length)
return false;
sample = matchChars.Aggregate(sample, (current, c) => current.Replace(c, ' '));
return sample.Trim().Length == 0;
}

Try this as a query... I am searching my ID where it contains 52C.
{ "_id": { $regex: '.*\Q52C\E.*', $options: 'i' } }
C# Equivalent:
await collection.Find(new BsonDocument("_id", new BsonDocument { { "$regex", $".*\Q{YourString}\E.*" },
{ "$options, "i" } })).ToListAsync();

Regex Replace to assist Orderby in LINQ

I'm using LINQ to SQL to pull records from a database, sort them by a string field, then perform some other work on them. Unfortunately the Name field that I'm sorting by comes out of the database like this
Name
ADAPT1
ADAPT10
ADAPT11
...
ADAPT2
ADAPT3
I'd like to sort the Name field in numerical order. Right now I'm using the Regex object to replace "ADAPT1" with "ADAPT01", etc. I then sort the records again using another LINQ query. The code I have for this looks like
var adaptationsUnsorted = from aun in dbContext.Adaptations
where aun.EventID == iep.EventID
select new Adaptation
{
StudentID = aun.StudentID,
EventID = aun.EventID,
Name = Regex.Replace(aun.Name,
#"ADAPT([0-9])$", #"ADAPT0$1"),
Value = aun.Value
};
var adaptationsSorted = from ast in adaptationsUnsorted
orderby ast.Name
select ast;
foreach(Adaptation adaptation in adaptationsSorted)
{
// do real work
}
The problem I have is that the foreach loop throws the exception
System.NotSupportedException was unhandled
Message="Method 'System.String Replace(System.String, System.String,
System.String)' has no supported translation to SQL."
Source="System.Data.Linq"
I'm also wondering if there's a cleaner way to do this with just one LINQ query. Any suggestions would be appreciated.

Force the hydration of the elements by enumerating the query (call ToList). From that point on, your operations will be against in-memory objects and those operations will not be translated into SQL.
List<Adaptation> result =
dbContext.Adaptation
.Where(aun => aun.EventID = iep.EventID)
.ToList();
result.ForEach(aun =>
aun.Name = Regex.Replace(aun.Name,
#"ADAPT([0-9])$", #"ADAPT0$1")
);
result = result.OrderBy(aun => aun.Name).ToList();

Implement a IComparer<string> with your logic:
var adaptationsUnsorted = from aun in dbContext.Adaptations
where aun.EventID == iep.EventID
select new Adaptation
{
StudentID = aun.StudentID,
EventID = aun.EventID,
Name = aun.Name,
Value = aun.Value
};
var adaptationsSorted = adaptationsUnsorted.ToList<Adaptation>().OrderBy(a => a.Name, new AdaptationComparer ());
foreach (Adaptation adaptation in adaptationsSorted)
{
// do real work
}
public class AdaptationComparer : IComparer<string>
{
public int Compare(string x, string y)
{
string x1 = Regex.Replace(x, #"ADAPT([0-9])$", #"ADAPT0$1");
string y1 = Regex.Replace(y, #"ADAPT([0-9])$", #"ADAPT0$1");
return Comparer<string>.Default.Compare(x1, y1);
}
}
I didn't test this code but it should do the job.

I wonder if you can add a calculated+persisted+indexed field to the database, that does this for you. It would be fairly trivial to write a UDF that gets the value as an integer (just using string values), but then you can sort on this column at the database. This would allow you to use Skip and Take effectively, rather than constantly fetching all the data to the .NET code (which simply doesn't scale).