Linq to Entities : using ToLower() on NText fields - c#

I'm using SQL Server 2005, with a case sensitive database..
In a search function, I need to create a Linq To Entities (L2E) query with a "where" clause that compare several strings with the data in the database with these rules :
The comparison is a "Contains" mode, not strict compare : easy as the string's Contains() method is allowed in L2E
The comparison must be case insensitive : I use ToLower() on both elements to perform an insensitive comparison.
All of this performs really well but I ran into the following Exception :
"Argument data type ntext is invalid for argument 1 of lower function" on one of my fields.
It seems that the field is a NText field and I can't perform a ToLower() on that.
What could I do to be able to perform a case insensitive Contains() on that NText field ?

Never use .ToLower() to perform a case-insensitive comparison. Here's why:
It's possibly wrong (your client collation could be, say, Turkish, and your DB collation not).
It's highly inefficient; the SQL Emitted is LOWER instead of = with a case-insensitive collation.
Instead, use StringComparison.OrdinalIgnoreCase or StringComparison.CurrentCultureIgnoreCase:
var q = from f in Context.Foos
where f.Bar.Equals("hi", StringComparison.OrdinalIgnoreCase)
select f;
But for Contains() there's a problem: Unlike Equals, StartsWith, etc., it doesn't have an overload for a StringComparison argument. Why? Good question; ask Microsoft.
That, combined with SQL Server's limitation on LOWER means there's no simple way to do what you want.
Possible workarounds might include:
Use a full text index, and do the search in a procedure.
Use Equals or StartsWith instead, if possible for your task
Change the default collation of the column?

Use a lambda expression here and create an intermediary list that can handle the lower clause.
var q = Context.Foos.ToList().Where(s => s.Bar.ToLower().Contains("hi"));
Not terribly efficient, but it does work. If you have additional predicates in your where clause then it works to your advantage:
var q = Context.Foos.Where(p => p.f1 == "foo" && p.f2 == "bar").
ToList().Where(s => s.Bar.ToLower().Contains("hi"));

as we known , this is a very "bugged" situation.
and it bugs me a lot.
Today, i decide to create a view as:
select * from tableName
where theColumn like '%key%'
then load this view into EF.
life is getting easy!

Related

String.ToLowerInvariant not working in lambda expression [duplicate]

I'm using SQL Server 2005, with a case sensitive database..
In a search function, I need to create a Linq To Entities (L2E) query with a "where" clause that compare several strings with the data in the database with these rules :
The comparison is a "Contains" mode, not strict compare : easy as the string's Contains() method is allowed in L2E
The comparison must be case insensitive : I use ToLower() on both elements to perform an insensitive comparison.
All of this performs really well but I ran into the following Exception :
"Argument data type ntext is invalid for argument 1 of lower function" on one of my fields.
It seems that the field is a NText field and I can't perform a ToLower() on that.
What could I do to be able to perform a case insensitive Contains() on that NText field ?
Never use .ToLower() to perform a case-insensitive comparison. Here's why:
It's possibly wrong (your client collation could be, say, Turkish, and your DB collation not).
It's highly inefficient; the SQL Emitted is LOWER instead of = with a case-insensitive collation.
Instead, use StringComparison.OrdinalIgnoreCase or StringComparison.CurrentCultureIgnoreCase:
var q = from f in Context.Foos
where f.Bar.Equals("hi", StringComparison.OrdinalIgnoreCase)
select f;
But for Contains() there's a problem: Unlike Equals, StartsWith, etc., it doesn't have an overload for a StringComparison argument. Why? Good question; ask Microsoft.
That, combined with SQL Server's limitation on LOWER means there's no simple way to do what you want.
Possible workarounds might include:
Use a full text index, and do the search in a procedure.
Use Equals or StartsWith instead, if possible for your task
Change the default collation of the column?
Use a lambda expression here and create an intermediary list that can handle the lower clause.
var q = Context.Foos.ToList().Where(s => s.Bar.ToLower().Contains("hi"));
Not terribly efficient, but it does work. If you have additional predicates in your where clause then it works to your advantage:
var q = Context.Foos.Where(p => p.f1 == "foo" && p.f2 == "bar").
ToList().Where(s => s.Bar.ToLower().Contains("hi"));
as we known , this is a very "bugged" situation.
and it bugs me a lot.
Today, i decide to create a view as:
select * from tableName
where theColumn like '%key%'
then load this view into EF.
life is getting easy!

How do I calculate a checksum on all columns in a row using LINQ and Entity Framework?

The query I am trying to execute is similar to this:
var checksum = from i in db.Items
where i.Id == id
select SqlFunctions.Checksum("*");
However, this returns the checksum value of the string "*" rather than evaluating the wildcard. Is there a way to calculate the checksum of all the columns instead?
Update:
var checksum = db.Database.SqlQuery<int?>("SELECT CHECKSUM(*) FROM [Catalog].[Item] WHERE Id = #p0", id);
This gives me the result I want but seems dirty. Is there a way to do this without inline SQL?
This can be done with the SqlFunctions class. This Class allows for linq-to-entities code to include methods that are easily converted to Sql.
First of all in your current edit: Using inline SQL is not 'dirty' and is totally fine in most (if not all) cases. ORMs don't provide everything, especially if there isn't a good object-column mapping that exists. However, since you're using entity framework you might as well get aquanted with the SqlFunctions static methods.
In this case there are a lot of overloads for performing a checksum, however they must all be of the same type. Since you didn't post what types your columns or how many you have, I don't want to recommend the wrong overload in an example for you to use.
Here are your options:
SqlFunctions.Checksum():
bool?
char[]
DateTime?
DateTimeOffset?
Decimal?
double?
Guid?
TimeSpan?
String
All of the above have overloads to allow up to 3 parameters (of the same type).
SqlFunctions.AggregateChecksum():
IEnumerable<int>
IEnumerable<int?>
If you take a look at the documentation for these functions you'll see that the parameters that you're passing are VALUES, not column names. So you should be using them inside of a Select() clause. This is why when you passed "*" to the operation it checksummed the string containing a single asterisk instead of all columns. Also, keep in mind that these functions cannot be called directly, and must only be used within a Linq-To-Entities query.
Let's assume your columns named "ItemName" & "Description" are both strings, and you also want your id, which is an int:
var checksum = db.Items.Where(i => i.Id == id)
.Select(i => SqlFunctions.Checksum(i.Id.ToString(), i.ItemName, i.Description));
Unfortunately, as you see in the above example we had to cast our int to a string. There are no overloads that allow for different typed parameters for computing a checksum, nor are there any options that allow for more than 3 parameters in the checksum function; however, as I mentioned above sometimes you need to do an inline SQL command and this is OK.

Case-insensitive "contains" in Linq

I have a mvc project which I use linq in it.
In my database there is some records, for example "Someth ing","SOmeTH ing","someTh ing","SOMETH ING","someTH ING"
I want to do this:
SELECT * FROM dbo.doc_dt_records WHERE name LIKE '%' + #records.Name + '%'
However if I run this code, list.Count returns 0. What should I do?
records.Name = "someth ing"; //for example
var rec = db.Records.ToList();
var lists = rec.Where(p => p.Name.Contains(records.Name)).ToList();
if (lists.Count > 0)
{
// do sthng
}
Thanks for your helps...
the easy way is to use ToLower() method
var lists = rec.Where(p => p.Name.ToLower().Contains(records.Name.ToLower())).ToList();
a better solution (based on this post: Case insensitive 'Contains(string)')
var lists = rec.Where(p =>
CultureInfo.CurrentCulture.CompareInfo.IndexOf
(p.Name, records.Name, CompareOptions.IgnoreCase) >= 0).ToList();
That is totally not a LINQ issue.
Case sensitiivty on the generated SQL depends on the collation relevant for the table. Which in your case likely is case insensitive.
You would get the same result from any SQL you emit.
use IndexOf and StringComparison.OrdinalIgnoreCase:
p.Name.IndexOf(records.Name, StringComparison.OrdinalIgnoreCase) >= 0;
You can create an extension function like this:
public static bool Contains(this string src, string toCheck, StringComparison comp)
{
return src.IndexOf(toCheck, comp) >= 0;
}
To my understanding, this question does not have an unambiguous answer. The matter is that the best way of doing this depends on details which aren't provided in the question. For instance, what exact ORM do you use and what precise DB server you are connected to. For example, if you use Entity Framework against MS SQL Server, you better do not touch your LINQ expression at all. All you need to do is to set the case-insensitive collation on the database/table/column you compare your string with. That will do the trick much better than any change of your LINQ expression. The matter is that when LINQ is translated to SQL, it better be the straight comparison of the column having case-insensitive collation to your string than anything else. Just because it usually works quicker and it is the natural way to do the trick.
You do not want the final query to be something like:
SELECT *
FROM AspNetUsers U
WHERE UPPER(U.Name) LIKE '%SOMETHING%';
It is much better to come up with something like:
SELECT *
FROM AspNetUsers U
WHERE U.Name LIKE '%SOMETHING%';
But with a case-insensitive collation of [Name] column. The difference is that if you have let's say index containing [Name] column, the second query might use it, the first one would do the full scan of the table anyway.
So if let's say records references to DBSet<T> and the record is just one object of type T. You code would be like this:
var lists = records.Where(p => p.Name.Contains(record.Name)).ToList();
And you do the rest on SQL-server. Or if all you need to know is there any value in the list and do not need these values, it would be even better to do like this:
if (records.Any(p => p.Name.Contains(record.Name)))
{
// do something
}
Generally speaking, if you use any sort of ORM connected to any sort of SQL server, you better do case-insensitivity by setting up appropriate parameters of your server/database/table/column. And only if it is impossible or by far too expensive, you consider other possibilities. Otherwise, you might bang into some unexpected and very unpleasant behaviour. For instance, Entity Framework Core 2.x if it cannot translate your LINQ expression straightway into SQL query, is doing different tricks replacing server-side operations with client-side ones. So you can end up with a solution which fetches all data from the table to the client and filter it there. It might be quite a problem if your table is big enough.
As for the situation when LINQ query is processed locally, there are a lot of ways to do the trick. My favourite one is the next:
var lists = records.Where(p => p.Name
.Contains(record.Name, StringComparison.InvariantCultureIgnoreCase))
.ToList();
try this
var lists = rec.Where(p => String.Equals(p.Name,records.Name,StringComparison.OrdinalIgnoreCase)).ToList();
refer here for documentation

Equal sign in LINQ

The SQL server doing queries based COLLATE option, so you can define how comparision will be performed (case sensitive or not). You can do it when you creating table or during query execution.
How can I control collation during my LINQ to SQL queries? Will my queries be allways case insensitive when I will do table.Column == stringValue comparison?
I don't work with the COLLATE option much, but will take my best stab at this question.
According to this article:
LINQ to SQL does not consider server settings when it translates queries.
If COLLATE is a database/table/column setting, it should just be set in the database and be ready to go when you connect.
If COLLATE is a connection setting, you can acquire the connection of your datacontext and run the command to set it. A good place to do this might be in the partial void OnCreated method.
You have to remember that L2S is a Object Relational Mapping system, so its trying to to compare objects, and translate to SQL. In L2S, if you want to compare two strings you have to ToLower() both of them for comparison.
Another thing that was a 'gotcha' for me was that in L2S, string comparison will not evaluate correctly if the comparison value you supply is null. So, in your example, if table.Column is null and stringValue is too, your query will not return the correct results (I am basing this on the assumption that stringValue is a variable defined in your code). In order to compare a string to null in L2S, you have to compare it explicitly to null: table.Column == null.
check out this article

Case-insensitive sort ordering in NHibernate

Consider the following criteria query:
var x = SomeCriteria.AddOrder(new Order("Name", true)).List();
This will order the result set by the Name property, but case sensitive:
"A1"
"B1"
"a2"
Any ideas how to add the order case insensitive so result "a2" will end up before "B1"?
You should be able to accomplish this by ordering on a projection that normalizes the case for you. For example, Oracle has a "lower" function that will lower case string data types like varchar2 and nvarchar2; so I will use this sql function to form a projection that will order appropriately.
var projection = Projections.SqlFunction("lower",
NHibernateUtil.String,
Projections.Property("Name"));
var x = SomeCriteria.AddOrder(Orders.Asc(projection)).List()
If you're using SQL Server, I'd recommend using the "upper" function instead of "lower" for efficiency. Microsoft has optimized its native code for performing uppercase comparisons, where the rest of the world seems to have optimized on lowercase.
Hibernate (Java) has an "ignoreCase()" method on the "Order" class, but it looks like NHibernate does not have this method on its "Order."
This is how I was thinking you could do it:
var x = SomeCriteria.AddOrder(new Order("Name", true).IgnoreCase()).List();
But unfortunately, there is no IgnoreCase().
As a workaround, you could use an HQL or SQL query - either of those should allow you to order case-insensitive.
This probably depends on a case-sensitivity setting on your database server. I suspect that NHibernate just issues an "ORDER BY" clause; at least, I can't imagine what else it would do. For SQL Server, the default sort order (collation) is dictionary order, case insensitive.
This article gives some techniques for performing case sensitive searches in SQL Server. However, my advice is to sort the list that is returned by the query in code. That solution preserves the database independence of NHibernate and let's you customize the sort order per your needs.
As I know the responses to my query are always fairly small, I ended up querying the data as normal and sorting them afterwards using Linq. It works, so why bother tweaking NHibernate ;) (Using SQLite, btw)

Categories