We Keep Coding

sorting, filtering, and paging with entity framework core 2

I'm creating a tabular grid of data and will have every column header sortable (both ascending and descending) as well as filterable by a textbox underneath the column name.
I've done some digging to find out the best approach to generating my LINQ queries for entity framework and best I found was this: https://learn.microsoft.com/en-us/aspnet/core/data/ef-mvc/sort-filter-page?view=aspnetcore-3.0
That just uses a big switch / case (for ordering only...doesn't even consider multiple filtering options beyond one). Surely there's got to be a better, more concise approach than this?
I'm half tempted to just use my old sproc and call the sproc with EF if this is really the path I'll have to go down.
Ideally, I'd want something like this:
return myContext.Where(x =>
x.%thisisadynamicfield%.Contains("%somefiltervalue%" &&
x.%anotherdynamicfield%.Contains("%someothervalue%" && ...)
.OrderBy(x => x.%someorderybydynamicfield%)
I'm ok using a conditional to determine ordering ascending or descending, but I'd really like to compact everything else as much as possible and that MS article I linked above looks like code vomit and it would only cover part of my needs.
EDIT: as is almost always the case, as soon as I post here and try one more searching effort, I find what I'm looking for. I just found this: https://github.com/harshwPro/EntityFrameworkPaginate which looks very promising. I will report back here if that solves my issue.
EDIT 2: well the github project I referenced in my first edit is certainly a cleaner solution than the MS article I posted originally but it still leaves me with writing a huge conditional to generate my filters by property. Using that solution, I'd have to write something like this:
dataPagingModel.Filters.Add(true, x => x.MyCoolProperty.Contains("blah")); but would have to conditionally add new filters based on my angular presentation layer's posted parameters - switch / case, or some huge if / else block. What I'd ideally like is a way to just loop over all of the posted filters list and dynamically add those filters in a loop. So I guess my question now boils down to, is there a way to access properties of an object dynamically (generically) in a lambda expression?
EDIT 3: I think this SO post has the answers I need: Generic method to filter a list object

SQLite bitwise query to EF Core Linq version?

So, I know my SQL much better than my EF Core linq, so my starting query was just a FromSQL like this:
var links = context.PortalLinks.FromSql("SELECT * FROM portal_link WHERE grant_bits & {0} AND linkSet_id={1} ORDER BY sortKey",claim,0).ToList();
Now I am trying to translate it into the proper linq style query
var links = context.PortalLinks.Where(x => ((x.GrantBits & claim) != 0) && x.LinkSetId == 0).OrderBy(s => s.SortKey);
I've come up with this, but I am not happy that I am being forced to explicitly test for !=0 when in SQL that is implicit, and while this instance is harmless I can imagine other cases where there can be a more ... amusing ... mismatch between what I can express in C# and what I want to express in SQL.
I guess, is there some other way to express more idiomatic SQL queries for Linq to parse thats less bound by c#'s own assumptions as to what constitutes valid logic?

I am trying to figure what you mean by idiomatic... you can use string interpolation of C# 6.0 inside the query and pass your var:
var links = context.PortalLinks.Where(x => ((x.GrantBits & claim) != Int32.Parse($"{myVar}")) && x.LinkSetId == Int32.Parse($"{myVar2}").OrderBy(s => s.SortKey);
Notice that I use string interpolation as there will be no easy way to achieve this logic with an integer.
you can relate to this post for another source of information.
Edit:
Now that I understand your question I can address your inquiry:
What you describe isn't going to work with native LINQ functionality however!
there is this library which will help you extend the dynamic linq and achieve your goal:
Dynamic LINQ

Case-insensitive "contains" in Linq

I have a mvc project which I use linq in it.
In my database there is some records, for example "Someth ing","SOmeTH ing","someTh ing","SOMETH ING","someTH ING"
I want to do this:
SELECT * FROM dbo.doc_dt_records WHERE name LIKE '%' + #records.Name + '%'
However if I run this code, list.Count returns 0. What should I do?
records.Name = "someth ing"; //for example
var rec = db.Records.ToList();
var lists = rec.Where(p => p.Name.Contains(records.Name)).ToList();
if (lists.Count > 0)
{
// do sthng
}
Thanks for your helps...

the easy way is to use ToLower() method
var lists = rec.Where(p => p.Name.ToLower().Contains(records.Name.ToLower())).ToList();
a better solution (based on this post: Case insensitive 'Contains(string)')
var lists = rec.Where(p =>
CultureInfo.CurrentCulture.CompareInfo.IndexOf
(p.Name, records.Name, CompareOptions.IgnoreCase) >= 0).ToList();

That is totally not a LINQ issue.
Case sensitiivty on the generated SQL depends on the collation relevant for the table. Which in your case likely is case insensitive.
You would get the same result from any SQL you emit.

use IndexOf and StringComparison.OrdinalIgnoreCase:
p.Name.IndexOf(records.Name, StringComparison.OrdinalIgnoreCase) >= 0;
You can create an extension function like this:
public static bool Contains(this string src, string toCheck, StringComparison comp)
{
return src.IndexOf(toCheck, comp) >= 0;
}

To my understanding, this question does not have an unambiguous answer. The matter is that the best way of doing this depends on details which aren't provided in the question. For instance, what exact ORM do you use and what precise DB server you are connected to. For example, if you use Entity Framework against MS SQL Server, you better do not touch your LINQ expression at all. All you need to do is to set the case-insensitive collation on the database/table/column you compare your string with. That will do the trick much better than any change of your LINQ expression. The matter is that when LINQ is translated to SQL, it better be the straight comparison of the column having case-insensitive collation to your string than anything else. Just because it usually works quicker and it is the natural way to do the trick.
You do not want the final query to be something like:
SELECT *
FROM AspNetUsers U
WHERE UPPER(U.Name) LIKE '%SOMETHING%';
It is much better to come up with something like:
SELECT *
FROM AspNetUsers U
WHERE U.Name LIKE '%SOMETHING%';
But with a case-insensitive collation of [Name] column. The difference is that if you have let's say index containing [Name] column, the second query might use it, the first one would do the full scan of the table anyway.
So if let's say records references to DBSet<T> and the record is just one object of type T. You code would be like this:
var lists = records.Where(p => p.Name.Contains(record.Name)).ToList();
And you do the rest on SQL-server. Or if all you need to know is there any value in the list and do not need these values, it would be even better to do like this:
if (records.Any(p => p.Name.Contains(record.Name)))
{
// do something
}
Generally speaking, if you use any sort of ORM connected to any sort of SQL server, you better do case-insensitivity by setting up appropriate parameters of your server/database/table/column. And only if it is impossible or by far too expensive, you consider other possibilities. Otherwise, you might bang into some unexpected and very unpleasant behaviour. For instance, Entity Framework Core 2.x if it cannot translate your LINQ expression straightway into SQL query, is doing different tricks replacing server-side operations with client-side ones. So you can end up with a solution which fetches all data from the table to the client and filter it there. It might be quite a problem if your table is big enough.
As for the situation when LINQ query is processed locally, there are a lot of ways to do the trick. My favourite one is the next:
var lists = records.Where(p => p.Name
.Contains(record.Name, StringComparison.InvariantCultureIgnoreCase))
.ToList();

try this
var lists = rec.Where(p => String.Equals(p.Name,records.Name,StringComparison.OrdinalIgnoreCase)).ToList();
refer here for documentation

LINQ to Entities does not recognize the method 'Int32 IndexOf(System.String, System.StringComparison)' method

I have executed a linq query by using Entityframework like below
GroupMaster getGroup = null;
getGroup = DataContext.Groups.FirstOrDefault(item => keyword.IndexOf(item.Keywords,StringComparison.OrdinalIgnoreCase)>=0 && item.IsEnabled)
when executing this method I got exception like below
LINQ to Entities does not recognize the method 'Int32 IndexOf(System.String, System.StringComparison)' method, and this
method cannot be translated into a store expression.
Contains() method by default case sensitive so again I need to convert to lower.Is there any method for checking a string match other than the contains method and is there any method to solve the indexOf method issue?

The IndexOf method Of string class will not recognized by Entity Framework, Please replace this function with SQLfunction or Canonical functions
You can also take help from here or maybe here
You can use below code sample:
DataContext.Groups.FirstOrDefault(item =>
System.Data.Objects.SqlClient.SqlFunctions.CharIndex(item.Keywords, keyword).Value >=0 && item.IsEnabled)

You really only have four options here.
Change the collation of the database globally. This can be done in several ways, a simple google search should reveal them.
Change the collation of individual tables or columns.
Use a stored procedure and specify the COLATE statement on your query
perform a query and return a large set of results, then filter in memory using Linq to Objects.
number 4 is not a good option unless your result set is pretty small. #3 is good if you can't change the database (but you can't use Linq with it).
numbers 1 and 2 are choices you need to make about your data model as a whole, or if you only want to do it on specific fields.
Changing the Servers collation:
http://technet.microsoft.com/en-us/library/ms179254.aspx
Changing the Database Collation:
http://technet.microsoft.com/en-us/library/ms179254.aspx
Changing the Columns Collation:
http://technet.microsoft.com/en-us/library/ms190920(v=sql.105).aspx
Using the Collate statement in a stored proc:
http://technet.microsoft.com/en-us/library/ms184391.aspx

Instead you can use this method below for lowering the cases:
var lowerCaseItem = item.ToLower();
If your item is of type string. Then this might get you through that exception.

Erik Funkenbush' answer is perfectly valid when looking at it like a database problem. But I get the feeling that you need a better structure for keeping data regarding keywords if you want to traverse them efficiently.
Note that this answer isn't intended to be better, it is intended to fix the problem in your data model rather than making the environment adapt to the current (apparently flawed, since there is an issue) data model you have.
My main suggestion, regardless of time constraint (I realize this isn't the easiest fix) would be to add a separate table for the keywords (with a many-to-many relationship with its related classes).
[GROUPS] * ------- * [KEYWORD]
This should allow for you to search for the keyword, and only then retrieve the items that have that keyword related to it (based on ID rather than a compound string).
int? keywordID = DataContext.Keywords.Where(x => x.Name == keywordFilter).Select(x => x.Id).FirstOrDefault();
if(keywordID != null)
{
getGroup = DataContext.Groups.FirstOrDefault(group => group.Keywords.Any(kw => kw.Id == keywordID));
}
But I can understand completely if this type of fix is not possible anymore in the current project. I wanted to mention it though, in case anyone in the future stumbles on this question and still has the option for improving the data structure.

Searching Multiple Fields with LINQ Contains or Other

I'm using LINQ to search multiple fields on a single phrase and I'm using Contains() to do this. Up until today, when I noticed that the find method isn't working correctly.
So I'd like to be able to search each of the fields where there is any match. I've done some searching on Google and through SO and found suggestions of using Any()? Does anyone have any suggestions?
var l = _getData().Select(_marshallData).ToList();
return l.Where(x => x.Name.Contains(what) || x.Pumpkin.Contains(what) || x.Orange.Contains(what) || x.Tomato.Contains(what)).ToList();
Please excuse the stupid field names, I've had to change them for confidentiality.

Since you're materializing the data already you could do this:
var l = _getData().Select(_marshallData).AsEnumerable();
return l.Where(x => new[] { x.Name, x.Pumpkin, x.Orange, x.Tomato }.Any(s => s.Contains(what)));
But if you're using an ORM (like Entity Framework) this trick probably won't work (since there's a good chance this can't be converted to a SQL expression). In this case, you're original solution is fine, but it would probably be better to do this before you materialize the data.
return _getData()
.Where(x =>
x.Name.Contains(what) || x.Pumpkin.Contains(what) ||
x.Orange.Contains(what) || x.Tomato.Contains(what))
.Select(_marshallData)
.AsEnumerable();
Of course the field names here might be different, since the parameter x is probably a different type. Without further information, it's up to you to write this filter correctly.
Note that in both examples, I've eliminated the calls to ToList. Unless you absolutely need the result to a List<T> this is probably unnecessary.