We Keep Coding

Entity Framework Core to SQL Server IN Clause

We are converting from LINQ to SQL to Entity Framework Core 2.2 and are finding that the translation of Contains operations do not become IN clauses in SQL Server. What is happening is that EF Core is pulling back all of the data using the other conditions and then filtering it down locally. This is not acceptable. We can use EF.Functions.Contains but this requires us to enable Full Text Search in SQL Server, which is over kill.
Any idea how to get a statement like the following to translate to an IN clause in SQL Server?
var myValues = new [] { 1, 2, 3, 4, 5 };
var qry = _context.Table.Where(t => myValues.Contains(t.TableProperty));
Okay, maybe I over simplified the code to keep it simple for the question. The actual code looks like:
voterQuery = voterQuery.Where(v => voterFilter.VoterStatus.Select(p => p.Value).Contains(v.VotStatus.ToString()));
What is happening in our code is that we are building up an IQueryable from user selections on a filtering screen. voterFilters contains a collection of these selection criteria. VoterStatus is one of the selection criteria which is a List<CheckedListItem>, which are from a Winforms CheckedListItems control.
The selection of p.Value returns a List of strings. I have tried to project the list of strings to an array, with the same results of the IN clause not being created or a server query. Perhaps, EF Core does not allow strings to be used for the IN clause values. Any insight would be appreciated.

Currently (as of v2.2.3), EF Core requires the expression used for Contains to be a simple IEnumerable<T> variable (no LINQ operators) where the T is primitive type.
Which means you need to move the voterFilter.VoterStatus.Select(p => p.Value) into variable outside the query expression tree and use that variable inside:
var voterStatusFilter = voterFilter.VoterStatus.Select(p => p.Value);
voterQuery = voterQuery.Where(v => voteStatusFilter.Contains(v.VotStatus.ToString()));

How Dapper.NET works internally with .Count() and SingleOrDefault()?

I am new to Dapper though I am aware about ORMs and DAL and have implemented DAL with NHibernate earlier.
Example Query: -
string sql = "SELECT * FROM MyTable";
public int GetCount()
{
var result = Connection.Query<MyTablePoco>(sql).Count();
return result;
}
Will Dapper convert this query (internally) to SELECT COUNT(*) FROM MyTable looking at .Count() at the end?
Similarly, will it convert to SELECT TOP 1 * FROM MyTable in case of SingleOrDefault()?
I came from NHibernate world where it generates query accordingly. I am not sure about Dapper though. As I am working with MS Access, I do not see a way to check the query generated.

No, dapper will not adjust your query. The immediate way to tell this is simply: does the method return IEnumerable... vs IQueryable...? If it is the first, then it can only use local in-memory mechanisms.
Specifically, by default, Query will actually return a fully populated List<>. LINQ's Count() method recognises that and just accesses the .Count property of the list. So all the data is fetched from the database.
If you want to ask the database for the count, ask the database for the count.
As for mechanisms to view what is actually sent to the database: we use mini-profiler for this. It works great.
Note: when you are querying exactly one row: QueryFirstOrDefault (and the other variants you would expect) exist and have optimizations internally (including hints to ADO.NET, although not all providers can act on those things) to do things as efficiently as possible, but it does not adjust your query. In some cases the provider itself (not dapper) can help, but ultimately: if you only want the first row, ask the database for the first row (TOP or similar).

lightswitch LINQ PreprocessQuery

I use the PreprocessQuery method to extend a query in lightswitch.
Something like this:
query = (from item in query
where (validIDs.Contains(item.tableIDs.myID)) &&
elementCount[item.ID] <= maxEleCount)
select item);
Where validIDs is a HashSet(int) and elementCount is a Dictionary(int, int).
the first where clause is working fine, but the second -> elementCount[item.ID] <= maxEleCount
is not working.
What i want to do is to filter a table by some IDs (validIDs) and check also if in another table the number of entries for every of this IDs does not exceed a limit.
Any ideas?
EDIT
I found a solution. Instead of a Dictionary I also used a HashSet for the second where clause. It seems it is not possible to do the Dictionary lookup inside the LINQ statement for some reason (?)

First, although being a bit pedantic, what you're doing in a PreProcessQuery method is "restricting" records in the query, not "extending" the query.
What you put in a LING query has to be able to be processed by the Entity Framework data provider (in the case of LS, the SQL Server Data Provider).
Sometimes you'll find that while your LINQ query compiles, it fails at runtime. This is because the data provider is unable to express it to the data store (again in this case SQL Server).
You're normally restricted to "primitive" values, so if you hadn't said that using a Dictionary actually worked, I would have said that it wouldn't.
Any time you have a static (as in non-changing) value, I'd suggest that you create a variable outside of your LINQ query, then use the variable in the LINQ query. By doing this, you're simply passing a value, the data provider doesn't have to try to figure out how to pass it to the data store.
Reading your code again, this might not be what you're doing, but hopefully this explanation will still be helpful.

LinQ to SQL query constructed at runtime

I have an xml file which contains query details like
<queries>
<query>
<name>GetStudentById</name>
<statement>select student_name from student where student_id=#id</statement>
</query>
</queries>
In C# code, the user will call a method ExecuteQuery(string queryname, hashtable params)
I will retrieve the query statement from the xml file using the query name.
The params hash table will contain values like
params['#id']=5
I will then use ADO .NET to construct the query and execute it at runtime using ExecuteReader after constructing the query statement and passing it the parameter list.
I want to know if it is possible to do it by using LinQ. Can someone show me how to execute the statement as a LinQ query?
Thanks,
David

Have a look at this SO post. Look at the answer from john skeet where he explains how to use the CSharpCodeProvider to execute the statement at runtime.
You will however have to convert your statement from sql to linq yourself. If there is no particular reason for it having to be linq, I would suggest keeping it in sql because at the end of the day you are going to parse sql to linq and then back to sql.
Even with dynamic linq you will have a problem because you will always have to specify the from class in your query.
Another option is that if you want to use this with your entity framework model, it would be way easier to use entity sql, have a look here and here for more info.

It's a considerable leap to go from using string query statements via ADO.NET to Linq to SQL. In the first instance you need to build your Linq to SQL entity model/DataContext. How you go about parsing a sql statement from text to Linq I'm not entirely sure, but if I wanted to look into how to achieve this I would be reading up on expression trees with a view to building a parser that takes your string query and referers to your entity model in order to produce a linq query.

Found something called LinqTextQueryBuilder at codeplex and another MS Library.
Links below
LinqTextQueryBuilder
http://msdn.microsoft.com/en-us/vstudio/bb894665.aspx

How do I implement a search feature with Entity Framework?

I have a blog application that models a database using Entity Framework. The problem with this blog is that it has become difficult to find things I'm looking for. It needs a search function, but I'm not sure how to implement this with SQL and/or LINQ to Entities.
Right now I am searching my database with this LINQ query but it seems like it should be better.
public IEnumerable<BlogPost> SearchBlogPosts(string query, int page, int itemsPerPage)
{
var result = _dataContext.BlogPosts
.Where(BlogPostContains(query))
.OrderByDescending(x => x.PostedDate)
.Skip((page - 1) * itemsPerPage)
.Take(itemsPerPage),
return result;
}
private Func<BlogPost, bool> BlogPostContains(string query)
{
return x => x.Title.Contains(query) || x.Body.Contains(query) || x.Author.Contains(query);
}
One big problem with this is that the search is case sensitive.
Question 1) Is there a better way to do searching with LINQ to Entities?
Question 2) What about with just plain SQL? How would I write a search stored procedure in SQL Server so that I can map and use that in EF instead of LINQ?
I just want a case-insensitive search that is performed in the database so as to maintain good performance.
Thanks in advance.

The standard approach for this would be a SQL fulltext search. You will have to enable fulltext on the DB, designate column(s) to be fulltext indexed and then will then be able to use SQL Contains queries using these columns.
Fulltext search queries are currently not supported by Linq to Entities - you will have to resort to standard SQL queries for this. You can use ExecuteStoreQuery() to at least map the search results to typed result.

The default collation of SQL Server is case-insensitive which means that a where clause like this (which is what LINQ to Entities would create out of Contains)...
where Name like '%JOHN%'
...should find "john Wayne". I believe that your query is not executed on the server but actually with LINQ to Objects in memory - and there the search is case sensitive. It's LINQ to Objects because you are not returning an Expression from your BlogPostContains. The signature should be:
private Expression<Func<BlogPost, bool>> BlogPostContains(string query)
If your are only returning Func<BlogPost, bool> you are working with the IEnumerable (and not the IQueryable) overload of the Where extension method which in turn causes the whole BlogPosts table loaded first into memory. Then the filter is applied in memory with LINQ to Objects.
Would be interesting to know if the case-sensitivity disappears if you return an Expression.
(Just as a note about your Case-Sensitivity problem, not a solution to your general question about the best way to implement a Search feature.)

I would look at Lucene.Net for a good search provider.

As per your searches using LINQ, you can try something like this and it works just fine:
(x.Title).ToUpper().Contains(query.ToUpper())
Or the similar method of ToLower() should suffice as well.
For databases that truly consider an uppercase 'A' different from a lowercase 'a' as different values, which they are technically are, the above LINQ procedure will get you the results.
Hope this helps.