I have a search where I use LINQ with EF. When ever the search criteria are null or empty I need to return everything. Currently I've used if conditions as a solution. and from that I moved to a solution like this.
data = data
.Where(p => !string.IsNullOrEmpty(searchriteria1)? p.field1.Contains(searchriteria1) : true)
.Where(p => !string.IsNullOrEmpty(searchriteria2)? p.field2.Contains(searchriteria2) : true);
Is there a better way to do this? maybe use an extension or any better approach?
You could check the search criteria field previously and build up the query this way:
IQueryable<Foo> data = context.Foo.AsQueryable();
if(!string.IsNullOrEmpty(searchriteria1))
{
data = data.Where(p => p.field1.Contains(searchriteria1));
}
if (!string.IsNullOrEmpty(searchriteria2))
{
data = data.Where(p => p.field2.Contains(searchriteria2));
}
There are two parts to the question. How to filter dynamically and how to filter efficiently.
Dynamic criteria
For the first question, there's no need for a catch-all query when using LINQ. Catch-all queries result in inefficient execution plans, so it's best to avoid them.
LINQ isn't SQL though. You can construct your query part by part. The final query will be translated to SQL only when you try to enumerate it. This means you can write :
if(!String.IsNullOrEmpty(searchCriteria1))
{
query=query=.Where(p=>p.Field1.Contains(searchCriteria1);
}
You can chain multiple Where call to get the equivalent of multiple AND criteria.
To generate more complex queries using eg OR you'd have to construct the proper Expression<Func<...,bool>> objects, or use a library like LINQKit to make this bearable.
Efficiency
Whether you can write an efficient query depends on the search criteria. The clause field LIKE '%potato%' can't use any indexes and will end up scanning the entire table.
On the other hand, field LIKE 'potato% can take advantage of an index that covers field because it will be converted to a range search like field >='potato' and field<='potatp.
If you want to implement autocomplete or spell checking though, you often want to find text that has the fewest differences from the criteria.
Full Text Search
You can efficiently search for words, word variations and even full phrases using Full-Text Search indexes and FTS functions like CONTAINS or FREETEXT.
FTS is similar to how Google or ... StackOverflow searches for words or sentences.
Quoting form the docs:
CONTAINS can search for:
A word or phrase.
The prefix of a word or phrase.
A word near another word.
A word inflectionally generated from another (for example, the word drive is the inflectional stem of drives, drove, driving, and driven).
A word that is a synonym of another word using a thesaurus (for example, the word "metal" can have synonyms such as "aluminum" and "steel").
FREETEXT on the other hand is closer to how Google/SO work by searching for an entire phrase, returning close matches, not just exact matches.
Both CONTAINS and FREETEXT are available in EF Core 5 and later, through the DbFunctions.Contains and DbFunctions.FreeText functions.
This means that if you want to search for a word or phrase, you could construct a proper FTS argument and use :
var searchCriteria1="' Mountain OR Road '";
if(!String.IsNullOrEmpty(searchCriteria1))
{
query=query=.Where(p=>DbFunctions.Contains(p.Field1.Contains(searchCriteria1));
}
That's a lot easier than using LINQKit.
Or search for ride, riding, ridden with :
var searchCriteria1="' FORMSOF (INFLECTIONAL, ride) '";
shorter syntax
data.Where(p => (string.IsNullOrEmpty(searchriteria1) || p.field1.Contains(searchriteria1))
&& (string.IsNullOrEmpty(searchriteria2) || p.field2.Contains(searchriteria2)));
public static List<Test> getAll(Expression<Func<Test, bool>> filter = null)
{
return filter == null ? context.Set<Test>().ToList() : context.Set<Test>().Where(filter).ToList();
}
If you want to filter
var l=getAll(p => p.field1.Contains(searchriteria1)&&p.field2.Contains(searchriteria2));
no filter
var l=getAll();
Related
I am new to Lucene.net ,Here I would like to know how to make a lucene search query almost like an sql query .Lemme give more..
I have set of parameter values,Let assume like a stored procedure has set of parameters .Now I want to build a query with all this parameters.
searchParams.UseLast = Convert.ToBoolean(base.Arguments["UseLast"]);
searchParams.LastEditedFrom= Convert.ToDateTime(base.Arguments["LastEditedFrom"]);
searchParams.LastEditedTo = Convert.ToDateTime(base.Arguments["LastEditedTo"]);
searchParams.Reviewed = Convert.ToBoolean(base.Arguments["Reviewed"]);
searchParams.Approved = Convert.ToBoolean(base.Arguments["Approved"]);
searchParams.Include = Convert.ToBoolean(base.Arguments["Include"]);
searchParams.IsVisibleToUser = Convert.ToBoolean(base.Arguments["IsVisibleToUser"]);
searchParams.IsEntry = Convert.ToBoolean(base.Arguments["IsEntry"]);
searchParams.UserId = Convert.ToInt32(base.Arguments["UserId"]);
IEnumerable Categories = base.Arguments["Categories"] as IEnumerable;
IEnumerable Departments = base.Arguments["Departments"] as IEnumerable;
String mQuery = "How to construct it ....!!!" // Need help in this
var query = queryParser.Parse(mQuery);
indexSearcher.Search(query, collector);
Here I want to fetch all records from lucene index which has the value for all the above fields.
I'm unclear what you are using searchParams for, however in general you may construct your query string (mQuery) in this cases with any of the features of the Lucene query syntax. Here is a link to the documentation for Lucene.Net version 4.8 Query Parser Syntax.
In general, when multiple words are listed in the query they are treated with a logical OR but doc matches that contain all terms are ranked higher than docs with only one term. So for example white dog would match docs containing white dog or white or dog. You can put and in the statement if you only want docs that match all the terms so for example you could say small and white and dog if you only want docs that contain all three terms.
To specify the specific field to search you list the field name followed by a colon. So for example you can search UserId:ron and Categories:dogs. There is much more to the Lucene query syntax but hopefully that will get you started. For more details see the Lucene query syntax doc I referred to.
We recently discovered a bug in our system whereby any serial numbers that have been entered in lowercase have not been processed correctly.
To correct this, we need to add a one off function that will run through the database and re-process all items with lower case serial numbers.
In linq, is there a query I can run that will return a list of such items?
Note: I am not asking how to convert lowercase to uppercase or reverse, which is all google will return. I need to generate a list of all database entries where the serial number has been entered in lowercase.
EDIT: I am using Linq to MS SQL, which appears to be case insensitive.
Yes, there is. You can try something like this:
var result = serialnumber.Any(c => char.IsLower(c));
[EDIT]
Well, in case of Linq to Entities...
As is stated here: Regex in Linq (EntityFramework), String processing in database, there's few ways to workaround it.
Change database table structure. E.g. create table Foo_Filter which will link your entities to filters. And then create table Filters
which will contain filters data.
Execute query in memory and use Linq to Objects. This option will be slow, because you have to fetch all data from database to memory
Note: link to MSDN documentation has been added by me.
For example:
var result = context.Serials.ToList().Where(sn => sn.Any(c => char.IsLower(c)));
Another way is to use SqlMethods.Like Method
Finally, i'd strongly recommend to read this: Case sensitive search using Entity Framework and Custom Annotation
I have string with comma separated list of emails
var emails="a#gmail.com,b#gmail.com,c#gmail.com,d#gmail.com,e#gmail.com,";
var list=new List<string>();
list.Add("c#gmail.com");
list.Add("d#gmail.com");
how can I write ling query to find out if string(email) has any email address matching to
collection(list).
I am using EF and emails string is property in a class and list is independent collection.
This solution uses linq. Because your email addresses are comma separated (and ends in a comma) we can check if Any() of the items in the list are contained by the emails string. I used ToLower() to make it case insensitive (which email addresses typically are).
var hasMatch = list.Any(item => ","+emails.ToLower().Contains(","+item.ToLower()+","));
emails.Split(',').Any(e=>list.Contains(e));
alternatively:
emails.Split(',').Intersect(list).Any();
If you are using it to find database records, then you can do this:
db.MyTable.Where(l=>list.Any(e=>l.emails.StartsWith(e+",")) ||
list.Any(e=>l.emails.EndsWith(","+e)) ||
list.Any(e=>l.emails.Contains(","+e+",")) ||
list.Any(e=>l.emails==e)
)
or you can simplify it with:
db.MyTable.Where(l=>list.Any(e=>(","+l.emails+",").Contains(","+e+",")))
The 3rd option may perform better if you are looking for the first record as it might use any index you have on emails to quickly locate the record, but it will generate some really big SQL statements if the list is big (Current implementations of SQL LINQ provider unfortunately translates this to a CHARINDEX function instead of LIKE 'email%', but that could change).
The 4th option will generate simpler SQL, and will likely perform better if want to find all the records that match instead of just the first one.
You can do it following way
emails.Split(',').Any (y => list.Contains (y));
var emailsList = emails.Split(',');
list.Any( x => emailsList.Contains(x));
Antoher approach
HashSet<String> emailSet = New HashSet<String>(temp.Split(","));
list.Any(s => emailSet.Contains(s, StringComparer.Ordinal));
Use HashSet - Contains works faster (if emails string can be big) and remove duplicates
And comparing stringg in case-sensitive without creating new strings by ToLower()
I have a mvc project which I use linq in it.
In my database there is some records, for example "Someth ing","SOmeTH ing","someTh ing","SOMETH ING","someTH ING"
I want to do this:
SELECT * FROM dbo.doc_dt_records WHERE name LIKE '%' + #records.Name + '%'
However if I run this code, list.Count returns 0. What should I do?
records.Name = "someth ing"; //for example
var rec = db.Records.ToList();
var lists = rec.Where(p => p.Name.Contains(records.Name)).ToList();
if (lists.Count > 0)
{
// do sthng
}
Thanks for your helps...
the easy way is to use ToLower() method
var lists = rec.Where(p => p.Name.ToLower().Contains(records.Name.ToLower())).ToList();
a better solution (based on this post: Case insensitive 'Contains(string)')
var lists = rec.Where(p =>
CultureInfo.CurrentCulture.CompareInfo.IndexOf
(p.Name, records.Name, CompareOptions.IgnoreCase) >= 0).ToList();
That is totally not a LINQ issue.
Case sensitiivty on the generated SQL depends on the collation relevant for the table. Which in your case likely is case insensitive.
You would get the same result from any SQL you emit.
use IndexOf and StringComparison.OrdinalIgnoreCase:
p.Name.IndexOf(records.Name, StringComparison.OrdinalIgnoreCase) >= 0;
You can create an extension function like this:
public static bool Contains(this string src, string toCheck, StringComparison comp)
{
return src.IndexOf(toCheck, comp) >= 0;
}
To my understanding, this question does not have an unambiguous answer. The matter is that the best way of doing this depends on details which aren't provided in the question. For instance, what exact ORM do you use and what precise DB server you are connected to. For example, if you use Entity Framework against MS SQL Server, you better do not touch your LINQ expression at all. All you need to do is to set the case-insensitive collation on the database/table/column you compare your string with. That will do the trick much better than any change of your LINQ expression. The matter is that when LINQ is translated to SQL, it better be the straight comparison of the column having case-insensitive collation to your string than anything else. Just because it usually works quicker and it is the natural way to do the trick.
You do not want the final query to be something like:
SELECT *
FROM AspNetUsers U
WHERE UPPER(U.Name) LIKE '%SOMETHING%';
It is much better to come up with something like:
SELECT *
FROM AspNetUsers U
WHERE U.Name LIKE '%SOMETHING%';
But with a case-insensitive collation of [Name] column. The difference is that if you have let's say index containing [Name] column, the second query might use it, the first one would do the full scan of the table anyway.
So if let's say records references to DBSet<T> and the record is just one object of type T. You code would be like this:
var lists = records.Where(p => p.Name.Contains(record.Name)).ToList();
And you do the rest on SQL-server. Or if all you need to know is there any value in the list and do not need these values, it would be even better to do like this:
if (records.Any(p => p.Name.Contains(record.Name)))
{
// do something
}
Generally speaking, if you use any sort of ORM connected to any sort of SQL server, you better do case-insensitivity by setting up appropriate parameters of your server/database/table/column. And only if it is impossible or by far too expensive, you consider other possibilities. Otherwise, you might bang into some unexpected and very unpleasant behaviour. For instance, Entity Framework Core 2.x if it cannot translate your LINQ expression straightway into SQL query, is doing different tricks replacing server-side operations with client-side ones. So you can end up with a solution which fetches all data from the table to the client and filter it there. It might be quite a problem if your table is big enough.
As for the situation when LINQ query is processed locally, there are a lot of ways to do the trick. My favourite one is the next:
var lists = records.Where(p => p.Name
.Contains(record.Name, StringComparison.InvariantCultureIgnoreCase))
.ToList();
try this
var lists = rec.Where(p => String.Equals(p.Name,records.Name,StringComparison.OrdinalIgnoreCase)).ToList();
refer here for documentation
I have a blog application that models a database using Entity Framework. The problem with this blog is that it has become difficult to find things I'm looking for. It needs a search function, but I'm not sure how to implement this with SQL and/or LINQ to Entities.
Right now I am searching my database with this LINQ query but it seems like it should be better.
public IEnumerable<BlogPost> SearchBlogPosts(string query, int page, int itemsPerPage)
{
var result = _dataContext.BlogPosts
.Where(BlogPostContains(query))
.OrderByDescending(x => x.PostedDate)
.Skip((page - 1) * itemsPerPage)
.Take(itemsPerPage),
return result;
}
private Func<BlogPost, bool> BlogPostContains(string query)
{
return x => x.Title.Contains(query) || x.Body.Contains(query) || x.Author.Contains(query);
}
One big problem with this is that the search is case sensitive.
Question 1) Is there a better way to do searching with LINQ to Entities?
Question 2) What about with just plain SQL? How would I write a search stored procedure in SQL Server so that I can map and use that in EF instead of LINQ?
I just want a case-insensitive search that is performed in the database so as to maintain good performance.
Thanks in advance.
The standard approach for this would be a SQL fulltext search. You will have to enable fulltext on the DB, designate column(s) to be fulltext indexed and then will then be able to use SQL Contains queries using these columns.
Fulltext search queries are currently not supported by Linq to Entities - you will have to resort to standard SQL queries for this. You can use ExecuteStoreQuery() to at least map the search results to typed result.
The default collation of SQL Server is case-insensitive which means that a where clause like this (which is what LINQ to Entities would create out of Contains)...
where Name like '%JOHN%'
...should find "john Wayne". I believe that your query is not executed on the server but actually with LINQ to Objects in memory - and there the search is case sensitive. It's LINQ to Objects because you are not returning an Expression from your BlogPostContains. The signature should be:
private Expression<Func<BlogPost, bool>> BlogPostContains(string query)
If your are only returning Func<BlogPost, bool> you are working with the IEnumerable (and not the IQueryable) overload of the Where extension method which in turn causes the whole BlogPosts table loaded first into memory. Then the filter is applied in memory with LINQ to Objects.
Would be interesting to know if the case-sensitivity disappears if you return an Expression.
(Just as a note about your Case-Sensitivity problem, not a solution to your general question about the best way to implement a Search feature.)
I would look at Lucene.Net for a good search provider.
As per your searches using LINQ, you can try something like this and it works just fine:
(x.Title).ToUpper().Contains(query.ToUpper())
Or the similar method of ToLower() should suffice as well.
For databases that truly consider an uppercase 'A' different from a lowercase 'a' as different values, which they are technically are, the above LINQ procedure will get you the results.
Hope this helps.