I have a table that has two records (there will be many at runtime). The deviceId of the records are, “DEVICE1” and “DEVICE2”. I want to use a regular expression to extract records.
The code below compiles but fails to return a result. When I hover the cursor on the “devices.ToList()” statement I get the following error:
base {System.SystemException} = {"LINQ to Entities does not recognize the method 'System.Text.RegularExpressions.MatchCollection Matches(System.String)' method, and this method cannot be translated into a store expression."}”
Can anyone show me how I can modify my query so that this would return records based on the expression?
filterText = #"DEVICE.";
Regex searchTerm = new Regex(filterText);
using (var ctx = new MyEntities())
{
var devices = from d in ctx.Devices
let matches = searchTerm.Matches(d.DeviceId)
where matches.Count > 0
select ((Device)d);
return devices.ToList();
}
I don't believe you can use regular expressions with LINQ to Entities. However, it looks like you're just trying to find devices which start with "DEVICE", so the query would be:
return ctx.Devices.Where(d => d.DeviceId.StartsWith("DEVICE"))
.ToList();
EDIT: If you actually need the flexibility of a regular expression, you should probably first fetch the device IDs (and only the device IDs) back to the client, then perform the regular expression on those, and finally fetch the rest of the data which matches those queries:
Regex regex = new Regex(...);
var deviceIds = ctx.Devices.Select(d => DeviceId).AsEnumerable();
var matchingIds = deviceIds.Where(id => regex.IsMatch(id))
.ToList();
var devices = ctx.Devices.Where(d => matchingIds.Contains(d.DeviceId));
That's assuming it would actually be expensive to fetch all the data for all devices to start with. If that's not too bad, it would be a simpler option. To force processing to be performed in process, use AsEnumerable():
var devices = ctx.Devices.AsEnumerable()
.Where(d => regex.IsMatch(d.DeviceId))
.ToList();
You should always remember that your LinqToEntities queries must be translated to SQL queries. Since SQL Server has no support for regular expressions, this can not work.
As suggested in the comment by Paul Ruane, StartsWith will work. This can be translated by LinqToEntities into WHERE DeviceId LIKE 'DEVICE%'.
If StartsWith isn't enough because you may need to look for strings in the middle of database columns, Contains will also work:
var devices = from d in ctx.Devices
where d.DeviceId.Contains("DEVICE")
select d;
This will result in the following: WHERE DeviceId LIKE '%DEVICE%'.
Remember when using Entity Framework or Linq to Sql that your query ends up being translated to SQL. SQL doesn't understand your regular expression object, and can't use its matches on the server side. To use your RegEx easily you could instead retrieve all the devices from the server first, and then use your existing logic. e.g.
using (var ctx = new MyEntities())
{
var devices = from Device d in ctx.Devices select d;
// Retrieve all the devices:
devices = devices.ToList();
devices = from d in devices
let matches = searchTerm.Matches(d.DeviceId)
where matches.Count > 0
select ((Device)d);
return devices.ToList();
}
This does carry the cost of retrieving more data than you need, potentially much more. For complex logic you may want to consider a Stored Procedure, which could potentially use the same RegEx via CLR functions.
LinqToEntities does not support pushing Regex's down to the database. Just do Contains (which gets converted to sql... where DeviceId Like '%DEVICE%').
filterText = #"DEVICE.";
using (var ctx = new MyEntities())
{
var devices = from d in ctx.Devices
d.DeviceId.Contains(filterText)
select d;
return devices.ToList();
}
Related
I have a weird situation regarding EntityFramework 6 with .NET 4.5 (C#).
I have (almost) the same query in two different places. But one time it queries agains the database and the second time it queries against in-memory objects. And since I'm filtering for a substring, this is a crucial difference:
Database structure are tables Role, Right and a cross-table Role_Right
First time around I want to find all available rights that are not already assigned to the role plus (and that's where it gets complicated) a manual filter to reduce the result list:
Role role = ...;
string filter = ...;
var roleRightNames = role.Right.Select(roleRight => roleRight.RightName);
var filteredRights = context.Right.Where(right => !roleRightNames.Contains(right.RightName));
if (!string.IsNullOrWhiteSpace(filter))
{
filteredRights = filteredRights.Where(e => e.RightName.Contains(filter));
}
var result = filteredRights.ToList();
I cannot use IndexOf(filter, StringComparison.InvariantCultureIgnoreCase) >= 0) because this cannot be translated to SQL. But I'm fine with Contains because it produces the desired result (see below).
When enabling the SQL output I get:
SELECT [Extent1].[RightName] AS [RightName]
FROM [dbo].[Right] AS [Extent1]
WHERE ( NOT ([Extent1].[RightName] IN ('Right_A1', 'Right_A2', 'Right_B1'))) AND ([Extent1].[RightName] LIKE #p__linq__0 ESCAPE '~'
-- p__linq__0: '%~_a%' (Type = AnsiString, Size = 8000)
Which is exactly what I want, a case-insensitive search on the filter "_a" to find for example 'Right_A3'
The second time I want to filter the existing associated rights for the same filter:
Role role = ...;
string filter = ...;
var filteredRights = string.IsNullOrWhiteSpace(filter)
? role.Right
: role.Right.Where(e => e.RightName.IndexOf(filter, StringComparison.InvariantCultureIgnoreCase) >= 0);
var result = filteredRights.ToList();
This time it forces me to use IndexOf because it uses the Contains method of the string instead of translating it to an SQL LIKE and string.Contains is case-sensitive.
My problem is that I cannot - from looking at the code - predict when a query is executed against the database and when it is done in-memory and since I cannot use IndexOf in the first query and Contains in the second this seems to be a bit unpredictable to me. What happens when one day the second query is executed first and the data is not already in-memory?
Edit 10 Feb 2020
OK, so I figured out what the main difference is. context.Right is of type DbSet which is an IQueryable and so is the subsequent extension method Where. However userRole.Right returns an ICollection which is an IEnumerable and so is the subsequent Where. Is there a way to make the relationship property of an entity object to an IQueryable? AsQueryable did not work. Which means that all associated Right entities are always gotten from the database before doing an in-memory Where.
We're not talking about huge amounts of data and at least now this behaviour is predictable, but I find it unfortunate nonetheless.
My problem is that I cannot - from looking at the code - predict when
a query is executed against the database and when it is done in-memory
and since I cannot use IndexOf in the first query and Contains in the
second this seems to be a bit unpredictable to me.
You can use IndexOf and Contains in both queries, as long as you don't use the overload featuring a StringComparison. As pointed by #BrettCaswell, the case matching is fixed by the collation of your Database/Table/Column.
A query will be translated to SQL if its root is a context's DbSet and all method calls are translatable to SQL.
As soon as a method cannot be translated, the current state request is performed at SQL level and the remainder of the query is performed in the memory of the .Net application.
Also I think that p__linq__0 value should be '%~_a%' as _ is a special character in LIKE clauses.
OK, so I found two different solutions to always query against the database in case a relation contains a huge result set. Both solutions are not directly intuitive - IMHO - and you will need the DbContext variable which you hadn't needed before.
Solution one is using the Role table as a starting point and simply filtering for the entity with the correct Id.
Note You cannot use Single because then you deal with a single entity object and you're right back where you've started. You need to use Where and then a SelectMany even though it's counter-intuitive:
Role role = ...;
string filter = ...;
var filteredRights = context.Role.Where(e => e.RoleId == userRole.RoleId).SelectMany(e => e.Right);
if (!string.IsNullOrWhiteSpace(filter))
{
filteredRights = filteredRights.Where(e => e.RightName.Contains(filter));
}
var rights = filteredRights.ToList();
which results in an SQL query against the DB:
SELECT
[Extent1].[RightName] AS [RightName]
FROM [dbo].[Role_Right] AS [Extent1]
WHERE ([Extent1].[RoleId] = #p__linq__0) AND ([Extent1].[RightName] LIKE #p__linq__1 ESCAPE '~')
-- p__linq__0: '42' (Type = Int32, IsNullable = false)
-- p__linq__1: '%~_a%' (Type = AnsiString, Size = 8000)
The second solution I found here: https://stackoverflow.com/a/7552985/2334520
In my case this results in:
Role role = ...;
string filter = ...;
var filteredRights = context.Entry(userRole).Collection(e => e.Right).Query();
if (!string.IsNullOrWhiteSpace(filter))
{
filteredRights = filteredRights.Where(e => e.RightName.Contains(filter));
}
var rights = filteredRights.ToList();
and SQL
SELECT
[Extent1].[RightName] AS [RightName]
FROM [dbo].[Role_Right] AS [Extent1]
WHERE ([Extent1].[RoleId] = #EntityKeyValue1) AND ([Extent1].[RightName] LIKE #p__linq__0 ESCAPE '~')
-- EntityKeyValue1: '42' (Type = Int32, IsNullable = false)
-- p__linq__0: '%~_a%' (Type = AnsiString, Size = 8000)
I am trying to search record(s) from table by appying multiple search parameters.
as per below snap.
here by using various parameters as per above snap i want to filter the records.
here user could enter any combination of parameter(s) to search record.
i tried something like below code hich works for single condition but fails for combination of any search paramets.
public List<students> SearchStudents(students search)
{
var result = new List<students>();
var records= from stud in db.students
where stud.enrollmentNumber== search.enrollmentNumber
|| stud.enrollmentDate==search.enrollmenttDate
|| stud.enrollmentType==search.enrollmentType
|| stud.className==search.className
select new Search()
{
enrollmentNumber= stud.enrollmentNumber,
enrollmentDate = stud.enrollmentDate,
enrollmentType = stud.enrollmentType,
Name = stud.Name,
className=stud.className,
Description = stud.Description
};
result = records.ToList();
return result;
}
but this is not working properly. means it returns same result whatever parameters I pass.
Like in the table i ahve 20 records and the enrollment number is the unique value field in DB so here when i am passing enrollment number thats like "2018-0001" it returns all records when it should return only single reocrd.
can someone guide me with this?
Without further explanation in your question about how this isn't working, the best we can do is guess. However, one very plausible reason for this is because you're including parameters you don't want to be filtering on.
Because you're using ORs in your statement, if any of those other properties are defaulted in the database, you're going to be returning those records. What you need to be doing is conditionally including your pieces of the WHERE clauses for only the properties that you want to search on. Unfortunately, that is not possible with the "SQL syntax" version of LINQ, so you will need to convert your query to that. (Good news: It's slightly more performant as well as it usually has to convert the SQL to the method syntax.)
Because of deferred execution, your query will not be sent to the database until you call a .ToList() or something to actually start processing the results. This allows you to chain method calls together, even if they are completely different C# statements. This is what you'll want to do:
public List<students> SearchStudents(students search)
{
var query = db.students;
if (!string.IsNullOrWhiteSpace(search.enrollmentNumber))
{
query = query.Where(s => s.enrollmentNumber == search.enrollmentNumber);
}
if (search.enrollmentDate != DateTime.MinValue)
{
query = query.Where(s => s.enrollmentDate == search.enrollmentDate);
}
if (!string.IsNullOrWhiteSpace(search.enrollmentType))
{
query = query.Where(s => s.enrollmentType == search.enrollmentType);
}
if (!string.IsNullOrWhiteSpace(search.className))
{
query = query.Where(s => s.className == search.className);
}
return query.Select(stud => new Search
{
enrollmentNumber= stud.enrollmentNumber,
enrollmentDate = stud.enrollmentDate,
enrollmentType = stud.enrollmentType,
Name = stud.Name,
className=stud.className,
Description = stud.Description
})
.ToList();
}
You may need to adjust the if statements in there to accommodate different data types than what is intuitive from the names, but this will only add the filter if a value has been provided.
I am trying to mimic below statement in Linq to SQL.
WHERE (rtrim(posid) like '%101' or rtrim(posid) like '%532')
I statement basically determine if posid ends with 101 or 532. In the above example I am only making 2 comparisons but their could be 1 to N comparisons all joined with OR. I store the comparison values (101,532,...) in a generic list that I send to my Linq to SQL method.
I have tried to mimic above SQL using a where clause unsuccessfully (example below):
var PosNum = new List<string>();
PosNum.Add("101");
PosNum.Add("532");
var q = (from a in context.tbl_sspos select a);
q = q.Where(p => PosNum.Contains(p.posid.Trim()));
The issue with the above where clause is that it tries to do an exact match rather I want an ends with comparison.
How would I mimic the SQL statement in Linq to SQL.
Thank You in advance for any help / advice you can provide.
I would use String.EndsWith();
This will check the end of the string rather than entire contents of it.
var q = (from a in context.tbl_sspos select a);
q = q.Where(p => p.posid.EndsWith("102") || p.posid.EndsWith("532"));
In EF 4 you can use the StartsWith / EndsWith methods by now. Might also work in LINQ to SQL.
UPDATE
Just realized that you are trying todo this against multiple values (PosNum), I don't think that this is directly supported currently. You can however concatenate multiple Where()clauses to get the result.
UPDATE 2
As AdamKing pointed out concatenating the where clauses was filtering against all PosNum values, here is the corrected version:
var baseQuery = (from a in context.tbl_sspos select a);
IEnumerable<YourType> q = null;
foreach(var pos in PosNum)
{
if(q == null)
q = baseQuery.Where(a => a.posid.EndsWith(pos));
else
q = q.Union(baseQuery.Where(a => a.posid.EndsWith(pos)));
}
This is not as pretty anymore, but works nonetheless.
Why cant I do this:
usuariosEntities usersDB = new usuariosEntities();
foreach (DataGridViewRow user in dgvUsuarios.Rows)
{
var rowtoupdate =
usersDB.usuarios.Where(
u => u.codigo_usuario == Convert.ToInt32(user.Cells[0].Value)
).First();
rowtoupdate.password = user.Cells[3].Value.ToString();
}
usersDB.SaveChanges();
And have to do this:
usuariosEntities usersDB = new usuariosEntities();
foreach (DataGridViewRow user in dgvUsuarios.Rows)
{
int usercode = Convert.ToInt32(user.Cells[0].Value);
var rowtoupdate =
usersDB.usuarios.Where(u => u.codigo_usuario == usercode).First();
rowtoupdate.password = user.Cells[3].Value.ToString();
}
usersDB.SaveChanges();
I must admit it is a more readable code but why cant this be done?
The thing about it is that LINQ queries are transformed by the compiler into an expression tree. This expression tree is then converted into T-SQL and passed to the server. LINQ to SQL maps certain methods like String.Contains to the T-SQL equivalents.
In the first example, LINQ apparently does not map Convert.ToInt32 to anything and the exception is thrown. The reason it works in your second example is because the Convert.ToInt32 call is done outside of the query so it isn't part of the expression tree and doesn't need to be converted to T-SQL.
This MSDN page describes how LINQ to SQL translates various data types, operators, and methods into T-SQL. (Although the documentation does suggest Convert.ToInt32 is supported so I'm not sure what else might be going on here.)
Sorry just realized this is ADO.NET Entity Framework, not LINQ to SQL. This page lists the ADO.NET Entity Framework mappings. It is a bit more restrictive, mostly because it needs to work with multiple providers.
Because your LINQ to Ent. doesn't compile the query into MSIL with all the meta data, but simply translates the query into a few extantion methods and is bounded to lambda parsing abuilities of the languge. That means that
this code:
var results = from c in SomeCollection
where c.SomeProperty < someValue * 2
select new {c.SomeProperty, c.OtherProperty};
is the same as this:
var results =
SomeCollection
.Where(c => c.SomeProperty < someValue * 2)
.Select(c => new {c.SomeProperty, c.OtherProperty});
I know the following is possible with linq2db4o
from Apple a in db
where a.Color.Equals(Colors.Green)
select a
What I need however is something that allows me to build my query conditionally (like I can in other linq variants)
public IEnumerable<Apple> SearchApples (AppleSearchbag bag){
var q = db.Apples;
if(bag.Color != null){
q = q.Where(a=>a.Color.Equals(bag.Color));
}
return q.AsEnumerable();
}
In a real world situation the searchbag will hold many properties and building a giant if-tree that catches all possible combinations of filled in properties would be madman's work.
It is possible to first call
var q = (from Color c in db select c);
and then continue from there. but this is not exactly what I'm looking for.
Disclaimer: near duplicate of my question of nearly 11 months ago.
This one's a bit more clear as I understand the matter better now and I hope by now some of the db4o dev eyes could catch this on this:
Any suggestions?
Yes it's definitely possible to compose optimized LINQ queries using db4o. Granted that db is defined as follows:
IObjectContainer db;
Here is your query:
public IEnumerable<Apple> SearchApples (AppleSearchbag bag)
{
var query = db.Cast<Apple> ();
// query will be a Db4objects.Db4o.Linq.IDb4oLinqQuery<Apple>
if (bag.Color != null)
query = query.Where (a => a.Color == bag.Color);
return query;
}
In that case, the query will be executed whenever the returned enumerable is being iterated over.
Another possibility is to use the IQueryable mechanism, that has the advantage of being better recognized by developers:
public IQueryable<Apple> SearchApples (AppleSearchbag bag)
{
var query = db.AsQueryable<Apple> ();
// query will be a System.Linq.IQueryble<Apple>
if (bag.Color != null)
query = query.Where (a => a.Color == bag.Color);
return query;
}
In both cases, db4o will try to deduce an optimized query from the lambda expression upon execution, and if it fails, will fallback to LINQ to objects. The first one has the advantage of being more direct, by avoiding the queryable to LINQ to db4o transformation.