We Keep Coding

C# linq Contains method with List

I need help with Linq Contains method. Here's the code below.
This code does work but outputs an empty sets.
var query = _context.RegistrationCodes.Select(x => x);
if (request.OperatorId != null && request.OperatorId != Guid.Empty)
{
var checkOperator = _context.Operators.Include(a => a.OperatorLevel).Include(a => a.City).Include("City.StateRegion.Country").FirstOrDefault(a => a.Id == request.OperatorId);
List<String> Cities = new List<String>();
if (checkOperator.OperatorLevel.Name == "City")
{
Cities = await _context.Cities
.Where(a => (checkOperator.CityId) == (a.Id))
.Select(a => a.Code)
.ToListAsync();
}
else if (checkOperator.OperatorLevel.Name == "Regional")
{
Cities = await _context.Cities
.Where(a => checkOperator.City.StateRegionId == a.StateRegionId)
.Select(a => a.Code)
.ToListAsync();
}
else if (checkOperator.OperatorLevel.Name == "National")
{
List<Guid> StateRegion = await _context.StateRegions
.Where(a => checkOperator.City.StateRegion.CountryId == a.CountryId)
.Select(a => a.Id)
.ToListAsync();
Cities = await _context.Cities
.Where(a => StateRegion.Contains(a.StateRegionId))
.Select(a => a.Code)
.ToListAsync();
}
var nullableStrings = Cities.Cast<String?>().ToList();
query = query.Where(a => nullableStrings.Contains(a.Code));
}
I need to compare nullableStrings to a.Code which is something like this, but does not work.
query = query.Where(a => a.Code.Contains(nullableStrings));
Error : Argument 1: cannot convert from 'System.Collections.Generic.List' to 'char'
I need a method that would replace
query = query.Where(a => nullableStrings.Contains(a.Code));
A help would be appreciated. Thanks.

Looking at the code, my guess is the requirement is to get a list of operators depending on the current (check) operator's level. I suspect the issue you are encountering is that some cities may not have a code. You then want to apply all found codes to another query that you are building up.
My guess is that the crux of the problem is that some cities might not have a code, hence the concern for null-able strings, while others might have multiple codes hacked into a single-code intended field. The solution there would typically be to remove any null values
Firstly, this line:
var checkOperator = _context.Operators.Include(a => a.OperatorLevel).Include(a => a.City).Include("City.StateRegion.Country").FirstOrDefault(a => a.Id == request.OperatorId);
can be simplified to:
var checkOperator = _context.Operators
.Select(a => new
{
Level = a.OperatorLevel.Name,
CityId = a.City.Id,
CityCode = a.City.Code,
StateRegionId = a.City.StateRegion.Id,
CountryId = a.City.StateRegion.Country.Id
}).FirstOrDefault(a => a.Id == request.OperatorId);
This builds a faster query, rather than fetching an entire operator object graph, just select the fields from the object graph that we need.
Now to handle the operator level. Here I don't recommend trying to force every scenario into a single pattern. The goal is just to apply a filter to the built query, so have the scenarios do just that:
select (checkOperator.Level)
{
case "City":
query = query.Where(a => a.Code == checkOperator.CityCode);
break;
case "Regional":
var cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Id == checkOperator.StateRegionId)
.Select(a => a.Code)
.ToListAsync();
query = query.Where(a => cityCodes.Contains(a.Code));
break;
case "Country":
var cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Country.Id == checkOperator.CountryId)
.Select(a => a.Code)
.ToListAsync();
query = query.Where(a => cityCodes.Contains(a.Code));
break;
}
Now based on the comments it sounds like your data with cities and codes is breaking proper normalization where Code was intended as a 1-to-1 but later hacked to handle one city having multiple codes, so multiple values were concatenated with hyphens. (I.e. ABC-DEF) If this represents 2 Codes for the city then you will need to handle this..
private List<string> splitCityCodes(List<string> cityCodes)
{
if (cityCodes == null) throw NullReferenceException(nameof(cityCodes));
if (!cityCodes.Any()) throw new ArgumentException("At least one city code is expected.");
var multiCodes = cityCodes.Where(x => x.Contains("-")).ToList();
if (!multiCodes.Any())
return cityCodes;
var results = new List<string>(cityCodes);
results.RemoveRange(multiCodes);
foreach(var multiCode in multiCodes)
{
var codes = multiCode.Split("-");
results.AddRange(codes);
}
return results.Distinct();
}
That can probably be optimized, but the gist is to take the city codes, look for hyphenated values and split them up, then return a distinct list to remove any duplicates.
List<string> cityCodes = new List<string>();
select (checkOperator.Level)
{
case "City":
cityCodes = splitCityCodes(new []{checkOperator.CityCode}.ToList());
if(cityCodes.Count == 1)
query = query.Where(a => a.Code == cityCodes[0]);
else
query = query.Where(a => cityCodes.Contains(a.Code));
break;
case "Regional":
cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Id == checkOperator.StateRegionId)
.Select(a => a.Code)
.ToListAsync();
cityCodes = splitCityCodes(cityCodes);
query = query.Where(a => cityCodes.Contains(a.Code));
break;
case "Country":
cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Country.Id == checkOperator.CountryId)
.Select(a => a.Code)
.ToListAsync();
cityCodes = splitCityCodes(cityCodes);
query = query.Where(a => cityCodes.Contains(a.Code));
break;
}
... and I suspect that would about do it for handling the possibility of a city code containing multiple values.

If your search argument is in the form "ABC-DEF" and you want that to match "ABC" OR "DEF" then it can be done, but it is not clear from your data setup how that scenario comes about.
Lets assume these codes are airport codes, and that a city that has multiple airports has the City.Code as a hyphenated list of the Airport codes, then if the checkOperator is in Australia, and their OperatorLevel is "National" then this might build the following nullableStrings:
var Cities = new List<string> {
"PER",
"ADE",
"DRW",
"MEL-AVV",
"SYD",
"BNE",
"OOL",
"HBA"
};
If then your query is a listing of AirPorts and you want to search the airports by these codes, specifically to match both "MEL" and "AVV" then you can use syntax like this
var nullableStrings = Cities.Cast<String?>().ToList();
query = query.Where(ap => nullableStrings.Any(n => n.Contains(ap.Code)));
But if you intend this to be translated to SQL via LINQ to Entities (so be executed server-side) then we can make this query more efficient buy normalizing the search args so we can do an exact match lookup:
var nullableStrings = Cities.Where(x => !String.IsNullOrWhiteSpace (x))
.SelectMany(x => x.Split('-'))
.Cast<String?>()
.ToList();
query = query.Where(ap => nullableStrings.Contains(ap.Code));
As this routine is called as part of a larger set and your checkOperator goes out of scope, you should try to reduce the fields that you retrieve from the database to the specific set that this query needs through a projection
Using .Select() to project out specific fields can help improve the overall efficiency of the database, not just each individual query. If the additional fields are minimal or natural surrogate keys, and your projections are common to other query scenarios then they can make good candidates for specific index optimizations.
Instead of loading SELECT * from all these table in this include list:
var checkOperator = _context.Operators.Include(a => a.OperatorLevel)
.Include(a => a.City.StateRegion.Country)
.FirstOrDefault(a => a.Id == request.OperatorId);
So instead of all the fields from OperatorLevel, City, StateRegion, Country we can load just the fields that our logic needs:
var checkOperator = _context.Operators.Where(o => o.Id == request.OperatorId)
.Select(o => new {
OperatorLevelName = o.OperatorLevel.Name,
o.CityId,
o.City.StateRegionId,
o.City.StateRegion.CountryId
})
.FirstOrDefault();
So many of the EF has poor performance opinions out there stem from a lot of poorly defined examples that proliferate the web. Eagerly loading is the same as executing SELECT * FROM ... for simple tables it's only a bandwidth and memory waste, but for complex tables that have computed columns or custom expressions there can be significant server CPU costs.
It cannot be overstated the improvements that you can experience if you use projections to expose only the specific sub-set of the data that you need, especially if you will not be attempting to modify the results of the query.
Be a good corporate citizen, only take what you need!
So lets put this back into your logic:
if (request.OperatorId != null && request.OperatorId != Guid.Empty)
{
var checkOperator = _context.Operators.Where(o => o.Id == request.OperatorId)
.Select(o => new {
OperatorLevelName = o.OperatorLevel.Name,
o.CityId,
o.City.StateRegionId,
o.City.StateRegion.CountryId
})
.FirstOrDefault();
IQueryable<City> cityQuery = null;
if (checkOperator.OperatorLevelName == "City")
cityQuery = _context.Cities
.Where(a => checkOperator.CityId == a.Id);
else if (checkOperator.OperatorLevelName == "Regional")
cityQuery = _context. Cities
.Where(a => checkOperator.StateRegionId == a.StateRegionId);
else if (checkOperator.OperatorLevelName == "National")
cityQuery = _context. Cities
.Where(c => c.StateRegion.CountryId == checkOperator.CountryId);
// TODO: is there any default filter when operator level is something else?
if (cityQuery != null)
{
var nullableStrings = cityQuery.Select(a => a.Code)
.ToList()
.Where(x => !String.IsNullOrWhiteSpace(x))
.SelectMany(x => x.Split('-'))
.Cast<String?>()
.ToList();
query = query.Where(ap => nullableStrings.Contains(ap.Code));
}
}
If you don't want or need to normalize the strings, then you can defer this whole expression without realizing the city query at all:
// No nullable string, but we can still remove missing Codes
cityQuery = cityQuery.Where(c => c.Code != null);
query = query.Where(ap => cityQuery.Any(c => c.Code.Contains(ap.Code)));

how to send variable of type Func<T,decimal> to sum from IQueryable using ef core

I need to send a lamda in function parameter
but when i do that i've error say
Expression of type 'System.Func2[KokazGoodsTransfer.Models.Receipt,System.Decimal]' cannot be used for parameter of type 'System.Linq.Expressions.Expression1[System.Func2[KokazGoodsTransfer.Models.Receipt,System.Decimal]]' of method 'System.Decimal Sum[Receipt](System.Linq.IQueryable1[KokazGoodsTransfer.Models.Receipt], System.Linq.Expressions.Expression1[System.Func2[KokazGoodsTransfer.Models.Receipt,System.Decimal]])' (Parameter 'arg1')'
example of code
var sum = await context.Receipts.GroupBy(c => c.ClientId).Select(c => new { c.Key, Sum = c.Sum(c=>c.Amount) }).ToListAsync();
it's work fine
but when i try this i see the error
Func<Receipt, Decimal> func = c => c.Amount;
var sum = await context.Receipts.GroupBy(c => c.ClientId).Select(c => new { c.Key, Sum = c.Sum(func) }).ToListAsync();
thank you

EF usually requires an expression tree to be able to translate the code into actual SQL query.
You can try something like this (though not tested, but in some cases such tricks worked as far as I remember):
Expression<Func<Receipt, Decimal>> func = c => c.Amount;
var sum = await context.Receipts
.GroupBy(c => c.ClientId)
.Select(c => new { c.Key, Sum = c.AsQueryable().Sum(func) })
.ToListAsync();
Otherwise you maybe will need either to build select statement expression manually (which is not that easy) or look into 3rd party library like LINQKit which allows to use Func's with some magic. Something along this lines:
Expression<Func<Receipt, Decimal>> func = c => c.Amount;
var sum = await context.Receipts
.AsExpandable() // or WithExpressionExpanding on the context DI set up
.GroupBy(c => c.ClientId)
.Select(c => new { c.Key, Sum = c.Sum(func.Compile()) })
.ToListAsync();

You have to use not Func but Expression<Func<Receipt, Decimal>>. But it also will be not translatable without third-party extensions. I would suggest to use LINQKit. It needs just configuring DbContextOptions:
builder
.UseSqlServer(connectionString) // or any other provider
.WithExpressionExpanding(); // enabling LINQKit extension
Then your query will work in the following way:
Expression<Func<Receipt, Decimal>> func = c => c.Amount;
var sum = await context.Receipts.GroupBy(c => c.ClientId)
.Select(c => new { c.Key, Sum = c.Sum(x => func.Invoke(x)) })
.ToListAsync();

How to repair LINQ query? (not work on EF Core 3.0)

I have moved my application from dotnet 2.2 to 3.0. I've solved big and small problems, but the LINQ queries I wrote in the service layer are failing.
For example, this method (and linq query) not work:
public async Task<IList<PersonnelDto>> PersonnelChart(short year)
{
var result = await uow.Repository<Personnel>().Query()
.Where(x => !x.IsOpen)
.Include(i => i.CurrentMoney)
.GroupBy(gp => new { id = gp.CurrencyType})
.Select(s => new PersonnelChartDto
{
PersonnelId = s.Key.id,
PersonnelCode = s.Key.code,
PersonnelName = s.Key.name,
PersonnelCount = s.Count(),
SavingTotal = Math.Round(s.Sum(sm => sm.CurrentMoney
.Where(x => !x.IsDeleted)
.Select(ss => ss.SavingTotal)
.DefaultIfEmpty(0)
.Sum()), 2),
})
.ToListAsync();
return result;
}
When trigger this method, EF Core throw that error:
System.InvalidOperationException: Processing of the LINQ expression 'AsQueryable((Unhandled parameter: sm).CurrentMoney)' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core.
So, how can I fix this error?

How can I reuse a subquery inside a select expression?

In my database I have two tables Organizations and OrganizationMembers, with a 1:N relationship.
I want to express a query that returns each organization with the first and last name of the first organization owner.
My current select expression works, but it's neither efficient nor does it look right to me, since every subquery gets defined multiple times.
await dbContext.Organizations
.AsNoTracking()
.Select(x =>
{
return new OrganizationListItem
{
Id = x.Id,
Name = x.Name,
OwnerFirstName = (x.Members.OrderBy(member => member.CreatedAt).First(member => member.Role == RoleType.Owner)).FirstName,
OwnerLastName = (x.Members.OrderBy(member => member.CreatedAt).First(member => member.Role == RoleType.Owner)).LastName,
OwnerEmailAddress = (x.Members.OrderBy(member => member.CreatedAt).First(member => member.Role == RoleType.Owner)).EmailAddress
};
})
.ToArrayAsync();
Is it somehow possible to summarize or reuse the subqueries, so I don't need to define them multiple times?
Note that I've already tried storing the subquery result in a variable. This doesn't work, because it requires converting the expression into a statement body, which results in a compiler error.

The subquery can be reused by introducing intermediate projection (Select), which is the equivalent of let operator in the query syntax.
For instance:
dbContext.Organizations.AsNoTracking()
// intermediate projection
.Select(x => new
{
Organization = x,
Owner = x.Members
.Where(member => member.Role == RoleType.Owner)
.OrderBy(member => member.CreatedAt)
.FirstOrDefault()
})
// final projection
.Select(x => new OrganizationListItem
{
Id = x.Organization.Id,
Name = x.Organization.Name,
OwnerFirstName = Owner.FirstName,
OwnerLastName = Owner.LastName,
OwnerEmailAddress = Owner.EmailAddress
})
Note that in pre EF Core 3.0 you have to use FirstOrDefault instead of First if you want to avoid client evaluation.
Also this does not make the generated SQL query better/faster - it still contains separate inline subquery for each property included in the final select. Hence will improve readability, but not the efficiency.
That's why it's usually better to project nested object into unflattened DTO property, i.e. instead of OwnerFirstName, OwnerLastName, OwnerEmailAddress have a class with properties FirstName, LastName, EmailAddress and property let say Owner of that type in OrganizationListItem (similar to entity with reference navigation property). This way you will be able to use something like
dbContext.Organizations.AsNoTracking()
.Select(x => new
{
Id = x.Organization.Id,
Name = x.Organization.Name,
Owner = x.Members
.Where(member => member.Role == RoleType.Owner)
.OrderBy(member => member.CreatedAt)
.Select(member => new OwnerInfo // the new class
{
FirstName = member.FirstName,
LastName = member.LastName,
EmailAddress = member.EmailAddress
})
.FirstOrDefault()
})
Unfortunately in pre 3.0 versions EF Core will generate N + 1 SQL queries for this LINQ query, but in 3.0+ it will generate a single and quite efficient SQL query.

How about this:
await dbContext.Organizations
.AsNoTracking()
.Select(x =>
{
var firstMember = x.Members.OrderBy(member => member.CreatedAt).First(member => member.Role == RoleType.Owner);
return new OrganizationListItem
{
Id = x.Id,
Name = x.Name,
OwnerFirstName = firstMember.FirstName,
OwnerLastName = firstMember.LastName,
OwnerEmailAddress = firstMember.EmailAddress
};
})
.ToArrayAsync();

How about doing this like
await dbContext.Organizations
.AsNoTracking()
.Select(x => new OrganizationListItem
{
Id = x.Id,
Name = x.Name,
OwnerFirstName = x.Members.FirstOrDefault(member => member.Role == RoleType.Owner).FirstName,
OwnerLastName = x.Members.FirstOrDefault(member => member.Role == RoleType.Owner)).LastName,
OwnerEmailAddress = x.Members.FirstOrDefault(member => member.Role == RoleType.Owner)).EmailAddress
})
.ToArrayAsync();

ANY with ALL in Entity Framework evaluates locally

I have the following Entity Framework 2.0 query:
var user = context.Users.AsNoTracking()
.Include(x => x.UserSkills).ThenInclude(x => x.Skill)
.Include(x => x.UserSkills).ThenInclude(x => x.SkillLevel)
.FirstOrDefault(x => x.Id == userId);
var userSkills = user.UserSkills.Select(z => new {
SkillId = z.SkillId,
SkillLevelId = z.SkillLevelId
}).ToList()
Then I tried the following query:
var lessons = _context.Lessons.AsNoTracking()
.Where(x => x.LessonSkills.All(y =>
userSkills.Any(z => y.SkillId == z.SkillId && y.SkillLevelId <= z.SkillLevelId)))
.ToList();
This query evaluates locally and I get the message:
The LINQ expression 'where (([y].SkillId == [z].SkillId) AndAlso ([y].SkillLevelId <= [z].SkillLevelId))' could not be translated and will be evaluated locally.'.
I tried to solve it using userSkills instead of user.UserSkills but no luck.
Is there a way to run this query on the server?

You should try limiting the usage of in-memory collections inside LINQ to Entities queries to basically Contains on primitive value collection, which currently is the only server translatable construct.
Since Contains is not applicable here, you should not use the memory collection, but the corresponding server side subquery:
var userSkills = context.UserSkills
.Where(x => x.UserId == userId);
var lessons = context.Lessons.AsNoTracking()
.Where(x => x.LessonSkills.All(y =>
userSkills.Any(z => y.SkillId == z.SkillId && y.SkillLevelId <= z.SkillLevelId)))
.ToList();
or even embed the first subquery into the main query:
var lessons = context.Lessons.AsNoTracking()
.Where(x => x.LessonSkills.All(y =>
context.UserSkills.Any(z => z.UserId == userId && y.SkillId == z.SkillId && y.SkillLevelId <= z.SkillLevelId)))
.ToList();

Use Contains on the server then filter further on the client:
var userSkillIds = userSkills.Select(s => s.SkillId).ToList();
var lessons = _context.Lessons.AsNoTracking()
.Where(lsn => lsn.LessonSkills.All(lsnskill => userSkillIds.Contains(lsnskill.SkillId)))
.AsEnumerable() // depending on EF Core translation, may not be needed
.Where(lsn => lsn.LessonSkills.All(lsnskill => userSkills.Any(uskill => uskill.SkillId == lsnskill.SkillId && lsnskill.SkillLevelId <= uskill.SkillLevelId)))
.ToList();