We Keep Coding

C# linq Contains method with List

I need help with Linq Contains method. Here's the code below.
This code does work but outputs an empty sets.
var query = _context.RegistrationCodes.Select(x => x);
if (request.OperatorId != null && request.OperatorId != Guid.Empty)
{
var checkOperator = _context.Operators.Include(a => a.OperatorLevel).Include(a => a.City).Include("City.StateRegion.Country").FirstOrDefault(a => a.Id == request.OperatorId);
List<String> Cities = new List<String>();
if (checkOperator.OperatorLevel.Name == "City")
{
Cities = await _context.Cities
.Where(a => (checkOperator.CityId) == (a.Id))
.Select(a => a.Code)
.ToListAsync();
}
else if (checkOperator.OperatorLevel.Name == "Regional")
{
Cities = await _context.Cities
.Where(a => checkOperator.City.StateRegionId == a.StateRegionId)
.Select(a => a.Code)
.ToListAsync();
}
else if (checkOperator.OperatorLevel.Name == "National")
{
List<Guid> StateRegion = await _context.StateRegions
.Where(a => checkOperator.City.StateRegion.CountryId == a.CountryId)
.Select(a => a.Id)
.ToListAsync();
Cities = await _context.Cities
.Where(a => StateRegion.Contains(a.StateRegionId))
.Select(a => a.Code)
.ToListAsync();
}
var nullableStrings = Cities.Cast<String?>().ToList();
query = query.Where(a => nullableStrings.Contains(a.Code));
}
I need to compare nullableStrings to a.Code which is something like this, but does not work.
query = query.Where(a => a.Code.Contains(nullableStrings));
Error : Argument 1: cannot convert from 'System.Collections.Generic.List' to 'char'
I need a method that would replace
query = query.Where(a => nullableStrings.Contains(a.Code));
A help would be appreciated. Thanks.

Looking at the code, my guess is the requirement is to get a list of operators depending on the current (check) operator's level. I suspect the issue you are encountering is that some cities may not have a code. You then want to apply all found codes to another query that you are building up.
My guess is that the crux of the problem is that some cities might not have a code, hence the concern for null-able strings, while others might have multiple codes hacked into a single-code intended field. The solution there would typically be to remove any null values
Firstly, this line:
var checkOperator = _context.Operators.Include(a => a.OperatorLevel).Include(a => a.City).Include("City.StateRegion.Country").FirstOrDefault(a => a.Id == request.OperatorId);
can be simplified to:
var checkOperator = _context.Operators
.Select(a => new
{
Level = a.OperatorLevel.Name,
CityId = a.City.Id,
CityCode = a.City.Code,
StateRegionId = a.City.StateRegion.Id,
CountryId = a.City.StateRegion.Country.Id
}).FirstOrDefault(a => a.Id == request.OperatorId);
This builds a faster query, rather than fetching an entire operator object graph, just select the fields from the object graph that we need.
Now to handle the operator level. Here I don't recommend trying to force every scenario into a single pattern. The goal is just to apply a filter to the built query, so have the scenarios do just that:
select (checkOperator.Level)
{
case "City":
query = query.Where(a => a.Code == checkOperator.CityCode);
break;
case "Regional":
var cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Id == checkOperator.StateRegionId)
.Select(a => a.Code)
.ToListAsync();
query = query.Where(a => cityCodes.Contains(a.Code));
break;
case "Country":
var cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Country.Id == checkOperator.CountryId)
.Select(a => a.Code)
.ToListAsync();
query = query.Where(a => cityCodes.Contains(a.Code));
break;
}
Now based on the comments it sounds like your data with cities and codes is breaking proper normalization where Code was intended as a 1-to-1 but later hacked to handle one city having multiple codes, so multiple values were concatenated with hyphens. (I.e. ABC-DEF) If this represents 2 Codes for the city then you will need to handle this..
private List<string> splitCityCodes(List<string> cityCodes)
{
if (cityCodes == null) throw NullReferenceException(nameof(cityCodes));
if (!cityCodes.Any()) throw new ArgumentException("At least one city code is expected.");
var multiCodes = cityCodes.Where(x => x.Contains("-")).ToList();
if (!multiCodes.Any())
return cityCodes;
var results = new List<string>(cityCodes);
results.RemoveRange(multiCodes);
foreach(var multiCode in multiCodes)
{
var codes = multiCode.Split("-");
results.AddRange(codes);
}
return results.Distinct();
}
That can probably be optimized, but the gist is to take the city codes, look for hyphenated values and split them up, then return a distinct list to remove any duplicates.
List<string> cityCodes = new List<string>();
select (checkOperator.Level)
{
case "City":
cityCodes = splitCityCodes(new []{checkOperator.CityCode}.ToList());
if(cityCodes.Count == 1)
query = query.Where(a => a.Code == cityCodes[0]);
else
query = query.Where(a => cityCodes.Contains(a.Code));
break;
case "Regional":
cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Id == checkOperator.StateRegionId)
.Select(a => a.Code)
.ToListAsync();
cityCodes = splitCityCodes(cityCodes);
query = query.Where(a => cityCodes.Contains(a.Code));
break;
case "Country":
cityCodes = await _context.Cities
.Where(a => a.Code != null && a.StateRegion.Country.Id == checkOperator.CountryId)
.Select(a => a.Code)
.ToListAsync();
cityCodes = splitCityCodes(cityCodes);
query = query.Where(a => cityCodes.Contains(a.Code));
break;
}
... and I suspect that would about do it for handling the possibility of a city code containing multiple values.

If your search argument is in the form "ABC-DEF" and you want that to match "ABC" OR "DEF" then it can be done, but it is not clear from your data setup how that scenario comes about.
Lets assume these codes are airport codes, and that a city that has multiple airports has the City.Code as a hyphenated list of the Airport codes, then if the checkOperator is in Australia, and their OperatorLevel is "National" then this might build the following nullableStrings:
var Cities = new List<string> {
"PER",
"ADE",
"DRW",
"MEL-AVV",
"SYD",
"BNE",
"OOL",
"HBA"
};
If then your query is a listing of AirPorts and you want to search the airports by these codes, specifically to match both "MEL" and "AVV" then you can use syntax like this
var nullableStrings = Cities.Cast<String?>().ToList();
query = query.Where(ap => nullableStrings.Any(n => n.Contains(ap.Code)));
But if you intend this to be translated to SQL via LINQ to Entities (so be executed server-side) then we can make this query more efficient buy normalizing the search args so we can do an exact match lookup:
var nullableStrings = Cities.Where(x => !String.IsNullOrWhiteSpace (x))
.SelectMany(x => x.Split('-'))
.Cast<String?>()
.ToList();
query = query.Where(ap => nullableStrings.Contains(ap.Code));
As this routine is called as part of a larger set and your checkOperator goes out of scope, you should try to reduce the fields that you retrieve from the database to the specific set that this query needs through a projection
Using .Select() to project out specific fields can help improve the overall efficiency of the database, not just each individual query. If the additional fields are minimal or natural surrogate keys, and your projections are common to other query scenarios then they can make good candidates for specific index optimizations.
Instead of loading SELECT * from all these table in this include list:
var checkOperator = _context.Operators.Include(a => a.OperatorLevel)
.Include(a => a.City.StateRegion.Country)
.FirstOrDefault(a => a.Id == request.OperatorId);
So instead of all the fields from OperatorLevel, City, StateRegion, Country we can load just the fields that our logic needs:
var checkOperator = _context.Operators.Where(o => o.Id == request.OperatorId)
.Select(o => new {
OperatorLevelName = o.OperatorLevel.Name,
o.CityId,
o.City.StateRegionId,
o.City.StateRegion.CountryId
})
.FirstOrDefault();
So many of the EF has poor performance opinions out there stem from a lot of poorly defined examples that proliferate the web. Eagerly loading is the same as executing SELECT * FROM ... for simple tables it's only a bandwidth and memory waste, but for complex tables that have computed columns or custom expressions there can be significant server CPU costs.
It cannot be overstated the improvements that you can experience if you use projections to expose only the specific sub-set of the data that you need, especially if you will not be attempting to modify the results of the query.
Be a good corporate citizen, only take what you need!
So lets put this back into your logic:
if (request.OperatorId != null && request.OperatorId != Guid.Empty)
{
var checkOperator = _context.Operators.Where(o => o.Id == request.OperatorId)
.Select(o => new {
OperatorLevelName = o.OperatorLevel.Name,
o.CityId,
o.City.StateRegionId,
o.City.StateRegion.CountryId
})
.FirstOrDefault();
IQueryable<City> cityQuery = null;
if (checkOperator.OperatorLevelName == "City")
cityQuery = _context.Cities
.Where(a => checkOperator.CityId == a.Id);
else if (checkOperator.OperatorLevelName == "Regional")
cityQuery = _context. Cities
.Where(a => checkOperator.StateRegionId == a.StateRegionId);
else if (checkOperator.OperatorLevelName == "National")
cityQuery = _context. Cities
.Where(c => c.StateRegion.CountryId == checkOperator.CountryId);
// TODO: is there any default filter when operator level is something else?
if (cityQuery != null)
{
var nullableStrings = cityQuery.Select(a => a.Code)
.ToList()
.Where(x => !String.IsNullOrWhiteSpace(x))
.SelectMany(x => x.Split('-'))
.Cast<String?>()
.ToList();
query = query.Where(ap => nullableStrings.Contains(ap.Code));
}
}
If you don't want or need to normalize the strings, then you can defer this whole expression without realizing the city query at all:
// No nullable string, but we can still remove missing Codes
cityQuery = cityQuery.Where(c => c.Code != null);
query = query.Where(ap => cityQuery.Any(c => c.Code.Contains(ap.Code)));

Linq Count inside Select, nested, minimizing immediate execution database dependency calls

I have a massive LINQ query that fetches information that looks like this:
In other words, first-level categories, which own second-level categories, which own third level categories. For each category we retrieve the number of listings it contains.
Here is the query:
categories = categoryRepository
.Categories
.Where(x => x.ParentID == null)
.Select(x => new CategoryBrowseIndexViewModel
{
CategoryID = x.CategoryID,
FriendlyName = x.FriendlyName,
RoutingName = x.RoutingName,
ListingCount = listingRepository
.Listings
.Where(y => y.SelectedCategoryOneID == x.CategoryID
&& y.Lister.Status != Subscription.StatusEnum.Cancelled.ToString())
.Count(),
BrowseCategoriesLevelTwoViewModels = categoryRepository
.Categories
.Where(a => a.ParentID == x.CategoryID)
.Select(a => new BrowseCategoriesLevelTwoViewModel
{
CategoryID = a.CategoryID,
FriendlyName = a.FriendlyName,
RoutingName = a.RoutingName,
ParentRoutingName = x.RoutingName,
ListingCount = listingRepository
.Listings
.Where(n => n.SelectedCategoryTwoID == a.CategoryID
&& n.Lister.Status != Subscription.StatusEnum.Cancelled.ToString())
.Count(),
BrowseCategoriesLevelThreeViewModels = categoryRepository
.Categories
.Where(b => b.ParentID == a.CategoryID)
.Select(b => new BrowseCategoriesLevelThreeViewModel
{
CategoryID = b.CategoryID,
FriendlyName = b.FriendlyName,
RoutingName = b.RoutingName,
ParentRoutingName = a.RoutingName,
ParentParentID = x.CategoryID,
ParentParentRoutingName = x.RoutingName,
ListingCount = listingRepository
.Listings
.Where(n => n.SelectedCategoryThreeID == b.CategoryID
&& n.Lister.Status != Subscription.StatusEnum.Cancelled.ToString())
.Count()
})
.Distinct()
.OrderBy(b => b.FriendlyName)
.ToList()
})
.Distinct()
.OrderBy(a => a.FriendlyName)
.ToList()
})
.Distinct()
.OrderBy(x => x.FriendlyName == jobVacanciesFriendlyName)
.ThenBy(x => x.FriendlyName == servicesLabourHireFriendlyName)
.ThenBy(x => x.FriendlyName == goodsEquipmentFriendlyName)
.ToList();
This was fast enough on my dev machine, but alas! Deployed to Azure it's very slow. The reason seems to be that this query is making hundreds of dependency calls to the database, I'm pretty sure because of the immediate execution of the Count statements. Although the app and the database are in the same datacenter, the calls add up in a way they didn't on my dev machine (~40s vs < 1s). So what I'd like to do is send this whole thing off to the database, let it crunch, and get it all back in one hit, if it's possible. How do I do this? Also if I'm approaching this whole thing wrong please tell me. This is the biggest bottleneck in my web app so any help to make it more efficient is appreciated. Thank you! (I'm less concerned about web app memory usage than I am about the cumulative effect of all the database calls.)

This is my suggestion to your massive query.
Don't use ToList() inside the inner queries.
Don't use Count() inside the inner queries.
Try to retrieve all the data once without above IEnumerable operations.In other words fetch the data as IQueryable mode.After loading it in to the App's memory,you can create your data model as you wish.This process will give huge performance boost to your app.So try that and let us know.
Update : about Count()
If you have lot of columns on that list, just fetch a 1 column without Count() using projection.After that you can get the count() on your IEnumerable list.In other words on your app's memory after fetching it from the db.

Here's what I've got so far. It's working really well, but I'm still curious if I can do this in one DB trip, not two. That would seem to be complicated by the fact that each repository has its own DBContext. If you guys have any more thoughts I'd be more than happy to upvote you.
var allCategories = categoryRepository
.Categories
.Select(x => new
{
x.CategoryID,
x.FriendlyName,
x.RoutingName,
x.ParentID
})
.ToList();
var allListings = listingRepository
.Listings
.Where(x => x.Lister.Status != Subscription.StatusEnum.Cancelled.ToString())
.Select(x => new
{
x.SelectedCategoryOneID,
x.SelectedCategoryTwoID,
x.SelectedCategoryThreeID,
})
.ToList();
categories =
allCategories
.Where(x => x.ParentID == null)
.Select(a => new CategoryBrowseIndexViewModel
{
CategoryID = a.CategoryID,
FriendlyName = a.FriendlyName,
RoutingName = a.RoutingName,
ListingCount = allListings
.Where(x => x.SelectedCategoryOneID == a.CategoryID)
.Count(),
BrowseCategoriesLevelTwoViewModels =
allCategories
.Where(x => x.ParentID == a.CategoryID)
.Select(b => new BrowseCategoriesLevelTwoViewModel
{
CategoryID = b.CategoryID,
FriendlyName = b.FriendlyName,
RoutingName = b.RoutingName,
ParentRoutingName = a.RoutingName,
ListingCount = allListings
.Where(x => x.SelectedCategoryTwoID == b.CategoryID)
.Count(),
BrowseCategoriesLevelThreeViewModels =
allCategories
.Where(x => x.ParentID == b.CategoryID)
.Select(c => new BrowseCategoriesLevelThreeViewModel
{
CategoryID = c.CategoryID,
FriendlyName = c.FriendlyName,
RoutingName = c.RoutingName,
ParentRoutingName = b.RoutingName,
ParentParentID = a.CategoryID,
ParentParentRoutingName = a.RoutingName,
ListingCount = allListings
.Where(x => x.SelectedCategoryThreeID == c.CategoryID)
.Count()
})
.OrderBy(x => x.FriendlyName)
})
.OrderBy(x => x.FriendlyName)
})
.OrderBy(x => x.FriendlyName == jobVacanciesFriendlyName)
.ThenBy(x => x.FriendlyName == servicesLabourHireFriendlyName)
.ThenBy(x => x.FriendlyName == goodsEquipmentFriendlyName);

Finding the most specific matching item

User input will be like 'BY1 2PX', which will split and stored into list like below
var items = new List<string> {'BY1 2PX', 'BY12', 'BY1', 'BY'};
I have source list of Products
public class Product
{
public string Name {get;set;}
public string Id {get;set;}
}
Below is a sample product list. There is no guarentee on ordering, it could be in any order.
var products = new List<Product>{
new Product("1", "BY1 2PX"),
new Product("2", "BY12"),
new Product("3", "BY1"),
new Product("4", "BY"),
new Product("5", "AA2 B2X"),
//...etc
}
my output should fetch 1, because its most specific match. If Id = 1 is not there then it should have fetched Id =2 like that...etc Could anyone help me in writing a linq query. I have tried something like below, is this fine?
var result = items.Select(x => products.FirstOrDefault(p =>
string.Equals(p.Name.Trim(), x, StringComparison.OrdinalIgnoreCase)))
.FirstOrDefault();

Well, you can use dictionary with its fast lookups :
var productsDict = products.ToDictionary(p => p.Name, p => p);
var key = items.FirstOrDefault(i => productsDict.ContainsKey(i));
Product result = key != null ? productsDict[key] : null;
Or as Tim suggested, if you have multiple elements with same names you can use Lookup :
var productsDict = products.ToLookup(p => p.Name, p => p);
var key = items.FirstOrDefault(i => productsDict.Contains(i));
Product result = key != null ? productsDict[key] : null;

If you want to select the best-matching product you need to select from the product- not the string-list. You could use following LINQ approach that uses List.FindIndex:
Product bestProduct = products
.Select(p => new {
Product = p,
Index = items.FindIndex(s => String.Equals(p.Name, s, StringComparison.OrdinalIgnoreCase))
})
.Where(x => x.Index != -1)
.OrderBy(x => x.Index) // ensures the best-match logic
.Select(x => x.Product)
.FirstOrDefault();
The Where ensures that you won't get an arbitrary product if there is no matching one.
Update:
A more efficient solution is this query:
Product bestProduct = items
.Select(item => products.FirstOrDefault(p => String.Equals(p.Name, item, StringComparison.OrdinalIgnoreCase)))
.FirstOrDefault(p != null); // ensures the best-match logic

You can try to find resemblance of words by using a specific algorythm called Levenshtein's distance algorythm, which is mostly used on "Did you mean 'word'" on most search websites.
This solution can be found here:
https://stackoverflow.com/a/9453762/1372750
Once you find the distance difference, you can measure which word or phrase is more "like" the searched one.

This will find for each product what is the "most specific" (the longest) match in items and will return the product with the longest match (regardless to order of either of the collections)
var result = products
.Select(p => new
{
Product = p,
MostSpecific = items.Where(item => p.Name.Contains(item))
.OrderByDescending(match => match.Length
.FirstOrDefault()
})
.Where(x => x.MostSpecific != null)
.OrderByDescending(x => x.MostSpecific.Length)
.Select(x => x.Product)
.FirstOrDefault();

I want to convert this foreach loop to a LINQ statement

I am not an great at linq by any means but I usually have no issues with a problem of this sort. I want to convert this foreach statement to a LINQ statement:
var existingKeys = new List<int>();
foreach (var taskKey in request.Keys)
{
existingKeys.AddRange(_context.WebTaskGroups
.Where(x => x.TaskGroupNameKey == key && x.TaskKey == taskKey)
.Select(x => x.TaskGroupNameKey));
}
I thought this would do it:
var existingKeys = request.Keys.ForEach(taskKey => _context.WebTaskGroups
.Where(x => x.TaskGroupNameKey == key && x.TaskKey == taskKey)
.Select(x => x.TaskGroupNameKey));
That apparently returns a void not a list...
This:
var existingKeys = request.Keys.Select(taskKey =>
_context.WebTaskGroups
.Where(x => x.TaskGroupNameKey == key && x.TaskKey == taskKey)
.Select(keys => keys.TaskGroupNameKey));
Gives me an "IEnumerable<IQueryable<int>>. So what is the secret sauce that I am missing here?

You shouldn't be performing N database queries in the first place. Using LINQ to perform those N queries instead of a foreach loop doesn't fix that core problem.
You need to re-conceptualize your query so that you have just one query that gets all of the data that you need. In this case that means getting all of the items that match your collection of keys rather than trying to match a single key and then performing N of those queries.
var requestedKeys = request.Keys;
var existingKeys = _context.WebTaskGroups
.Where(x => x.TaskGroupNameKey == key &&
requestedKeys.Contains(x.TaskKey))
.Select(x => x.TaskGroupNameKey))
.ToList();

var existingKeys = request
.SelectMany(r => r.Keys)
.SelectMany(tk =>
_context.WebTaskGroups
.Where(x.TaskGroupNameKey == key && x.TaskKey == tk)
.Select(x => x.TaskGroupNameKey))
.ToList();

var existingKeys = _context.WebTaskGroups
.Where(x => x.TaskGroupNameKey == key && request.Keys.Contains(x.TaskKey))
.Select(x => x.TaskGroupNameKey)
.ToList();

ForEach return a void: http://msdn.microsoft.com/en-us/library/bwabdf9z(v=vs.110).aspx
ForEch: Performs the specified action on each element of the List.
So what to do, is for each item in the list of request.Keys to perform the action to add to the list of existingKeys.
For example:
request.Keys.ForEach(taskKey =>
existingKeys.AddRange(_context.WebTaskGroups
.Where(x => x.TaskGroupNameKey == key && x.TaskKey == taskKey)
.Select(x => x.TaskGroupNameKey));

Lambda expression: Where clause for Include list

So I got something like this:
var myObj = db.SomeObject
.Include("Tasks")
.SingleOrDefault(x => x.Id == someObjectId);
if (myObj != null)
{
myObj.Tasks = myObj.Tasks.OrderBy(x => x.Number).ToList();
}
Here I want to be able to put a condition (where) on my Include, for instance:
.where task.IsDeleted == false
Thusfar I have not found a solution.
I am aware of the fact that I could use the where together with where I order the tasks but this however, does not run on the database but in stead uses memory. I want it to run on the database.
Does anyone here know how I can do this?
If so, is there also a way to put the order by condition to the included list of tasks?

Something like this, returs you your original object with its child collection filtered and sorted.
SomeObject a = db.SomeObjects.Where(x => x.Id == someobjectid)
.Select(
x =>
new
{
someObject = x,
task = x.Tasks.Where(task => task.IsDeleted == false)
.OrderBy(task => whatever)
})
.Select(x => x.someObject).Single();
It's actually loosing the collection of activities in the last select so you can do this :
SomeObject a = db.SomeObjects.Where(x => x.Id == someobjectid)
.Select(
x =>
new
{
someObject = x,
task = x.Tasks.Where(task => task.IsDeleted == false)
.OrderBy(task => whatever)
});
return a.FirstOrDefault().someObject;

To do this is EF, you need to specify a projection with a Select clause.
Something like this will get just the data you want from the db:
var anonymous = db.SomeObject.Where( x => x.Id == someObjectId )
.Select( x => new
{
SomeObject = x,
Tasks = x.Tasks
.Where( o => !o.IsDeleted )
.OrderBy( o => ... )
}
)
.SingleOrDefault()
;
You will end up with an instance of the anonymous type , but you can easily fix that up on the client:
MyObject myObject = anonymous.SomeObject;
myObject.Tasks = anonymous.Tasks;

The simple answer is: You can't do that.
Why? Because you query SomeObject.
Each returned SomeObject contains all referenced data, because if it wouldn't it wouldn't represent the actual object in the database.

What about getting them separately:
var myObj = db.SomeObject
.SingleOrDefault(x => x.Id == someObjectId);
var tasks = db.SomeObject
.Where(x => x.Id == someObjectId)
.SelectMany(x => x.Tasks)
.Where(x => !x.Deleted);