We Keep Coding

Use dynamic linq filters in the Include and ThenInclude part of a entity framework core query

What i want to achieve:
Currently I'm working on a filter system which will based on user input apply different filters on entites. That is an easy task if you just focus on the root entity or on Many-to-one relationships. But as soon as you want to filter on collections it gets a little bit harder to express and also harder to query.
The Problem:
I want to filter on the root entity (Organization) and also want to filter on collections like Organization.Contracts or Contract.Licenses. I have the option to add a selection to the Include and ThenInclude clause (see my example). But i can only add fixed LINQ-Querys but not build dynamic functions to select the right rows.
The Exception:
System.ArgumentException: "Expression of type 'System.Func`2[ExpressionTreeTest.MinimalTest+Contract,System.Boolean]' cannot be used for parameter of type 'System.Linq.Expressions.Expression`1[System.Func`2[ExpressionTreeTest.MinimalTest+Contract,System.Boolean]]' of method 'System.Linq.IQueryable`1[ExpressionTreeTest.MinimalTest+Contract] Where[Contract](System.Linq.IQueryable`1[ExpressionTreeTest.MinimalTest+Contract], System.Linq.Expressions.Expression`1[System.Func`2[ExpressionTreeTest.MinimalTest+Contract,System.Boolean]])' Arg_ParamName_Name"
Based on the exception i saw, that Entity Framework Core wants a System.Linq.Expressions.Expression1[System.Func2[ExpressionTreeTest.MinimalTest+Contract,System.Boolean]] but i provide a System.Func`2[ExpressionTreeTest.MinimalTest+Contract,System.Boolean]. As soon as i change this to Expression<Func<Contract, bool>> ContractFilterExpression = c => c.EndDate > DateTime.Now.AddMonths(11); intellisense reports an error, that this can not be accepted.
Other awnsers:
I found many different questions about querying entity framework core and building dynamic queries. Based on this I was able to build my dynamic queries for the root entity (Organization). But for the nested ThenInclude list queries i could not find any dynamic example.
My minimal test case for you:
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Query.SqlExpressions;
using System.ComponentModel.DataAnnotations;
using System.Linq.Expressions;
namespace ExpressionTreeTest
{
public class MinimalTest
{
public void RunTest()
{
AppDbContext dbContext = new AppDbContext();
dbContext.Database.EnsureDeleted();
dbContext.Database.EnsureCreated();
// Setup some test data to check the query results
SetupTestData(dbContext);
// Pre build query with entity framework, which is working as expected
// Expectation:
// Get all Organizations with the Name == NameOfCompany and
// Include all Organization.Contracts which are Created > ateTime.Now.AddMonths(12)
// ThenInclude all Contract.Licenses which are l.Articel.Name == "TestArticelName"
IQueryable<Organization> preBuildQuery = dbContext.Organizations.Where(o => o.Name == "NameOfCompany").
Include(c => c.Contracts.Where(c => c.Created > DateTime.Now.AddMonths(12))).
ThenInclude(l => l.Licenses.Where(l => l.Articel.Name == "TestArticelName"));
// This prints 1, which is the desired result
Console.WriteLine("Query result count: " + preBuildQuery.ToList().Count());
// This is the dynamic filter funtion for the Include-Part of the query
// This function gets accepted by Visual Studio but throws an error by Entity Framework
Func<Contract, bool> ContractFilterFunction = c => c.EndDate > DateTime.Now.AddMonths(11);
// Build the above query dynamically based on user input
IQueryable<Organization> dynamicQuery = dbContext.Organizations.Where(BuildWhereQuery()).
Include(c => c.Contracts.Where(ContractFilterFunction)).
ThenInclude(l => l.Licenses.Where(l => l.Articel.Name == "TestArticelName"));
// This is the line with the error you will find in the question
// If i remove the ContractFilterFunction and replace it with an inline lambda
// the query gets executed, but i am not able to dynamically set the query parameters.
Console.WriteLine("Query result count: " + dynamicQuery.ToList().Count());
}
/// <summary>
/// This method creates based on input a query with different types. In the future there
/// should be some select based on the binaryExpression to use (Equal, Greater, etc.)
/// At the moment this is static for testing purposes
/// </summary>
/// <returns>A Func<T,bool> to parse to a Linq-Entity-Framewor query</returns>
private Expression<Func<Organization, bool>> BuildWhereQuery()
{
ParameterExpression rootEntity = Expression.Parameter(typeof(Organization));
MemberExpression fieldExpression = Expression.PropertyOrField(rootEntity, "Name");
ConstantExpression valueToCompare = Expression.Constant("NameOfCompany");
var binaryExpression = Expression.Equal(fieldExpression, valueToCompare);
return Expression.Lambda<Func<Organization, bool>>(binaryExpression, rootEntity);
}
public class AppDbContext : DbContext
{
public DbSet<Organization> Organizations { get; set; }
public DbSet<Contact> Contacts { get; set; }
public DbSet<Contract> Contracts { get; set; }
public DbSet<License> Licenses { get; set; }
public DbSet<Articel> Articels { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlite(#"Data Source=mydb.db");
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
}
}
public class BasicEntity
{
[Key]
public Guid Id { get; set; }
}
public class Organization : BasicEntity
{
public string Name { get; set; }
public List<Contract> Contracts { get; set; }
public List<Contact> Contacts { get; set; }
}
public class Articel : BasicEntity
{
public string Name { get; set; }
}
public class Contact : BasicEntity
{
public string Name { get; set; }
public Organization Organization { get; set; }
}
public class Contract : BasicEntity
{
public DateTime Created { get; set; }
public DateTime EndDate { get; set; }
public List<License> Licenses { get; set; }
public Organization Organization { get; set; }
}
public class License : BasicEntity
{
public string LicenseNumber { get; set; }
public Articel Articel { get; set; }
public Contract Contract { get; set; }
}
private static void SetupTestData(AppDbContext dbContext)
{
Organization org = new Organization
{
Name = "NameOfCompany",
};
dbContext.Add(org);
dbContext.Add(new Contact
{
Name = "Contact 1",
Organization = org
});
dbContext.Add(new Contact
{
Name = "Contact 2",
Organization = org
});
Articel articel = new Articel
{
Id = Guid.NewGuid(),
Name = "TestArticelName"
};
dbContext.Add(articel);
Contract contract = new Contract
{
Id = Guid.NewGuid(),
Created = DateTime.Now,
EndDate = DateTime.Now.AddMonths(12),
Organization = org
};
dbContext.Add(contract);
License license = new License
{
Id = Guid.NewGuid(),
LicenseNumber = "12345-12345",
Articel = articel,
Contract = contract
};
dbContext.Add(license);
dbContext.SaveChanges();
}
}
}
Note:
If you have any additional tips for me or even a workaround or other solution, I would be very happy. It's not a requirement to do it like this, but it's the only way I found.

Simplest way to solve your issue is to install LINQKit - LinqKit.Microsoft.EntityFrameworkCore
Add WithExpressionExpanding in OnConfiguring
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlite(#"Data Source=mydb.db");
optionsBuilder.WithExpressionExpanding();
}
Make dynamic filter function as a Expression, EF Core will not translate Func<> to the SQL and use LINQKit extension Invoke:
// This is the dynamic filter function for the Include-Part of the query
Expression<Func<Contract, bool>> ContractFilterFunction = c => c.EndDate > DateTime.Now.AddMonths(11);
IQueryable<Organization> dynamicQuery = dbContext.Organizations.Where(BuildWhereQuery())
.Include(c => c.Contracts.Where(c => ContractFilterFunction.Invoke(c)))
.ThenInclude(l => l.Licenses.Where(l => l.Articel.Name == "TestArticelName"));
LINQKit extension will expand LambdaExpression ContractFilterFunction and inject into final Expression Tree before processing by EF Core's LINQ Translator.

Sub-Query or Join with embedded/nested document using C# LINQ with MongoDB

I am trying to do something like bellow example but getting exception as -
System.ArgumentException: Expression of type 'System.Collections.Generic.IEnumerable1 ' cannot be used for parameter of type 'System.Linq.IQueryable1' of method.
Here is my code and related classes . how can i resolve this issue, is there any way except which a trying to do.
var channels = _channelService.Collection;
var tracks = _trackService.Collection;
var query = from b in tracks.AsQueryable()
select b;
var data = (from q in channels.AsQueryable()
from p in q.Episodes
//from x in trackcoll.AsQueryable()
select new
{
p,
Tracks = query.Where(w => p.Tracks.Contains(w.Id))
}).ToList();
// Related classes
public class ContentBase : IAuditable
{
public string Id { get; set ; }
public string CreatedBy { get ; set ; }
public string CreatedOn { get ; set ; }
public string UpdatedBy { get ; set ; }
public string UpdatedOn { get; set; }
}
public class Channel: ContentBase
{
public List<Episode> Episodes { get; set; }
}
public class Episode: ContentBase
{
// List of track Id
public List<string> Tracks { get; set; }
}
public class Track: ContentBase
{
public string TrackUrl { get; set; }
public string Duration { get; set; }
public string Size { get; set; }
public string ContentType { get; set;
}

MongoDB's LINQ support for joins is limited to equal joins as described here. Your expression cannot be translated into Aggregation Framework's $lookup since there's no equivalent syntax for .Contains().
Therefore you have to run an operation that's closer to Aggregation Framework syntax. One example is a fluent aggregate interface which allows you to run extension methods having the same name as Aggregation Framework's operators. Try:
var q = _channels.Aggregate()
.Unwind(x => x.Episodes)
.Lookup(
foreignCollectionName:"tracks",
localField:"Episodes.Tracks",
foreignField:"_id",
#as:"Tracks");
var result = q.ToList();
Above code will return a List<BsonDocument>

mickl's answer will get you there with the official driver, but if you don't like dealing with bsondocuments and would like to have some degree of type-safety, you can simply do the following with mongodb.entities library (which i'm the author of):
public class EpisodeWithTracks
{
public Track[] Tracks { get; set; }
}
var pipeline = new Template<Channel, EpisodeWithTracks>(#"
[
{
$unwind: '$<Episodes>'
},
{
$lookup: {
from: '<track_collection>',
localField: '<Episodes.Tracks>',
foreignField: '_id',
as: '<Tracks>'
}
}
]")
.Path(c => c.Episodes)
.Tag("track_collection", collectionName)
.Path(c => c.Episodes[0].Tracks)
.PathOfResult(ewt => ewt.Tracks);
var result = DB.Aggregate(pipeline)
.ToList();
here's the wiki page explaining how it works.

Trying to resolve "The Include path expression must refer to a navigation property defined on the type"

Currently, I'm working with an IQueryable() and cannot get part of the query to work. I'm getting a System.ArgumentException with a description that is in the title of this post.
I'll start with the TSQL query that I need but in IQueryable() code:
SELECT w.*,wr.*,c.*
FROM WorldRegion w
JOIN WorldRegionResource wr
ON w.Id = wr.Id
JOIN Culture c
ON wr.CultureId = c.Id
WHERE c.CultureName = 'en-US'
Now I'll break the code:
Here's the Entity structure:
public class WorldRegion
{
public virtual ICollection<WorldRegionResource> WorldRegionResources { get; set; }
}
public class WorldRegionResource
{
public Culture Culture { get; set; }
}
public class Culture
{
public string CultureName { get; set; }
}
Here's a working query (with other expressions removed for brevity), but I need to add a filter to this as this returns ALL available WorldRegionResources for the WorldRegion and I want to return ONE:
IQueryable<WorldRegion> query = repository.Queryable();
query.Include(w => w.WorldRegionResources.Select(wr => wr.Culture));
I need to add a filter:
Where Culture.CultureName == "en-US"
Initially I tried changing the query.Include (which threw the exception):
query.Include(w => w.WorldRegionResources.Where(wr => wr.Culture.CultureName == "en-US"));
I've been searching for a solution, but with no success.
I've even tried building an anonymous query:
query.Include(w => w.WorldRegionResources.Select(wr => wr.Culture));
query.Select(w => new
{
WorldRegion = w,
WorldRegionResource = w.WorldRegionResources.Select(wr => new
{
WorldRegionResource = wr,
Culture = wr.Culture.CultureName == "en-US"
})
});
But this produced the same result as the initial query. SQL Server Profiler doesn't even recognize the 'en-US' parameter.
Thank you in advance for your time and any assistance is appreciated,
Ronnie

I believe that you have to tell the enities which entity they map to.
Try adding the foreign key attribute to the foreign key id.
public class WorldRegion
{
public virtual ICollection<WorldRegionResource> WorldRegionResources { get; set; }
}
public class WorldRegionResource
{
[ForeignKey(nameof(Culture))]
public int CultureId { get; set; }
public virtual Culture Culture { get; set; }
}
public class Culture
{
public string CultureName { get; set; }
}

Using Lambda to insert derived attribute into IQueryable dataset

I have the following query:
IQueryable<BarcodeQuery> barcodes = db.Barcodes.Select(b => new BarcodeQuery
{
id = b.id,
category_id = b.category_id,
...
checkout = b.Checkouts.Select(c => new CheckoutChild
{
id = c.id,
loanee_id = c.loanee_id,
...
})
.Where(c => c.datein == null)
.FirstOrDefault()
});
And so on. It's based on this model:
public class BarcodeQuery
{
public int id { get; set; }
public int category_id { get; set; }
...
public CheckoutChild checkout { get; set; }
public CheckoutStatus checkoutStatus { get; set; }
}
My question is about CheckoutStatus down there at the bottom. It looks like this:
public class CheckoutStatus
{
public string status { get; set; }
public int daysUntilDue { get; set; }
public int daysOverdue { get; set; }
}
All of those values are derived from information I get from the query--none of them are in the database itself. What is the best way of inserting the CheckoutStatus values into each barcode record?
I have a function that creates the CheckoutStatus values themselves, I just don't know how to get them into the barcode records.
Thanks!

If b has just be created with new, how can b.Checkouts contain something? I do not really understadn what you are trying to do.
EF is converting the lambda expression into a SQL statement. Therefore you can only use expressions that can actually be translated to SQL. Just query the barcodes from the DB and then add the missing information to the barcodes returned in a loop.
var barcodes = db.Barcodes.Select(...).ToList();
foreach (Barcode b in barcodes) {
b.Checkouts = ...
}

linq/ef: How to form the query which connects multiple tables to return result?

I'm extremely new to ASP .NET and LINQ so please forgive me for my ignorance.
I've a Region class:
public class Region
{
[Key]
public int Region_ID { get; set; }
public string Region_Name { get; set; }
}
And a Service class:
public class Service
{
[Key]
public int Service_ID { get; set; }
public string Service_Name { get; set; }
}
And a mapping class which stores the many-many mapping of service_IDs with region_IDs:
public class Mapping_ServiceToRegion
{
[Key]
public int Service_ID { get; set; }
public int Region_ID { get; set; }
}
Now I want to create an API function which outputs Region_Name based on given Service_ID. This is what I have so far in my RegionsController:
// GET api/Regions/Service_ID
[ResponseType(typeof(Region))]
public async Task<IHttpActionResult> GetRegion(int id)
{
var region_id = from sr in db.Mapping_ServiceToRegions
where sr.Service_ID == id
select sr.Region_ID;
var region = await db.Regions.Select(r =>
new Region()
{
Region_ID = r.Region_ID,
Region_Name = r.Region_Name
}).SingleOrDefaultAsync(r => r.Region_ID == region_id); //ERROR
if (region == null)
{
return NotFound();
}
return Ok(region);
}
The error I'm getting is:
Cannot convert lambda expression because it is not a delegate type.
I realize that my region_id variable will have multiple region_ids based on a service_id. How can I modify the code to account for this? Is there an IN operator that I can use to say r.Region_ID IN region_id?
And does the above code look correct otherwise?
Thanks.

You should change the SingleOrDefaultAsync() call using Contains() method like below since your region_id is of IEnumerable<T> and not a single value and so you can't perform direct equality comparison.
SingleOrDefaultAsync(r => region_id.Contains(r.Region_ID))
Ahh!!! here Region is one of EF mapped entity and you are trying to construct that and thus the error. You should either chose to select an Anonymous type (or) use a custom viewmodel/DTO object like
var region = await db.Regions.Select(r =>
new
{
Region_ID = r.Region_ID,
Region_Name = r.Region_Name
}).SingleOrDefaultAsync(r => region_id.Contains(r.Region_ID));