We Keep Coding

How to deal with multiple queries for each resource?

I want to create a GraphQL API with .NET Core using the GraphQL and GraphiQL package. I created one query for users and one for tasks
public sealed class UserQuery : ObjectGraphType
{
private List<User> users = new List<User>();
public UserQuery()
{
Field<ListGraphType<UserType>>(
"users",
"Returns a collection of users.",
resolve: context => users);
}
}
public sealed class TaskQuery : ObjectGraphType
{
private List<Task> tasks = new List<Task>();
public TaskQuery()
{
Field<ListGraphType<TaskType>>(
"tasks",
"Returns a collection of tasks.",
resolve: context => tasks);
}
}
I created two schemas to "register" the queries
public sealed class UserSchema : GraphQL.Types.Schema
{
public UserSchema(UserQuery userQuery)
{
Query = userQuery;
}
}
public sealed class TaskSchema : GraphQL.Types.Schema
{
public TaskSchema(TaskQuery taskQuery)
{
Query = taskQuery;
}
}
Unfortunately I've seen that this is not possible because I have to register ISchema as a singleton service
serviceCollection
.AddSingleton<ISchema, UserSchema>()
.AddSingleton<ISchema, TaskSchema>();
So the TaskSchema overwrites the UserSchema. One solution would be to have one single schema with one single query containing the content of UserQuery and TaskQuery but things get messy really fast, no?
Any better ideas to split the resources?
I think things might get very complex when dealing with 100 fields or even more.

Every GraphQL schema needs to contain exactly one query type: https://graphql.org/learn/schema/
And one endpoint can only surface one schema. If you want to split it up into multiple files, you could use partial classes.
You can use partial classes, but you might have to do some reflection magic to distribute the Code from the ctor. One could use reflection to call any method in the class that starts with "Init" for example. Then the partial classes can declare init methods and to what they need
Alternatively, you could also explore hot chocolate, there you do not need to declare a single schema. You can add all the different query types and they are merged automatically.

EF Core queries all columns in SQL when mapping to object in Select

While trying to organize some data access code using EF Core I noticed that the generated queries were worse than before, they now queried columns that were not needed. The basic query is just selecting from one table and mapping a subset of columns to a DTO. But after rewriting it now all columns are fetched, not just the ones in the DTO.
I created a minimal example with some queries that show the problem:
ctx.Items.ToList();
// SELECT i."Id", i."Property1", i."Property2", i."Property3" FROM "Items" AS i
ctx.Items.Select(x => new
{
Id = x.Id,
Property1 = x.Property1
}
).ToList();
// SELECT i."Id", i."Property1" FROM "Items" AS i
ctx.Items.Select(x => new MinimalItem
{
Id = x.Id,
Property1 = x.Property1
}
).ToList();
// SELECT i."Id", i."Property1" FROM "Items" AS i
ctx.Items.Select(
x => x.MapToMinimalItem()
).ToList();
// SELECT i."Id", i."Property1", i."Property2", i."Property3" FROM "Items" AS i
ctx.Items.Select(
x => new MinimalItem(x)
).ToList();
// SELECT i."Id", i."Property1", i."Property2", i."Property3" FROM "Items" AS i
The objects are defined like this:
public class Item
{
public int Id { get; set; }
public string Property1 { get; set; }
public string Property2 { get; set; }
public string Property3 { get; set; }
}
public class MinimalItem
{
public MinimalItem() { }
public MinimalItem(Item source)
{
Id = source.Id;
Property1 = source.Property1;
}
public int Id { get; set; }
public string Property1 { get; set; }
}
public static class ItemExtensionMethods
{
public static MinimalItem MapToMinimalItem(this Item source)
{
return new MinimalItem
{
Id = source.Id,
Property1 = source.Property1
};
}
}
The first query queries all columns as intended, and the second query with an anonymous object only queries the selected queries, that works all fine. Using my MinimalItem DTO also works as long as it is created directly in the Select method. But the last two queries fetch all columns even though they do exactly the same thing as the third query, just moved to a constructor or an extension method, respectively.
Obviously EF Core can't follow this code and determine that it only needs the two columns if I move it out of the Select method. But I'd really like to do that to be able to reuse the mapping code, and make the actual query code easier to read. How can I extract this kind of straightforward mapping code without making EF Core inefficiently fetching all columns all the time?

This is fundamental problem with IQueryable from the very beginning, with no out of the box solution after so many years.
The problem is that IQueryable translation and code encapsulation/reusability are mutually exclusive. IQueryable translation is based on knowledge in advance, which means the query processor must be able to "see" the actual code, and then translate the "known" methods/properties. But the content of the custom methods / calculable properties is not visible at runtime, so query processors usually fail, or in limited cases where they support "client evaluation" (EF Core does that only for final projections) they generate inefficient translation which retrieves much more data than needed like in your examples.
To recap, neither C# compiler nor BCL helps solving this "core concern". Some 3rd party libraries are trying to address it in different level of degree - LinqKit, NeinLinq and similar. The problem with them is that they require refactoring your existing code additionally to calling a special method like AsExpandable(), ToInjectable() etc.
Recently I found a little gem called DelegateDecompiler, which uses another package called Mono.Reflection.Core to decompile method body to its lambda representation.
Using it is quite easy. All you need after installing it is to mark your custom methods / computed properties with custom provided [Computed] or [Decompile] attributes (just make sure you use expression style implementation and not code blocks), and call Decompile() or DecompileAsync() custom extension method somewhere in the IQueryable chain. It doesn't work with constructors, but all other constructs are supported.
For instance, taking your extension method example:
public static class ItemExtensionMethods
{
[Decompile] // <--
public static MinimalItem MapToMinimalItem(this Item source)
{
return new MinimalItem
{
Id = source.Id,
Property1 = source.Property1
};
}
}
(Note: it supports other ways of telling which methods to decompile, for instance all methods/properties of specific class etc.)
and now
ctx.Items.Decompile()
.Select(x => x.MapToMinimalItem())
.ToList();
produces
// SELECT i."Id", i."Property1" FROM "Items" AS i
The only problem with this approach (and other 3rd party libraries) is the need of calling custom extension method Decompile, in order to wrap the queryable with custom provider just to be able to preprocess the final query expression.
It would have been nice if EF Core allow plugging custom query expression preprocessor in its LINQ query processing pipeline, thus eliminating the need of calling custom method in each query, which could easily be forgotten, and also custom query providers does not play well with EF Core specific extensions like AsTracking, AsNoTracking, Include/ ThenInclude, so it should really be called after them etc.
Update (EF Core 7.0+):
EF Core 7.0 finally added Interception to modify the LINQ expression tree capability, so now the plumbing code is reduced to
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.EntityFrameworkCore.Diagnostics;
namespace Microsoft.EntityFrameworkCore
{
public static class DelegateDecompilerDbContextOptionsBuilderExtensions
{
public static DbContextOptionsBuilder AddDelegateDecompiler(this DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder.AddInterceptors(new DelegateDecompilerQueryPreprocessor());
}
}
namespace Microsoft.EntityFrameworkCore.Query
{
using System.Linq.Expressions;
using DelegateDecompiler;
public class DelegateDecompilerQueryPreprocessor : IQueryExpressionInterceptor
{
Expression IQueryExpressionInterceptor.QueryCompilationStarting(Expression queryExpression, QueryExpressionEventData eventData)
=> DecompileExpressionVisitor.Decompile(queryExpression);
}
}
Original:
Currently there is an open issue Please open the query translation pipeline for extension #19748 where I'm trying to convince the team to add an easy way to add expression preprocessor. You can read the discussion and vote up.
Until then, here is my solution for EF Core 3.1:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.Extensions.DependencyInjection;
namespace Microsoft.EntityFrameworkCore
{
public static partial class CustomDbContextOptionsExtensions
{
public static DbContextOptionsBuilder AddQueryPreprocessor(this DbContextOptionsBuilder optionsBuilder, IQueryPreprocessor processor)
{
var option = optionsBuilder.Options.FindExtension<CustomOptionsExtension>()?.Clone() ?? new CustomOptionsExtension();
if (option.Processors.Count == 0)
optionsBuilder.ReplaceService<IQueryTranslationPreprocessorFactory, CustomQueryTranslationPreprocessorFactory>();
else
option.Processors.Remove(processor);
option.Processors.Add(processor);
((IDbContextOptionsBuilderInfrastructure)optionsBuilder).AddOrUpdateExtension(option);
return optionsBuilder;
}
}
}
namespace Microsoft.EntityFrameworkCore.Infrastructure
{
public class CustomOptionsExtension : IDbContextOptionsExtension
{
public CustomOptionsExtension() { }
private CustomOptionsExtension(CustomOptionsExtension copyFrom) => Processors = copyFrom.Processors.ToList();
public CustomOptionsExtension Clone() => new CustomOptionsExtension(this);
public List<IQueryPreprocessor> Processors { get; } = new List<IQueryPreprocessor>();
ExtensionInfo info;
public DbContextOptionsExtensionInfo Info => info ?? (info = new ExtensionInfo(this));
public void Validate(IDbContextOptions options) { }
public void ApplyServices(IServiceCollection services)
=> services.AddSingleton<IEnumerable<IQueryPreprocessor>>(Processors);
private sealed class ExtensionInfo : DbContextOptionsExtensionInfo
{
public ExtensionInfo(CustomOptionsExtension extension) : base(extension) { }
new private CustomOptionsExtension Extension => (CustomOptionsExtension)base.Extension;
public override bool IsDatabaseProvider => false;
public override string LogFragment => string.Empty;
public override void PopulateDebugInfo(IDictionary<string, string> debugInfo) { }
public override long GetServiceProviderHashCode() => Extension.Processors.Count;
}
}
}
namespace Microsoft.EntityFrameworkCore.Query
{
public interface IQueryPreprocessor
{
Expression Process(Expression query);
}
public class CustomQueryTranslationPreprocessor : RelationalQueryTranslationPreprocessor
{
public CustomQueryTranslationPreprocessor(QueryTranslationPreprocessorDependencies dependencies, RelationalQueryTranslationPreprocessorDependencies relationalDependencies, IEnumerable<IQueryPreprocessor> processors, QueryCompilationContext queryCompilationContext)
: base(dependencies, relationalDependencies, queryCompilationContext) => Processors = processors;
protected IEnumerable<IQueryPreprocessor> Processors { get; }
public override Expression Process(Expression query)
{
foreach (var processor in Processors)
query = processor.Process(query);
return base.Process(query);
}
}
public class CustomQueryTranslationPreprocessorFactory : IQueryTranslationPreprocessorFactory
{
public CustomQueryTranslationPreprocessorFactory(QueryTranslationPreprocessorDependencies dependencies, RelationalQueryTranslationPreprocessorDependencies relationalDependencies, IEnumerable<IQueryPreprocessor> processors)
{
Dependencies = dependencies;
RelationalDependencies = relationalDependencies;
Processors = processors;
}
protected QueryTranslationPreprocessorDependencies Dependencies { get; }
protected RelationalQueryTranslationPreprocessorDependencies RelationalDependencies { get; }
protected IEnumerable<IQueryPreprocessor> Processors { get; }
public QueryTranslationPreprocessor Create(QueryCompilationContext queryCompilationContext)
=> new CustomQueryTranslationPreprocessor(Dependencies, RelationalDependencies, Processors, queryCompilationContext);
}
}
You don't need to understand that code. Most (if not all) of it is a boilerplate plumbing code to support the currently missing IQueryPreprocessor and AddQueryPreprocesor (similar to recently added interceptors). I'll update it if EF Core adds that functionality in the future.
Now you can use it to plug the DelegateDecompiler into EF Core:
using System.Linq.Expressions;
using Microsoft.EntityFrameworkCore.Query;
using DelegateDecompiler;
namespace Microsoft.EntityFrameworkCore
{
public static class DelegateDecompilerDbContextOptionsExtensions
{
public static DbContextOptionsBuilder AddDelegateDecompiler(this DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder.AddQueryPreprocessor(new DelegateDecompilerQueryPreprocessor());
}
}
namespace Microsoft.EntityFrameworkCore.Query
{
public class DelegateDecompilerQueryPreprocessor : IQueryPreprocessor
{
public Expression Process(Expression query) => DecompileExpressionVisitor.Decompile(query);
}
}
A lot of code just to be able to call
DecompileExpressionVisitor.Decompile(query)
before EF Core processing, but it is what it is.
Now all you need is to call
optionsBuilder.AddDelegateDecompiler();
in your derived context OnConfiguring override, and all your EF Core LINQ queries will be preprocessed and decompiled bodies injected.
With you examples
ctx.Items.Select(x => x.MapToMinimalItem())
will automatically be converted to
ctx.Items.Select(x => new
{
Id = x.Id,
Property1 = x.Property1
}
thus translated by EF Core to
// SELECT i."Id", i."Property1" FROM "Items" AS I
which was the goal.
Additionally, composing over projection also works, so the following query
ctx.Items
.Select(x => x.MapToMinimalItem())
.Where(x => x.Property1 == "abc")
.ToList();
originally would have generated runtime exception, but now translates and runs successfully.

Entity Framework does not know anything about your MapToMinimalItem method and how to translate it into SQL, so it fetches whole entity and performs the Select on the client side.
If you take a closer look at the EF LINQ method signatures, you will see, that IQueryable operates with Expression's of Func (Select for example) instead of Funcs as it's IEnumerable counterpart, so underlying provider could analyze the code and generate what is needed (SQL in this case).
So if you want to move the projection code into separate method this method should return Expression, so EF could transform it into SQL. For example:
public static class ItemExtensionMethods
{
public static readonly Expression<Func<Item, MinimalItem>> MapToMinimalItemExpr =
source => new MinimalItem
{
Id = source.Id,
Property1 = source.Property1
};
}
though it will have limited usability caused you will not able to reuse it nested projections, only in simple like this:
ctx.Items.Select(ItemExtensionMethods.MapToMinimalItemExpr)

Is it possible to implement an authorised DbContext?

Given a DbContext and a ClientContext (custom session data about the user) is it possible create a DbContext that is "authorised": where only a subset of the rows on each "table" is available?
With an authorised DbContext I'm trying to have a central row-level authorisation implementation.
I've researched it a bit and the only way to filter out a DbSet would be to use something like Queryable.Where but that returns an IQueryable<T> and there doesn't seem to be a way to return a filtered DbSet<T> (except maybe for global queries that you can setup in Startup but they don't have access to injected dependencies like ClientContext).
Is it possible to define DbSet<T> authorisation filters via an injected scoped dependency like ClientContext?

There are model-level query filters: https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-2.0#model-level-query-filters
From the link:
This feature allows LINQ query predicates (a boolean expression
typically passed to the LINQ Where query operator) to be defined
directly on Entity Types in the metadata model (usually in
OnModelCreating). Such filters are automatically applied to any LINQ
queries involving those Entity Types, including Entity Types
referenced indirectly, such as through the use of Include or direct
navigation property references.
Example from the link:
public class BloggingContext : DbContext
{
public DbSet<Blog> Blogs { get; set; }
public DbSet<Post> Posts { get; set; }
public int TenantId { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Post>().HasQueryFilter(
p => !p.IsDeleted
&& p.TenantId == this.TenantId);
}
}
You can use this for simple scenarios. You define an instance property in your DbContext and in OnModelCreating you specify HasQueryFilter on any entity you want to filter. The property is an instance property, so if you have a scoped DbContext, the correct property value from that request would be used, which is handy if you want to filter by something from your UserContext. I have personally never tried this so I don't know how complex it allows your implementation to be, but you can play with it.

I'm not sure about EF and EF core, but we abstract the DbContext away into functional specific 'logic' blocks.
e.g:
class DbContext()
{
public DbSet<PeopleEntity> peoples;
}
class PeopleLogic()
{
DbContext _context;
PeopleLogic(DbContext context)
{
_context = context;
}
IEnumerable GetAllPeoples()
{
// create context,
// apply filters
// return result
}
}
We ofcourse have a base for simple CRUD operations;
public void AddOrUpdate(){
lock (SyncDatabaseWriteObject)
{
try
{
using (var context = CreateContext())
{
//insert the entity and add it to the db context
context.Set<TEntity>().AddOrUpdate((TEntity)entity);
context.SaveChanges();
}
return entity;
}
catch (Exception ex)
{
throw new DatabaseAccessException("Error occured while getting saving.", ex);
}
}
}
And instead of passing the dbcontext around, we pass around logics.
e.g. we seperate the logic for the database and the access to the database into 2 seperate projects, the business layer then only uses the dbAccess layer.

Nhibernate: Multiple classmaps for one entityclass

So, I'm writing a reusable library. And Nhibernate mapping by code is used for ORM operations. There will be multiple services making use of this library so I want the library to behave as dynamically as possible.
There will be multiple services and for every service there will be specific tables found in the database, these are prefixed by their service name. Unity will inject this prefix and that all works nice and dandy when using only one service.
But now I'm at the point where I have to write a service that will read and combine from multiple services. So this libdummy item will have to be mapped multiple times with different table prefixes.
public class LibDummy
{
public virtual int Id { get; set; }
public virtual string Guid { get; set; }
}
public class LibDummyMapping : ClassMapping<LibDummy>
{
public LibDummyMapping(ServiceName service)
{
Table($"{service.Name}_LibDummy");
Id(o => o.Id, m => m.Column("Id"));
Property(o => o.Guid, m => m.Column("Guid"));
}
}
I tried doing it like this:
public class FirstLibDummyMapping : LibDummyMapping
{
public FirstLibDummyMapping (ServiceName service) : base(service)
{
}
}
public class SecondLibDummyMapping : LibDummyMapping
{
public SecondLibDummyMapping (ServiceName service) : base(service)
{
}
}
But this will throw a "Duplicate class/entity mapping" error.
And with 2 different classmaps for the same entity it will throw a Collection already mapped error.
Ideally I would have one dynamic classmap that can just be natively used in the library, but that's not an option i guess??
Any ideas for this, or is this something that is just not going to work?

Any NHibernate guru's with a definitive answer?

Virtual Navigation Properties and Multi-Tenancy

I have a standard DbContext with code like the following:
public DbSet<Interest> Interests { get; set; }
public DbSet<User> Users { get; set; }
I've recently implemented multi-tenancy by creating a TenantContext that contains the following:
private readonly DbContext _dbContext;
private readonly Tenant _tenant;
public TenantContext(Tenant tenant)
: base("name=DefaultConnection") {
this._tenant = tenant;
this._dbContext = new DbContext();
}
public IQueryable<User> Users { get { return FilterTenant(_dbContext.Users); } }
public IQueryable<Interest> Interests { get { return FilterTenant(_dbContext.Interests); } }
private IQueryable<T> FilterTenant<T>(IQueryable<T> values) where T : class, ITenantData
{
return values.Where(x => x.TenantId == _tenant.TenantId);
}
So far, this has been working great. Whenever any of my services creates a new TenantContext, all queries directly off of that context are filtered through this FilterTenant method that guarantees I'm only returning tenant-relevant entities.
The problem that I'm encountering is my usage of navigation properties that do not take this into account:
using (var db = CreateContext()) // new TenantContext
{
return db.Users.
Include(u => u.Interests).FirstOrDefault(s => s.UserId == userId);
}
This query pulls up the tenant-specific Users, but then the Include() statement pulls in Interests for that user only - but across all tenants. So if a user has Interests across multiple Tenants, I get all of the user's Interests with the above query.
My User model has the following:
public int UserId { get; set; }
public int TenantId { get; set; }
public virtual ICollection<Interest> Interests { get; set; }
Is there any way that I can somehow modify these navigation properties to perform tenant-specific queries? Or should I go and tear out all navigation properties in favor of handwritten code?
The second option scares me because a lot of queries have nested Includes. Any input here would be fantastic.

As far as I know, there's no other way than to either use reflection or query the properties by hand.
So in your IQueryable<T> FilterTenant<T>(IQueryable<T> values) method, you'll have to inspect your type T for properties that implement your ITenantData interface.
Then you're still not there, as the properties of your root entity (User in this case) may be entities themselves, or lists of entities (think Invoice.InvoiceLines[].Item.Categories[]).
For each of the properties you found by doing this, you'll have to write a Where() clause that filters those properties.
Or you can hand-code it per property.
These checks should at least happen when creating and editing entities. You'll want to check that navigation properties referenced by an ID property (e.g. ContactModel.AddressID) that get posted to your repository (for example from an MVC site) are accessible for the currently logged on tenant. This is your mass assignment protection, which ensures a malicious user can't craft a request that would otherwise link an entity to which he has permissions (a Contact he is creating or editing) to one Address of another tenant, simply by posting a random or known AddressID.
If you trust this system, you only have to check the TenantID of the root entity when reading, because given the checks when creating and updating, all child entities are accessible for the tenant if the root entity is accessible.
Because of your description you do need to filter child entities. An example for hand-coding your example, using the technique explained found here:
public class UserRepository
{
// ctor injects _dbContext and _tenantId
public IQueryable<User> GetUsers()
{
var user = _dbContext.Users.Where(u => u.TenantId == _tenantId)
.Select(u => new User
{
Interests = u.Interests.Where(u =>
u.TenantId == _tenantId),
Other = u.Other,
};
}
}
}
But as you see, you'll have to map every property of User like that.

Just wanted to offer an alternative approach to implementing multi-tenancy, which is working really well in a current project, using EF5 and SQL 2012. Basic design is (bear with me here...):
Every table in the database has a column (ClientSid binary, default constraint = SUSER_SID()) and is never queried directly, only ever via a dedicated view
Each view is a direct select over the table with WHERE (ClientSid = SUSER_SID()) but doesn't select the ClientSid (effectively exposing the interface of the table)
EF5 model is mapped to the VIEW, not the TABLE
The connection string is varied based on the context of the tenant (user / client whatever multi-tenant partition requirement may be)
That's pretty much it - though it might be useful to share. I know it's not a direct answer to your question, but this has resulted in basically zero custom code in the C# area.