I'm currently exploring the GraphQL development and I'm currently exploring what kind of SQL queries are Generated via EF Core and I observed that no matter that my GraphQL query includes only a few fields the EF Core sends SQL Select for all fields of the Entity.
This is the code I'm using now:
public class DoctorType : ObjectGraphType<Doctors>
{
public DoctorType()
{
Field(d => d.PrefixTitle);
Field(d => d.FName);
Field(d => d.MName);
Field(d => d.LName);
Field(d => d.SufixTitle);
Field(d => d.Image);
Field(d => d.EGN);
Field(d => d.Description);
Field(d => d.UID_Code);
}
}
public class Doctors : ApplicationUser
{
public string Image { get; set; }
[StringLength(50)]
public string UID_Code { get; set; }
}
the query I'm using is
{
doctors{
fName
lName
}
}
The SQL generated selects all fields of the Doctor entity.
Is there any way to further optimize that the generated SQL query from EF Core?
I'm guessing this happens because the DoctorType inherits from ObjectGraphType<Doctors> and not from some Projection of the Doctor, but I can't think of a clever workaround of this?
Any suggestions?
EDIT:
I'm using GraphQL.NET (graphql-dotnet) by Joe McBride version 2.4.0
EDIT 2:
Either I'm doing it wrong or I don't know.
As one of the comments suggested i downloaded GraphQL.EntityFramework Nuget package by SimonCropp
I did all the configuration needed for it:
services.AddDbContext<ScheduleDbContext>(options =>
{
options.UseMySql(Configuration.GetConnectionString("DefaultConnection"));
});
using (var myDataContext = new ScheduleDbContext())
{
EfGraphQLConventions.RegisterInContainer(services, myDataContext);
}
My Object graph Type is looking as follows
public class SpecializationType : EfObjectGraphType<Specializations>
{
public SpecializationType(IEfGraphQLService graphQlService)
:base(graphQlService)
{
Field(p => p.SpecializationId);
Field(p => p.Code);
Field(p => p.SpecializationName);
}
}
My query looks is:
public class RootQuery : EfObjectGraphType
{
public RootQuery(IEfGraphQLService efGraphQlService,
ScheduleDbContext dbContext) : base(efGraphQlService)
{
Name = "Query";
AddQueryField<SpecializationType, Specializations>("specializationsQueryable", resolve: ctx => dbContext.Specializations);
}
}
and I'm using this graphQL query
{
specializationsQueryable
{
specializationName
}
}
The debug log show that the generated SQL query is
SELECT `s`.`SpecializationId`, `s`.`Code`, `s`.`SpecializationName`
FROM `Specializations` AS `s`
even though I want only specializationName field and I'm expecting it to be:
SELECT `s`.`SpecializationName`
FROM `Specializations` AS `s`
UPDATE
I guess so far I didn't understand how graphQL really worked. I thought that there is some behind the scene fetch of data but there isn't.
The primary fetch is done in the query's field resolver :
FieldAsync<ListGraphType<DoctorType>>("doctors", resolve: async ctx => await doctorServices.ListAsync());
and as long the result to the resolver is the full object in my case the resolver return List of Doctors entity, it will query the Database for the whole entity (all fields). No optimisations are done out of the box from GraphQL doesn't matter if you return IQueryable or else of the entity you are querying.
Every conclusion here is thought of mine it is not 100% guaranteed right
So what I've did is create a group of Helper methods which are creating an selection Expression to use in the LINQ query. The helpers are using resolver's context.SubFields property to get the fields needed.
The problem is that you need for every level of the query only the leaves, say some query "specializations" with "SpecializationName" and "Code" and the "Doctors" with their "Name" and else. In this case in the RootQuery specializations field's resolver you need only the Specializations entity projection so: SpecializationName and Code , then when it goes to fetch all Doctors from the "doctors" Field in SpecializationType the resolver's context has different SubFields which should be used for the projection of the Doctor.
The problem with the above is, when you use query batches i guess even if you dont the thing is that the Doctors Field in SpecializationType needs the SpecializationId fetched in the RootQuery specializations Field.
I guess i didn't explain good what i went through.
Base line is as far as I understand we have to dynamically create selectors which the linq should use to project the entity.
I'm posting my approach here:
public class RootQuery : EfObjectGraphType
{
public RootQuery(IEfGraphQLService efGraphQlService, ISpecializationGraphQlServices specializationServices,
IDoctorGraphQlServices doctorServices, ScheduleDbContext dbContext) : base(efGraphQlService)
{
Name = "Query";
FieldAsync<ListGraphType<SpecializationType>>("specializations"
, resolve: async ctx => {
var selectedFields = GraphQLResolverContextHelpers.GetFirstLevelLeavesNamesPascalCase(ctx.SubFields);
var expression = BuildLinqSelectorObject.DynamicSelectGenerator<Specializations>(selectedFields.ToArray());
return await specializationServices.ListAsync(selector: expression);
});
}
}
SpecializationType
public class SpecializationType : EfObjectGraphType<Specializations>
{
public SpecializationType(IEfGraphQLService graphQlService
, IDataLoaderContextAccessor accessor, IDoctorGraphQlServices doctorServices)
: base(graphQlService)
{
Field(p => p.SpecializationId);
Field(p => p.Code);
Field(p => p.SpecializationName);
Field<ListGraphType<DoctorType>, IEnumerable<Doctors>>()
.Name("doctors")
.ResolveAsync(ctx =>
{
var selectedFields = GraphQLResolverContextHelpers.GetFirstLevelLeavesNamesPascalCase(ctx.SubFields);
selectedFields = GraphQLResolverContextHelpers.AppendParrentNodeToEachItem(selectedFields, parentNode: "Doctor");
selectedFields = selectedFields.Union(new[] { "Specializations_SpecializationId" });
var expression = BuildLinqSelectorObject.BuildSelector<SpecializationsDoctors, SpecializationsDoctors>(selectedFields);
var doctorsLoader = accessor.Context
.GetOrAddCollectionBatchLoader<int, Doctors>(
"GetDoctorsBySpecializationId"
, (collection, token) =>
{
return doctorServices.GetDoctorsBySpecializationIdAsync(collection, token, expression);
});
return doctorsLoader.LoadAsync(ctx.Source.SpecializationId);
});
}
}
DoctorsServices:
public class DoctorGraphQlServices : IDoctorGraphQlServices
{
public ScheduleDbContext _dbContext { get; set; }
public DoctorGraphQlServices(ScheduleDbContext dbContext)
{
_dbContext = dbContext;
}
public async Task<List<Doctors>> ListAsync(int? specializationId = null)
{
var doctors = _dbContext.Doctors.AsQueryable();
if(specializationId != null)
{
doctors = doctors.Where(d => d.Specializations.Any(s => s.Specializations_SpecializationId == specializationId));
}
return await doctors.ToListAsync();
}
public async Task<ILookup<int, Doctors>> GetDoctorsBySpecializationIdAsync(IEnumerable<int> specializationIds, CancellationToken token, Expression<Func<SpecializationsDoctors, SpecializationsDoctors>> selector = null)
{
var doctors = await _dbContext.SpecializationsDoctors
.Include(s => s.Doctor)
.Where(spDocs => specializationIds.Any(sp => sp == spDocs.Specializations_SpecializationId))
.Select(selector: selector)
.ToListAsync();
return doctors.ToLookup(i => i.Specializations_SpecializationId, i => i.Doctor);
}
}
SpecializationServices
public class SpeciaizationGraphQlServices : ISpecializationGraphQlServices
{
public ScheduleDbContext _dbContext { get; set; }
public SpeciaizationGraphQlServices(ScheduleDbContext dbContext)
{
_dbContext = dbContext;
}
public async Task<dynamic> ListAsync(string doctorId = null, Expression<Func<Specializations, Specializations>> selector = null)
{
var specializations = _dbContext.Specializations.AsQueryable();
if (!string.IsNullOrEmpty(doctorId))
{
specializations = specializations.Where(s => s.Doctors.Any(d => d.Doctors_Id == doctorId));
}
return await specializations.Select(selector).ToListAsync();
}
public async Task<ILookup<string, Specializations>> GetSpecializationsByDoctorIdAsync(IEnumerable<string> doctorIds, CancellationToken token)
{
var specializations = await _dbContext.SpecializationsDoctors
.Include(s => s.Specialization)
.Where(spDocs => doctorIds.Any(sp => sp == spDocs.Doctors_Id))
.ToListAsync();
return specializations.ToLookup(i => i.Doctors_Id, i => i.Specialization);
}
public IQueryable<Specializations> List(string doctorId = null)
{
var specializations = _dbContext.Specializations.AsQueryable();
if (!string.IsNullOrEmpty(doctorId))
{
specializations = specializations.Where(s => s.Doctors.Any(d => d.Doctors_Id == doctorId));
}
return specializations;
}
}
This post has become pretty large, sorry for the span..
For DoctorType, check the defined ObjectGraphType which is used to return Doctors.
For example, I have PlayerType like below:
public class PlayerType : ObjectGraphType<Player>
{
public PlayerType(ISkaterStatisticRepository skaterStatisticRepository)
{
Field(x => x.Id);
Field(x => x.Name, true);
Field(x => x.BirthPlace);
Field(x => x.Height);
Field(x => x.WeightLbs);
Field<StringGraphType>("birthDate", resolve: context => context.Source.BirthDate.ToShortDateString());
Field<ListGraphType<SkaterStatisticType>>("skaterSeasonStats",
arguments: new QueryArguments(new QueryArgument<IntGraphType> { Name = "id" }),
resolve: context => skaterStatisticRepository.Get(context.Source.Id), description: "Player's skater stats");
}
}
And I return Field<ListGraphType<PlayerType>> by
public class NHLStatsQuery : ObjectGraphType
{
public NHLStatsQuery(IPlayerRepository playerRepository, NHLStatsContext dbContext)
{
Field<ListGraphType<PlayerType>>(
"players",
resolve: context => {
return dbContext.Players.Select(p =>new Player { Id = p.Id, Name = p.Name });
//return playerRepository.All();
});
}
}
For the query and its columns, it is controlled by resolve in Field.
No matter what fields you want to return, make sure the columns defined in PlayerType are returned in resolve.
I suggest you:
1-use dto models and map them with database models
This means that you need to convert input dto model in database model to save in db; and also convert database models got from entity framework database select into dto model.
This is the classic approach used when you made a generic api, that for example get dto model data in input request, convert dto to save data in database, and viceversa.
2-map dto model to graphqltypes (objectgraphtype and inputobjectgraphtype)
This means that for each dto model could be necessary write 1 objectgraphtype and 1 inputobjectgraphtype.
TO DO THIS I'VE CREATE AN AUTOMATIC DTO TO GRAPHTYPE CONVERTER, so you don't need to write K and K of codes!! (see link at the end)
3-DON'T USE ADDDBCONTEXT! Graphql middleware use a singleton pattern; everything used via Dependecy injection in graphql is singleton externally, even if it is register as scoped (AddDbContext means "scoped").
This means that you have 1 connection opened to startup. You can't do 2 db operation in the same time!
In the real life you can't use AddDbContext with Graphql!
You can use factory pattern to do this. So, don't pass dbcontext in Dependency injection, but a Func and instantiate dbcontext explicitally.
Here a complete implementation example:
https://github.com/graphql-dotnet/graphql-dotnet/issues/576#issuecomment-626661695
There was a talk about GraphQL with EF Core 6 by #jeremylikness on .NET Conf 2021. I would recommend using .NET 6 and check his talk out:
https://devblogs.microsoft.com/dotnet/get-to-know-ef-core-6/#graphql
https://aka.ms/graphql-efcore
https://www.youtube.com/watch?v=GBvTRcV4PVA
https://www.youtube.com/watch?v=4nqjB_z5CU0
Here is an example implementation using Hot Chocolate GraphQL server:
https://chillicream.com/docs/hotchocolate/integrations/entity-framework
This is what Microsoft wrote about GraphQL for EF Core 6.0 in their High-level plan:
GraphQL has been gaining traction over the last few years across
a variety of platforms. We plan to investigate the space and find ways
to improve the experience with .NET. This will involve working with
the community on understanding and supporting the existing ecosystem.
It may also involve specific investment from Microsoft, either in the
form of contributions to existing work or in developing complimentary
pieces in the Microsoft stack.
https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-6.0/plan#graphql
To enable automatic fields projection, having your DB context set up as usual for .NET 6, add hotchocolate server:
dotnet add package HotChocolate.Data.EntityFramework
Expose some data to it:
public class MyQueries
{
[UseProjection] // Enables field projection
public IQueryable<Book> Books([Service] MyContext db) => db.Books;
}
Enable it in Program.cs:
builder.Services.AddGraphQLServer().AddQueryType<MyQueries>().AddProjections();
...
app.MapGraphQL("/graphql");
That should be enough to ensure automatic db fields projection. Now you can run GraphQL queries via builder generated at /graphql/, while monitoring SQL via MyContext.Database.Log = Console.Write;
I'm using GraphQL.NET (graphql-dotnet) by Joe McBride version 2.4.0
First of all, I'd recommend updating at least to v4.6 - there are lots of fixes and useful updates.
Secondly, if you don't have data mutation (means - update/delete/insert data), I'd say that better to not use EF for fetching data. Based on the same GraphQL.Net lib you can take a look, for example, NReco.GraphQL is using lightweight-ORM to fetch and map data (you just need to define a schema in json-file).
Related
After migration from EF Core 3 to EF Core 6 this query:
private async Task<Variation[]> GetPizzasInOrder(Uuid[] productsInOrder, CancellationToken ct)
{
return await _clientCheckupsGatewayContext.MetaProducts
.SelectMany(mp => mp.Variations)
.Where(v => productsInOrder.Contains(v.Id))
.Include(v => v.MetaProduct)
.ToArrayAsync(ct);
}
started to throw error:
System.InvalidOperationException: A tracking query is attempting to project an owned entity without a corresponding owner in its result, but owned entities cannot be tracked without their owner. Either include the owner entity in the result or make the query non-tracking using 'AsNoTracking'.
Changing to 'AsNoTracking()' gives another error:
private async Task<Variation[]> GetPizzasInOrder(Uuid[] productsInOrder, CancellationToken ct)
{
return await _clientCheckupsGatewayContext.MetaProducts
.AsNoTracking()
.SelectMany(mp => mp.Variations)
.Where(v => productsInOrder.Contains(v.Id))
.Include(v => v.MetaProduct)
.ToArrayAsync(ct);
}
System.InvalidOperationException: The Include path
'MetaProduct->Variations' results in a cycle. Cycles are not allowed
in no-tracking queries; either use a tracking query or remove the
cycle.
public class MetaProduct
{
public Uuid Id { get; }
public IReadOnlyList<Variation> Variations => _variations.ToArray();
private List<Variation> _variations = null!;
}
public class Variation
{
public Uuid Id { get; }
public MetaProduct? MetaProduct { get; }
}
Relationship configuration:
private static void MapMetaProducts(ModelBuilder modelBuilder)
{
var tagsConverter = new ValueConverter<string[], string>(
v => JsonConvert.SerializeObject(v),
v => JsonConvert.DeserializeObject<string[]>(v)
);
var builder = modelBuilder.Entity<MetaProduct>().ToTable("metaproducts");
builder.HasKey(p => p.Id);
builder.Property(p => p.Id).HasColumnName("Id");
builder.OwnsMany(mp => mp.Variations,
vBuilder =>
{
vBuilder.ToTable("metaproducts_variations");
vBuilder.WithOwner(v => v.MetaProduct!);
vBuilder.Property(v => v.Id).HasColumnName("Id");
vBuilder.HasKey("Id");
});
}
How to fix it?
It seems to me that all you're trying to do is return an array of Variations where the Variation Id is in the list.
Also, your variation should not have a navigation property back to MetaProduct when it is owned. It should really only be retrieved in the context of its owner.
If you really want to navigate from Variation to MetaProduct then you should reconsider whether Variation really is an 'owned' entity or just a related entity.
The problem is that you are entering the query from MetaProducts but then trying to Include it again. If you can do away with the navigation from Variation to MetaProduct then the following will work:
return await _clientCheckupsGatewayContext.MetaProducts
.AsNoTracking()
.SelectMany(mp => mp.Variations)
.Where(v => productsInOrder.Contains(v.Id))
.ToArrayAsync(ct);
If you really want to navigate the other way, then you'd need to promote Variation to a related entity (HasMany instead of OwnsMany), and then expose Variations as a DbSet on your context.
So I just get started,
I have an entity which keeps the data from other entities not by a direct relation but with keeping the EntityId and EntityType(Enum).
When I read these records from GraphQL I expect to resolve a field with a resolver as follow,
public class AssignmentResolver
{
public object GetEntity( Assignment assignment, AppDbContext context)
{
if(assignment.EntityType == AssignmentEntityType.PERSON)
{
return context.People.FirstOrDefault(x => x.Id == assignment.EntityId);
}
// And more checks
return null;
}
}
Then I can say
public class AssignmentQueryType: ObjectType<Assignment>
{
protected override void Configure(IObjectTypeDescriptor<Assignment> descriptor)
{
descriptor.Field("entity").ResolveWith<AssignmentResolver>(x => x.GetEntity(default!, default!));
}
}
I wanna know if this is right or is there a better way...
I mean the better way would be using a document database for this but that's not an option for now.
I also maybe instead of putting the EntityType and EntityId can simply set an actual relation to those other entities but I wanna see if this current way is possible.
Well that was fast.
I found my problem. It seems that in the resolver I cannot just return an object because the schema should be clear when being read.
So from the resolver if I return a viewModel which is shared between all those entities then we are good to go.
So the GetEntity code will change to
public EntityViewModel GetEntity( Assignment assignment, [Service] AppDbContext context)
{
if(assignment.EntityType == AssignmentEntityType.PERSON)
{
var entity = context.People.FirstOrDefault(x => x.Id == assignment.EntityId);
return new EntityViewModel(entity);
}
// And more checks
return null;
}
While trying to organize some data access code using EF Core I noticed that the generated queries were worse than before, they now queried columns that were not needed. The basic query is just selecting from one table and mapping a subset of columns to a DTO. But after rewriting it now all columns are fetched, not just the ones in the DTO.
I created a minimal example with some queries that show the problem:
ctx.Items.ToList();
// SELECT i."Id", i."Property1", i."Property2", i."Property3" FROM "Items" AS i
ctx.Items.Select(x => new
{
Id = x.Id,
Property1 = x.Property1
}
).ToList();
// SELECT i."Id", i."Property1" FROM "Items" AS i
ctx.Items.Select(x => new MinimalItem
{
Id = x.Id,
Property1 = x.Property1
}
).ToList();
// SELECT i."Id", i."Property1" FROM "Items" AS i
ctx.Items.Select(
x => x.MapToMinimalItem()
).ToList();
// SELECT i."Id", i."Property1", i."Property2", i."Property3" FROM "Items" AS i
ctx.Items.Select(
x => new MinimalItem(x)
).ToList();
// SELECT i."Id", i."Property1", i."Property2", i."Property3" FROM "Items" AS i
The objects are defined like this:
public class Item
{
public int Id { get; set; }
public string Property1 { get; set; }
public string Property2 { get; set; }
public string Property3 { get; set; }
}
public class MinimalItem
{
public MinimalItem() { }
public MinimalItem(Item source)
{
Id = source.Id;
Property1 = source.Property1;
}
public int Id { get; set; }
public string Property1 { get; set; }
}
public static class ItemExtensionMethods
{
public static MinimalItem MapToMinimalItem(this Item source)
{
return new MinimalItem
{
Id = source.Id,
Property1 = source.Property1
};
}
}
The first query queries all columns as intended, and the second query with an anonymous object only queries the selected queries, that works all fine. Using my MinimalItem DTO also works as long as it is created directly in the Select method. But the last two queries fetch all columns even though they do exactly the same thing as the third query, just moved to a constructor or an extension method, respectively.
Obviously EF Core can't follow this code and determine that it only needs the two columns if I move it out of the Select method. But I'd really like to do that to be able to reuse the mapping code, and make the actual query code easier to read. How can I extract this kind of straightforward mapping code without making EF Core inefficiently fetching all columns all the time?
This is fundamental problem with IQueryable from the very beginning, with no out of the box solution after so many years.
The problem is that IQueryable translation and code encapsulation/reusability are mutually exclusive. IQueryable translation is based on knowledge in advance, which means the query processor must be able to "see" the actual code, and then translate the "known" methods/properties. But the content of the custom methods / calculable properties is not visible at runtime, so query processors usually fail, or in limited cases where they support "client evaluation" (EF Core does that only for final projections) they generate inefficient translation which retrieves much more data than needed like in your examples.
To recap, neither C# compiler nor BCL helps solving this "core concern". Some 3rd party libraries are trying to address it in different level of degree - LinqKit, NeinLinq and similar. The problem with them is that they require refactoring your existing code additionally to calling a special method like AsExpandable(), ToInjectable() etc.
Recently I found a little gem called DelegateDecompiler, which uses another package called Mono.Reflection.Core to decompile method body to its lambda representation.
Using it is quite easy. All you need after installing it is to mark your custom methods / computed properties with custom provided [Computed] or [Decompile] attributes (just make sure you use expression style implementation and not code blocks), and call Decompile() or DecompileAsync() custom extension method somewhere in the IQueryable chain. It doesn't work with constructors, but all other constructs are supported.
For instance, taking your extension method example:
public static class ItemExtensionMethods
{
[Decompile] // <--
public static MinimalItem MapToMinimalItem(this Item source)
{
return new MinimalItem
{
Id = source.Id,
Property1 = source.Property1
};
}
}
(Note: it supports other ways of telling which methods to decompile, for instance all methods/properties of specific class etc.)
and now
ctx.Items.Decompile()
.Select(x => x.MapToMinimalItem())
.ToList();
produces
// SELECT i."Id", i."Property1" FROM "Items" AS i
The only problem with this approach (and other 3rd party libraries) is the need of calling custom extension method Decompile, in order to wrap the queryable with custom provider just to be able to preprocess the final query expression.
It would have been nice if EF Core allow plugging custom query expression preprocessor in its LINQ query processing pipeline, thus eliminating the need of calling custom method in each query, which could easily be forgotten, and also custom query providers does not play well with EF Core specific extensions like AsTracking, AsNoTracking, Include/ ThenInclude, so it should really be called after them etc.
Update (EF Core 7.0+):
EF Core 7.0 finally added Interception to modify the LINQ expression tree capability, so now the plumbing code is reduced to
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.EntityFrameworkCore.Diagnostics;
namespace Microsoft.EntityFrameworkCore
{
public static class DelegateDecompilerDbContextOptionsBuilderExtensions
{
public static DbContextOptionsBuilder AddDelegateDecompiler(this DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder.AddInterceptors(new DelegateDecompilerQueryPreprocessor());
}
}
namespace Microsoft.EntityFrameworkCore.Query
{
using System.Linq.Expressions;
using DelegateDecompiler;
public class DelegateDecompilerQueryPreprocessor : IQueryExpressionInterceptor
{
Expression IQueryExpressionInterceptor.QueryCompilationStarting(Expression queryExpression, QueryExpressionEventData eventData)
=> DecompileExpressionVisitor.Decompile(queryExpression);
}
}
Original:
Currently there is an open issue Please open the query translation pipeline for extension #19748 where I'm trying to convince the team to add an easy way to add expression preprocessor. You can read the discussion and vote up.
Until then, here is my solution for EF Core 3.1:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.Extensions.DependencyInjection;
namespace Microsoft.EntityFrameworkCore
{
public static partial class CustomDbContextOptionsExtensions
{
public static DbContextOptionsBuilder AddQueryPreprocessor(this DbContextOptionsBuilder optionsBuilder, IQueryPreprocessor processor)
{
var option = optionsBuilder.Options.FindExtension<CustomOptionsExtension>()?.Clone() ?? new CustomOptionsExtension();
if (option.Processors.Count == 0)
optionsBuilder.ReplaceService<IQueryTranslationPreprocessorFactory, CustomQueryTranslationPreprocessorFactory>();
else
option.Processors.Remove(processor);
option.Processors.Add(processor);
((IDbContextOptionsBuilderInfrastructure)optionsBuilder).AddOrUpdateExtension(option);
return optionsBuilder;
}
}
}
namespace Microsoft.EntityFrameworkCore.Infrastructure
{
public class CustomOptionsExtension : IDbContextOptionsExtension
{
public CustomOptionsExtension() { }
private CustomOptionsExtension(CustomOptionsExtension copyFrom) => Processors = copyFrom.Processors.ToList();
public CustomOptionsExtension Clone() => new CustomOptionsExtension(this);
public List<IQueryPreprocessor> Processors { get; } = new List<IQueryPreprocessor>();
ExtensionInfo info;
public DbContextOptionsExtensionInfo Info => info ?? (info = new ExtensionInfo(this));
public void Validate(IDbContextOptions options) { }
public void ApplyServices(IServiceCollection services)
=> services.AddSingleton<IEnumerable<IQueryPreprocessor>>(Processors);
private sealed class ExtensionInfo : DbContextOptionsExtensionInfo
{
public ExtensionInfo(CustomOptionsExtension extension) : base(extension) { }
new private CustomOptionsExtension Extension => (CustomOptionsExtension)base.Extension;
public override bool IsDatabaseProvider => false;
public override string LogFragment => string.Empty;
public override void PopulateDebugInfo(IDictionary<string, string> debugInfo) { }
public override long GetServiceProviderHashCode() => Extension.Processors.Count;
}
}
}
namespace Microsoft.EntityFrameworkCore.Query
{
public interface IQueryPreprocessor
{
Expression Process(Expression query);
}
public class CustomQueryTranslationPreprocessor : RelationalQueryTranslationPreprocessor
{
public CustomQueryTranslationPreprocessor(QueryTranslationPreprocessorDependencies dependencies, RelationalQueryTranslationPreprocessorDependencies relationalDependencies, IEnumerable<IQueryPreprocessor> processors, QueryCompilationContext queryCompilationContext)
: base(dependencies, relationalDependencies, queryCompilationContext) => Processors = processors;
protected IEnumerable<IQueryPreprocessor> Processors { get; }
public override Expression Process(Expression query)
{
foreach (var processor in Processors)
query = processor.Process(query);
return base.Process(query);
}
}
public class CustomQueryTranslationPreprocessorFactory : IQueryTranslationPreprocessorFactory
{
public CustomQueryTranslationPreprocessorFactory(QueryTranslationPreprocessorDependencies dependencies, RelationalQueryTranslationPreprocessorDependencies relationalDependencies, IEnumerable<IQueryPreprocessor> processors)
{
Dependencies = dependencies;
RelationalDependencies = relationalDependencies;
Processors = processors;
}
protected QueryTranslationPreprocessorDependencies Dependencies { get; }
protected RelationalQueryTranslationPreprocessorDependencies RelationalDependencies { get; }
protected IEnumerable<IQueryPreprocessor> Processors { get; }
public QueryTranslationPreprocessor Create(QueryCompilationContext queryCompilationContext)
=> new CustomQueryTranslationPreprocessor(Dependencies, RelationalDependencies, Processors, queryCompilationContext);
}
}
You don't need to understand that code. Most (if not all) of it is a boilerplate plumbing code to support the currently missing IQueryPreprocessor and AddQueryPreprocesor (similar to recently added interceptors). I'll update it if EF Core adds that functionality in the future.
Now you can use it to plug the DelegateDecompiler into EF Core:
using System.Linq.Expressions;
using Microsoft.EntityFrameworkCore.Query;
using DelegateDecompiler;
namespace Microsoft.EntityFrameworkCore
{
public static class DelegateDecompilerDbContextOptionsExtensions
{
public static DbContextOptionsBuilder AddDelegateDecompiler(this DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder.AddQueryPreprocessor(new DelegateDecompilerQueryPreprocessor());
}
}
namespace Microsoft.EntityFrameworkCore.Query
{
public class DelegateDecompilerQueryPreprocessor : IQueryPreprocessor
{
public Expression Process(Expression query) => DecompileExpressionVisitor.Decompile(query);
}
}
A lot of code just to be able to call
DecompileExpressionVisitor.Decompile(query)
before EF Core processing, but it is what it is.
Now all you need is to call
optionsBuilder.AddDelegateDecompiler();
in your derived context OnConfiguring override, and all your EF Core LINQ queries will be preprocessed and decompiled bodies injected.
With you examples
ctx.Items.Select(x => x.MapToMinimalItem())
will automatically be converted to
ctx.Items.Select(x => new
{
Id = x.Id,
Property1 = x.Property1
}
thus translated by EF Core to
// SELECT i."Id", i."Property1" FROM "Items" AS I
which was the goal.
Additionally, composing over projection also works, so the following query
ctx.Items
.Select(x => x.MapToMinimalItem())
.Where(x => x.Property1 == "abc")
.ToList();
originally would have generated runtime exception, but now translates and runs successfully.
Entity Framework does not know anything about your MapToMinimalItem method and how to translate it into SQL, so it fetches whole entity and performs the Select on the client side.
If you take a closer look at the EF LINQ method signatures, you will see, that IQueryable operates with Expression's of Func (Select for example) instead of Funcs as it's IEnumerable counterpart, so underlying provider could analyze the code and generate what is needed (SQL in this case).
So if you want to move the projection code into separate method this method should return Expression, so EF could transform it into SQL. For example:
public static class ItemExtensionMethods
{
public static readonly Expression<Func<Item, MinimalItem>> MapToMinimalItemExpr =
source => new MinimalItem
{
Id = source.Id,
Property1 = source.Property1
};
}
though it will have limited usability caused you will not able to reuse it nested projections, only in simple like this:
ctx.Items.Select(ItemExtensionMethods.MapToMinimalItemExpr)
We are using EntityFramework and have already gone through a long journey with Database First. We have multiple contexts and many, many entities. While we entertain the notion of undergoing a project of code first, we want to explore the alternate possibilities available with EF.
We are using the latest version of EF as of current, 6.0.2
We have quite a few tables with common fields such as Auditing fields "CreatedBy", "CreatedDate", "UpdatedBy" and "UpdatedDate" and we thought that it would be perfect to use a ComplexType here.
In a table that uses those fields, when we generate the code from the database, it pulls the fields in raw. We then delete it in the model browser, add the complex type and then maps the complex properties to the field names in the DB.
In this experiment, we ran the "Generate database from model" without mapping the fields to see what were the resulting column names and if any convention or auto-magic would allow that behavior bi-directionally(Turning complex type into columns and recognizing columns as complex types).
This resulted in column names with the complex type suffixed with an '_' and the field name. We tested and it didn't pull the complex type back from the database in the model when we regenerated the model.
Is there a proper way to have EF detect Complex types in tables with database first?
Is there some code factory, builder, strategy or template that I can write?
Our main motivation is that we have quite a number of tables to support, we embrace frequent changes and we want to prevent people on our team neglecting this step and breaking the code base.
Much appreciation for your time StackOverflowians!
-- Edit --
While this doesn't solve the problem using the auto-detection of complex types, this has resolve the issues of Developer's breaking the model each time they run the T4 template updates.
http://www.ninjacrab.com/2015/02/07/update-entityframework-detecting-complex-type-with-database-first/
What I did was a Generic semi-Repository pattern over EF. I also used intefaces to allow identification and automatic use of methods if they match.
Auditable Interface:
public interface IDbAuditable : IDbEntity
{
Guid CreatedById { get; set; }
DateTime CreatedOn { get; set; }
Guid ModifiedById { get; set; }
DateTime ModifiedOn { get; set; }
Guid? DeletedById { get; set; }
DateTime? DeletedOn { get; set; }
}
(I personally like the idea that if CreatedOn == ModifiedOn then it's never been modified, some people like DateTime?, doesn't really matter they serve the same purpose.)
Since this example is in a small project, I simply wrapped the EF Context and didn't use any IoC/DI. This is a small subset of all the actual code (some methods are missing and some interfaces are missing, but it should make perfect sense).
public sealed class MyDb : IDisposable
{
private MyContext _context;
public MyDb(string connectionString, Lazy<Guid?> currentUserIdFunc)
{
this._context = new MyContext(connectionString);
Database.SetInitializer<MyContext>(new DatabaseInitializer());
this._context.Database.Initialize(true);
this._currentUserIdFunc = currentUserIdFunc;
}
public async Task<T> GetEntityAsync<T>(Func<IQueryable<T>, IQueryable<T>> entityQuery) where T : class, IDbEntity
{
var query = entityQuery(this._context.Set<T>());
if (typeof(T) is IDbAuditable)
{
query = query.Cast<IDbAuditable>()
.Where(a => !a.DeletedById.HasValue)
.Cast<T>();
}
return await query.FirstOrDefaultAsync();
}
public async Task<int> UpdateAsync<T>(T entity) where T : class, IDbEntity
{
if (entity is IDbDoNotModify)
{
throw new DoNotModifyException("Entity cannot be Modified (IDoNotModify).");
}
this._context.Set<T>().Attach(entity);
var entry = this._context.Entry<T>(entity);
entry.State = EntityState.Unchanged;
var entityType = entity.GetType();
var metadata = entityType.GetCustomAttributes(typeof(MetadataTypeAttribute)).FirstOrDefault() as MetadataTypeAttribute;
if (metadata != null)
{
var type = metadata.MetadataClassType;
var properties = type.GetProperties(BindingFlags.Instance | BindingFlags.Public)
.Select(p => new
{
Name = p.Name,
ScaffoldColumn = p.GetCustomAttributes(typeof(ScaffoldColumnAttribute), true).FirstOrDefault() as ScaffoldColumnAttribute,
Readonly = entityType.GetProperty(p.Name).GetCustomAttributes(typeof(ReadOnlyAttribute), true).FirstOrDefault() as ReadOnlyAttribute
})
.Where(p => (p.ScaffoldColumn == null || p.ScaffoldColumn.Scaffold)
&& (p.Readonly == null || !p.Readonly.IsReadOnly))
.ToList();
foreach (var property in properties)
{
entry.Property(property.Name).IsModified = true;
}
}
else
{
entry.State = EntityState.Modified;
}
var auditable = entity as IDbAuditable;
if (auditable != null)
{
this.Modified(auditable, this._currentUserIdFunc.Value);
entry.Property("ModifiedOn").IsModified = true;
entry.Property("ModifiedById").IsModified = true;
}
return await this._context.SaveChangesAsync();
}
private void Modified(IDbAuditable instance, Guid? currentUserId)
{
instance.ModifiedById = currentUserId.Value;
instance.ModifiedOn = DateTime.Now;
}
}
Here is how I would use it:
// returns the first car with a model of ford
var car = MyDb.EntityAsync<Car>((query) = query
.Where(c => c.Model.Equals("ford")
);
// returns the first dog with an ownerId of id
var car = MyDb.EntityAsync<Dog>((query) => query
.Where(d => d.OwnerId == Id)
);
UpdateAsync(car);
This example application is very small so I didn't implement any UnitOfWork but it could be easily modified for such a case. Additionally, the MyDb class could be turned into a MyDb<T> if you have multiple Contexts pretty easily. I allow heavy use of DataAnnotations.
I'm building CMS like application.
For example my BlogPost page contains several widget areas. Each widget hosts a serie of "related" blog posts.
All my views are pure presentational, i build urls, convert datetime and ints into strings in my service layer. I find this approach easier to maintain, since views have zer0 logic. All logic is consolidated into AutoMapper's resolvers, converters and custom transformation logic.
So lets come closer to the problem at hand.
To create Url i need 2 parameters: BlogId and BlogSlug, my urls look like b/{id}/{slug}.html
Im quite happy with that.
In my CSM i use so called "source models", a model that is not view model, but an intermediate representation of it. Why do i have to resort to such wicked solutions ?
Well, lets take a look how a typical data retrieval code can look like in my project:
.Select(x =>
{
Id = x.Id,
BlogId = x.Blog.Id,
BlogSlug = x.Blog.Slug,
// Here is the trap, LINQ provider will throw an exception, since he doesn't know how to translate function into expression
BlogUrl = Url.Action("RenderPost", "BlogController", new { Id = x.Blog.Id, slug = x.Blog.Slug })
}
So thats not an option.
Luckily, we can do this
.Select(x => new
{
Id = x.Id,
BlogId = x.Blog.Id,
BlogSlug = x.Blog.Slug
}
.ToList()
.Select(x => new
{
// This works
BlogUrl = Url.Action("RenderPost", "BlogController", new { Id = x.BlogId, slug = x.BlogSlug })
}
Copy paste this stuff into each and every action method that renders different "intresting blog" parts (they have different visual representation as well cant use same view model) ? Not a good way, so i came up with a solution.
I created "source model", so the code will be
.Select(x => new BlogPostSourceViewModel
{
Id = x.Id,
BlogId = x.Blog.Id,
BlogSlug = x.Blog.Slug
}
.ToList()
.Select(x => x.ToBlogPostViewModel()) // Extension method { return Mapper.Map<>() }
.ToList();
This surely looks better, but i have many different models like BlogPostSourceViewModel, BlogAuthorSourceViewModel, BlogCommentSourceViewModel. They all need this link building logic.
Ok, i extract the needed source data (BlogId, BlogSlug) into an interface
BlogPostSourceViewModel : IBlogPostUrl
BlogAuthorSourceViewModel: IBlogPostUrl
BlogCommentSourceViewModel : IBlogPostUrl
Then i define the mappings
Mapper.CreateMap<BlogPostSourceViewModel, BlogPostViewModel>
.ForMember(dest => dest.BlogUrl, opt => opt.ResolveUsing<BlogPostUrlResolver>())
Mapper.CreateMap<BlogAuthorSourceViewModel, BlogAuthorViewModel>
.ForMember(dest => dest.BlogUrl, opt => opt.ResolveUsing<BlogPostUrlResolver>())
Mapper.CreateMap<BlogCommentSourceViewModel, BlogCommentViewModel>
.ForMember(dest => dest.BlogUrl, opt => opt.ResolveUsing<BlogPostUrlResolver>())
Resolver:
BlogPostUrlResolver : ValueResolver<IBlogPostUrl, String>
// Here goes the url building logic
As you see the more models i have that need blog url the more identical mappings i have to add. This is ok for now, but as project grows it will be painful.
Ideally i would want to have it like this:
Mapper.CreateMap<IBlogPostUrl, SomeOtherInterfaceWithBlogUrlAsString>
.ForMember(dest => dest.BlogUrl, opt => opt.ResolveUsing<BlogPostUrlResolver>())
but Automapper doesn't understand it. And i dont know how if there is other way to do it.
Any ideas ?
If I understand the question correctly...
In order to use AutoMapper mapping capabilities across all of your classes you can provide base class that will use Generics:
public abstract class Base<Entity, ViewModel>
where Entity : EntityObject
where ViewModel : BaseViewModel
{
// you will call this method from your operations class using base
public SomeViewModel GetData()
{
public Entity entityObject = db.Entity.SingleOrDefault();
public ViewModel yourViewModelName = base.Map(entityObject);
return yourViewModelName;
}
....
// this will be defined only once for each mapping direction
// ie. there will be multiple of these.
public static Entity Map(ViewModel typeViewModel)
{
try
{
Mapper.CreateMap<ViewModel, Entity>();
Entity t = Mapper.Map<ViewModel, Entity>(typeViewModel);
return t;
}
catch (Exception exc)
{
throw exc;
}
}
}
// this will be your class for database operations on specific Entity
// that inherits generic base, with its AutoMapping setup.
public class DataBaseOperationsClass : Base<SomeEntity, SomeViewModel>
{
public SomeViewModel Get()
{
return base.GetData();
}
}
Hope this helps !