We Keep Coding

Including multiple many-to-many collection navigation properties

Description
When attempting to eagerly .Include() Many-to-Many ICollection properties on an already nested ICollection, I receive a EntityCommandCompilationException with the innerException: System.NotSupportedException: APPLY joins are not supported.
Are there any (suitable) workarounds, or is this simply not supported by EF6/SQLite? If not supported, would migrating to EntityFramework.Core work?
If some of you can confirm that this is a limitation, I'd create an issue on the SQLite issue tracker
Quick Problem Synopsis
// Bare minimum model definitions - see below for a full implementation
public class Mode
{
[Key] public int Id { get; set; }
}
public class Type
{
[Key] public int Id { get; set; }
}
public class Preset
{
[Key] public string PresetId { get; set; }
/* Note: Both Types and Modes are Many-To-Many Entity Relationship configured
* using Join Table/Fluent API */
public ICollection<Type> Types { get; set; } = new List<Type>();
public ICollection<Mode> Modes { get; set; } = new List<Mode>();
}
public class Plugin
{
[Key] public int Id { get; set; }
public ICollection<Preset> Presets { get; set; } = new List<Preset>();
}
public void test ()
{
context.Plugins
.Include("Presets.Modes")
.Include("Presets.Types")
.Load();
}
// EntityCommandCompilationException An error occurred while preparing the command definition. See the inner exception for details.
// innerException System.NotSupportedException: APPLY joins are not supported
This is the basic code which fails. See below for the complete code and how the models are implemented.
In contrast, including just one of the two ICollection navigation properties works fine:
context.Plugins
.Include("Presets.Modes")
.Load();
Also, including more than one ICollection on the same level works fine:
context.Presets
.Include("Modes")
.Include("Types")
.Load();
Further Information, Environment
The problem only affects EntityFramework6
together with System.Data.SQLite.EF6. As far as I know EntityFramework.SqlServerCompact is not affected.
I am using EF6 Code-First, I have not tried EF6 Model-First or Database-First (and switching to either of those is not an applicable solution)
The problem only occurs with many-to-many collection navigation properties, and only if I attempt to load multiple ones on the same level
I have tried common alternatives, including Select(), which yields the same exception
I have not tried building manual joins or queries, as this would result in extreme code maintenance loss in my project
At this point, I believe the root cause is a limitation of the SQLite EF6 implementation (which doesn't seem to be documented) rather than something I could fix easily with code
I've only found one similar StackOverflow question regarding multiple .Include(), and the (awful to use) SQLite bugtracker as well as the mailing lists don't seem to mention this particular issue
The full source code, including a test database with test data can be found in my GitHub EntityFrameworkResearch repository. Specifically the Model definitions, the DbContext and the repro code
Code Reference
public class Mode
{
[Key] public int Id { get; set; }
public ICollection<Plugin> Plugins { get; set; }
public ICollection<Preset> Presets { get; set; }
public string Name { get; set; }
}
public class Type
{
[Key] public int Id { get; set; }
public ICollection<Plugin> Plugins { get; set; } = new List<Plugin>();
public ICollection<Preset> Presets { get; set; } = new List<Preset>();
public string Name { get; set; }
public string SubTypeName { get; set; }
}
public class Plugin
{
[Key] public int Id { get; set; }
public ICollection<Preset> Presets { get; set; } = new List<Preset>();
public ICollection<Type> DefaultTypes { get; set; } = new List<Type>();
public ICollection<Mode> DefaultModes { get; set; } = new List<Mode>();
}
public class Preset
{
[Key] public string PresetId { get; set; } = Guid.NewGuid().ToString();
[ForeignKey("Plugin")] public int PluginId { get; set; }
public Plugin Plugin { get; set; }
public ICollection<Type> Types { get; set; } = new List<Type>();
public ICollection<Mode> Modes { get; set; } = new List<Mode>();
}
public class ApplicationDatabaseContext : DbContext
{
// Constructor and further database initialization omitted for clarity
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Plugin>()
.HasMany(p => p.DefaultModes)
.WithMany(q => q.Plugins)
.Map(mc => mc.MapLeftKey("PluginId").MapRightKey("ModeId").ToTable("PluginModes"));
modelBuilder.Entity<Plugin>()
.HasMany(p => p.DefaultTypes)
.WithMany(q => q.Plugins)
.Map(mc => mc.MapLeftKey("PluginId").MapRightKey("TypeId").ToTable("PluginTypes"));
modelBuilder.Entity<Preset>()
.HasMany(p => p.Types)
.WithMany(q => q.Presets)
.Map(mc =>mc.MapLeftKey("PresetId").MapRightKey("TypeId").ToTable("PresetTypes"));
modelBuilder.Entity<Preset>()
.HasMany(p => p.Modes)
.WithMany(q => q.Presets)
.Map(mc => mc.MapLeftKey("PresetId").MapRightKey("ModeId").ToTable("PresetModes"));
}
public DbSet<Plugin> Plugins { get; set; }
public DbSet<Preset> Presets { get; set; }
public DbSet<Mode> Modes { get; set; }
public DbSet<Type> Types { get; set; }
}

EF Core: Circular entity reference

I would say that it takes a lot of time to get to know ASP.NET Core to understand how to achieve things then previous versions with webforms, but I understand that ASP.NET Core is bigger and you are able to build more complex solutions.
I'm quite new to ASP.NET Core and I'm trying to understand EF Core and related data. I'm using https://learn.microsoft.com/en-us/aspnet/core/data/ef-mvc/intro to learn the basics and create my first ASP.NET Core application.
I have a Entity "Standard" that can have multiple Forms (Form entity). The entities share a couple of same properties so I've made them both inherit from a master class called MasterDocument. Previously called Document.
Standard:
namespace Skjemabasen.Models.Document
{
public class Standard : MasterDocument
{
[Display(Name = "Kategori")]
public virtual Category Category { get; set; }
[Display(Name = "Dokumenter")]
public ICollection<Form> Forms { get; set; }
}
}
Form:
public class Form : MasterDocument
{
public Format Format { get; set; }
public virtual Member Assignee { get; set; }
public String Status { get; set; }
[ForeignKey("Standard")]
public int StandardId { get; set; }
public Standard Standard { get; set; }
public ICollection<Customer.Subscription> Subscribers { get; set; }
}
MasterDocument:
namespace Skjemabasen.Models.Document
{
public class MasterDocument : IDocument
{
public int ID { get; set; }
[Required]
[Display(Name = "EStandard")]
[StringLength(50)]
public string EStandard { get; set; }
[Required]
[Column("Betegnelse")]
[Display(Name = "Betegnelse")]
[StringLength(60)]
public string Betegnelse { get; set; }
[Display(Name = "Kommentar")]
public string Comment { get; set; }
}
}
I understand that this can cause circular request or circular deletion so I inserted a DeleteBehavior.Restrict on Standard:
modelBuilder.Entity<Standard>()
.HasOne(d => d.Forms)
.WithMany()
.OnDelete(DeleteBehavior.Restrict);
My complete context class:
namespace Skjemabasen.Data
{
public class SkjemabasenContext : DbContext
{
public SkjemabasenContext(DbContextOptions<SkjemabasenContext> options) :base(options)
{
}
public DbSet<Member> Members { get; set; }
public DbSet<Category> Categories { get; set; }
public DbSet<Standard> Standards { get; set; }
public DbSet<Form> Forms { get; set; }
public DbSet<Customer> Customers { get; set; }
public DbSet<Revision> Revisions { get; set; }
public DbSet<Subscription> Subscriptions { get; set; }
public DbSet<MasterDocument> Documents { get; set; }
public IQueryable<Customer> CurrentCustomers
{
get { return Customers.Where(c => c.Inactive == false); }
}
public IQueryable<Customer> InActiveCustomers
{
get { return Customers.Where(c => c.Inactive == true); }
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Member>().ToTable("Member");
modelBuilder.Entity<Category>().ToTable("Category");
modelBuilder.Entity<Standard>().ToTable("Standard");
modelBuilder.Entity<Form>().ToTable("Form");
modelBuilder.Entity<Customer>().ToTable("Customer");
modelBuilder.Entity<Revision>().ToTable("Revision");
modelBuilder.Entity<Subscription>().ToTable("Subscription");
modelBuilder.Entity<MasterDocument>().ToTable("Document");
modelBuilder.Entity<Standard>()
.HasOne(d => d.Forms)
.WithMany()
.OnDelete(DeleteBehavior.Restrict);
}
}
}
When I try to run the application I get the error:
System.ArgumentException: 'The entity type
'System.Collections.Generic.ICollection`1[Skjemabasen.Models.Document.Form]'
provided for the argument 'clrType' must be a reference type.' Because
all Forms must have a parent Standard and both 'Standard' and 'Form'
inherits from MasterDocument, I understand that ASP.NET Core warns
about circular deletion, but I'm not sure how to achieve this. The
error says something about ICollection of 'Forms' not being a
reference type. Is something missing in 'Standard' related to the
relation between and 'Form'.
Based on https://learn.microsoft.com/en-us/aspnet/core/data/ef-mvc/intro I can't figure out what I'm missing here.

I'm assuming you don't actually want to have polymorphic entities by inheriting from MasterDocument. So, from what I see, you want Form and Standard to share the same properties of MasterDocument while MasterDocument being itself an Entity. If that's the case, just abstract away those properties to a base class:
public abstract class MasterBaseDocument
{
public int ID { get; set; }
[Required]
[Display(Name = "EStandard")]
[StringLength(50)]
public string EStandard { get; set; }
[Required]
[Column("Betegnelse")]
[Display(Name = "Betegnelse")]
[StringLength(60)]
public string Betegnelse { get; set; }
[Display(Name = "Kommentar")]
public string Comment { get; set; }
}
public class Form : MasterBaseDocument
{
...
}
public class Standard : MasterBaseDocument
{
...
}
public class MasterDocument : MasterBaseDocument
{
// right now, empty here...
}
That should fix it.
Another approach to your model would be to have a MasterDocument FK on Forms and Standard. That way you don't get the duplicates fields on the tables.
Further improving: Also, keep in mind that you can achieve all those configurations you have using attributes with FluentAPI. This way your classes are keep and decouple from EF stuff. That just adds noise and makes it very hard to read. Should be examples on Fluent API on EF docs as well.

Entity Framework - Invalid column name 'CourseLesson_Id'

After trying to execute the following query:
List<CourseLesson> courseLessons = (from cl in context.CourseLessons
.Include(x => x.CourseLessonTestQuestions)
select cl).ToList();
I get the the error Invalid column name 'CourseLesson_Id'.
My models and DataContext looks like this(this is from a test project I've created to repreduce the problem)
public class CourseLesson
{
public int Id { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public ICollection<CourseLessonTestQuestion> CourseLessonTestQuestions { get; set; }
}
public class CourseLessonTestQuestion
{
public int Id { get; set; }
public int CourseLessonId { get; set; }
[ForeignKey(nameof(CourseLessonId))]
public CourseLesson CourseLesson { get; set; }
public int? ReturnCourseLessonId { get; set; }
[ForeignKey(nameof(ReturnCourseLessonId))]
public CourseLesson ReturnCourseLesson { get; set; }
}
I have 2 foreign keys that point to the same table and I'm assuming EF is trying to create or map something that doesn't really exist.
After reading for a while I've found a way to fix my problem in (this answer) with the following code:
modelBuilder.Entity<CourseLessonTestQuestion>()
.HasOptional(cltq => cltq.ReturnCourseLesson)
.WithMany(x => x.CourseLessonTestQuestions);
What really bugs me about this situation is why everything works when I use the Fluent API, but it doesn't work with the ForeignKey attribute? This looks like something that could lead to future problems and I want to know what is really happening.
And the real question is there a solution for fixing this problem without the Fluent API? Like using attributes or some other convention?
I'm using Entity Framework 6.1.3

Solution without Fluent API, but with the help of InversePropertyAttribute, whose constructor's argument is the name of corresponding CourseLessonTestQuestion's property:
public class CourseLesson
{
public int Id { get; set; }
public string Title { get; set; }
public string Content { get; set; }
[InverseProperty("CourseLesson")]
public ICollection<CourseLessonTestQuestion> CourseLessonTestQuestions { get; set; }
[InverseProperty("ReturnCourseLesson")]
public ICollection<CourseLessonTestQuestion> ReturnCourseLessons { get; set; }
}

Data annotations not create one-to-many object references

I'm missing something when using the data annotations.
This is my first class
[Table("PriceFeed")]
public class PriceFeed : History
{
public PriceFeed()
{
this.Votes = new List<PriceVote>();
this.History = new List<PriceFeed__History>();
}
[Key]
public long Id { get; set; }
[ForeignKey("Store")]
public long Store_Id { get; set; }
[ForeignKey("Item")]
public long Item_Id { get; set; }
[Required]
public decimal Price { get; set; }
public Store Store { get; set; }
public Item Item { get; set; }
public virtual ICollection<PriceFeed__History> History { get; set; }
}
And this is my second class
[Table("PriceFeed__History")]
public class PriceFeed__History : History
{
[Key]
public long Id { get; set; }
[ForeignKey("PriceFeed")]
public long PriceFeed_Id { get; set; }
[Required]
public decimal Price { get; set; }
public virtual PriceFeed PriceFeed { get; set; }
}
When I run the add-migration, it creates the database correctly but when I try to access PriceFeed.History it gives me an error
{"Message":"An error has occurred.","ExceptionMessage":"A specified Include path is not valid. The EntityType 'Verdinhas.Web.Contexts.PriceFeed' does not declare a navigation property with the name 'PriceFeed__History'."
I always worked with API Fluent and typed by myself the code like
.Entity<Student>()
.HasRequired<Standard>(s => s.Standard)
.WithMany(s => s.Students)
.HasForeignKey(s => s.StdId);
But now I'm using the data annotations and when I generate the migration, it does not create the "withmany" like the above.
What am I doing wrong?

The issue has nothing to do with Data Annotations which seems to be correct in your model.
As mentioned in the comments, the exception is caused by a code that tries to use Include method with string "'PriceFeed__History" - you seem to think that you should specify the related entity types, but in fact you need to specify the navigation property names, which in your case is "History".

Loading Related Data with Entity Framework Core

I am creating a new project with ASP.NET Core & Entity Framework Core. (I think this is entity framework 7?)
I think MS have removed Lazy Loading, so my virtual collections are not loading automatically. So then I decided to try ".include" however this method was missing. So I included "System.Data.Entity" and now I have .include but it still loads the collections as null. The entites definitely have related data with in the database.
My Code is as follows:
//Property Table
public class Property
{
public int Id { get; set; }
public string PropertyName { get; set; }
public string Address1 { get; set; }
public string Address2 { get; set; }
public string City { get; set; }
public string Postcode { get; set; }
public string State { get; set; }
public string Country { get; set; }
public string PhoneNumber { get; set; }
public virtual ICollection<BodyCorpMember> BodyCorpMembers { get; set; }
public bool Deleted { get; set; }
}
//Body Corp Table
public class BodyCorpMember
{
public int Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string Email { get; set; }
public bool IsSecretary { get; set; }
public int Property_Id { get; set; }
[ForeignKey("Property_Id")]
public virtual Property Property { get; set; }
}
This is some of the things I tried in OnModelCreating with no success
//builder.Entity()
// .HasOne(a => a.Property)
// .WithMany(a => a.BodyCorpMembers)
// .HasForeignKey("Property_Id");
//builder.Entity<Property>()
// .HasMany(a => a.BodyCorpMembers)
// .WithOne(a => a.Property);
//Here is my main problem, Ive tried multiple options for ".include" but it always returns null for BodyCorpMembers
property = _dbContext.Properties.Include(a=>a.BodyCorpMembers).Where(a=>a.Id ==id).FirstOrDefault();
it should also be noted that I am using the built in IdentityDbContext as my Db Context. ie:
public class ApplicationDbContext : IdentityDbContext<ApplicationUser>

I didn't figure out why it wasn't working, but I just upgraded EF Core to 1.1 and used explicit loading.
ie:
property = _dbContext.Properties.Single(a=>a.Id == id);
_dbContext.Entry(property).Collection(a => a.BodyCorpMembers).Load();
This works fine.
I do hope MS brings back Lazy loading however.
update
After updating to Entity framework core 1.1 .include started working also.