We Keep Coding

EF Core - Producing unfiltered queries on all members of dbcontext

I'm writing this here, in the hope that someone has experienced this before and has a thorough answer to this problem.
We have a C# Web API that handles CRUD operations where we are using EF Core as our ORM to manipulate data.
From inspecting the SQL logs we can see that there are many select statements that are unfiltered (no where clause). That don't seem to be coming from the code base.
What is strange about this is some of the tables that are being called have no other references in the code other than being on the DBContext itself.
For example.
public class MyContext : DbContext{
public DbSet<MyDbSet> MyDBSet { get; set; }
}
This seems to periodically output a select statement on the table even if its not used in the code.
SELECT [m].[Id], [m].[Column1], [m].[Column2]
FROM [MyDBSet] AS [m]
The problem with this is, that some of these tables have large amounts of data in which I would like to avoid an unfiltered query.
From further inspection it seems that the order in which these unfiltered queries occur is the same order they are found on the DBContext.
Example 2
public class MyContext : DbContext{
public DbSet<MyDbSet> MyDBSet { get; set; }
public DbSet<MyDbSet2> MyDBSet2 { get; set; }
public DbSet<MyDbSet3> MyDBSet3 { get; set; }
}
SELECT [m].[Id], [m].[Column1], [m].[Column2]
FROM [MyDBSet] AS [m]
SELECT [m].[Id], [m].[Column1], [m].[Column2]
FROM [MyDBSet2] AS [m]
SELECT [m].[Id], [m].[Column1], [m].[Column2]
FROM [MyDBSet3] AS [m]
This happens even when the table is not read or referenced in the codebase, and there are no references to these tables that don't have a where clause.
My first assumption is that EF core is taking a snapshot of the table? Perhaps query caching or something similar?
I guess my question is, does anyone know what could be causing such odd behaviour and or is there something in EF core that is responsible.
NOTE - We are not using Lazy Loading Proxies package and are using eager loading throughout the application

After scrutinising a significant memory dump of the system - The package that was causing the reported unfiltered queries was Serilog.
https://github.com/RehanSaeed/Serilog.Exceptions
From here you will find the following details around versions of Serilog exceptions and it appears that being on below version 8 in conjunction with EF core, caused Serilog to try and destructure values that implemented IQueryable - in newer versions this does not happen.
WARNING: In versions of Serilog.Exceptions older than 8.0.0, if you are using Entity Framework Core with Serilog.Exceptions you must add this, otherwise in certain cases your entire database will be logged! This is because the exceptions in Entity Framework Core have properties that link to the entire database schema in them (See #100, aspnet/EntityFrameworkCore#15214). Newer versions of Serilog.Exceptions avoids this issue by preventing the destructure of properties that implement IQueryable preventing their execution.

Issue with ASP.NET MVC 5 Web Application Entity Framework

I am working on MIT open source license example ASP.NET MVC web applications, and adding them as github public repos, I am also planning to have private github repos for my applications I plan to make money with in the future. I have a developer account with github.com.
I created a BOOKS MVC 5 web application using a TSQL script I was provided during a previous job interview some years ago, and am using GUID for the primary key ID fields with a default value of NEWID(), instead of an INT with IDENTITY, the solution is an ASP.NET MVC 5 web application with database first Entity Framework. I am using LocalDB for my SQL Server with this project, the script to create and populate the database is in my SQL-Server repo and is called BOOKS_Create.sql
I published the solution to my GitHub at the following URL:
https://github.com/ABION-Technology/Books
The TSQL scripts are available in the following repo:
https://github.com/ABION-Technology/SQL-Server
I added links the the shared layout view to show the index view for all Authors in the database, and also links to Index views for the TITLE and SALE EF models.
THe AUTHORS link works just fine, and lists all the authors in the database. But when I click the TITLES link, I get a SQL Exception of 'Author_ID' invalid column name, I did a search through my entire solution and did not find any variable named 'Author_ID' and did not find a C# class with a property called 'Author_ID". So this issue has me very confused there does not appear to be a good way to debug this issue. Thanks.

EF will follow some default conventions to work out FK relationships. The error you are seeing is due to Author having a Titles collection and EF is attempting to automatically set up the 1-to-many between the two. It's expecting to find an "Author_ID" on the Title, which doesn't exist because your schema is set up with a joining table called TitleAuthor.
To resolve this, you will need to map the TitleAuthor entity, in which the Author will contain a collection of TitleAuthors which refer themselves to an Author and Title entity. EF can automatically map joining tables given those tables consist of just the two FKs. As soon as you want to introduce additional fields, then you need to define the joining entity.
public class TitleAuthor
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int ID { get; internal set;}
public virtual Title Title { get; internal set;}
public virtual Author Author { get; internal set;}
// add other properties as needed..
}
So from your Author entity:
public virtual ICollection<TitleAuthor> Authors {get; internal set;} = new List<TitleAuthor>();
To access the titles for the author:
author.Titles.Select(x => x.Title);
I would recommend reading up on many-to-many mapping with EF. I invariably use deliberate mapping with EF rather than relying on it's conventions. It just helps make it more predictable.
If you are using defaults for PKs then you need to tell EF via the DatabaseGenerated attribute. This isn't needed for read operations, but it will be needed when you go to insert records.
Also, with SQL Server, consider using NewSequentialId() as the default for your UUID PKs. These are more index-friendly than NewId().
The above example using internal (private works too) setters to promote DDD style use of entities. Public setters can lead to misuse/abuse of entities in the sense that the context will diligently attempt to save whatever you set. It's generally a good idea to restrict functionality that would alter an entity's state to a method in the entity with required arguments to be validated, or a repository. I use internal scoping to allow unit tests to still initialize entities. (leveraging InternalsVisibleTo between domain and unit test assemblies)

Reason is you are getting that Author ID error is, you have list of Titles in Author Class. Then there should be relationship between Author and Title entities, which is not exists in your data context. Comment public virtual ICollection<Title> Titles { get; set; } . And it should work.
Reason for you cant search this attribute is, its automatically generated by entity framework. (TableName_PrimaryKey)
If you want to keep this, create relationship in database using foreign keys and add that to your data context also. You may refer this

Dynamic Property Binding in EF?

Apparently, the same columns value-type differs across environments for the same database entity (table) & they refuse to update to a common type - don't ask why!
I am using Entity Framework (version 6.1.3) alongside a Unit of Work for data-access. And, as you can guess, I am getting errors because the DEV & the QA database definitions do not match for the same column.
THE GOOD NEWS:
We do not save into these particular tables - we only query those particular tables.
SAMPLE MODEL:
There are obviously more columns than this.
public partial class Transactions
{
[Key]
public int TransactionId { get; set; }
public float Amount { get; set; } //<-- This type differs between database environments
}
MY QUESTION:
Is there a way to dynamically bind the value for a column in Entity Framework?
Or, can I treat it as a dynamic under-the-hood...and transform it to an expected type which is constant to my model?
OPTIMALLY - AND TO BE CLEAR:
I would like to define the property concretely, and have Entity Framework "convert" from the unknown type & into the concrete type - but under-the-hood.
Any help is appreciated.

If the types of the columns are compatible (i.e. are all numbers) you can have a common type on the class then disable model checking (and migration). This solution could work on some DBMSs and not on some Others (depends on the provider).
You can write a view with cast and map it in your model (and not the table).
You can write a direct query with EF (official docs https://msdn.microsoft.com/en-us/data/jj592907.aspx similar to Dapper solution with same advantages and defects, in your case lazy load won't work).

Entity Framework native sql mapping to class

I got a code first EF and I want to use native sql for the more complex select statements.
When I try to execute:
using (VaultsDbContext db = new VaultsDbContext())
{
var contracts = db.Contracts.SqlQuery("select * from Contracts").ToList<Contract>();
}
I got:
Cannot create a value for property 'MetaProps' of type
'DskVault.Models.DbModels.MetaProps'. Only properties of primitive or
enumeration types are supported.
MetaProps is a class that holds deleteflag, creator etc. and it's a property of all my classes. It's not mapped to a different table, every table has deleteflag, createor, etc.
public class Contract
{
public long Id { get; set; }
...
public MetaProps MetaProps { get; set; }
}
Is there a way to map from the native sql to the class if the class contains a complex type or does EF not support that? Also what if the complex type is entity mapped to another table(join)?
Edit:
Version: Entity Framework 6

I know from experience not all the fields in your table have to be contained in your model. This is a good thing when it comes to installing updates into production.
Have you tried reverse engineering your tables on a SEPARATE temporary project using the Entity Framework Power tools? This is a Nuget package that I have found to be extremely useful in code first programming. Reverse engineering will overwrite existing files, so make sure not to do this on your live code.

Nhibernate updating version column on foreign key entity

I am using c# with Fluent NHibernate and auto mapping.
Here is some code (truncated for clarity), then I'll explain the problem.
public class Company
{
public virtual string Description { get; set; }
}
public class Stock
{
public virtual Product Product { get; set; }
public virtual Company Company { get; set; }
}
Mapping
mappings.Conventions.Add<CascadeConvention>()
.Conventions.Add<CustomForeignKeyConvention>()
.Conventions.Add<HasManyConvention>()
.Conventions.Add<VersionConvention>()
CascadeConvention just sets everything to All.
CustomForeignKeyConvention removes the _id that NHibernate usually
appends to foreign key id columns.
HasManyConvention sets all HasMany's to inverse.
VersionConvention convertion looks like this:
instance.Column("Version");
instance.Default(1);
The problem is that when I insert a new stock record, Nhibernate also updates the version number on the related Company.
If I had an IList<Stock> property on the Company then that would make sense but I don't.
I've done a lot of reading around:
NHibernate emitting extraneous update statements regardless of proper Inverse (fluent nhibernate) settings on relations
Cascade Save-Update in NHibernate and version column
NHibernate Reference - Chapter 17. Example: Parent/Child
Ayende # Rahien - NHibernate Mapping
From these, I've tried a whole bunch of things including adding .Not.OptimisticLock() all over the place. I even added an IList<Stock> property on Company so that I could specifically set it as Inverse, Not.OptimisticLock, etc. Nothing I do seems to make any difference.

We eventually sorted this by moving to a Session-per-request paradigm. Not sure why it was going wrong or why this fixed it. I wrote numerous unit tests to try and reproduce the behaviour in a more controlled environment without success.
In any case, it works now. There are good reasons session-per-request is often given as the best practice way to manage NHibernate sessions in a web application.