We Keep Coding

EntityFramework core - Update a collection of data without selecting the entities

How would you Upsert without select? the upsert would be a collection of entities received by a method which contains DTOs that may not be available in the database so you can NOT use attach range for example.
One way theoretically is to load the ExistingData partially with a select like dbContext.People.Where(x => x exists in requested collection).Select(x => new Person { Id = x.Id, State = x.State }).ToList() which just loads a part of the entity and not the heavy parts. But here if you update one of these returned entityItems from this collection it will not update because of the new Person its not tracking it and you also cannot say dbContext.Entry<Person>(person).State = Modified because it will throw an error and will tell you that ef core is already "Tracking" it.
So what to do.
One way would be to detach all of them from the ChangeTracker and then do the state change and it will do the update but not just on one field even if you say dbContext.Entry<Person>(person).Property(x => x.State).Modified = true. It will overwrite every fields that you haven't read from the database to their default value and it will make a mess in the database.
The other way would be to read the ChangeTracker entries and update them but it will also overwrite and it will consider like everything is chanaged.
So techinically I don't know how ef core can create the following SQL,
update People set state = 'Approved' where state != 'Approved'
without updating anything else. or loading the person first completely.
The reason for not loading your data is that you may want to update like 14000 records and those records are really heavy to load because they contain byte[] and have images stored on them for example.
BTW the lack of friendly documentation on EFCore is a disaster compare to Laravel. Recently it has cost us the loss of a huge amount of data.
btw, the examples like the code below will NOT work for us because they are updating one field which they know that it exists in database. But we are trying to upsert a collection which some of those DTOs may not be available in the database.
try
{
using (var db = new dbContext())
{
// Create new stub with correct id and attach to context.
var entity = new myEntity { PageID = pageid };
db.Pages.Attach(entity);
// Now the entity is being tracked by EF, update required properties.
entity.Title = "new title";
entity.Url = "new-url";
// EF knows only to update the propeties specified above.
db.SaveChanges();
}
}
catch (DataException)
{
// process exception
}
Edit: The used ef core version is #3.1.9

Fantastic, I found the solution (You need to also take care about your unit tests).
Entityframework is actually working fine it can be just a lack of experience which I'm documenting here in case anyone else got into the same issue.
Consider that we have an entity for Person which has a profile picture saved as Blob on it which causes that if you do something like the following for let's say 20k people the query goes slow even when you've tried to have enough correct index on your table.
You want to do this query to update these entities based on a request.
var entityIdsToUpdate = request.PeopleDtos.Select(p => p.Id);
var people = dbContext.People.Where(x => entityIdsToUpdate.Contains(x.Id)).ToList();
This is fine and it works perfectly, you will get the People collection and then you can update them based on the given data.
In these kind of updates you normally will not need to update images even if you do, then you need to increase the `TimeOut1 property on your client but for our case we did not need to update the images.
So the above code will change to this.
var entityIdsToUpdate = request.PeopleDtos.Select(p => p.Id);
var people = dbContext.People
.Select(p => new Person {
Id = p.Id,
Firstname = p.Firstname,
Lastname = p.Lastname,
//But no images to load
})
.Where(p => entityIdsToUpdate.Contains(p.Id)).ToList();
But then with this approach, EntityFramework will lose the track of your entities.
So you need to attach it like this and I will tell you how NOT to attach it.
This is the correct way for a collection
dbContext.People.AttachRange(people); //These are the people you've already queried
Now DO NOT do this, you may want to do this because you get an error from the first one from EntityFramework which says the entity is already being tracked, trust it because it already is. I will explain after the code.
//Do not do this
foreach(var entry in dbContext.ChangeTracker.Entries())
{
entry.State = EntityState.Detached;
}
//and then on updating a record you may write the following to attach it back
dbContext.Entry(Person).State = EntityState.Modified;
The above code will cause EntityFramework not to follow the changes on the entities anymore and by the last line you will tell it literally everything edited or not edited is changed and will cause you to LOSE your unedited properties like the "image".
Note: Now what can u do by mistake that even messes up the correct approach.
Well since you are not loading your whole entity, you may assume that it is still fine to assign values to the unloaded ones even if the value is not different than the one in the database. This causes entity framework to assume that something is changed and if you are setting a ModifiedOn on your records it will change it for no good reason.
And now about testing:
While you test, you may get something out from database and create a dto from that and pass the dto with the same dbContext to your SystemUnderTest the attach method will throw an error here which says this entity is already bein tracked because of that call in your test method. The best way would be create a new dbContext for each process and dispose them after you are done with them.
BTW in testing it may happen that with the same dbContext you update an entity and after the test you want to fetch if from the database. Please take note that this one which is returning to you is the "Cached" one by EntityFramework and if you have fetched it in the first place not completely like just with Select(x => ) then you will get some fields as null or default value.
In this case you should do DbContext.Entry(YOUR_ENTRY).Reload().
It is a really complete answer it may not directly be related to the question but all of the things mentioned above if you don't notice them may cause a disaster.

Does EF automatically load many to many references collections

Imagine we have the following db structure
Organization
{
Guid OrganizationId
//....
}
User
{
Guid UserId
}
OrganizationUsers
{
Guid OrganizationId
Guid UserId
}
When the edmx generated this class it abstracts away the OrganizationUsers into a many to many references. So no POCO class will be generated for it.
Say I'm loading data from my context, but to avoid Cartesian Production, I don't use an include I make two seperate queries.
using(var context = new EntitiesContext())
{
var organizationsQuery = context.Where(FilterByParent);
var organizations = organizationsQuery.ToList();
var users = organizationsQuery.SelectMany(x => x.Users).Load();
}
Is it safe to assume that the connected entitites are loaded?
Would this make any difference if I loaded the users directly from the DBSet?

From database point of view:
Is it safe to assume that the connected entitites are loaded?
Yes It's safe, because first organizations being tracked by EF Change Tracker and then by calling Load in next statement EF knows that results should be attach to tracked entities
Would this make any difference if I loaded the users directly from the DBSet?
In fact using Load this way does nothing better than Include!
If you use Include EF translate it to LEFT JOIN, if you use Load it will be translated to INNER JOIN, and if you fetch Users directly by their ids using Contains method it will be translated to IN on Sql side.
In Load and Contains cases you execute two query (in two pass) on Sql, but in Include case it's being done in one pass, so overally it's outperform your approach.
You can compare these approaches yourself using Sql Profiler tool.
Update:
Based on conversations I realized that the main issue of Johnny is just existence of OrganizationUsers object. So I suggest to change your approach from DB First to Code first then this object explicitly could be exist! See this to help you on this way
Also another approach that I guess maybe work is customizing T4 Template that seems harder but not impossible!

Load a single related object

I need to load a single related object from an navigation property (ICollection) to send to my MVC View.
This can save me from unnecessary db access and load.
I've found this article about loading related objects but didn't figure out how to load one single related object from the list.
To be short, I need the object and inside its navigation property one single related object.
How to achive this?

You can't do this using Include. Include will bring back all related entities for the navigation property. You can either write two separate queries or write a join in your query.
Writing two queries:
var princess = context.Princesses.Find(id);
var unicorns = context.Unicorns.Where(u => u.PrincessId == id && u.UnicornName == "Blinky");
princess.Unicorns = unicorns.ToList();

Doing a Count in a Linq to SQL Query (created as IQueryable) gets all the rows

Class Customer has the following method, which returns an IQueryable with the linq query to get the active sales for a customer:
public IQueryable<SalesHeader> QueryCurrentSales()
{
// this.SalesHeaders is an association defined in the DBML between
// the Customer and SalesHeader tables (SalesHeader.CustomerId <-> Customer.Id).
return this.SalesHeaders
.Where(sh => sh.Status == 1).AsQueryable();
}
The idea is to centralise the query definition in a single point, so that later we can get the SalesHeader rows, do a count, paginate using the PaginatedList class (from NerdsDinner), and add further Where conditions when searching Sales within active SalesHeaders.
However, after running the SQL Sever profiler, I've discovered that doing the following, gets all the rows and then does the Count in memory.
// cust is an object of type Customer.
int rows = cust.QueryCurrentSales().count();
I guess that the problem lies in the fact that we have used the AsQueryable() method, but I don't know what would be the correct way to accomplish what we intended.

You haven't shown what SalesHeaders is. The fact that you feel you need to do AsQueryable suggests that it's returning an IEnumerable<T> rather than an IQueryable<T>... which is probably the problem. If you could show SalesHeaders, we may be able to help.
It's possible that instead of using AsQueryable you may be able to just cast to IQueryable<SalesHaeder> if the SalesHeaders property is returning an expression which could actually be returned as IQueryable<SalesHeaders> in the first place - but in that case I'd suggest changing the property type instead.
EDIT: Okay, if it's an entity set then that's probably tricky to fix directly.
I know it's ugly, but how about something like:
public IQueryable<SalesHeader> QueryCurrentSales()
{
// You *may* be able to get away with this in the query; I'm not sure
// what LINQ to SQL would do with it.
int id = this.Id;
return context.SalesHeaders
.Where(sh => sh.Status == 1 && sh.CustomerId == id);
}
Here I'm assuming you have some way of getting to the data context in question - I can't remember offhand whether there's an easy way of doing this in LINQ to SQL, but I'd expect so.

This works right. Doing count() on SQL Server.
// cust is an object of type Customer.
int rows = cust.QueryCurrentSales().AsQueryable<SalesHeaders>().count();

Entity Framework: Adding a scalar property which is equal to a FK id

Short Question:
Basically I am trying to add a scalar property to my entity which holds the ID of a FK entity.
What I have tried to do thus far:
What I have tried so far is adding the scalar property (called ChildId) and mapped it to the matching column in the database. Now as you can imagine I get some exceptions when I try and do this because entity framework complains that the FK id is being managed in two places, once through x.ChildId and the other through x.Child.ChildId.
Now I get why it is doing this but I need some why to be able to have a scalar property which is automatically populated with the ChildId.
What I know I could do but really don't want to:
I realize that I could write a linq query that does something like the following (where I have implemented the other half of the partial class and added a property there called ChildId):
from x in db.Parent
select new Parent { ParentName = x.ParentName, ..., ChildId = x.Child.ChildId }
But this is extremely messy, particular when I have 30 odd queries that return a parent object, this mapping would need to be repeated for each query...
Also I realize that after I have executed the query I could go something like:
var childId = parent.Child.Id;
But this would cause either an extra query to be triggered, or if I was proactively loading child, and in either case I would be pulling out a lot more data than I need when I only want the ID...
The required end result:
So how do I get around some of these limitations so that I can write my queries like so (or something very similar):
from x in db.Parent
select x
And have it so that I can either go:
var childId = parent.Child.Id; //Where in this case the only property retrieved would be the Id
//Or
var childId = parent.ChildId;
Cheers
Anthony
EDIT:
Hey thanks for the reply...
I just figured this out for myself as well. Basically I was thinking that if EF supports lazy loading it must be storing the ID somewhere. Then it clicked that it must be in the reference... Hence for me it worked out being something like:
destination.PlanTypeId = (int)source.PlanTypeReference.EntityKey.EntityKeyValues[0].Value;
Also thanks for the idea of creating the extension property... will be very useful.

Did you try using parent.ChildReference.EntityKey? It doesn't need to be additionaly loaded and is holding FK. You can write extension method to get key easier.