We Keep Coding

Entity Framework Core - efficient way to update Entity that has children based on JSON representation of entity being passed in via Web API

I am writing an .NET Core Web API which is backed by Entity Framework Core and having a PostgreSQL database underneath (AWS Aurora). I have been able to learn and successfully work with EF Core for inserts and querying data, which is great, but starting to look into UPDATES of existing entities and it's become unclear to me the most efficient way to achieve what I'm after. I have my Database Entities as a result of my Database First scaffolding exercise. I have an entity that is similar to the below (simplified).
Customer -< Addresses
-< ContactInformation
So a Customer, which may have multiple Addresses and / or multiple ContactInformation records. I am expecting my Web API to pass in a JSON payload that can be converted to a Customer with all associated information. I have a method which converts my CustomerPayload to a Customer that can be added to the database:
public class CustomerPayload : Payload, ITransform<CustomerPayload, Customer>
{
[JsonProperty("customer")]
public RequestCustomer RequestCustomer { get; set; }
public Customer Convert(CustomerPayload source)
{
Console.WriteLine("Creating new customer");
Customer customer = new Customer
{
McaId = source.RequestCustomer.Identification.MembershipNumber,
BusinessPartnerId = source.RequestCustomer.Identification.BusinessPartnerId,
LoyaltyDbId = source.RequestCustomer.Identification.LoyaltyDbId,
Title = source.RequestCustomer.Name.Title,
FirstName = source.RequestCustomer.Name.FirstName,
LastName = source.RequestCustomer.Name.Surname,
Gender = source.RequestCustomer.Gender,
DateOfBirth = source.RequestCustomer.DateOfBirth,
CustomerType = source.RequestCustomer.CustomerType,
HomeStoreId = source.RequestCustomer.HomeStoreId,
HomeStoreUpdated = source.RequestCustomer.HomeStoreUpdated,
StoreJoined = source.RequestCustomer.StoreJoinedId,
CreatedDate = DateTime.UtcNow,
UpdatedDate = DateTime.UtcNow,
UpdatedBy = Functions.DbUser
};
Console.WriteLine("Creating address");
if (source.RequestCustomer.Address != null)
{
customer.Address.Add(new Address
{
AddressType = "Home",
AddressLine1 = source.RequestCustomer.Address.AddressLine1,
AddressLine2 = source.RequestCustomer.Address.AddressLine2,
Suburb = source.RequestCustomer.Address.Suburb,
Postcode = source.RequestCustomer.Address.Postcode,
Region = source.RequestCustomer.Address.State,
Country = source.RequestCustomer.Address.Country,
CreatedDate = DateTime.UtcNow,
UpdatedDate = DateTime.UtcNow,
UpdatedBy = Functions.DbUser,
UpdatingStore = null, // Not passed by API at present
AddressValidated = false, // Not passed by API
AddressUndeliverable = false, // Not passed by API
});
}
Console.WriteLine("Creating marketing preferences");
if (source.RequestCustomer.MarketingPreferences != null)
{
customer.MarketingPreferences = source.RequestCustomer.MarketingPreferences
.Select(x => new MarketingPreferences()
{
ChannelId = x.Channel,
OptIn = x.OptIn,
ValidFromDate = x.ValidFromDate,
UpdatedBy = Functions.DbUser,
CreatedDate = DateTime.UtcNow,
UpdatedDate = DateTime.UtcNow,
ContentTypePreferences = (from c in x.ContentTypePreferences
where x.ContentTypePreferences != null
select new ContentTypePreferences
{
TypeId = c.Type,
OptIn = c.OptIn,
ValidFromDate = c.ValidFromDate,
ChannelId = x.Channel // Should inherit parent marketing preference channel
}).ToList(),
UpdatingStore = null // Not passed by API
})
.ToList();
}
Console.WriteLine("Creating contact information");
if (source.RequestCustomer.ContactInformation != null)
{
// Validate email if present
var emails = (from e in source.RequestCustomer.ContactInformation
where e.ContactType.ToUpper() == ContactInformation.ContactTypes.Email && e.ContactValue != null
select e.ContactValue);
if (!emails.Any()) throw new Exception("At least 1 email address must be provided for a customer registration.");
foreach (var email in emails)
{
Console.WriteLine($"Validating email {email}");
if (!IsValidEmail(email))
{
throw new Exception($"Email address {email} is not valid.");
}
}
customer.ContactInformation = source.RequestCustomer.ContactInformation
.Select(x => new ContactInformation()
{
ContactType = x.ContactType,
ContactValue = x.ContactValue,
CreatedDate = DateTime.UtcNow,
UpdatedBy = Functions.DbUser,
UpdatedDate = DateTime.UtcNow,
Validated = x.Validated,
UpdatingStore = x.UpdatingStore
})
.ToList();
}
else
{
throw new Exception("Minimum required elements not present in POST request");
}
Console.WriteLine("Creating external cards");
if (source.RequestCustomer.ExternalCards != null)
{
customer.ExternalCards = source.RequestCustomer.ExternalCards
.Select(x => new ExternalCards()
{
CardNumber = x.CardNumber,
CardStatus = x.Status,
CardDesign = x.CardDesign,
CardType = x.CardType,
UpdatingStore = x.UpdatingStore,
UpdatedBy = Functions.DbUser
})
.ToList();
}
Console.WriteLine($"Converted customer object --> {JsonConvert.SerializeObject(customer)}");
return customer;
}
I would like to be able to look up an existing customer by McaId (which is fine, I can do that via)
var customer = await loyalty.Customer
.Include(c => c.ContactInformation)
.Include(c => c.Address)
.Include(c => c.MarketingPreferences)
.Include(c => c.ContentTypePreferences)
.Include(c => c.ExternalCards)
.Where(c => c.McaId == updateCustomer.McaId).FirstAsync();
But then be able to neatly update that Customer and associated tables with any different values for any properties contained in the Customer - OR - its related entities. So, in pseudo code:
CustomerPayload (cust: 1234) Comes in.
Convert CustomerPayload to Customer(1234)
Get Customer(1234) current entity and related data from Database.
Check changed values for any properties of Customer(1234) compared to Customer(1234) that's come in.
Generate the update statement:
UPDATE Customer(1234)
Set thing = value, thing = value, thing = value.
UPDATE Address where Customer = Customer(1234)
Set thing = value
Save to Database.
Can anyone help as to the best way to achieve this?
EDIT: Updated with attempt. Code below:
public static async void UpdateCustomerRecord(CustomerPayload customerPayload)
{
try
{
var updateCustomer = customerPayload.Convert(customerPayload);
using (var loyalty = new loyaltyContext())
{
var customer = await loyalty.Customer
.Include(c => c.ContactInformation)
.Include(c => c.Address)
.Include(c => c.MarketingPreferences)
.Include(c => c.ContentTypePreferences)
.Include(c => c.ExternalCards)
.Where(c => c.McaId == updateCustomer.McaId).FirstAsync();
loyalty.Entry(customer).CurrentValues.SetValues(updateCustomer);
await loyalty.SaveChangesAsync();
//TODO expand code to cover scenarios such as an additional address on an udpate
}
}
catch (ArgumentNullException e)
{
Console.WriteLine(e);
throw new CustomerNotFoundException();
}
}
All I've changed is the last name of the customer. No errors occur, however the record is not updated in the database. I have the following settings on so was expecting to see the generated SQL statements in my log, but no statements were generated:
Entity Framework Core 3.1.4 initialized 'loyaltyContext' using provider 'Npgsql.EntityFrameworkCore.PostgreSQL' with options: SensitiveDataLoggingEnabled
That's the only entry I have listed in my log.

You are working with graphs of disconnected entities. Here is the documentation section that might be of interest to you.
Example:
var existingCustomer = await loyalty.Customer
.Include(c => c.ContactInformation)
.Include(c => c.Address)
.Include(c => c.MarketingPreferences)
.Include(c => c.ContentTypePreferences)
.Include(c => c.ExternalCards)
.FirstOrDefault(c => c.McaId == customer.McaId);
if (existingCustomer == null)
{
// Customer does not exist, insert new one
loyalty.Add(customer);
}
else
{
// Customer exists, replace its property values
loyalty.Entry(existingCustomer).CurrentValues.SetValues(customer);
// Insert or update customer addresses
foreach (var address in customer.Address)
{
var existingAddress = existingCustomer.Address.FirstOrDefault(a => a.AddressId == address.AddressId);
if (existingAddress == null)
{
// Address does not exist, insert new one
existingCustomer.Address.Add(address);
}
else
{
// Address exists, replace its property values
loyalty.Entry(existingAddress).CurrentValues.SetValues(address);
}
}
// Remove addresses not present in the updated customer
foreach (var address in existingCustomer.Address)
{
if (!customer.Address.Any(a => a.AddressId == address.AddressId))
{
loyalty.Remove(address);
}
}
}
loyalty.SaveChanges();

LINQ Expression cannot be translated - C#

I wrote simple query (issues is when I try to set an Address below ProductCode):
var query = _connectDBContext.Products
.Join(_context.CustomerRelations,
Product => Product.ForeignKeyId,
CustomerRelation => CustomerRelation.CompanyId,
(Product, CustomerRelation) => new { Product, CustomerRelation })
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
//Address = new AddressDTO // THIS WORKS BUT LOOKS UGLY :(
//{
// Address = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Address,
// Country = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Country,
// Zip = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Zip,
//},
})
.Where(x => x.Id == x.CustomerRelationId);
// Rest of the code
}
private AddressDTO Map(Address address)
{
return new AddressDTO
{
Address = address.Address,
Country = address.Country,
Zip = address.Zip,
};
}
This code above breaks on this line:
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
It says that linq cannot be translated and advicing me to rewrite a query..
But this commented code here which does almost the same works, so if I remove calling Map method in Select and if I uncomment this code in Select everything would work, but I would like to get rid of this - writing too many OrderBy for each prop, I would like to Order it once and after that I would like to use it.. but unfortunatelly I don't know how.. :
//Address = new AddressDTO
//{
// Address = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Address,
// Country = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Country,
// Zip = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Zip,
//},
Thanks in advance,
Cheers

Presumably Entity Framework is translating this code to SQL, and your custom Map() method is unknown to SQL. Fortunately that method doesn't do much, so you should be able to move its functionality directly to the query.
You can use .Select() to project the collection into a new type (just like you already do for building your ProductDto).
For example:
//...
Address = x.Product.Addresses.OrderBy(x=> x.CreatedDate)
.Select(x=> new AddressDTO
{
Address = x.Address,
Country = x.Country,
Zip = x.Zip
})
.FirstOrDefault()
//...

You can't call the Map function as it is, but you can modify it a bit to work on Queryables instead
var query = _connectDBContext.Products
.Join(_context.CustomerRelations,
Product => Product.ForeignKeyId,
CustomerRelation => CustomerRelation.CompanyId,
(Product, CustomerRelation) => new { Product, CustomerRelation })
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses),
})
.Where(x => x.Id == x.CustomerRelationId);
// Rest of the code
}
private AddressDTO Map(IQueryable<Address> addresses)
{
return addresses.OrderBy(x => x.CreatedDate).Select(x => new AddressDTO
{
Address = address.Address,
Country = address.Country,
Zip = address.Zip
}).FirstOrDefault();
}

See David's answer above for explanation why it does not work.
For a workaround, you can call .ToList() before the mapping so the query is executed on the server:
_connectDBContext.Products
.Join(...)
.Where(x => x.Id == x.CustomerRelationId)
.ToList() // !!!
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
...
});

Entity Framework: How to query a number of related tables in a database making a single trip

I have a query that is currently far too slow.
I am trying to search a Code (a string) on the main page that will bring the user the relevant info.
Eg. The user can search a code from the main page and this will search for the code in Job, Work Phase, Wbs, Work Element, EA, Jobcard and Estimate and return the relevant info.
I make a number of trips to the database to collect the data i need when I believe it can be done in just one.
I have a number of tables that are all linked:
Contracts, Jobs, WorkPhases, Wbss, Engineering Activities, Jobcards and Estimates.
Contracts have a list of Jobs,
Jobs have a list of Workphases,
Workphases have a list of Wbss etc
Is there a quicker way to do this?
public Result Handle(Query query)
{
query.Code = query.Code ?? string.Empty;
var result = new Result();
//result.SetParametersFromPagedQuery(query);
result.Items = new List<Item>();
if (query.SearchPerformed)
{
var contracts = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(contracts.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Contract,
ContractName = x.Name,
Url = string.Format("Admin/Contract/Edit/{0}", x.Id)
})).ToList();
var jobs = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jobs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
ContractName = x.Contract.Name,
Type = MainPageSearchEnum.Job,
Url = string.Format("Admin/Job/Edit/{0}", x.Id)
})).ToList();
//var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code.ToLower() == query.Code.ToLower());
var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code);
result.Items = result.Items.Concat(workPhases.Select(x => new Item()
{
Code = x.ContractPhase.Code,
Id = x.Id,
Name = x.ContractPhase.Name,
Type = MainPageSearchEnum.WorkPhase,
Url = string.Format("Admin/WorkPhase/Edit/{0}", x.Id)
})).ToList();
var wbss = _db.WBSs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(wbss.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.WBS,
Url = string.Format("Admin/WBS/Edit/{0}", x.Id)
})).ToList();
var eas = _db.EngineeringActivities.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(eas.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/EngineeringActivity/Edit/{0}", x.Id)
})).ToList();
var jcs = _db.Jobcards.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jcs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/JobCard/Edit/{0}", x.Id)
})).ToList();
var estimates = _db.Estimates.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(estimates.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Estimate,
Url = string.Format("Estimation/Estimate/Edit/{0}", x.Id)
})).ToList();
}
return result;
}

Disclaimer: I'm the owner of the project Entity Framework Plus
This library has a Query Future feature which allows batching multiple queries in a single roundtrip.
Example:
// using Z.EntityFramework.Plus; // Don't forget to include this.
var ctx = new EntitiesContext();
// CREATE a pending list of future queries
var futureCountries = ctx.Countries.Where(x => x.IsActive).Future();
var futureStates = ctx.States.Where(x => x.IsActive).Future();
// TRIGGER all pending queries in one database round trip
// SELECT * FROM Country WHERE IsActive = true;
// SELECT * FROM State WHERE IsActive = true
var countries = futureCountries.ToList();
// futureStates is already resolved and contains the result
var states = futureStates.ToList();
Wiki: EF+ Query Future

Have you tried the Union / UnionAll operator?
It's purpose is exactly like you wish - combine the identical data from different sources.
Furthermore due to the concept of deferred execution your query will only be executed when you actually iterate over the result (or call a method that does that, for example - .ToList()
var contractsQuery = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new {Code=x.Code, Id=x.Id, ...});
var jobsQuery = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
var workPhasesQuery = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
// and so on
var combinedQuery = contractsQuery.UnionAll(jobsQuery).UnionAll(workPhasesQuery ).UnionAll(...
var result = combinedQuery.ToList();
A similar question is Union in linq entity framework
Another code sample can be found here
Please notice that this is exactly the same concept of manipulating data as in T-SQL union, and under the covers you will get an sql query using a union operator

Yes there most certainly is a way to query multiple tables. You can use the Include() method extension for your query. for instance:
var examplelist = _db.Contracts.(v => v.id == "someid" && v.name == "anotherfilter").Include("theOtherTablesName").ToList();
You can include as many tables as you like this way. This is the recommended method.
You can also use the UnionAll() method but you'd have to define your queries separately for this

how to sort data from database in listview using c# and entity framework

I'm using Empty Web Application as my host and i have different classes, such as users in it.
I also have a client (Windows Forms app).
Please notice this is Entity framework database!
Ok here's the thing. I want to display sorted data by name in listview.
In my Empty Web app, i have Webservice where i have the code for displaying users.
This is the code, and it works
public Users[] getUsers()
{
List<Users> userList = new List<Users>();
using (var db = new DataBase())
{
var query = from x in db.userList
select new
{
ID = x.UserID,
name = x.Name,
lName = x.LastName,
age = x.Age,
club = x.Club,
price = x.Price
};
foreach (var user in query)
{
Users usrLst = new Users();
usrLst.UserID = ID;
usrLst.Name = user.name;
usrLst.LastName = user.lName;
usrLst.Age = user.age;
usrLst.Club = user.club;
usrLst.Price = user.price;
userList.Add(usrLst);
}
}
return userList.ToArray();
This displays data from my entitiy framework database to listview in my client (windows forms application).
Can you guys please help me fix this, so it gets sorted by name
I would really appreciate any input!

Try this:
var query = from x in db.userList
orderby x.Name
select new
{
ID = x.UserID,
name = x.Name,
lName = x.LastName,
age = x.Age,
club = x.Club,
price = x.Price
};

At the return statement.
instead of return userList.ToArray();
Try changing to
return userList.OrderBy(x => x.Name).ToArray();

Data binding directly to a store query (DbSet, DbQuery, DbSqlQuery, DbRawSqlQuery) is not supported [duplicate]

This question already has answers here:
Data binding directly to a store query (DbSet, DbQuery, DbSqlQuery) is not supported
(3 answers)
Closed 7 years ago.
There error is as the title states: Data binding directly to a store query (DbSet, DbQuery, DbSqlQuery, DbRawSqlQuery) is not supported. I've tried searching for possible fixes, but couldn't find the one that was working for me.
Here is my DataLayer:
public virtual IQueryable<T> All()
{
return this.DbSet.AsQueryable();
}
My Controller:
public IQueryable<Movie> GetAllMovies()
{
var data = this.Data.Movies.All().Select(x => new Movie
{
Id = x.Id,
Name = x.Name,
ReleaseDate = x.ReleaseDate,
Rating = x.Rating,
Duration = x.Duration,
Director = x.Director,
Writer = x.Writer,
Cost = x.Cost,
Type = x.Type
}).OrderBy(x => x.Id);
return data;
}
And my GUI where I am calling it:
public MovieManagementGUI()
{
InitializeComponent();
***this.radListView1.DataSource = movieCtr.GetAllMovies();*** //<-- Here I am getting the error
this.radListView1.ViewType = ListViewType.IconsView;
this.radListView1.AllowEdit = false;
this.radListView1.AllowRemove = false;
ImagePrimitive searchIcon = new ImagePrimitive();
searchIcon.Alignment = ContentAlignment.MiddleRight;
SetupIconsView();
}
I am trying to fetch all the data from the the DB and post it in a ListView. Can somebody look at the code and try fixing me up and if you need additional code, let me know.
Thank you,
Marius J.

I managed to fix my problem, I created a new DTO class and edited my controller:
public IEnumerable<MovieDTO> GetAllMovies()
{
var data = this.Data.Movies.All().Select(x => new MovieDTO
{
Id = x.Id,
Name = x.Name,
ReleaseDate = x.ReleaseDate,
Rating = x.Rating,
Duration = x.Duration,
Director = x.Director,
Writer = x.Writer,
Cost = x.Cost,
Type = x.Type
}).OrderBy(x => x.Id).ToList();
return data;
}