I wrote simple query (issues is when I try to set an Address below ProductCode):
var query = _connectDBContext.Products
.Join(_context.CustomerRelations,
Product => Product.ForeignKeyId,
CustomerRelation => CustomerRelation.CompanyId,
(Product, CustomerRelation) => new { Product, CustomerRelation })
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
//Address = new AddressDTO // THIS WORKS BUT LOOKS UGLY :(
//{
// Address = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Address,
// Country = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Country,
// Zip = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Zip,
//},
})
.Where(x => x.Id == x.CustomerRelationId);
// Rest of the code
}
private AddressDTO Map(Address address)
{
return new AddressDTO
{
Address = address.Address,
Country = address.Country,
Zip = address.Zip,
};
}
This code above breaks on this line:
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
It says that linq cannot be translated and advicing me to rewrite a query..
But this commented code here which does almost the same works, so if I remove calling Map method in Select and if I uncomment this code in Select everything would work, but I would like to get rid of this - writing too many OrderBy for each prop, I would like to Order it once and after that I would like to use it.. but unfortunatelly I don't know how.. :
//Address = new AddressDTO
//{
// Address = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Address,
// Country = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Country,
// Zip = x.Product.Addresses.OrderBy(x => x.CreatedDate).FirstOrDefault().Zip,
//},
Thanks in advance,
Cheers
Presumably Entity Framework is translating this code to SQL, and your custom Map() method is unknown to SQL. Fortunately that method doesn't do much, so you should be able to move its functionality directly to the query.
You can use .Select() to project the collection into a new type (just like you already do for building your ProductDto).
For example:
//...
Address = x.Product.Addresses.OrderBy(x=> x.CreatedDate)
.Select(x=> new AddressDTO
{
Address = x.Address,
Country = x.Country,
Zip = x.Zip
})
.FirstOrDefault()
//...
You can't call the Map function as it is, but you can modify it a bit to work on Queryables instead
var query = _connectDBContext.Products
.Join(_context.CustomerRelations,
Product => Product.ForeignKeyId,
CustomerRelation => CustomerRelation.CompanyId,
(Product, CustomerRelation) => new { Product, CustomerRelation })
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses),
})
.Where(x => x.Id == x.CustomerRelationId);
// Rest of the code
}
private AddressDTO Map(IQueryable<Address> addresses)
{
return addresses.OrderBy(x => x.CreatedDate).Select(x => new AddressDTO
{
Address = address.Address,
Country = address.Country,
Zip = address.Zip
}).FirstOrDefault();
}
See David's answer above for explanation why it does not work.
For a workaround, you can call .ToList() before the mapping so the query is executed on the server:
_connectDBContext.Products
.Join(...)
.Where(x => x.Id == x.CustomerRelationId)
.ToList() // !!!
.Select(x => new ProductDto
{
Title = x.Product.Title,
ProductCode = x.Product.Code,
Address = Map(x.Product.Addresses.OrderBy(x=> x.CreatedDate).FirstOrDefault()),
...
});
Related
I have a method that returns data by viewing models from multiple tables. The problem occurs when I try to select a column while data is not exiting. Heres my code
public async Task<List<CustomerViewModel>> GetCustomers()
{
return await _context.Customers.Select(x => new CustomerViewModel
{
Id = x.Id,
Name = x.Name,
Phone = x.Phone,
Address = x.Address,
Payment = x.Payment,
Due = x.Due,
Received = x.Received,
DateOfReg = x.DateOfReg,
Quantity = _context.BSells.Where(_ => _.CustomerId == x.Id).Select(_=>_.Quantity).Sum(),
Unitcost = _context.BSells.Where(_ => _.CustomerId == x.Id).FirstOrDefault().UnitPrice
}).ToListAsync();
}
The problem occurs on lines 14, and 15 because the BSell table doesn't have data of the particular customer. Now, what should I do now?
I may not be able to express the problem clearly. Sorry for that.
Alright I've solved the issue by changing code little bit
public async Task<List<CustomerViewModel>> GetCustomers()
{
return await _context.Customers.Select(x => new CustomerViewModel
{
Id = x.Id,
Name = x.Name,
Phone = x.Phone,
Address = x.Address,
Payment = x.Payment,
Due = x.Due,
Received = x.Received,
DateOfReg = x.DateOfReg,
Quantity = _context.BSells.Where(_ => _.CustomerId == x.Id).Select(_=>_.Quantity).Sum(),
Unitcost = _context.BSells.Where(_ => _.CustomerId == x.Id).Select(_ => _.UnitPrice).FirstOrDefault()
}).ToListAsync();
}
I should have selected the column before taking the first value. Is it? I will be really grateful if anyone addresses my mistakes.
Thank you.
I'm trying to make some queries using EF-Core and I have the following code
public List<Visitation> GetAllVisitations()
{
return this.hospital.Visitations
.Where(v => v.DoctorId == this.Doctor.Id)
.Select(v => new Visitation
{
Doctor = v.Doctor,
Patient = v.Patient,
Date = v.Date,
Comments = v.Comments
})
.ToList();
}
public List<Visitation> GetVisitationByPatient(int id)
{
var patient = this.GetPatientById(id);
return this.hospital.Visitations
.Where(v => v.PatientId == patient.Id)
.Select(v => new Visitation
{
Doctor = v.Doctor,
Patient = v.Patient,
Date = v.Date,
Comments = v.Comments
})
.ToList();
}
It is pretty obvious that the Select statement is the same in both methods. However I know that EF Core uses Expression<Func>, rather than Func therefore I do not know how to make an Expression, which can be used in both Select statements.
The query won't execute until you call .ToList(). So you may take the partial query up to the .Where() and pass it to a function that adds the Select() portion.
Something like this:
public List<Visitation> GetAllVisitations()
{
var query = this.hospital.Visitations
.Where(v => v.DoctorId == this.Doctor.Id);
return this.addTransformation(query)
.ToList();
}
public List<Visitation> GetVisitationByPatient(int id)
{
var patient = this.GetPatientById(id);
var query = this.hospital.Visitations
.Where(v => v.PatientId == patient.Id)
return this.addTransformation(query)
.ToList();
}
public IQueriable<Visitation> AddTransformation(IQueriable<Visitation> query)
{
return query.Select(v => new Visitation
{
Doctor = v.Doctor,
Patient = v.Patient,
Date = v.Date,
Comments = v.Comments
});
}
I am writing an .NET Core Web API which is backed by Entity Framework Core and having a PostgreSQL database underneath (AWS Aurora). I have been able to learn and successfully work with EF Core for inserts and querying data, which is great, but starting to look into UPDATES of existing entities and it's become unclear to me the most efficient way to achieve what I'm after. I have my Database Entities as a result of my Database First scaffolding exercise. I have an entity that is similar to the below (simplified).
Customer -< Addresses
-< ContactInformation
So a Customer, which may have multiple Addresses and / or multiple ContactInformation records. I am expecting my Web API to pass in a JSON payload that can be converted to a Customer with all associated information. I have a method which converts my CustomerPayload to a Customer that can be added to the database:
public class CustomerPayload : Payload, ITransform<CustomerPayload, Customer>
{
[JsonProperty("customer")]
public RequestCustomer RequestCustomer { get; set; }
public Customer Convert(CustomerPayload source)
{
Console.WriteLine("Creating new customer");
Customer customer = new Customer
{
McaId = source.RequestCustomer.Identification.MembershipNumber,
BusinessPartnerId = source.RequestCustomer.Identification.BusinessPartnerId,
LoyaltyDbId = source.RequestCustomer.Identification.LoyaltyDbId,
Title = source.RequestCustomer.Name.Title,
FirstName = source.RequestCustomer.Name.FirstName,
LastName = source.RequestCustomer.Name.Surname,
Gender = source.RequestCustomer.Gender,
DateOfBirth = source.RequestCustomer.DateOfBirth,
CustomerType = source.RequestCustomer.CustomerType,
HomeStoreId = source.RequestCustomer.HomeStoreId,
HomeStoreUpdated = source.RequestCustomer.HomeStoreUpdated,
StoreJoined = source.RequestCustomer.StoreJoinedId,
CreatedDate = DateTime.UtcNow,
UpdatedDate = DateTime.UtcNow,
UpdatedBy = Functions.DbUser
};
Console.WriteLine("Creating address");
if (source.RequestCustomer.Address != null)
{
customer.Address.Add(new Address
{
AddressType = "Home",
AddressLine1 = source.RequestCustomer.Address.AddressLine1,
AddressLine2 = source.RequestCustomer.Address.AddressLine2,
Suburb = source.RequestCustomer.Address.Suburb,
Postcode = source.RequestCustomer.Address.Postcode,
Region = source.RequestCustomer.Address.State,
Country = source.RequestCustomer.Address.Country,
CreatedDate = DateTime.UtcNow,
UpdatedDate = DateTime.UtcNow,
UpdatedBy = Functions.DbUser,
UpdatingStore = null, // Not passed by API at present
AddressValidated = false, // Not passed by API
AddressUndeliverable = false, // Not passed by API
});
}
Console.WriteLine("Creating marketing preferences");
if (source.RequestCustomer.MarketingPreferences != null)
{
customer.MarketingPreferences = source.RequestCustomer.MarketingPreferences
.Select(x => new MarketingPreferences()
{
ChannelId = x.Channel,
OptIn = x.OptIn,
ValidFromDate = x.ValidFromDate,
UpdatedBy = Functions.DbUser,
CreatedDate = DateTime.UtcNow,
UpdatedDate = DateTime.UtcNow,
ContentTypePreferences = (from c in x.ContentTypePreferences
where x.ContentTypePreferences != null
select new ContentTypePreferences
{
TypeId = c.Type,
OptIn = c.OptIn,
ValidFromDate = c.ValidFromDate,
ChannelId = x.Channel // Should inherit parent marketing preference channel
}).ToList(),
UpdatingStore = null // Not passed by API
})
.ToList();
}
Console.WriteLine("Creating contact information");
if (source.RequestCustomer.ContactInformation != null)
{
// Validate email if present
var emails = (from e in source.RequestCustomer.ContactInformation
where e.ContactType.ToUpper() == ContactInformation.ContactTypes.Email && e.ContactValue != null
select e.ContactValue);
if (!emails.Any()) throw new Exception("At least 1 email address must be provided for a customer registration.");
foreach (var email in emails)
{
Console.WriteLine($"Validating email {email}");
if (!IsValidEmail(email))
{
throw new Exception($"Email address {email} is not valid.");
}
}
customer.ContactInformation = source.RequestCustomer.ContactInformation
.Select(x => new ContactInformation()
{
ContactType = x.ContactType,
ContactValue = x.ContactValue,
CreatedDate = DateTime.UtcNow,
UpdatedBy = Functions.DbUser,
UpdatedDate = DateTime.UtcNow,
Validated = x.Validated,
UpdatingStore = x.UpdatingStore
})
.ToList();
}
else
{
throw new Exception("Minimum required elements not present in POST request");
}
Console.WriteLine("Creating external cards");
if (source.RequestCustomer.ExternalCards != null)
{
customer.ExternalCards = source.RequestCustomer.ExternalCards
.Select(x => new ExternalCards()
{
CardNumber = x.CardNumber,
CardStatus = x.Status,
CardDesign = x.CardDesign,
CardType = x.CardType,
UpdatingStore = x.UpdatingStore,
UpdatedBy = Functions.DbUser
})
.ToList();
}
Console.WriteLine($"Converted customer object --> {JsonConvert.SerializeObject(customer)}");
return customer;
}
I would like to be able to look up an existing customer by McaId (which is fine, I can do that via)
var customer = await loyalty.Customer
.Include(c => c.ContactInformation)
.Include(c => c.Address)
.Include(c => c.MarketingPreferences)
.Include(c => c.ContentTypePreferences)
.Include(c => c.ExternalCards)
.Where(c => c.McaId == updateCustomer.McaId).FirstAsync();
But then be able to neatly update that Customer and associated tables with any different values for any properties contained in the Customer - OR - its related entities. So, in pseudo code:
CustomerPayload (cust: 1234) Comes in.
Convert CustomerPayload to Customer(1234)
Get Customer(1234) current entity and related data from Database.
Check changed values for any properties of Customer(1234) compared to Customer(1234) that's come in.
Generate the update statement:
UPDATE Customer(1234)
Set thing = value, thing = value, thing = value.
UPDATE Address where Customer = Customer(1234)
Set thing = value
Save to Database.
Can anyone help as to the best way to achieve this?
EDIT: Updated with attempt. Code below:
public static async void UpdateCustomerRecord(CustomerPayload customerPayload)
{
try
{
var updateCustomer = customerPayload.Convert(customerPayload);
using (var loyalty = new loyaltyContext())
{
var customer = await loyalty.Customer
.Include(c => c.ContactInformation)
.Include(c => c.Address)
.Include(c => c.MarketingPreferences)
.Include(c => c.ContentTypePreferences)
.Include(c => c.ExternalCards)
.Where(c => c.McaId == updateCustomer.McaId).FirstAsync();
loyalty.Entry(customer).CurrentValues.SetValues(updateCustomer);
await loyalty.SaveChangesAsync();
//TODO expand code to cover scenarios such as an additional address on an udpate
}
}
catch (ArgumentNullException e)
{
Console.WriteLine(e);
throw new CustomerNotFoundException();
}
}
All I've changed is the last name of the customer. No errors occur, however the record is not updated in the database. I have the following settings on so was expecting to see the generated SQL statements in my log, but no statements were generated:
Entity Framework Core 3.1.4 initialized 'loyaltyContext' using provider 'Npgsql.EntityFrameworkCore.PostgreSQL' with options: SensitiveDataLoggingEnabled
That's the only entry I have listed in my log.
You are working with graphs of disconnected entities. Here is the documentation section that might be of interest to you.
Example:
var existingCustomer = await loyalty.Customer
.Include(c => c.ContactInformation)
.Include(c => c.Address)
.Include(c => c.MarketingPreferences)
.Include(c => c.ContentTypePreferences)
.Include(c => c.ExternalCards)
.FirstOrDefault(c => c.McaId == customer.McaId);
if (existingCustomer == null)
{
// Customer does not exist, insert new one
loyalty.Add(customer);
}
else
{
// Customer exists, replace its property values
loyalty.Entry(existingCustomer).CurrentValues.SetValues(customer);
// Insert or update customer addresses
foreach (var address in customer.Address)
{
var existingAddress = existingCustomer.Address.FirstOrDefault(a => a.AddressId == address.AddressId);
if (existingAddress == null)
{
// Address does not exist, insert new one
existingCustomer.Address.Add(address);
}
else
{
// Address exists, replace its property values
loyalty.Entry(existingAddress).CurrentValues.SetValues(address);
}
}
// Remove addresses not present in the updated customer
foreach (var address in existingCustomer.Address)
{
if (!customer.Address.Any(a => a.AddressId == address.AddressId))
{
loyalty.Remove(address);
}
}
}
loyalty.SaveChanges();
I have a query that is currently far too slow.
I am trying to search a Code (a string) on the main page that will bring the user the relevant info.
Eg. The user can search a code from the main page and this will search for the code in Job, Work Phase, Wbs, Work Element, EA, Jobcard and Estimate and return the relevant info.
I make a number of trips to the database to collect the data i need when I believe it can be done in just one.
I have a number of tables that are all linked:
Contracts, Jobs, WorkPhases, Wbss, Engineering Activities, Jobcards and Estimates.
Contracts have a list of Jobs,
Jobs have a list of Workphases,
Workphases have a list of Wbss etc
Is there a quicker way to do this?
public Result Handle(Query query)
{
query.Code = query.Code ?? string.Empty;
var result = new Result();
//result.SetParametersFromPagedQuery(query);
result.Items = new List<Item>();
if (query.SearchPerformed)
{
var contracts = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(contracts.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Contract,
ContractName = x.Name,
Url = string.Format("Admin/Contract/Edit/{0}", x.Id)
})).ToList();
var jobs = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jobs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
ContractName = x.Contract.Name,
Type = MainPageSearchEnum.Job,
Url = string.Format("Admin/Job/Edit/{0}", x.Id)
})).ToList();
//var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code.ToLower() == query.Code.ToLower());
var workPhases = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code);
result.Items = result.Items.Concat(workPhases.Select(x => new Item()
{
Code = x.ContractPhase.Code,
Id = x.Id,
Name = x.ContractPhase.Name,
Type = MainPageSearchEnum.WorkPhase,
Url = string.Format("Admin/WorkPhase/Edit/{0}", x.Id)
})).ToList();
var wbss = _db.WBSs.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(wbss.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.WBS,
Url = string.Format("Admin/WBS/Edit/{0}", x.Id)
})).ToList();
var eas = _db.EngineeringActivities.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(eas.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/EngineeringActivity/Edit/{0}", x.Id)
})).ToList();
var jcs = _db.Jobcards.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(jcs.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.EA,
Url = string.Format("Admin/JobCard/Edit/{0}", x.Id)
})).ToList();
var estimates = _db.Estimates.AsEnumerable().Where(x => x.Code == query.Code);
result.Items = result.Items.Concat(estimates.Select(x => new Item()
{
Code = x.Code,
Id = x.Id,
Name = x.Name,
Type = MainPageSearchEnum.Estimate,
Url = string.Format("Estimation/Estimate/Edit/{0}", x.Id)
})).ToList();
}
return result;
}
Disclaimer: I'm the owner of the project Entity Framework Plus
This library has a Query Future feature which allows batching multiple queries in a single roundtrip.
Example:
// using Z.EntityFramework.Plus; // Don't forget to include this.
var ctx = new EntitiesContext();
// CREATE a pending list of future queries
var futureCountries = ctx.Countries.Where(x => x.IsActive).Future();
var futureStates = ctx.States.Where(x => x.IsActive).Future();
// TRIGGER all pending queries in one database round trip
// SELECT * FROM Country WHERE IsActive = true;
// SELECT * FROM State WHERE IsActive = true
var countries = futureCountries.ToList();
// futureStates is already resolved and contains the result
var states = futureStates.ToList();
Wiki: EF+ Query Future
Have you tried the Union / UnionAll operator?
It's purpose is exactly like you wish - combine the identical data from different sources.
Furthermore due to the concept of deferred execution your query will only be executed when you actually iterate over the result (or call a method that does that, for example - .ToList()
var contractsQuery = _db.Contracts.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new {Code=x.Code, Id=x.Id, ...});
var jobsQuery = _db.Jobs.AsEnumerable().Where(x => x.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
var workPhasesQuery = _db.WorkPhases.AsEnumerable().Where(x => x.ContractPhase.Code == query.Code).Select(x=>new{Code=x.Code, Id=x.Id, ...});
// and so on
var combinedQuery = contractsQuery.UnionAll(jobsQuery).UnionAll(workPhasesQuery ).UnionAll(...
var result = combinedQuery.ToList();
A similar question is Union in linq entity framework
Another code sample can be found here
Please notice that this is exactly the same concept of manipulating data as in T-SQL union, and under the covers you will get an sql query using a union operator
Yes there most certainly is a way to query multiple tables. You can use the Include() method extension for your query. for instance:
var examplelist = _db.Contracts.(v => v.id == "someid" && v.name == "anotherfilter").Include("theOtherTablesName").ToList();
You can include as many tables as you like this way. This is the recommended method.
You can also use the UnionAll() method but you'd have to define your queries separately for this
I'm reading a simple csv file from my program. Here's what my csv file looks like:
NASDQ,O,
OTC,O,
NYSE,N,
TSE,T,
Here's my code to read the csv file:
string csvFile = #"x:\tech\SQL_IntlPricing\ExchangeLookup.csv";
string[] csvLines = File.ReadAllLines(csvFile);
var csvValues = csvLines
.Select(l => new {
Exchange = l.Split(',').First(),
Lookup = l.Split(',').Skip(1).First ()});
So far, everything is fine with the code. I'm using the following LINQ query:
from comp in Companies
where !comp.Coverage_status.Contains("drop")
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
But I'm getting the following error:
NotSupportedException: Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
Basically, I'm trying to get the following results:
AAPL-O
MSFT-O
Is there a way for me to achieve the results I want using my LINQ query?
If Companies are not a lot, then the simple solution would be:
from comp in Companies.Where(c => !c.Coverage_status.Contains("drop")).AsEnumerable()
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
Otherwise you could do the filtering there like;
from comp in Companies.Where( c =>
csvValues.Select(cs => cs.Exchange).Contains(comp.Exchange) &&
!c.Coverage_status.Contains("drop")
).AsEnumerable()
select new
{
FSTick = string.Format("{0}-{1}", comp.Ticker,
csvValues
.Where(v => v.Exchange.Contains(comp.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
};
Following my comment above, if converting the linq-sql expression into its extension methods syntax form is ok, you could do the following:
I created a list of companies just for the sake of the example. Company was defined as
public class Company
{
public string Coverage_status { get; set; }
public string Exchange { get; set; }
public string Ticker { get; set; }
}
Here's a full sample of how the code will would look like:
List<string> csvLines = new List<string>
{
"NASDQ,O,",
"OTC,O,",
"NYSE,N,",
"TSE,T,"
};
var csvValues = csvLines
.Select(l => new
{
Exchange = l.Split(',').First(),
Lookup = l.Split(',').Skip(1).First()
});
List<Company> companies = new List<Company>
{
new Company { Coverage_status = "aaa", Ticker = "123", Exchange = "NASDQ"},
new Company { Coverage_status = "1521drop422", Ticker = "1251223", Exchange = "aaaaaaaa"},
new Company { Coverage_status = "f2hdjjd", Ticker = "15525221123", Exchange = "TSE"}
};
var result = companies
.Where(c => !c.Coverage_status.Contains("drop"))
.Select(n => new
{
FSTick = string.Format("{0}-{1}", n.Ticker,
csvValues
.Where(v => v.Exchange.Contains(n.Exchange))
.Select(v => v.Lookup).FirstOrDefault())
});
foreach (var r in result)
Console.WriteLine(r.FSTick);
For the record, this code is definitely not performance-wise.
Output: