I've developed an UWP app where I use a SQLite database to store datas that are synced.
Among these data, there a lot of tables that contain translated data. For example, for various cases, we have:
a "businness" table, which contains the id that is really used in the database
a "translation" table, which contains transaltion for the business table
The models of the "business" tables are defined like this:
public class Failure_Type : BasePoco
{
[PrimaryKey, NotNull]
public int failure_id { get; set; }
public int? function_type_id { get; set; }
public int? component_category_id { get; set; }
[MaxLength(200), NotNull]
public string description { get; set; }
public DateTime? disable_date { get; set; }
[Ignore]
public string _descriptionTr { get; set; }
}
The field "description" stores the english/default description, and the "_descriptionTr" field will store the translated description.
The models of the "translation" tables are defined like this:
public class Failure_Type_Translation : BasePoco
{
[PrimaryKey, NotNull]
public int failure_type_translation_id { get; set; }
public int? failure_type_id { get; set; }
[MaxLength(2)]
public string language { get; set; }
[MaxLength(200), NotNull]
public string description { get; set; }
}
The field "failure_type_id" is related to the business table, the other fields store the language code and the related translation.
So, after syncing datas in the SQLite database, I refresh the "translated" datas in the app and this can take a long moment. The load of the the 2 tables from the SQLite is very quickly, but the update of the "_descriptionTr" field can be very slow:
var failureType = ServiceLocator.Current.GetInstance<IRepository>().GetAll<Failure_Type>();
var failureTypeTranslations = ServiceLocator.Current.GetInstance<IRepository>().GetAll<Failure_Type_Translation>();
FailureType = new ObservableCollection<Failure_Type>();
foreach (var ft in failureType)
{
var ftt = failureTypeTranslations.FirstOrDefault(i => i.failure_type_id == ft.failure_id && i.language.ToLower().Equals(lang));
if (ftt != null)
ft._descriptionTr = ftt.description;
else
ft._descriptionTr = ft.description;
FailureType.Add(ft);
}
Is there a better way for doing this?
How could I optimize it?
Edit :
the "business" table contains 550 rows
the "translation" table contains 3500 rows
the duration of the loop is nearly 1 minute
A couple of suggestions:
Create the observable collection at once ...
FailureType = new ObservableCollection<Failure_Type>(failureType);
... so the individual additions don't fire notifications. Now use FailureType in the loop.
Instead of fetching all translations, filter them by lang:
var failureTypeTranslations = ServiceLocator.Current.GetInstance<IRepository>()
.GetAll<Failure_Type_Translation>()
.Where(l => i.language == lang);
Create a dictionary for lookup of known translations:
var dict = failureTypeTranslations.ToDictionary(ftt => ftt.failure_id);
foreach (var ft in FailureType)
{
Failure_Type_Translation ftt;
if (dict.TryGetValue(ft.failure_id, out ftt)
ft._descriptionTr = ftt.description;
else
ft._descriptionTr = ft.description;
}
I think that esp. the part failureTypeTranslations.FirstOrDefault kills performance. The query is executed for each iteration of the loop.
Related
Fairly new to EF.Core and I'm having some issues as my tables start getting more complex. Here's an example of what I have defined for my classes. Note ... there are many more columns and tables than what I have defined below. I've paired them down for brevity.
public class User
{
public int Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public bool Active { get; set; }
}
Followed by
public class JournalEntry
{
public int Id { get; set; }
public int UserId { get; set; }
public string Details { get; set; }
public DateTime DateEntered { get; set; }
public virtual User User { get; set; }
}
I want to be able to issue the following query and INCLUDE the User Table so that I can then populate a ViewModel with columns from the User Table without having to do another lookup and also to sort the data while retrieving it:
public IQueryable<JournalEntry> GetByUser(int userId)
{
return _DbContext.JournalEntries.Where(j => j.UserId == userId)
.Include(u => u.User)
.OrderBy(u=> u.User.FirstName)
.ThenBy(j => j.DateEntered);
}
My controller would then have something similar to the following:
public IActionResult List(int userId)
{
var journalEntries = new _dbRepository.GetByUser(userId);
var myViewModel = new MyViewModel();
myViewModel.UserName = ($"{journalEntries.User.FirstName} {journalEntries.User.LastName}");
myViewModel.Entries = journalEntries;
etc ....
return View(myViewModel);
}
I'm loading the user's first and last name in the View Model and whatever other attributes from the various tables that are referenced. The problem that I'm having is that I'm getting errors on the Migration creation "Foreign key constraint may cause cycle or multiple cascade paths." And of course, if I remove the line reading public virtual User User { get; set; } from the JournalEntry class then the problem goes away (as one would expect).
I believe that the way I'm doing the models is incorrect. What would be the recommended way that I should code these models? I've heard of "lazy loading". Is that what I should be moving towards?
Thanks a bunch.
--- Val
Your query returns an IQueryable<JournalEntry> not a JournalEntry.
Change the code to get the user details from the first object:
var myViewModel.UserName = ($"{journalEntries.First().User.FirstName} {journalEntries.First().User.LastName}");
In the line above I'm calling First() on your journal entries collection and that would have a User. Then I can access FirstName and LastName.
Also, don't bother with LazyLoading since you are learning. It could cause select n+1 issues if used incorrectly
I have two classes:
public partial class TSS_Filter
{
[Key]
public int filter_id { get; set; }
public int? rackId { get; set; }
public virtual TSS_Rack TSS_Rack { get; set; }
}
AND
public partial class TSS_Rack
{
public TSS_Rack()
{
TSS_Filter = new HashSet<TSS_Filter>();
}
[Key]
public int rack_id { get; set; }
public virtual ICollection<TSS_Filter> TSS_Filter { get; set; }
}
When I try to save the filter with a Rack object it takes the copy of the Rack object in the database and re creates it with the next sequential primary key and all of the rest of the data is the same.
Example:
Rack Table:
1 - rack 1 - datais here <- the rack row I want to use for the foreign key
2 - rack 1 - datais here <- the created copy row that actually get assigned
Why is sql server/entity framework behaving this way?
Save code:
using (ScaleManagerEntities SSDb = new ScaleManagerEntities())
{
TSS_Filter filter = new TSS_Filter();
filter.TSS_Rack = SelectedRack;
SSDb.SaveChanges();
}
using (ScaleManagerEntities SSDb = new ScaleManagerEntities())
{
TSS_Rack rack = SSDb.TSS_Rack.Where(r => r.rack_id == SelectedRack.rack_id).FirstOrDefault();
foreach (TSS_Filter filter in Filters)
{
filter.createdAt = DateTime.Now;
filter.isUsed = false;
filter.TSS_Rack = rack;
SSDb.TSS_Filter.Add(filter);
}
SSDb.SaveChanges();
}
If I looked up with rack within the same context as I did save it the Database didn't have any trouble knowing it was the same object. Seems weird it wouldn't just know that they were the same row in the DB based on the RackId.
Would appreciate any that could make the database recognize the Object outside of the same using session. Thanks.
I'm trying to optimize my EF queries and I'm stuck with this one.
Let's say I have a model like this:
public class House
{
public int ID { get; set; }
public ICollection<Window> Windows { get; set; }
}
public class Window
{
public int ID { get; set; }
public string Color { get; set; }
public WindowKind Kind { get; set; }
}
public class WindowKind
{
public int ID { get; set; }
public string Name { get; set; }
public string Description { get; set; }
}
What I would like to do is to explicitly load all windows and to specify what should be populated in WindowKind property.
I know I can do it with .Include() like this:
var house = Context.Houses.Single(h => h.ID == id);
var windows = Context.Entry(house).Collection(h => h.Windows).Query().Include(w => w.Kind).Load();
However, this will create a query that will load all WindowKind properties and I need only Name, for example. I was hoping something like this would work but it does not, Windows collection is empty, although the generated query looks good.
var house = Context.Houses.Single(h => h.ID = id);
var windows = Context.Entry(house).Collection(h => h.Windows).Query().Select(w => { new w.Color, w.Kind.Name }).Load();
Is it possible to have fine grained control when loading child collections?
you can't load only a part of the scalar (int, string,...) properties of an entity by loading the entity.
In you case, something like the following should do:
from
w in Context.Windows
where
w.House.ID == id // here a navigation property is missing, but (imho) more clear for the sample
select new {
windows = w,
kName = w.Kind.Name
}
But in this case you will not get context attached entities.
I am running into a very odd error with EF6. I have uploaded ~38K records on my first pass. Then on my second round, I query the table with a conditional linq statement. That line of code takes about 4 mins to run. This are my entities.
[Table("RAW_ADWORDS")]
public class AdWord
{
[Key]
public int ID { get; set; }
public bool Processed { get; set; }
public string Client { get; set; }
public long ClientID { get; set; }
public bool Active { get; set; }
public bool ProcessedAllFile { get; set; }
public DateTime LastTimeRun{ get; set; }
public DateTime? LastDateTimeProcessed { get; set; }
public virtual List<AdWordCampaign> Campaigns { get; set; }
}
[Table("foobar")]
public class AdWordCampaign
{
[Key]
public int ID { get; set; }
public string Campaign { get; set; }
public long CampaignID { get; set; }
public string Day { get; set; }
public long Clicks { get; set; }
public string CampaignStatus { get; set; }
public long Cost { get; set; }
public long Impressions { get; set; }
public double CTR { get; set; }
public long AvgCPC { get; set; }
public double AvgPosition { get; set; }
public DateTime DownloadDate { get; set; }
}
}
First I run this:
AdWord objAdWord = adwordsContext.AdWords.Where(c => c.ClientID == iCampaignID).FirstOrDefault();
Then
AdWordCampaign objAdWordCampaign = objAdWord.Campaigns.Where(c => c.CampaignID == iElementCampaignID && c.Day == sElementDate).FirstOrDefault();
The line above seems to load ALL the records first before it does the query. Also it still takes 4 mins if I add a Take(5) in the query.
I hope this info will be useful.
Try to add indexes to fields of your table you are including to WHERE in LINQ.
You can always can create extra Views and add it into EF model and do LINQ to them. It will reduce time as well.
If you expect 1 record always try to use SingleOrDefault
Try:
objAdWord.Campaigns.FirstOrDefault(c => c.CampaignID == iElementCampaignID && c.Day == sElementDate)
.Where is an O(n) operation, I'm not sure if the Where then FirstOrDefault clause would be optimized but if it's not you're wasting a lot of time using Where. To improve performance further, ensure that CampaignID is indexed
You need to watch the queries that are generated & executed on the server and make sure they're optimized.
If you're using MS SQL Server, you want to run the SQL Server Profiler tool. Put breakpoints in your code before you call the method that executes the query. Clear the profiler's display, then execute the method. You can capture the SQL from there, then put it into SSMS and view the plan. If the query doesn't use indexes, you need to add indexes that it will use the next time it runs.
I've only ever used Database First, not Code First, so I don't know how you tell Entity Framework to create indexes in the Code First scenario, sorry. But you still need to optimize all of your queries.
I've seen this before with EF when referencing linked objects through a "primary object" - i.e. when you do
AdWordCampaign objAdWordCampaign = objAdWord.Campaigns.Where(...).FirstOrDefault();.
Quite simply it iterates all records one-by-one - and hence the slow query.
If you change to the following, you should get an almost instant response:
AdWord objAdWord = adwordsContext.AdWords.Where(c => c.ClientID == iCampaignID).FirstOrDefault();
AdWordCampaign objAdWordCampaign = <adwordsContext>.Campaigns
.Where(c => <c.AdwordId = objAdWord.Id> && c.CampaignID == iElementCampaignID && c.Day == sElementDate)
.FirstOrDefault();
I've put changes in angular brackets and I'm not sure which property within an AdWordCampaign marks the Id of the AdWord for the relationship from a glance at your model, but I'm sure you get the idea - go direct to Capaigns table via the context, using the AdWord as an addiitonal Where clause, rather than via the AdWord's Campaigns collection.
I've got three classes.
Event > Workshop > Workshop Times
I'm currently looking for best way of inserting records into the Workshop Times, this is running through code first using ICollections.
Looking for something along the lines of this, but I know it doesn't work:
//Create connection
var db = new Context();
var Event = db.Events
.Include("Workshops")
.Include("Workshops.Times")
.Where(ev => ev.GUID == EventGUID).FirstOrDefault();
Event.Workshops.Add(new Workshop
{
Name = tbWorkshopName.Text,
Description = tbWorkshopDescription.Text,
Times.Add(new WorkshopTime{
//Information for times
})
});
db.SaveChanges();
Chopped down classes:
public class Workshops{
public int id { get; set; }
public string name { get; set; }
public ICollection<WorkshopTimes> Times{get;set;}
}
public class Events {
public int id { get; set; }
public string name { get; set; }
public ICollection<Workshops> WorkShops { get; set; }
}
public class WorkshopTimes {
public int id { get; set; }
public DateTime time { get; set; }
}
You are definitely on the right track with your query, however your include statements appear incorrect. From your model I would expect:
var Event = db.Events
.Include("WorkShops")
.Include("WorkShops.events")
.Where(ev => ev.GUID == EventGUID).FirstOrDefault();
Note this uses the property names not the types. This will ensure that the entities in the listed nav properties will be included in the result.
In addition you can use a lambda to do the same thing (but its typesafe)
Check out here for how to do a very similar scenario to yours:
EF Code First - Include(x => x.Properties.Entity) a 1 : Many association
or from rowan miller (from EF team)
http://romiller.com/2010/07/14/ef-ctp4-tips-tricks-include-with-lambda/
And make sure you are using System.Data.Entities for lambda based includes ( Where did the overload of DbQuery.Include() go that takes a lambda? )