I convert my DB objects to interface objects when extracting data from the database and found that when I convert objects that have child objects, I see a significant performance impact.
My model
public class NumberRange
{
public NumberRange()
{
ReservedNumbers = new HashSet<ReservedNumber>();
}
[Key]
public Guid Id {get; set;}
public virtual ICollection<ReservedNumber> ReservedNumbers { get; set; }
}
public class ReservedNumber
{
[Key]
public Guid Id {get; set;}
public string Number { get; set; }
public Guid NumberRangeId { get; set; }
public virtual NumberRange NumberRange { get; set; }
}
Establishing the foreign Key as follows
modelBuilder.Entity<NumberRange>()
.HasMany(e => e.ReservedNumbers)
.WithRequired(e => e.NumberRange)
.HasForeignKey(e => e.NumberRangeId);
And the code to convert to interface models (extension methods)
public static Models.NumberRange ToApiObject(this Model.NumberRange obj, bool includeAssociatedLocations = false, bool includeReservedNumbers = true)
{
var apiObj = new Models.NumberRange { Id = obj.Id };
if (includeReservedNumbers)
apiObj.ReservedNumbers = obj.ReservedNumbers.OrderBy(n => n.Number).AsEnumerable().Select(x => x.ToApiObject()).ToList();
else
apiObj.ReservedNumbers = new List<AUDMService.Models.ReservedNumber>();
return apiObj;
}
public static Models.ReservedNumber ToApiObject(this Model.ReservedNumber obj)
{
return new Models.ReservedNumber { Id = obj.Id, Number = obj.Number, NumberRangeId = obj.NumberRangeId };
}
If I now extract a list of NumberRanges with ReservedNumbers
var myList = context.NumberRanges.OrderBy(n => n.Name).AsEnumerable().Select(x => x.ToApiObject(includeDependentElements, true));
I see a long delay with large data sets (I have ranges that have ~1000 reserved numbers). I figured I could include ReservedNumbers
var myList = context.NumberRanges.OrderBy(n => n.Name).Include(n => n.ReservedNumbers).AsEnumerable().Select(x => x.ToApiObject(includeDependentElements, true));
yet, that has no discernible impact.
I then came up with this, which solves the performance issue
var myList = context.NumberRanges.OrderBy(n => n.Name).Include(n => n.ReservedNumbers).AsEnumerable().Select(x => x.ToApiObject(includeDependentElements, false));
addReservedNumbers(myList, context);
private void addReservedNumbers(IEnumerable<Models.NumberRange> ranges, DatabaseContext context)
{
var rangeIds = ranges.Select(x => x.Id);
var reservedNumbers = context.AccessibleReservedNumbers.Where(x => rangeIds.Contains(x.NumberRangeId)).AsEnumerable();
var convertedNumbers = reservedNumbers.Select(x => x.ToApiObject());
foreach (var range in ranges)
range.ReservedNumbers.AddRange(convertedNumbers.Where(x => x.NumberRangeId == range.Id));
}
So I'm wondering.. isn't there a way to get my first approach to run as fast as the second one?
Related
I have the following query:
var catInclude = _db.Cat
.Where(x => x.ProvId == request.ProvId)
.Include(x => x.CatItems)
.SingleOrDefault(p => p.Id == request.ProvId
cancellationToken: cancellationToken);
As I don't want to get all properties from CatItems with Include(), I have created the following query:
var catSelect = _db.Cat
.Where(x => x.ProvId == request.ProvId)
.Select(p ==> new
{ Provider = p,
Items = p.CatItems.Select(x => new List<CatItems> { new CatItems
{ Id = x.Id, Name = x.Name, Price = x.Price } }
})})
SingleOrDefault(cancellationToken: cancellationToken);
But something is wrong in the 2nd query because here return _mapper.ProjectTo<CatDto>(cat) I get the following error:
Argument 1: cannot convert from '<anonymous type: Db.Entities.Cat Prov, System.Colletions.Generic.IEnumerable<System.Colletions.Generic.List<Models.CatItems> > Items>' to 'System.Linq.IQueryable'
Here is my CatDto:
public class CatDto
{
public int ProvId { get; set; }
public List<CatItems> CatItems { get; set; }
}
Here are my entities:
public class Prov
{
public int Id { get; set; }
public Cat Cat { get; set; }
}
public class Cat
{
public int Id { get; set; }
public int ProvId { get; set; }
public List<CatItems> CatItems { get; set; }
}
public class CatItems
{
public int Id { get; set; }
public int CatId { get; set; }
public DateTime CreatedOn { get; set; }
}
Is there a way to recreate the 2nd query and use it?
Main difference that instead of returning List of CatItems, your code returns IEnumerable<List<CatItems>> for property Items.
So, just correct your query to project to List:
var catSelect = await _db.Cat
.Where(x => x.ProvId == request.ProvId)
.Select(p => new CatDto
{
ProvId = p.ProvId,
Items = p.CatItems.Select(x => new CatItems
{
Id = x.Id,
Name = x.Name,
Price = x.Price
})
.ToList()
})
.SingleOrDefaultAsync(cancellationToken: cancellationToken);
I mean, even the exception is pretty self-explanatory. Nevertheless:
You are performing a .Select(...). It returns an Anonymous type. So, your catSelect is an anonymous type, thus the AutoMapper fails.
The quickest fix is to just cast (Cat)catSelect before mapping.
Or, you can dig deeper into how does AutoMapper play with anonymous types.
I feel like you can make most of the classes inherent Id and why is public cat CAT {get; set;} i thought you were supposed to initialize some kind of value
I have a use-case with a deeply nested class hierarchy, for example like this:
public class Parent
{
public int Id { get; set; }
public List<ChildOne> Children { get; set; }
}
public class ChildOne
{
public int Id { get; set; }
public int ParentId { get; set; }
public List<ChildTwo> ChildrenTwo { get; set; }
}
public class ChildTwo
{
public int Id { get; set; }
public int Priority { get; set; }
public int ChildOneId { get; set; }
public List<ChildThree> ChildrenThree { get; set; }
}
public class ChildThree
{
public int Id { get; set; }
public int ChildTwoId { get; set; }
}
If I want to load all parent-objects and their related children levels, I'd do this:
var objects = context.Parent
.Include(parent => parent.Children)
.ThenInclude(childOne => childOne.ChildrenTwo)
.ThenInclude(childTwo => childTwo.ChildrenThree)
.ToList();
But what if I want my ChildrenTwo entities in the eager-loaded navigational property of ChildOne to be ordered by their Priority? I've done some research, and from the links below (and some others), it is apparently not directly possible in EF Core (yet):
https://github.com/aspnet/EntityFrameworkCore/issues/9445
https://github.com/aspnet/EntityFrameworkCore/issues/2919
https://github.com/aspnet/EntityFrameworkCore/issues/9067
So, how can you achieve the ordering of the ChildrenTwo above (by Priority) in a good/clean way that is fast? That probably means most of the work should happen on the DB server and not on the .NET client side. What's the best approach here?
Though it is very late to answer, but it may help the future readers:
I will explain the code:
var authorArticles = await _context.AuthorArticles
.Include(a => a.Author)
.ThenInclude(p => p.Person)
.ThenInclude(pq => pq.Qualifications)
.ThenInclude(q => q.QualificationSubject)
.Include(a => a.Author)
.ThenInclude(p => p.Person)
.ThenInclude(pp => pp.Professions)
.Include(a => a.Author)
.ThenInclude(p => p.Person)
.ThenInclude(pp => pp.Professions)
.ThenInclude(prof => prof.Profession)
.Where(aa => aa.ArticleId == articleId)
.Select(s => new AuthorArticle
{
Author = new Author
{
Affiliation = s.Author.Affiliation,
AvailableAsReviewer = s.Author.AvailableAsReviewer,
Person = new Person
{
Email = s.Author.Person.Email,
FirstName = s.Author.Person.FirstName,
LastName = s.Author.Person.LastName,
MiddleName = s.Author.Person.MiddleName,
Title = s.Author.Person.Title,
FullName = s.Author.Person.FullName,
UserId = s.Author.Person.UserId,
Professions = new Collection<PersonProfession>
{
new PersonProfession
{
// using sorting here!!
Organization = s.Author.Person.Professions
.OrderByDescending(pid => pid.ProfessionId)
.FirstOrDefault().Organization,
Profession = s.Author.Person.Professions
.OrderByDescending(pid => pid.ProfessionId)
.FirstOrDefault().Profession
}
},
Qualifications = new Collection<PersonQualification>
{
new PersonQualification
{
QualificationSubject = s.Author.Person.Qualifications
.OrderByDescending(q => q.QualificationLevelId)
.FirstOrDefault().QualificationSubject,
QualificationLevelId = s.Author.Person.Qualifications
.OrderByDescending(q => q.QualificationLevelId)
.FirstOrDefault().QualificationLevelId
}
}
}
},
IsCorresponding = s.IsCorresponding,
AuthorPosition = s.AuthorPosition
}).ToListAsync();
return authorArticles;
If you simply eager loaded the entities, then at the time of projection; which means when you are selecting the items from the query, you can recreate the object that has already been provided in slightly different way. In my case, I wanted only one profession of the person out of many and same goes for the qualification of the person.
Took help of select from Another SO great answer!
My setup is that I have a basket which contains items. An item is made up of a product and a size. Products have a many to many relationship with sizes so that I can verify that a given size is valid for a given product. I would like to be able to add an item to the basket, perform some validation and save to the database.
I have created a demo program to demonstrate the problem I am having. When the program runs there is already a basket saved to the database (see the DBInitializer). It has one item which is a large foo. In the program you can see that I load the basket, load a small size and a bar product. I add the large bar to the basket. The basket does some internal validation and I save to the database. This works without error.
The problem comes when I try to add a product that already exists in the database with a different size. Hence if we try to add a large bar to the basket and save we get a null reference exception. This is not the behaviour I would like because a basket which contains 2 items, a large foo and a small foo, is perfectly valid.
I'm pretty sure the problem is to do with the fact that we have already loaded foo in the basket through eager loading. I've tried commenting out the eager loading for the basketitems and this works. However if possible I would like a solution which keeps the eager loading.
Notes: I have added an extra method to my dbcontext class which is int SaveChanges(bool excludeReferenceData). This stops extra product and size records being saved back to the database. I've made all my constructors, getters and setters public to make it easier to replicate my problem. My demo code was created on a console app targeting .net framework 4.5.2. The version of Entity framework is 6.2.
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
using System.Linq;
using static Demo.Constants;
namespace Demo
{
public static class Constants
{
public static int BasketId => 1;
public static int SmallId => 1;
public static int LargeId => 2;
public static int FooId => 1;
public static int BarId => 2;
}
public class Program
{
public static void Main()
{
using (var context = new AppContext())
{
var customerBasket = context.Baskets
.Include(b => b.Items.Select(cbi => cbi.Product))
.Include(b => b.Items.Select(cbi => cbi.Size))
.SingleOrDefault(b => b.Id == BasketId);
var size = context.Sizes.AsNoTracking()
.SingleOrDefault(s => s.Id == SmallId);
context.Configuration.ProxyCreationEnabled = false;
var product = context
.Products
.AsNoTracking()
.Include(p => p.Sizes)
.SingleOrDefault(p => p.Id == BarId);
//changing BarId to FooId in the above line results in
//null reference exception when savechanges is called.
customerBasket.AddItem(product, size);
context.SaveChanges(excludeReferenceData: true);
}
Console.ReadLine();
}
}
public class Basket
{
public int Id { get; set; }
public virtual ICollection<Item> Items { get; set; }
public Basket()
{
Items = new Collection<Item>();
}
public void AddItem(Product product, Size size)
{
if (itemAlreadyExists(product, size))
{
throw new InvalidOperationException("item already in basket");
}
var newBasketItem = Item.Create(
this,
product,
size);
Items.Add(newBasketItem);
}
private bool itemAlreadyExists(Product product, Size size)
{
return Items.Any(a => a.ProductId == product.Id && a.SizeId == size.Id);
}
}
public class Item
{
public Guid Id { get; set; }
public int BasketId { get; set; }
public virtual Product Product { get; set; }
public int ProductId { get; set; }
public virtual Size Size { get; set; }
public int SizeId { get; set; }
public Item()
{
}
public string getDescription()
{
return $"{Product.Name} - {Size.Name}";
}
internal static Item Create(Basket basket
, Product product,
Size size)
{
Guid id = Guid.NewGuid();
if (!product.HasSize(size))
{
throw new InvalidOperationException("product does not come in size");
}
var basketItem = new Item
{
Id = id,
BasketId = basket.Id,
Product = product,
ProductId = product.Id,
Size = size,
SizeId = size.Id
};
return basketItem;
}
}
public class Product : IReferenceObject
{
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<ProductSize> Sizes { get; set; }
public Product()
{
Sizes = new Collection<ProductSize>();
}
public bool HasSize(Size size)
{
return Sizes.Any(s => s.SizeId == size.Id);
}
}
public class ProductSize : IReferenceObject
{
public int SizeId { get; set; }
public virtual Size Size { get; set; }
public int ProductId { get; set; }
}
public class Size : IReferenceObject
{
public int Id { get; set; }
public string Name { get; set; }
}
public class AppContext : DbContext
{
public DbSet<Basket> Baskets { get; set; }
public DbSet<Product> Products { get; set; }
public DbSet<Size> Sizes { get; set; }
public AppContext()
: base("name=DefaultConnection")
{
Database.SetInitializer(new DBInitializer());
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Basket>()
.HasMany(c => c.Items)
.WithRequired()
.HasForeignKey(c => c.BasketId)
.WillCascadeOnDelete(true);
modelBuilder.Entity<Item>()
.Property(c => c.Id)
.HasDatabaseGeneratedOption(DatabaseGeneratedOption.None);
modelBuilder.Entity<Item>()
.HasKey(c => new { c.Id, c.BasketId });
modelBuilder.Entity<ProductSize>()
.HasKey(c => new { c.ProductId, c.SizeId });
base.OnModelCreating(modelBuilder);
}
public int SaveChanges(bool excludeReferenceData)
{
if(excludeReferenceData)
{
var referenceEntries =
ChangeTracker.Entries<IReferenceObject>()
.Where(e => e.State != EntityState.Unchanged
&& e.State != EntityState.Detached);
foreach (var entry in referenceEntries)
{
entry.State = EntityState.Detached;
}
}
return SaveChanges();
}
}
public interface IReferenceObject
{
}
public class DBInitializer: DropCreateDatabaseAlways<AppContext>
{
protected override void Seed(AppContext context)
{
context.Sizes.Add(new Size { Id = LargeId, Name = "Large" });
context.Sizes.Add(new Size { Id = SmallId, Name = "Small" });
context.Products.Add(
new Product
{
Id = FooId,
Name = "Foo",
Sizes = new Collection<ProductSize>()
{
new ProductSize{ProductId = FooId, SizeId = LargeId},
new ProductSize{ProductId = FooId, SizeId =SmallId}
}
});
context.Products.Add(new Product { Id = BarId, Name = "Bar",
Sizes = new Collection<ProductSize>()
{
new ProductSize{ProductId = BarId, SizeId = SmallId}
}
});
context.Baskets.Add(new Basket
{
Id = BasketId,
Items = new Collection<Item>()
{
new Item
{
Id = Guid.NewGuid(),
BasketId =BasketId,
ProductId = FooId,
SizeId = LargeId
}
}
});
base.Seed(context);
}
}
}
When you use AsNoTracking, this tells EF to not include the objects being loaded into the DbContext ChangeTracker. You normally want to do this when you load data to be returned and know you aren't going to want to save it back at that point. Thus, I think you just need to get rid of AsNoTracking on all of your calls and it should work fine.
I have a table called InvestigatorGroup and a table called InvestigatorGroupUsers which is used to see what groups have what users. I am trying to get the common investigator group between two users
My query is as follows:
public InvestigatorGroup GetCommonGroup(string userId, string investigatorUserId)
{
using (GameDbContext entityContext = new GameDbContext())
{
string[] ids = new[] { userId, investigatorUserId };
return entityContext.InvestigatorGroups
.Where(i => i.IsTrashed == false)
.Include(i => i.InvestigatorGroupUsers)
.Where(i => i.InvestigatorGroupUsers.Any(e => ids.Contains(e.UserId)))
.OrderByDescending(i => i.InvestigatorGroupId)
.GroupBy(i => i.InvestigatorGroupId)
.Where(i => i.Count() > 1)
.SelectMany(group => group).FirstOrDefault();
}
}
The entity InvestigatorGroup is as follows:
public class InvestigatorGroup : IIdentifiableEntity
{
public InvestigatorGroup()
{
this.InvestigatorGroupGames = new HashSet<InvestigatorGroupGame>();
this.InvestigatorGroupUsers = new HashSet<InvestigatorGroupUser>();
}
// Primary key
public int InvestigatorGroupId { get; set; }
public string InvestigatorGroupName { get; set; }
public bool HasGameAssignment { get; set; }
public string GroupRoleName { get; set; }
public bool IsTrashed { get; set; }
// Navigation property
public virtual ICollection<InvestigatorGroupUser> InvestigatorGroupUsers { get; private set; }
public virtual ICollection<InvestigatorGroupGame> InvestigatorGroupGames { get; private set; }
public int EntityId
{
get { return InvestigatorGroupId; }
set { InvestigatorGroupId = value; }
}
}
The problem is that it keeps returning a value of 0. It doesn't see the shared group with a count of 2 between the two users.
I did a test to return the groups (I removed the count>1 condition) and it returned all the groups for both users not only the one they have in common
I believe the issue is with this line: .Where(i => i.InvestigatorGroupUsers.Any(e => ids.Contains(e.UserId)))
Thanks for the help!
I've resolved this by changing my query so that it searches for the rows containing one of the UserId's. Then it queries through those selected rows and selects the ones containing the other UserId (InvestigatorUserId). This way only the rows containing both are returned
My new code is as follows:
public InvestigatorGroup GetCommonGroup(string userId, string investigatorUserId)
{
using (GameDbContext entityContext = new GameDbContext())
{
IEnumerable<InvestigatorGroup> userGroups = entityContext.InvestigatorGroups
.Where(i => i.IsTrashed == false)
.Include(i => i.InvestigatorGroupUsers)
.Where(i => i.InvestigatorGroupUsers.Any(e => e.UserId.Contains(userId)))
.OrderByDescending(i => i.InvestigatorGroupId);
return userGroups.Where(i => i.InvestigatorGroupUsers.Any(e => e.UserId.Contains(investigatorUserId))).FirstOrDefault();
}
}
I'm trying to update related database on many to many relationship using ADO.net
this is my database design:
as you guys notice, entity framework wont mapping the class_student & subject_course, i've been searching the method and found this website: http://www.asp.net/mvc/tutorials/getting-started-with-ef-using-mvc/updating-related-data-with-the-entity-framework-in-an-asp-net-mvc-application
the website told me to make a viewModel, and i do so:
namespace Test.Models.ViewModels
{
public class AssignedStudentData
{
public int ID { get; set; }
public string course_code { get; set; }
public bool Assigned { get; set; }
}
}
It's work flawlessly, but my problem is this line of code:
private void PopulateAssignedStudentData(ms_class ms_class)
{
var allStudent = db.ms_student; //this line is the problem
var ClassStudent = new HashSet<int>(ms_class.ms_student.Select(c => c.ID));
var viewModel = new List<AssignedStudentData>();
foreach (var student in allStudent)
{
viewModel.Add(new AssignedStudentData
{
ID = student.ID,
course_code = student.ms_course.course_name,
Assigned = ClassStudent.Contains(student.ID)
});
}
ViewBag.Students = viewModel;
}
in var allStudent, i've tried to make so the system not generate all the student, but instead, student THAT ASSIGNED WITH A SUBJECT so for example:
private void PopulateAssignedStudentDataBySubject(ms_class ms_class, int subject_id)
{
//var allStudent = db.ms_student; //this line is the problem
//My Version:
var allStudentByCourse = db.ms_student.Include(m => m.ms_course).Where(m => m.ms_course.ms_subject.subject_id == subject_id); //this code is not working
var ClassStudent = new HashSet<int>(ms_class.ms_student.Select(c => c.ID));
var viewModel = new List<AssignedStudentData>();
foreach (var student in allStudentByCourse )
{
viewModel.Add(new AssignedStudentData
{
ID = student.ID,
course_code = student.ms_course.course_name,
Assigned = ClassStudent.Contains(student.ID)
});
}
ViewBag.Students = viewModel;
}
i think the code won't work because the ms_course and ms_subject is a many-to-many relationship..
Thank you very much
Class
public partial class ms_course
{
public ms_course()
{
this.ms_student = new HashSet<ms_student>();
this.ms_subject = new HashSet<ms_subject>();
}
public int course_id { get; set; }
public string course_code { get; set; }
public string course_name { get; set; }
public virtual ICollection<ms_student> ms_student { get; set; }
public virtual ICollection<ms_subject> ms_subject { get; set; }
}
I understand that you're looking for students having a course that has at least one specific subject assigned to it. That would be:
db.ms_student
.Where(s => s.ms_course.ms_subject
.Any(sb => sb.subject_id == subject_id)))
It always helps me to articulate the problem clearly in terms of the object model first, as I did in the first sentence. It usually reveals what the query should look like.
What does the error message say?
You can try tis:
var allStudentByCourse = db.ms_student.Include(m => m.ms_course).Include("ms_course.ms_subject").Where(m => m.ms_course.ms_subject.subject_id == subject_id);
alternativ2 (this only works if ms_course has a fk property to ms_subject):
var allStudentByCourse = db.ms_student.Include(m => m.ms_course).Where(m => m.ms_course.subject_id == subject_id);
Update:
var allStudentByCourse = db.ms_student.Include(m => m.ms_course).Include("ms_course.ms_subject").Where(m => m.ms_course.ms_subject.Any(s => s.subject_id == subject_id));