We Keep Coding

Problems with Generic Repository Pattern and Entity Framework

There are a large number of examples on how to use the Generic Repository Pattern over Entitry Framework but none ever go into great detail. More often than not the example they use for accessing data from a webapi is a one to one mapping to the model.
For Eg. Get a single model object _carRepository<Car>.Get(id); or get all cars _carRepository<Car>.GetAll();
The way I have it implemented is in seperate layers like below:
Controller -> Service Layer -> Repository Layer -> DataAccess
I have my service layer using AutoMapper where I map from the model(entity) to a DTO and that gets returned to the controller who returns that.
The issue that I cant get my head around is how do I return an object from the repository layer which isnt of type < T>.
Eg. I want to return some data for a grid lets say a combination of multipe entities like a customers last number of purchases. Am I supposed to pass the Customer, Product, Orders repository into the service and make multiple calls to the database and aggregate? Or I was thinking in the cutomers repository just make a method which returns like a view dto of sorts already aggreating the data where it does a linq query and projects to this dto. The issue I have is that I'm sure my repository shouldnt know about dto's?
Would appreciate anyone who has thoughts on the correct way of doing this?

In short, I don't ever recommend using the Generic Repository (anti)pattern. The trouble is that it ends up being far too restrictive and inefficient. Methods like GetAll() serve only to end up materializing entire tables into memory. They don't accommodate things like projection (essentially what you are looking for) as well as eager loading related data, filtering, sorting, or things like supporting pagination.
The way I recommend looking at repositories is as a one-to-one supplier of data for a Controller in the MVC pattern. If you have a CarController you have a CarRepository to serve all data for that controller. If you break it down to CarController and AddCarController then similarly you can build repositories to serve each of these. These repositories are not Generic in that they don't just serve Car entities, but any and all entities from the DbContext that this controller (or service) needs rather than trying to marry repositories to a specific entity. This gives them only one concern for their existance, so a CarController only needs to worry about primarily one repository, and that repository only has to worry about serving that controller. (Rather than every controller that might want information about a Car)
Regarding using a Service between the controller and repository, I would only suggest this if there is a distinct requirement to provide that separation. For example, if you want to support both an MVC controller and a public facing WebAPI and you want the inputs and outputs provided to these to be 100% consistent. The data transported from Service to Consumer (Controllers, etc.) would be DTOs. This adds the complexity of needing to send details like sorting, pagination, and filtering etc. from the consumer to the service.
Removing the service can make interacting with the repository a lot easier, and you can leverage projection to populate the DTOs/view models actually sent to the view via the controller actions/endpoints. The way I facilitate this is leveraging IQueryable<TEntity> in the repository. This lets the repository handle low level filtering/rules if necessary and lets the consumer (controller) handle determining how it wants to consume the data.
For example if we have a CarController with a CarRepository and CarRepository has a method:
public IQueryable<Car> GetCars(bool includeInactive = false)
{
var query = _context.Cars.AsQueryable();
if (!includeInactive)
query = query.Where(x => x.IsActive);
return query;
}
When it comes time for the controller to request cars from the repository, it has full control over how to consume it:
var cars = await CarRepository.GetCars()
.Where(x => x.Make == make && x.Model == model)
.OrderBy(x => x.ModelYear)
.Select(x => new CarSummary
{
Make = x.Make,
Model = x.Model,
ModelYear = x.ModelYear,
Color = x.Color,
Features = x.Features.Select(...)
}.Skip(pageSize * (pageNumber-1))
.Take(pageSize)
.ToListAsync();
The controller has full control over how the data should come back including whether to run an async query or synchronous one all without adding any complexity to the repository. So here we can project (Select, or leverage Automapper and ProjectTo) to get whatever data we need from the entities and their related data. This leads to building far faster and memory/network efficient queries because the projections only worry about the data the end consumer actually needs. When we want to load an entity and its relations to perform an update, we can fetch the entity and eager load the related data we need to inspect/validate/update. Eager loading data is faster than lazy loading, but both use a fair bit of memory so we don't want to be serializing and transmitting entire object graphs. However, when doing an update we are typically only dealing with a single top-level object at a time.
The question then becomes "Why use a repository then?" The repository provides a nice abstraction for unit testing, and it can help standardize core rules you want in the system such as Active/Inactive (soft delete) and things like authorization in systems using multi-tenancy or otherwise having distinct access controls for the current user as to what data they should ever be able to see. If you don't have either of these requirements, then there isn't really much point to using a repository at all. The DbContext and it's DbSets essentially serve that role.
If you do have plans for multiple consumers, then the repository remains the same but the above consuming code happens in the Service, returning the DTO/ViewModel. This means that your controller will need to package and transmit standardized requests to the service or parameters covering whether it expects any sorting, pagination, etc. Again, I wouldn't recommend taking on that overhead and complexity unless there is a very real requirement justifying it. It just serves to make the code harder to work with.

This may not be complete answer but few things while working with Generic Repository.
Assumption I made that your generic repository return type of T and T is the entity.
Following is not exact interface but based on your question.
public class IRepository<T>
{
T Get(int Id);
IEnumrable<T> GetAll();
}
So most of the scenario above will suffice the thing.
Now let's assume that you have requirement that will only return specific attribute of T or partial of T.
Generic repository implementation never say that you don't need any extra or extended repository.
So you can do like this.
public interface ICarRepository : IRepository<Car>
{
IList<int> GetIds();
IList<string> GetAllModels();
bool IsActiveInProduction(int carId);
}
Implement above just like you have implemented IRepository for EF core. Inject this in your services and Use ICarRepository instead of IRepository
for your development.
Note: This is personal preference but for large project GenericRepository become issue sometime or may not completely fit but Repository is useful pattern and it overall depends on how you use it.
Update 1
In Car Repository if you want to return some other DTO but make sure it related to Car Entity otherwise it is not reponsibility of Repo layer.
public interface ICarRepository : IRepository<Car>
{
IList<int> GetIds();
IList<string> GetAllModels();
bool IsActiveInProduction(int carId);
MyDto GetMyDto(int carId); // MyDto is just for explanation.
}

Below is the BaseRespository.cs code that is use that I use as Generic class for entity framework. I inherit this class as parent with my other RepositoryClass.
public class BaseRepository
{
#region Ctor(s)
protected RuhBotEntities RuhBotEntities;
public BaseRepository(RuhBotEntities entities)
{
this.RuhBotEntities = entities;
}
#endregion
#region Protected & Private Members
protected async Task<T> Get<T>(Guid id) where T : class
{
return await this.RuhBotEntities.Set<T>().FindAsync(id);
}
protected IQueryable<T> GetAll<T>(out int totalCount ,bool isDeleted, string searchField = null, string searchTerm = null,
int? pageIndex = null, int? pageSize = null, string sortBy = null, SortOrder? sortOrder = null) where T : class
{
IQueryable<T> entities = this.RuhBotEntities.Set<T>();
var sortByProperty = typeof(T).GetProperty(sortBy);
if (!string.IsNullOrWhiteSpace(sortBy) && sortOrder.HasValue && sortByProperty != null)
{
switch (sortByProperty.PropertyType.FullName)
{
case "System.Int32":
entities = this.RuhBotEntities.Set<T>().Sort<T, int>(sortBy, sortOrder);
break;
case "System.String":
entities = this.RuhBotEntities.Set<T>().Sort<T, string>(sortBy, sortOrder);
break;
}
}
PropertyInfo[] entityProperties = typeof(T).GetProperties();
var propertyList = entityProperties.Where(p => p.PropertyType.FullName == "System.String").Select(p => p.Name).ToList();
if (!string.IsNullOrWhiteSpace(searchTerm) && !string.IsNullOrWhiteSpace(searchField))
{
entities = entities.Where(string.Format("{0}.Contains(#0)", searchField), searchTerm);
}
else if (!string.IsNullOrWhiteSpace(searchTerm) && propertyList.Count > 0)
{
var searchFieldList = new List<string>();
propertyList.ForEach(p => searchFieldList.Add(string.Format("{0}.Contains(#0)", p)));
var propertySearchField = String.Join("||", searchFieldList);
entities = entities.Where(propertySearchField, searchTerm);
}
if (isDeleted)
{
entities = entities.Where(string.Format("{0}.Equals(#0)", "IsDeleted"), false);
}
totalCount = entities.Count();
if (pageIndex.HasValue && pageSize.HasValue)
{
entities = entities.OrderBy(sortBy).Skip(pageIndex.Value * pageSize.Value).Take(pageSize.Value);
}
return entities;
}
protected void Add<T>(T entity) where T : class
{
this.RuhBotEntities.Set<T>().Add(entity);
}
public async Task<T> AddIfNotExists<T>(T entity, Expression<Func<T, bool>> predicate = null) where T : class, new()
{
var dbSet = this.RuhBotEntities.Set<T>();
var exists = predicate != null ? dbSet.Any(predicate) : dbSet.Any();
return exists ? await dbSet.FirstOrDefaultAsync(predicate) : dbSet.Add(entity);
}
protected void Update<T>(T entity, params string[] fieldsExcludedForUpdation) where T : class
{
var dbEntityEntry = this.AttachEntity(entity);
var createdOn = dbEntityEntry.Entity.GetType().GetProperty("CreatedOn");
var createdBy = dbEntityEntry.Entity.GetType().GetProperty("CreatedBy");
if (createdOn != null)
{
dbEntityEntry.Property("CreatedOn").IsModified = false;
}
if (createdBy != null)
{
dbEntityEntry.Property("CreatedBy").IsModified = false;
}
if (fieldsExcludedForUpdation != null && fieldsExcludedForUpdation.Length > 0)
{
foreach (var field in fieldsExcludedForUpdation)
{
dbEntityEntry.Property(field).IsModified = false;
}
}
dbEntityEntry.State = EntityState.Modified;
}
protected void Delete<T>(Guid id) where T : class
{
var entity = this.RuhBotEntities.Set<T>().Find(id);
this.RuhBotEntities.Set<T>().Remove(entity);
}
protected async Task SaveChanges()
{
await this.RuhBotEntities.SaveChangesAsync();
}
private DbEntityEntry AttachEntity(object entity)
{
var entry = this.RuhBotEntities.Entry(entity);
if (entry.State == EntityState.Detached)
{
var set = this.RuhBotEntities.Set(entity.GetType());
object attachedEntity = set.Find(entity.GetType().GetProperty("Id").GetValue(entity));
if (attachedEntity != null)
{
var attachedEntry = this.RuhBotEntities.Entry(attachedEntity);
attachedEntry.CurrentValues.SetValues(entity);
return attachedEntry;
}
else
{
entry.State = EntityState.Modified;
}
}
else
{
this.RuhBotEntities.Set(entity.GetType()).Attach(entity);
this.RuhBotEntities.Entry(entity).State = EntityState.Modified;
}
return entry;
}
#endregion
}

Problem with getting inserted Id from database

I am trying to get the last inserted Id from database via Entity Framework, however my problem is somehow unique and I am not able to find any solution, but to rewrite my whole infrastructure and business layer, so all my Ids are Guids, which I am able to create manually, or get last record with another database call after commit.
Here is the problem. I have a three-layered architecture, where I am using UoW, repository, services and facades. I will show my code from top to bottom, so you can understand.
Here is my facade, where uow.Commit is calling SaveChanges()
public async Task<int> RegisterUserAsync(UserCreateDto user)
{
using (var uow = UnitOfWorkProvider.Create())
{
var id = _userService.Create(user);
await uow.Commit();
return id;
}
}
As you can see I am sending only my DTO into the service, where I process is like this, also I am mapping inside service
public virtual int Create(TCreateDto entityDto)
{
var entity = Mapper.Map<TEntity>(entityDto);
Repository.Create(entity);
return entity.Id;
}
and finally my repository looks like this
public TKey Create(TEntity entity)
{
Context.Set<TEntity>().Add(entity);
return entity.Id;
}
Is there some elegant solution to this? Like I said my only idea is to switch all Ids to Guid or second call for Id after commit, which I find as not very good solution, because when I want to connect two or more tables in one transaction it would be impossible.

EF Core solution of the problem is simple - auto generated PK of an entity instance is available after calling SaveChanges[Async]. e.g.
var entity = Mapper.Map<TEntity>(entityDto);
Context.Add(entity);
// Here entity.Id contains auto-generated temporary value
// Other stuff...
Context.SaveChanges();
// Here entity.Id contains actual auto-generated value from the db
So the problem is more in the design of your infrastructure - all these (unnecessary) UoW, repository, services and facades simply hide that functionality.
The only relatively simple and elegant solution I see with your architecture is to change the service return type from int to Func<int>, e.g.
public virtual Func<int> Create(TCreateDto entityDto)
{
var entity = Mapper.Map<TEntity>(entityDto);
Repository.Create(entity);
return () => entity.Id; // <--
}
Then in your façade you could use
public async Task<int> RegisterUserAsync(UserCreateDto user)
{
using (var uow = UnitOfWorkProvider.Create())
{
var id = _userService.Create(user);
await uow.Commit();
return id(); // <-- the actual id!
}
}
Edit:
Actually EF Core provides another option which would allow you to keep your current design intact - the HiLo key generation strategy, but only if the database provider supports it. I can say for sure Microsoft.EntityFrameworkCore.SqlServer and Npgsql.EntityFrameworkCore.PostgreSQL do support it with respectively ForSqlServerUseSequenceHiLo and ForNpgsqlUseSequenceHiLo fluent APIs.

nested Owned Type not saved when Updating in the database

I have following classes:
Order:
public class Order {
private Order()
{
//some code to initialise object
}
//more properties
public Prepayment Prepayment { get; private set; }
//more methods and properties
}
Prepayment:
public class Prepayment:ValueObject<Prepayment>
{
private Prepayment()
{
}
public Money AmountPrepaid { get; private set; }
public bool HasPrepaymentBeenTaken => AmountPrepaid.Amount > 0;
}
Money:
public class Money {
private Money()
{
}
private Money(decimal amount)
: this()
{
Amount = amount;
}
public decimal Amount { get; private set; }
}
Then I the Order class is mapped to the database in following way:
modelBuilder.Entity<Order>()
.OwnsOne(x => x.Prepayment,
prepayment =>
{
prepayment.OwnsOne(x => x.AmountPrepaid,
amountPrepaid =>
{
amountPrepaid.Property(x => x.Amount)
.HasColumnName("PrepaymentAmount")
.HasColumnType("decimal(7,2)");
});
});
Repository code to SaveChanges:
public async Task<int> SaveAsync(Order order)
{
if (order.Id == 0)
{
await _context.AddAsync(order);
}
else
{
_context.Update(order);
}
return await _context.SaveChangesAsync();
}
Please understand, I removed all the not important properties from the code, to make the example more clear.
Above code works well for INSERT scenario, where Prepayment -> Money -> Amount is properly saved into the database. UPDATE though doesn't seem to be reflected in the database.
Please note, I have quite a few Owned Types in that model, and all of them are working well. The only difference as far as I can say is the fact that Prepayment property has another nested Owned type - Money.
Order class object passed to repository, is first pulled from the database, then changes are applied on that instance, and finally saved back to the database. Other properties like Customer not mentioned in the example, is also a OwnedType and UPDATE works as expected.
Just in case - the code used to retrieve the object prior to update:
public async Task<Order> GetOrderByIdAsync(int orderId)
{
var result = (from order in _context.Orders
where order.Id == orderId
select order).Include(x => x.OrderLines);
return await result.FirstOrDefaultAsync();
}
The exact version of Entity Framework Core I am using is: 2.2.0
Any help would be highly appreciated.
Thank you.
EDIT:
The code that updates the data looks like this:
public async Task<int> Handle(EditOrderCommand request, CancellationToken cancellationToken)
{
var order = await _orderRepository.GetOrderByIdAsync(request.Id);
var customer = new Customer(
request.FirstName,
request.LastName,
request.TelephoneNumber);
var prepayment = new Prepayment(
Money.SomeMoney(
request.PrepaymentAmount
)
);
order.ApplyChanges(
request.UserId,
request.AdditionalInformation,
collectionDate,
customer,
prepayment);
await _orderRepository.SaveAsync(order);
return order.Id;
}
And part of the ApplyChanges method that sets prepayment:
private void SetPrepayment(Prepayment prepayment)
{
Prepayment = prepayment ?? throw new ArgumentNullException(nameof(prepayment));
}

The issue has something in common with the nested owned entity type.
But the general problem is that the way you are using owned entity types is violating the EF Core rules, thus the behavior is undefined - sometimes it might work, sometimes not, sometimes even throw exceptions etc.
Owned entity types cannot be used to implement value objects, because even they are "owned", by EF Core terminology they are still "entities" (with hidden shadow PK), so they are tracked by reference and follow the same rules as other entity references -most importantly, there must be only one object instance per defining navigation PK.
In short, they are supposed to be allocated once and then mutated via their primitive properties. While what are you doing is mutating the references with immutable objects.
It's hard to give you good advice because of the aforementioned EF Core rule violations. The only working workaround is to make sure all original object references are not tracked by the context.
For instance, if GetOrderByIdAsync implementation uses AsNoTracking query, hence neither order nor order.Prepayment and order.Prepayment.AmountPrepaid instances are tracked by the _context, then _context.Update(order) will work.
It will also work if you manually detach them before calling ApplyChanges (requires access to the db context):
_context.Entry(order).State = EntityState.Detached;
_context.Entry(order.Prepayment).State = EntityState.Detached;
_context.Entry(order.Prepayment.AmountPrepaid).State = EntityState.Detached;
order.ApplyChanges(...);
await _orderRepository.SaveAsync(order); // _context.Update(order);
Looks like AsNoTracking is the better option. You can make it default for all queries by setting ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking; inside your db context constructor and use AsTracking() where needed.

Web API 2: Updating Navigational Properites (Entities)

I am looking for some guidance with Web API functionality that was a breeze in MVC in regards to updating entities that have navigational properties.
In MVC, it was accomplished as:
[AcceptVerbs(HttpVerbs.Post)]
[ValidateAntiForgeryToken]
public virtual async Task<ActionResult> Update(Page page)
{
Guard.IsNotNull(page, "page");
var pageToUpdate = await this.repository.Query.Include(p => p.Tags).Include(p => p.Name).SingleOrDefaultAsync(p => p.Pk == page.Pk);
if (pageToUpdate == null)
{
return this.RedirectToRoute(H.Constants.Routes.Error.Index, new
{
view = H.Constants.Views.Error.ViewPages.NotFound
});
}
if (this.TryUpdateModel(pageToUpdate))
{
this.repository.BeginTransaction();
this.repository.Update(pageToUpdate); // Updates related entities!
await this.repository.CommitTransactionAsync();
return this.RedirectToRoute(H.Constants.Routes.Data.Read);
}
return this.View(H.Constants.Views.FolderNames.ViewPages.FormatWith(H.Constants.Views.Data.ViewPages.Update), pageToUpdate);
}
All navigational properties would be updated, and life was well.
When attempting this exact thing in Web API, not so much. The related entities are not updated. Example:
[HttpPatch]
[HttpPut]
public virtual async Task<IHttpActionResult> Update(int pk, Page page)
{
Guard.IsNotNegativeOrZero(pk, "pk");
if (this.ModelState.IsValid)
{
if (page.Pk == pk)
{
try
{
this.repository.BeginTransaction();
this.repository.Update(page); // Doesn't update related entities.
await this.repository.CommitTransactionAsync();
return this.StatusCode(HttpStatusCode.NoContent);
}
catch (DbUpdateConcurrencyException dbUpdateConcurrencyException)
{
if (this.repository.Query.Any(p => p.Pk == pk))
{
return this.InternalServerError(dbUpdateConcurrencyException);
}
return this.NotFound();
}
}
return this.BadRequest();
}
return this.BadRequest(this.ModelState);
}
Is this possible to do today in Web API? Or is Web API currently an incomplete Microsoft product?
Edit: Updated with example and not referencing WCF

TryUpdateModel doesn't exist in WebAPI. You may also find you have issues with seralization of entities. For both those reasons I use AutoMapper to map ViewModels onto EF entities and you can use Mappings in automapper to deal with your navigation properites.
The web API method looks something like this:
public HttpResponseMessage Post(MyViewModel model)
{
if (someValidationCheckHere)
return new HttpResponseMessage(HttpStatusCode.BadRequest);
_myService.Update(Mapper.Map<MyViewModel, MyEntity>(model));
return new HttpResponseMessage(HttpStatusCode.OK);
}
As others have said the service and ultimately the repository does the update. In this example I'm ignoring the navigation property using automapper, but you can configure automapper to handle them how you like:
Mapper.CreateMap<MyViewModel, MyEntity>()
.ForMember(x => x.NavigationProperty, opt => opt.Ignore());
I set up all my mappings in a static class called on application_start in global.asax.

Best way to update a LINQ model coming from a form

I'm designing forms in my ASP.NET MVC application that will receive objects. Here's what a typical edit action looks like:
[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Edit(int id, Person person)
{
peopleService.SavePerson(person);
return Redirect("~/People/Index");
}
The SavePerson call in the service does this:
public void SavePerson(Person person)
{
_peopleRepository.SavePerson(person);
}
And the SavePerson call in the repository does this:
public void SavePerson(Person person)
{
using (var dc = new SIGAPDataContext())
{
if (person.IsNew)
{
dc.People.InsertOnSubmit(person);
}
else
{
dc.People.Attach(person, true);
}
dc.SubmitChanges();
}
}
Now, this works well when I create the new record. However, when I update it without all the form elements, it nulls other fields. For instance, my Person model has a NationalityID property that is nulled if it doesn't show on the edit form.
What's the best practice to make the model update with just the fields that come from the form? Do I have to get the record from the database first and update it's properties manually like:
Person persistedPerson = peopleService.GetPerson(person.ID);
persistedPerson.Name = person.Name;
persistedPerson.DateOfBirth = person.DateOfBirth
// etc...
Or is there any other, cleaner way to do this?

Stephen Walther just posted an article describing this very thing in ASP.NET MVC. I would recommend you actually create your own set of DTO's and business objects on top of LINQ to SQL and treat LINQ to SQL as a language-integrated object database due to the gross complexities introduced by managing the DataContext, which according to Pro LINQ, should be kept alive as little as possible by design.
If you are going to use your LINQ to SQL entities as your DTO's, you should get the entity first, detach it, update it, re-attach it, then submit it.

In your controller, retrieve the existing person from the repository, then use UpdateModel/TryUpdateModel and pass in a whitelist of properties that you want to be updated. This would use the ValueProvider on the controller so there is no need to take the Person object in the action parameter list.
public ActionResult Edit( int id )
{
Person person = peopleService.GetPerson(id);
UpdateModel(person,new string[] { list of properties to update } );
peopleService.SavePerson(person);
...
}