I am creating an OData endpoint using the NuGet Microsoft.AspNetCore.OData within my ASP.Net Core website.
I had the idea to not make my database models public and map them using the LINQ Select function, to have more control about which properties are able to be shown to the outside, such as who created what.
In the future some more logic will be added to the IEnumerable because of permissions granted on the data.
Everything is working fine; I am able to get, filter and expand data on my ODataController. Except for one thing I noticed using the debugging options for Entity Framework Core; everything I send to my ODataController is being done in memory and not via an expression to Entity Framework as I expected. It requests everything each time. Of course, as the grows, this will become more and more a problem over time.
Some code resembling what I am doing
public class ExampleModel {
public Guid Id { get; set; }
public string Name { get; set; }
}
public class ExampleDbModel{
public Guid Id { get; set; }
public string Name { get; set; }
}
public class ExampleRepository {
public IEnumerable<ContentItem> Get(string userId)
{
return this._dbContext.Example
.Select(ExampleMapper.Map) // Mapper is pure property to property in this example, same names
.Where(e => e.UserID == userId); // Permissions will be more complex
}
}
public class ExampleController : ODataController
{
// Constructor, properties, DI, ...
[EnableQuery]
public IActionResult Get()
{
return Ok(_exampleRepository.Get(SomeStaticVariableForUserId));
}
}
When calling the controller I always notice following query being done by Entity Framework Core
SELECT [ExampleDbModel].[Id], [ExampleDbModel].[Name]
FROM ExampleDbModel
Even when I am doing localhost:5050/odata/examplemodel?$select=id, it still performs the same query loading everything.
I am wondering what I am able to do to translate my OData queries straight to Entity Framework Core, to load as less as possible.
Related
Supposedly, I have a simple DbContext with Blog and Post models:
public class Blog
{
public int BlogId { get; set; }
public string Name { get; set; }
public ICollection<Post> Posts { get; set; }
}
public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public int BlogId { get; set; }
public Blog Blog { get; set; }
}
Let's say I have a stored procedure that returns some DTO:
[Keyless]
public class BlogPostDto
{
public string PostTitle { get; init; }
public string BlogName { get; init; }
}
Today I put the following into DbContext:
public class AppDbContext : DbContext {
public virtual DbSet<BlogPostDto> NeverUseIt { get; set; }
partial void OnModelCreatingPartial(ModelBuilder modelBuilder) {
modelBuilder.Entity<BlogPostDto>().ToView(null);
}
}
And then I can get Stored Procedure results shaped in the way I want:
List<BlogPostDto> results = await db.Set<BlogPostDto>().FromSqlRaw($"EXEC MyProc").ToListAsync();
So, my question is, do I have to add my BlogPostDto into DbContext? I know that in EF Core 3 I did; but there were a large number of improvements since then. Creating a bogus DbSet and mapping it to non-existent view just feels counter-intuitive!
The closest I found in most current documentation is here. The very first example of context is Serving as the return type for raw SQL queries. - but the article assumes that I have a matching view already in the database.
UPDATE: It looks like ToView(null) is not necessary - just DbSet<>
Nothing has changed in that regard so far from what you see in the EF Core 6.0 documentation and SO posts you are referring to.
Just to be crystal clear, you don't need a DbSet<T> returning property in your context. But you do need to include the type (keyless or not) in the model using the modelBuilder.Entity<T>() call, and also ToView(null) to prevent EF Core migrations associate database table and/or view with it, and optionally HasNoKey() in case you don;t want to use EF Core dependent attributes like [Keyless] in your data classes.
So the minimum requirement for your example is this line
modelBuilder.Entity<BlogPostDto>().ToView(null);
Now, this is a long time requested feature (which exists in the "obsolete" EF6 which the "modern" EF Core is supposed to replace), tracked by Support raw SQL queries without defining an entity type for the result #10753 issue in EF Core issue tracker. It was initially planned to be included in the upcoming EF Core 7.0 release (Nov 2022), but later has been cut for (eventually) EF Core 8.0 (Nov 2023). So until then you have to use the "register model" approach, or use 3rd party library(!) like Dapper for the same task, as suggested by one of the EF Core team members(?!).
guys.
I have a question about EF Core DbCommandInterceptor.
Let's have a class with 2 fields like this
public class User
{
public Guid Id { get; set; }
public string SameData { get; set; }
}
public class TestClass
{
public Guid Id { get; set; }
public Guid TestClassId { get; set; }
[NotMapperd, MyAttr]
public TestClass TestClass { get; set; }
}
where User and TestClass are both located in the different contexts (for example, UserDbContext, TestDbContext). MyAttr is the marker attribute, nothing more.
So, I want to write an interceptor that raises up each time we try to get info about TestClasses, but after data got it should get an additional data about User with cross-request to UserDbContext (It possible, because I have User Id after the command execution and can use this Id in the request)
I know, that it should be DbCommandInterceptor.ReaderExecuted or DbCommandInterceptor.ReaderExecutedAsync in this case, but I cannot understand how to get information about objects in the result (I can get rows but I cannot understand what should I do, how should I map it). I can use additional libraries in the project if needed (like Dapper and others).
Could anyone helps me to get
Result Type - concrete entity type or entity collection type?
Result as a C# object (POCO or POCO collection)?
Thank you.
I know, that it should be DbCommandInterceptor.ReaderExecuted or DbCommandInterceptor.ReaderExecutedAsync in this case, but I cannot understand how to get information about objects in the result
The EF Command interceptors don't support that. You can replace the DataReader with a different one. But the query was generated to fill a particular object graph, which is not exposed by the interceptor API.
I have my Unit of Measure which users fill in and save, they can then save a list of Unit Sizes which has its own table and is a foreign key to the Unit Of Measure. When I am fetching all the data back, the Unit Size value is coming back blank.
I have read a half dozen ways to do this and I am not comprehending them. The one that makes the most sense to me is using a Queryable extension so I am trying to go that route but my code still hasn't quite gotten there.
Here is where I am at - these are my entities:
namespace Mudman.Data.Entities
{
[Table("UnitOfMeasure")]
public class UnitOfMeasure : IEntityBase, IAuditBase
{
[Key]
[Column("UnitOfMeasureId")]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public string Id { get; set; }
[Required]
[ForeignKey("TenantId")]
public string TenantId { get; set; }
[JsonIgnore]
public virtual Tenant Tenant { get; set; }
public string Name { get; set; }
public virtual IEnumerable<UnitOfMeasureSize> UnitSize { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime CreateDate { get; set; } = DateTime.UtcNow;
[StringLength(255)]
public string CreateUserId { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime UpdateDate { get; set; }
[StringLength(255)]
public string UpdateUserId { get; set; }
}
}
Unit Of Measure size entity:
namespace Mudman.Data.Entities
{
[Table("UnitOfMeasureSize")]
public class UnitOfMeasureSize : IEntityBase, IAuditBase
{
[Key]
[Column("UnitOfMeasureSize")]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public string Id { get; set; }
[Required]
[ForeignKey("TenantId")]
public string TenantId { get; set; }
[JsonIgnore]
public virtual Tenant Tenant { get; set; }
[Required]
[ForeignKey("UnitOfMeasureId")]
public string UnitOfMeasureId { get; set; }
public virtual UnitOfMeasure UnitOfMeasure { get; set; }
[Required]
public int UnitSize { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime CreateDate { get; set; } = DateTime.UtcNow;
[StringLength(255)]
public string CreateUserId { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime UpdateDate { get; set; }
[StringLength(255)]
public string UpdateUserId { get; set; }
}
}
Unit Of Measure Repository including Unit Size:
namespace Mudman.Repository
{
public class UnitOfMeasureRepository : EntityBaseRepository<UnitOfMeasure>,
IUnitOfMeasureRepository
{
MudmanDbContext context;
public UnitOfMeasureRepository(MudmanDbContext context) : base(context)
{
{ this.context = context; };
}
public IEnumerable<UnitOfMeasure> GetAllUnitsOfMeasure(string TenantId)
{
var result = context.UnitOfMeasure
.Where( uom => uom.TenantId == TenantId)
.Include(uom => uom.UnitSize);
return result;
}
}
}
My GetAllAsync method in my service:
public Task<IEnumerable<UnitOfMeasureViewModel>> GetAllAsync()
{
var result = _unitOfMeasureRepo.GetAllUnitsOfMeasure(TenantId);
result.OrderBy(r => r.Name);
return _mapper.Map<List<UnitOfMeasure>, List<UnitOfMeasureViewModel>>(result.ToList());
}
AutoMapper Code:
CreateMap<UnitOfMeasure, UnitOfMeasureViewModel>().ReverseMap()
.ForMember(dest => dest.UnitSize, uos => uos.Ignore())
.ForMember(uom => uom.UnitSize, src => src.MapFrom(uom => uom.UnitSize));
There are a few issues with your attempts so far.
Firstly, your GetAllAsync looks like it wants to be an async method but you have it making entirely synchronous calls, and hasn't been marked as async. I would avoid diving into asynchronous methods until you have the fundamentals of retrieving your data down.
What we cannot see from your example is the mapping between your unit of measure entity and the view model. The entity has a one-to-many relationship between unit of measure and UnitSizes, so what gets updated depends on how the view model is laid out and configured for mapping. This is most likely the root of your problem where your view model mapping from the entity is likely relying on a convention that isn't pairing up with the data you expect.
Performance wise, this approach will run into problems as your data model grows in terms of entities and rows. The fundamental problem with using a repository like this is that a method like this:
IEnumerable<UnitOfMeasure> GetAllUnitsOfMeasure(string TenantId)
will load all data into memory and you explicitly need to include related entities, whether the consumer will want them or not, which adds to the amount of work the queries need to do and the memory required. If TenantId is for something like a multi-tenant database such as in a SaaS application with multiple tenants using a single data source, this is a good reason to adopt a Repository pattern, but I would not pass tenantIds around as parameters. Instead, have the repository accept a dependency that can validate and resolve the current TenantId from the session. This way the repository can always ensure that the current tenant rules are validated and applied for every query without worrying about where the caller might have gotten a TenantId from. (I.e accepting a TenantId from a POST request would be bad as that value could easily be tampered with)
To address performance and probably touch on what you had read about IQueryable extensions, rather than returning IEnumerable<TEntity> from a repository, you can return IQueryable<TEntity>. The advantages here are that you can still have the repository add base filtering rules like the tenantID, and allow the consumer to handle things like sorting and projection.
For example, the repository looks more like:
public class UnitOfMeasureRepository : IUnitOfMeasureRepository
{
private readonly MudmanDbContext _context;
private readonly ICurrentUserLocator _currentUserLocator;
public UnitOfMeasureRepository(MudmanDbContext context, ICurrentUserLocator currentUserLocator )
{
_context = context ?? throw new ArgumentNullException("context");
_currentUserLocator = currentUserLocator ?? throw new ArgumentNullException("currentUserLocator");
}
public IQueryable<UnitOfMeasure> GetUnitsOfMeasure()
{
var tenantId = _currentUserLocator.CurrentUserTenantId; // Checks session for current user and retrieves a tenant ID or throws an exception. (no session, etc.)
var query = _context.UnitOfMeasure
.Where( uom => uom.TenantId == tenantId)
return query;
}
}
The changes to note here is that we do away with the base generic repository class. This was confusing as you were passing the context to a base class then setting a local context instance as well. Generic repositories with EF are a bad code smell as they lead to either very complex code, very poor performing code, or both. There is a CurrentUserLocator with the container can inject which is a simple class that can verify that a user is currently authenticated and can return their Tenant ID. From there we will return an IQueryable<UnitOfMeasure> which has a base filter for the TenantID which will allow our consumers to make up their own minds how they want to consume it. Note that we do not need to use Include for related entities, again the consumers can decide what they need.
Calling the new repository method and projecting your view models looks fairly similar to what you had. It looks like you are using Automapper, rather than using .Map() we can use .ProjectTo() with the IQueryable and Automapper can essentially build a Select() expression to pull back only the data that the view model will need. To use ProjectTo extension method we do need to provide it with the MappingConfiguration that was used to create your mapper and that will tell it how to build the ViewModel. (So rather than having a dependency of type 'Mapper' you will need one for the MapperConfiguration you set up for that mapper.)
public IEnumerable<UnitOfMeasureViewModel> GetAll()
{
var models = _unitOfMeasureRepo.GetUnitsOfMeasure()
.OrderBy(r => r.Name)
.ProjectTo<UnitOfMeasureViewModel>(_mapperConfiguration)
.ToList();
}
What this does is call our repository method to get the IQueryable, which we can then append the ordering we desire, and call ProjectTo to allow Automapper to populate the view models before executing the query with ToList(). When using Select or ProjectTo we don't need to worry about using Include to eager load related data that might be mapped, these methods take care of loading data related entities if/when needed automatically.
Even in cases where we want to use a method like this to update entities with related entities, using IQueryable works there to:
public void IncrementUnitSize(string unitOfMeasureId)
{
var unitOfMeasure = _unitOfMeasureRepo.GetUnitsOfMeasure()
.Include(r => r.UnitSizes)
.Where(r => r.Id == unitOfMeasureId)
.Single();
foreach(var unitSize in unitOfMeasure.UnitSizes)
unitSize.UnitSize += 1;
_context.SaveChanges();
}
Just as an example of fetching related entities as needed, versus having a method that returns IEnumerable and needs to eager load everything just in case some caller might need it.
These methods can very easily be translated into an asyncronous method without touching the repository:
public async Task<IEnumerable<UnitOfMeasureViewModel>> GetAll()
{
var models = await _unitOfMeasureRepo.GetAllUnitsOfMeasure(TenantId)
.OrderBy(r => r.Name)
.ProjectTo<UnitOfMeasureViewModel>(_mapperConfiguration)
.ToListAsync();
}
... and that is all! Just remember that async doesn't make the call faster, if anything it makes it a touch slower. What it does is make the server more responsive by allowing it to move the request handling to a background thread and free the request thread to pick up a new server request. That is great for methods that are going to take a bit of time, or are going to get called very frequently to avoid tying down all of the server request threads leading to timeouts for users waiting for a response from the server. For methods that are very fast and aren't expected to get hammered by a lot of users, async doesn't add a lot of value and you need to ensure every async call is awaited or you can end up with whacky behaviour and exceptions.
I have a function in the repository, GetForms, the purpose of the function is to call a stored procedure and return rows with data. Everything is working fine until now.
Function
public IEnumerable<FormBO> GetForms()
{
var id = "1"
var Query= _context.FormBO.FromSqlRaw("dbo.SP_Core #pin_ID={0}", id)
.AsNoTracking().ToList(); //3K line of sp
return Query;
}
Model
public class FormBO
{
[Key]
public int? ID { get; set; }
public int? secondid { get; set; }
......
}
DbContext
Added this code, so context thinks it is a table in the database and, I don't have to do more stuff
public virtual DbSet<FormBO> FormBO { get; set; }
The problem
Whenever we scaffold the database and the db context, it regenerates all the files and code, so it removes the
public virtual DbSet<FormBO> FormBO { get; set; }
And we have to add this line manually is there any way I can change the logic, so I don't have to add this code (DBset<FormBO>) to DbContext every time a dba updates the database...
What I found
I found that if I change the model to ".Database" and FromSqlRaw to ExecuteSqlRaw, but it is just returning the count as int not a list of rows.
public IEnumerable<FormBO> GetForms()
{
var id = "1"
var Query = _context.Database.ExecuteSqlRaw("dbo.SP_Core #pin_ID={0}", id)
.AsNoTracking().ToList(); //3K line of sp
return Query;
}
If it is possible it automatically add the DBSet to context whenever we update the code which I don't think we will able to do.
or
Get the query result without the dbset model and then I will use foreach loop to add it in FormBO model it is just 10 rows
Since the table doesn't actually exist in the database, the built in scaffolding process won't attempt to create it.
However you could probably replace the IScaffoldingModelFactory service, with an implementation that extends RelationalScaffoldingModelFactory, and use the code-first fluent api to define meta data for tables that don't really exist.
You could probably use this type of approach to define types for all table values in the database. Since EF Core 5 is adding support for table values, maybe they'll do it for you, but I haven't tested that.
public class MyModelFactory : RelationalScaffoldingModelFactory
{
public MyModelFactory(
IOperationReporter reporter,
ICandidateNamingService candidateNamingService,
IPluralizer pluralizer,
ICSharpUtilities cSharpUtilities,
IScaffoldingTypeMapper scaffoldingTypeMapper,
LoggingDefinitions loggingDefinitions)
: base(reporter, candidateNamingService, pluralizer, cSharpUtilities, scaffoldingTypeMapper, loggingDefinitions)
{
}
protected override ModelBuilder VisitDatabaseModel(ModelBuilder modelBuilder, DatabaseModel databaseModel)
{
modelBuilder.Entity<FormBO>(entity =>
{
// ...
});
return base.VisitDatabaseModel(modelBuilder, databaseModel);
}
}
services.AddDbContextPool<ContextType>(o =>
{
o.ReplaceService<IScaffoldingModelFactory, MyModelFactory>();
// ...
});
Of course there's an easy answer too. The scaffolded context is a partial class. Just define your other DbSet in another source file.
I'm getting the following content when I invoke my API. It kind of breaks up in the middle when the tenant entity that member is linked to, will start listing its member entities.
{
"id":"00000000-7357-000b-0001-000000000000",
"tenantId":"00000000-7357-000a-0001-000000000000",
"userName":"user1",
"tenant":{
"id":"00000000-7357-000a-0001-000000000000",
"name":"First Fake Org",
"members":[
I configured the lazy loading like this.
services.AddDbContext<Context>(config => config
.UseLazyLoadingProxies()
.UseSqlServer(Configuration.GetConnectionString("Register")));
How should I change the code so that the lazily loaded entities don't get served? I was hoping that it would simply return an empty list to the client. Should I use a DTO for that purpose and not return from the DB like this? There's talk about not using lazy loading for APIs at all here.
[HttpGet("test1/{username}"), AllowAnonymous]
public IActionResult GetStuff(string userName)
{
Member output;
output = Context.Members
.Include(e => e.Tenant)
.Single(e => e.UserName == userName);
return Ok(output);
}
I'm not sure what to google for and all the hits I got were pointing to the UseLazyLoadingProxies() invokation.
This will probably be somewhat long winded: But here goes.
It sounds like you have Entities which look something like:
public partial class Member
{
public virtual long Id { get; set; }
public virtual List<Tenant> Tenants { get; set; } //tables have fk relationship
}
public partial class Tenant
{
public virtual long Id { get; set; }
public virtual List<Member> Members{ get; set; } //tables have another fk relationship?
}
And then for this method:
[HttpGet("test1/{username}"), AllowAnonymous]
public IActionResult GetStuff(string userName)
{
Member output;
output = Context.Members
.Include(e => e.Tenant)
.Single(e => e.UserName == userName);
return Ok(output);
}
I see a few issues, but I'll try to keep it short:
I wouldn't have the controller do this directly. But it should work.
What I think you over looked is exactly what the .Include statement does. When the object is instantiated, it will get all of those related entities. Includes essentially converts your where statement to a left join, where the foreign keys match (EF calls these navigation properties).
If you don't want the Tenant property, then you can omit the .Include statement. Unless this is meant to be more generic (In which case, an even stronger reason to use a different pattern and auto mapper).
Hopefully your database doesn't truly have a FK relationship both ways, if it does, fix that ASAP.
The next issue is that you might not want a list of child properties, but it is in the model so they will be "there". Although your List Tenants might be null. And while this might be fine to you, right now. As a general rule when I see an API returning a property, I expect something to be either not there (This Member doesn't have tenants) or something is wrong, like perhaps there is a second parameter I missed. This probably isn't a problem 93.284% of the time, but it is something to be mindful of.
This starts to get into why an AutoMapper is great. Your Database Models, Business Models and views are likely different. And as much as you shouldn't return the database models directly. Taking control of how the data is represented for each part of the application is a great idea.
You could reduce the code easily, and remove the navitation properties:
[HttpGet("test1/{username}"), AllowAnonymous]
public IActionResult GetStuff(string userName)
{
return Ok(Context.Members
.Include(e => e.Tenant)
.Single(e => e.UserName == userName));
}
But again, a business layer would be better:
[HttpGet("test1/{username}"), AllowAnonymous]
public IActionResult GetStuff(string userName)
{
return Ok(MemberRepository.GetMember(userName));
}
The main point I'd stress though, is creating a view model.
For example, Let's say a user Detail:
public class MemberDetail
{
public string UserName {get; set;}
public long UserId { get; set; }
public string FullName { get; set; }
}
This way the view always receives exactly what you want to see, and not the extra data. Add this with the fact that you can know exactly how every use of Member to MemberDetail will map.