We Keep Coding

Using Queryable Extension to get back Foreign Key Data Value

I have my Unit of Measure which users fill in and save, they can then save a list of Unit Sizes which has its own table and is a foreign key to the Unit Of Measure. When I am fetching all the data back, the Unit Size value is coming back blank.
I have read a half dozen ways to do this and I am not comprehending them. The one that makes the most sense to me is using a Queryable extension so I am trying to go that route but my code still hasn't quite gotten there.
Here is where I am at - these are my entities:
namespace Mudman.Data.Entities
{
[Table("UnitOfMeasure")]
public class UnitOfMeasure : IEntityBase, IAuditBase
{
[Key]
[Column("UnitOfMeasureId")]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public string Id { get; set; }
[Required]
[ForeignKey("TenantId")]
public string TenantId { get; set; }
[JsonIgnore]
public virtual Tenant Tenant { get; set; }
public string Name { get; set; }
public virtual IEnumerable<UnitOfMeasureSize> UnitSize { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime CreateDate { get; set; } = DateTime.UtcNow;
[StringLength(255)]
public string CreateUserId { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime UpdateDate { get; set; }
[StringLength(255)]
public string UpdateUserId { get; set; }
}
}
Unit Of Measure size entity:
namespace Mudman.Data.Entities
{
[Table("UnitOfMeasureSize")]
public class UnitOfMeasureSize : IEntityBase, IAuditBase
{
[Key]
[Column("UnitOfMeasureSize")]
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public string Id { get; set; }
[Required]
[ForeignKey("TenantId")]
public string TenantId { get; set; }
[JsonIgnore]
public virtual Tenant Tenant { get; set; }
[Required]
[ForeignKey("UnitOfMeasureId")]
public string UnitOfMeasureId { get; set; }
public virtual UnitOfMeasure UnitOfMeasure { get; set; }
[Required]
public int UnitSize { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime CreateDate { get; set; } = DateTime.UtcNow;
[StringLength(255)]
public string CreateUserId { get; set; }
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public DateTime UpdateDate { get; set; }
[StringLength(255)]
public string UpdateUserId { get; set; }
}
}
Unit Of Measure Repository including Unit Size:
namespace Mudman.Repository
{
public class UnitOfMeasureRepository : EntityBaseRepository<UnitOfMeasure>,
IUnitOfMeasureRepository
{
MudmanDbContext context;
public UnitOfMeasureRepository(MudmanDbContext context) : base(context)
{
{ this.context = context; };
}
public IEnumerable<UnitOfMeasure> GetAllUnitsOfMeasure(string TenantId)
{
var result = context.UnitOfMeasure
.Where( uom => uom.TenantId == TenantId)
.Include(uom => uom.UnitSize);
return result;
}
}
}
My GetAllAsync method in my service:
public Task<IEnumerable<UnitOfMeasureViewModel>> GetAllAsync()
{
var result = _unitOfMeasureRepo.GetAllUnitsOfMeasure(TenantId);
result.OrderBy(r => r.Name);
return _mapper.Map<List<UnitOfMeasure>, List<UnitOfMeasureViewModel>>(result.ToList());
}
AutoMapper Code:
CreateMap<UnitOfMeasure, UnitOfMeasureViewModel>().ReverseMap()
.ForMember(dest => dest.UnitSize, uos => uos.Ignore())
.ForMember(uom => uom.UnitSize, src => src.MapFrom(uom => uom.UnitSize));

There are a few issues with your attempts so far.
Firstly, your GetAllAsync looks like it wants to be an async method but you have it making entirely synchronous calls, and hasn't been marked as async. I would avoid diving into asynchronous methods until you have the fundamentals of retrieving your data down.
What we cannot see from your example is the mapping between your unit of measure entity and the view model. The entity has a one-to-many relationship between unit of measure and UnitSizes, so what gets updated depends on how the view model is laid out and configured for mapping. This is most likely the root of your problem where your view model mapping from the entity is likely relying on a convention that isn't pairing up with the data you expect.
Performance wise, this approach will run into problems as your data model grows in terms of entities and rows. The fundamental problem with using a repository like this is that a method like this:
IEnumerable<UnitOfMeasure> GetAllUnitsOfMeasure(string TenantId)
will load all data into memory and you explicitly need to include related entities, whether the consumer will want them or not, which adds to the amount of work the queries need to do and the memory required. If TenantId is for something like a multi-tenant database such as in a SaaS application with multiple tenants using a single data source, this is a good reason to adopt a Repository pattern, but I would not pass tenantIds around as parameters. Instead, have the repository accept a dependency that can validate and resolve the current TenantId from the session. This way the repository can always ensure that the current tenant rules are validated and applied for every query without worrying about where the caller might have gotten a TenantId from. (I.e accepting a TenantId from a POST request would be bad as that value could easily be tampered with)
To address performance and probably touch on what you had read about IQueryable extensions, rather than returning IEnumerable<TEntity> from a repository, you can return IQueryable<TEntity>. The advantages here are that you can still have the repository add base filtering rules like the tenantID, and allow the consumer to handle things like sorting and projection.
For example, the repository looks more like:
public class UnitOfMeasureRepository : IUnitOfMeasureRepository
{
private readonly MudmanDbContext _context;
private readonly ICurrentUserLocator _currentUserLocator;
public UnitOfMeasureRepository(MudmanDbContext context, ICurrentUserLocator currentUserLocator )
{
_context = context ?? throw new ArgumentNullException("context");
_currentUserLocator = currentUserLocator ?? throw new ArgumentNullException("currentUserLocator");
}
public IQueryable<UnitOfMeasure> GetUnitsOfMeasure()
{
var tenantId = _currentUserLocator.CurrentUserTenantId; // Checks session for current user and retrieves a tenant ID or throws an exception. (no session, etc.)
var query = _context.UnitOfMeasure
.Where( uom => uom.TenantId == tenantId)
return query;
}
}
The changes to note here is that we do away with the base generic repository class. This was confusing as you were passing the context to a base class then setting a local context instance as well. Generic repositories with EF are a bad code smell as they lead to either very complex code, very poor performing code, or both. There is a CurrentUserLocator with the container can inject which is a simple class that can verify that a user is currently authenticated and can return their Tenant ID. From there we will return an IQueryable<UnitOfMeasure> which has a base filter for the TenantID which will allow our consumers to make up their own minds how they want to consume it. Note that we do not need to use Include for related entities, again the consumers can decide what they need.
Calling the new repository method and projecting your view models looks fairly similar to what you had. It looks like you are using Automapper, rather than using .Map() we can use .ProjectTo() with the IQueryable and Automapper can essentially build a Select() expression to pull back only the data that the view model will need. To use ProjectTo extension method we do need to provide it with the MappingConfiguration that was used to create your mapper and that will tell it how to build the ViewModel. (So rather than having a dependency of type 'Mapper' you will need one for the MapperConfiguration you set up for that mapper.)
public IEnumerable<UnitOfMeasureViewModel> GetAll()
{
var models = _unitOfMeasureRepo.GetUnitsOfMeasure()
.OrderBy(r => r.Name)
.ProjectTo<UnitOfMeasureViewModel>(_mapperConfiguration)
.ToList();
}
What this does is call our repository method to get the IQueryable, which we can then append the ordering we desire, and call ProjectTo to allow Automapper to populate the view models before executing the query with ToList(). When using Select or ProjectTo we don't need to worry about using Include to eager load related data that might be mapped, these methods take care of loading data related entities if/when needed automatically.
Even in cases where we want to use a method like this to update entities with related entities, using IQueryable works there to:
public void IncrementUnitSize(string unitOfMeasureId)
{
var unitOfMeasure = _unitOfMeasureRepo.GetUnitsOfMeasure()
.Include(r => r.UnitSizes)
.Where(r => r.Id == unitOfMeasureId)
.Single();
foreach(var unitSize in unitOfMeasure.UnitSizes)
unitSize.UnitSize += 1;
_context.SaveChanges();
}
Just as an example of fetching related entities as needed, versus having a method that returns IEnumerable and needs to eager load everything just in case some caller might need it.
These methods can very easily be translated into an asyncronous method without touching the repository:
public async Task<IEnumerable<UnitOfMeasureViewModel>> GetAll()
{
var models = await _unitOfMeasureRepo.GetAllUnitsOfMeasure(TenantId)
.OrderBy(r => r.Name)
.ProjectTo<UnitOfMeasureViewModel>(_mapperConfiguration)
.ToListAsync();
}
... and that is all! Just remember that async doesn't make the call faster, if anything it makes it a touch slower. What it does is make the server more responsive by allowing it to move the request handling to a background thread and free the request thread to pick up a new server request. That is great for methods that are going to take a bit of time, or are going to get called very frequently to avoid tying down all of the server request threads leading to timeouts for users waiting for a response from the server. For methods that are very fast and aren't expected to get hammered by a lot of users, async doesn't add a lot of value and you need to ensure every async call is awaited or you can end up with whacky behaviour and exceptions.

Avoiding Linq-To-Objects but allowing reutilization of code

Imagine these three EF classes
class Instrument
{
public long InstrumentId { get; set; }
public string Model { get; set; }
public dynamic AsJson()
{
return new
{
instrumentId = this.InstrumentId,
model = this.Model
}
}
}
class Musician
{
public long MusicianId { get; set; }
public virtual Instrument Instrument { get; set; } // notice navigation
public string Name { get; set; }
public dynamic AsJson()
{
return new
{
musicianId = this.MusicianId,
name = this.Name,
instrument = this.Instrument.AsJson()
}
}
}
class MusicBand
{
public long MusicBandId { get; set; }
public string Name { get; set; }
public virtual List<Musician> Members { get; set; }
}
Imagine now that we need multiple actions, all similar with each other, that return JSON.
lets call this Approach (A)
// ajax/Bands/Members
//
public JsonResult Members(long musicBandId)
{
MusicBand g = db.MusicBands.SingleOrDefault(g => g.MusicBandId == musicBandId);
if (g == null)
return null;
return Json(new
{
error = false,
message = "",
members = from p in g.Members.ToList() select p.AsJson()
}, JsonRequestBehavior.AllowGet);
}
The problem with it is that the ToList() is required for the methods AsJson() to be used... so there's a considerable amount of work done in memory
In the following approach, lets call it Approach (B), this is not a problem.. the work done in memory is the minimum, and most of the work is done in SQL. In fact, it is one large SQL query that contains everything needed..
// ajax/Bands/Members
//
public JsonResult Members(long musicBandId)
{
MusicBand g = db.MusicBands.SingleOrDefault(g => g.MusicBandId == musicBandId);
if (g == null)
return null;
return Json(new
{
error = false,
message = "",
persons = from p in g.Members select new
{
musicianId = p.MusicianId,
name = p.Name,
instrument = select new
{
instrumentId = instrument.InstrumentId,
model = instrument.Model
}
}
}, JsonRequestBehavior.AllowGet);
}
Approach A.
Pros: Neat code, I can reuse code in other actions, less coupling
Cons: performance issues (work done in memory!)
Approach B:
Pros: Uggly code, if other action needs something similar, I'll end up copy pasting code! which brings coupling (modify similar code multiple times)
Cons: No performance issues (work done in SQL!)
Finally, the question:
Im looking for another approach (C), that has the better of both (A) and (B),
Reutilization and no performance issues
I'd like to hear how big systems, with many navigation properties, achieve this.
How they manage to reduce coupling when different JSONS (that share subparts) are required
Also, a sub-question:
In approach (A), will the following make a difference?
db.MusicBands.Include(g => g.Members.Select(m => m.Instrument)).SingleOrDefault(g => g.MusicBandId == musicBandId)
or it wont? (keeping the rest of the code with no change)

The starting point of your question is these AsJson() methods in the entity classes. But that's something I wouldn't do in the first place. Firstly, it introduces a transport concept in your domain and secondly, the representation of an entity may depend on the use case: maybe in other cases you only want to show musicians without instruments.
If you want to serialize entities as JSON you generally want to disable proxy generation, which disables lazy loading and materializes the original entity types...
db.ProxyCreationEnabled = false;
...and eagerly load everything you want to return...
MusicBand g = db.MusicBands
.Include(mb => mb.Members.Select(m => m.Instrument))
.SingleOrDefault(mb => mb.MusicBandId == musicBandId);
...and return g as JSON.
This does the job in one SQL query. In your alternatives there will always be at least two queries, because you first fetch the MusicBand and then the members (and instruments) by lazy loading.
If you want to serialize other representations of entities you should map then to DTO (or view model) objects. Here a tool like AutoMapper comes in handy.

Easier way of avoiding duplicates in entity framework

Can anyone provide an easier more automatic way of doing this?
I have the following save method for a FilterComboTemplate model. The data has been converted from json to a c# model entity by the webapi.
So I don't create duplicate entries in the DeviceProperty table I have to go through each filter in turn and retrieve the assigned DeviceFilterProperty from the context and override the object in the filter. See the code below.
I have all the object Id's if they already exist so it seems like this should be handled automatically but perhaps that's just wishful thinking.
public void Save(FilterComboTemplate comboTemplate)
{
// Set the Device Properties so we don't create dupes
foreach (var filter in comboTemplate.Filters)
{
filter.DeviceProperty = context.DeviceFilterProperties.Find(filter.DeviceFilterProperty.DeviceFilterPropertyId);
}
context.FilterComboTemplates.Add(comboTemplate);
context.SaveChanges();
}
From here I'm going to have to check whether any of the filters exist too and then manually update them if they are different to what's in the database so as not to keep creating a whole new set after an edit of a FilterComboTemplate.
I'm finding myself writing a lot of this type of code. I've included the other model classes below for a bit of context.
public class FilterComboTemplate
{
public FilterComboTemplate()
{
Filters = new Collection<Filter>();
}
[Key]
public int FilterComboTemplateId { get; set; }
[Required]
public string Name { get; set; }
[Required]
public ICollection<Filter> Filters { get; set; }
}
public class Filter
{
[Key]
public int FilterId { get; set; }
[Required]
public DeviceFilterProperty DeviceFilterProperty { get; set; }
[Required]
public bool Exclude { get; set; }
[Required]
public string Data1 { get; set; }
}
public class DeviceFilterProperty
{
[Key]
public int DeviceFilterPropertyId { get; set; }
[Required]
public string Name { get; set; }
}

Judging from some similar questions on SO, it does not seem something EF does automatically...
It's probably not a massive cut on code but you could do something like this, an extension method on DbContext (or on your particular dataContext):
public static bool Exists<TEntity>(this MyDataContext context, int id)
{
// your code here, something similar to
return context.Set<TEntity>().Any(x => x.Id == id);
// or with reflection:
return context.Set<TEntity>().Any(x => {
var props = typeof(TEntity).GetProperties();
var myProp = props.First(y => y.GetCustomAttributes(typeof(Key), true).length > 0)
var objectId = myProp.GetValue(x)
return objectId == id;
});
}
This will check if an object with that key exists in the DbContext. Naturally a similar method can be created to actually return that entity as well.
There are two "returns" in the code, just use the one you prefer. The former will force you to have all entities inherit from an "Entity" object with an Id Property (which is not necessarily a bad thing, but I can see the pain in this... you will also need to force the TEntity param: where TEntity : Entity or similar).
Take the "reflection" solution with a pinch of salt, first of all the performance may be a problem, second of all I don't have VS running up now, so I don't even know if it compiles ok, let alone work!
Let me know if that works :)

It seems that you have some common operations for parameters after it's bound from request.
You may consider to write custom parameter bindings to reuse the code. HongMei's blog is a good start point: http://blogs.msdn.com/b/hongmeig1/archive/2012/09/28/how-to-customize-parameter-binding.aspx
You may use the code in Scenario 2 to get the formatter binding to deserialize the model from body and perform the operations your want after that.
See the final step in the blog to specify the parameter type you want customize.

Does AsQueryable() on ICollection really makes lazy execution?

I am using Entity Framework CodeFirst where I have used Parent Child relations using ICollection as
public class Person
{
public string UserName { get;set}
public ICollection<Blog> Blogs { get; set;}
}
public class Blog
{
public int id { get; set; }
public string Subject { get; set; }
public string Body { get; set; }
}
Ok, so far everything is working ok, but my concern is, whenever I want to get the Blogs of a person, I get it as
var thePerson = _context.Persons.Where(x => x.UserName = 'xxx').SingleOrDefault();
var theBlogs = thePerson.Blogs.OrderBy(id).Take(5);
Now, I understand that, when the line is executed, all Blogs for that person is loaded into the memory and then sorting and selecting is done from memory. That is not ideal for a record of Person who has large number of blogs. I want to make the Blog Child as IQueryable so that the Sorting and Selecting is done in SQL database before pulling to Memory.
I know I could declare the Blogs as IQueryable in my context so that I could directly query as
var theBlogs = _context.Blogs.Where(.....)
but that is not feasible for me due to design choice, I want to avoid any circular reference as much as possible due to serialization problem. So, I did not make any reference of the parent entity in my child.
I found that, i can call AsQueryable() method on the blogs as
var theBlogs = thePerson.Blogs.AsQueryable().OrderBy(id).Take(5);
That looks like a magic for me and seems too good to be true. So my question. Does this AsQueryable really make the ICollection as IQueryable in reality and makes all Query process in SQL Server (Lazy loading) OR it is just a casting where Blogs are loaded into memory as like before, but change the interface from ICollection to IQueryable ?

So actually it appears that writing your navigation property as IQueryable<T> is not possible.
What you could do is adding a navigation property to Blog:
public class Blog
{
public int id { get; set; }
public string Subject { get; set; }
public string Body { get; set; }
public virtual Person Owner { get; set; }
}
From that, you can query as follows so it won't load everything into memory:
var thePerson = _context.Persons.Where(x => x.UserName = 'xxx').SingleOrDefault();
var results = _context.Blogs.Where(z => z.Person.Name = thePerson.Name).OrderBy(id).Take(5)
I suggest you to try LINQPad to see how LINQ is translated into SQL, and what is actually requested from the DB.

A better approach is described in Ladislav's answer. In your case:
var theBlogs = _context.Entry(thePerson)
.Collection(x => x.Blogs)
.Query()
.OrderBy(x => x.id)
.Take(5);

Fluent NHibernate - 'failed to lazily initialize a collection' - Querying Over a Collection

I have a self-referential model class:
public class Word
{
public virtual int Id { get; set; }
public virtual string Text { get; set; }
public virtual IList<Word> Synonyms { get; set; }
public virtual int Extra { get; set; }
}
I am trying to query for all synonyms of a word where Extra is 1 and returning the list of words in JSON format in my MVC 3 app:
[HttpPost]
public JsonResult Synonyms(string wordText)
{
using (var session = ...)
{
using (var tx = session.BeginTransaction())
{
var word = session.QueryOver<Word>()
.Where(w => w.Text == wordText)
.SingleOrDefault();
var results = new SynonymsResults()
{
Words = word.Synonyms
.Where(x => x.Extra == 1)
.Select(x => x.Text)
};
return Json(results);
}
}
}
I'm getting an error that it fails to lazily initialize the collection. I'm not sure why though, since I am still in the same session here and even using a transaction.

The result executes much later, after the action has finished running and outside of the session. The fact that you have returned Json(results) doesn't mean that these results will be immediately serialized into JSON. The action will first finish executing, then the ASP.NET MVC pipeline will handle the execution to the result (OnResultExecuting) and it is at this point that the JavaScriptSerializer will touch the collection. At that point of time sessions and transactions are long gone.
So either instruct your ORM to eagerly fetch the dependent collections or even better take a look at the following series of blog posts and use view models.

To get rid of the error, install Nuget Pacakage Manager Newton.JSON and map to the appropriate project and decorate the property with [JsonIgnore], this will skip the serialization issues and you won't get the error.