We Keep Coding

Best way to handle a many to many relationship in EF6 ASP.NET MVC 5 and retrieve property from navigation property to list

A few years ago I took over a project from a previous developer and have been working my way through this system and adding to it as well as reconfiguring some aspects of it. The latest thing I've uncovered was a many to many relationship that I haven't handled yet.
I have two tables:
ECAnalysis - which contains active and valid analyses for a given instrument.
ECAnalyte - which contains active and valid analytes.
One analysis can have many analytes whereas one analyte can be on many analyses (if I'm thinking about this in a correct way from what was left for me).
There is also an intermediate table in my MySQL database called ECAnalysisToECAnalyte which just contains the primary keys of each table.
The EF code is configured as such to be a generic repository because of the large number of entities. So to get data out of the system along with navigation properties there is a method that gets all of the data:
public virtual async Task<IList<T>> GetAllAsync<T>(params Expression<Func<T, object>>[] navigationProperties) where T : class
{
try
{
using (var db = new EntitiesContext())
{
List<T> list;
IQueryable<T> dbQuery = db.Set<T>();
foreach (Expression<Func<T, object>> navigationProperty in navigationProperties)
dbQuery = dbQuery.Include<T, object>(navigationProperty);
list = await dbQuery.AsNoTracking().ToListAsync<T>();
return list;
}
}
catch (ArgumentNullException ex)
{
throw new InvalidOperationException($"Invalid state: {typeof(T).Name} DbSet is null.", ex);
}
}
This method works correctly for all normal types of relationships and entities but it doesn't seem to work for a many to many relationship.
And to call it in my main application it's used in this manner:
var data = (from a in await Manager.GetAllAsync<ECAnalysis>(n => n.ECAnalyte)
select a);
My task now has been to get a list of analytes for a given analysis and pass them to a view. My first issue is when I use the method as is with the AsNoTracking() I get an exception stating that:
When an object is returned with a NoTracking merge option, Load can only be called when the EntityCollection or EntityReference does not contain objects.
If I remove the AsNoTracking() I get the correct list of analytes to my view but along with another exception:
The ObjectContext instance has been disposed and can no longer be used for operations that require a connection.
How I'm getting this list of analytes is like this...I know it's probably not the cleanest or most efficient so bear with me:
public async Task<JsonResult> GetAnalytes(string sampleType)
{
var analysis = (from a in await Manager.AllAsync<ECAnalysis>(b => b.ECAnalyte)
where a.Method == "IC" && a.Media == sampleType + " - ANIONS"
select a).ToList();
var ecAnalytes = analysis.FirstOrDefault().ECAnalyte.ToList();
var analyteList = new List<string>();
foreach (var item in ecAnalytes)
{
analyteList.Add(item.Name);
};
return Json(analysis, JsonRequestBehavior.AllowGet);
//this is being called from an Ajax method
}
So my questions are:
Is the many to many relationship configured properly for how I want to use it?
Is there a better way to get a list of analytes for a given analysis that doesn't give me an exception?

EntityFramework: Eager loading with excludes instead of includes?

My data model has a lot of nested entities and I would like to eagerly load the whole object tree ... except for a view entities that will be explicitly loaded on demand.
Using include paths I have to specify many paths and every time I add a new entity I will have to adapt those include paths. I currently use following method of my repository to load all entities of a type:
public virtual IQueryable<TEntity> All(string commaSeperatedIncludePropertyPaths = "")
{
IQueryable<TEntity> initialQuery = Context.Set<TEntity>();
string[] includePaths = commaSeperatedIncludePropertyPaths.Split(new[] { ','}, StringSplitOptions.RemoveEmptyEntries);
return includePaths.Aggregate(initialQuery, (currentQuery, includeProperty) => currentQuery.Include(includeProperty));
}
The passed include paths already fill a whole screen.
Therefore I would prefer that the EntityFramework automatically loads all navigation properties eagerly, except for the ones I specify with exclude paths:
public virtual IQueryable<TEntity> All(string commaSeperatedExcludePropertyPaths = "")
{
//... how to implement?
}
The exclude paths would help to avoid circular dependencies and to filter out the few entities I don't want to load eagerly. Specifying excludes instead of includes would reduce boilerplate code for me.
Is this possible with EF 6.1.3 or planned for EF 7? If not, what are reasons against that option?
Did anyone already try to read entity meta data and apply it for "auto eager loading" and failed?
Related (old) questions and articles:
Overview on options for loading navigation properties:
https://msdn.microsoft.com/en-us/magazine/hh205756.aspx
Auto eager load
Entity framework auto eager load
Entity Framework - Is there a way to automatically eager-load child entities without Include()?
Entity framework linq query Include() multiple children entities
Type save includes
Entity Framework .Include() with compile time checking?

Below is a first draft for a solution. I still have to find out if it's practicable ... and I'll consider to rework the loading approach (as Lanorkin suggested), too. Thank you for your comments.
Edit
It turned out that, while excludes might make sense when developing an application ...doing many changes to the domain model..., excludes are not more elegant than includes for a "real world example" that I just considered.
a) I went through my entities and counted the number of included and excluded navigation properties. The average number of excluded properties was not significantly smaller then the number of included properties.
b) If I do consider a distinct navigation property "foos" for the exclusions, I will be forced to consider exclusions for the sub entities of type Foo ... if I do not want to use its properties at all.
On the other hand, using inclusions, I just need to specify the navigation property "foos" and do not need to specify anything else for the sub entities.
Therefore, while excludes might save some specs for one level, they dent to require more specs for the next level ... (when excluding some intermediate entities and not only entities that are located at the leaves of the loaded object tree).
c) Furthermore, the includes/excludes might not only depend on the type of the entity but also on the path that is used to access it. Then an exclude needs to be specified like "exclude properties xy when loading the entity for one purpose and exclude properties z when loading the entity for another purpose".
=> As a result of this considerations I will go on using inclusions.
I implemented type save inclusions that are based on inclusion dictionaries instead of strings:
private static readonly Inclusions<Person> _personInclusionsWithCompanyParent = new Inclusions<Person>(typeof(Company))
{
{e => e.Company, false},
{e => e.Roles, true}
};
I have a method that creates the query from a list of inclusions. That method also checks if all existing navigation properties are considered in the dictionaries. If I add a new entity and forget to specify corresponding inclusions, an exception will be thrown.
Nevertheless, here is an experimental solution for using excludes instead of includes:
private const int MAX_EXPANSION_DEPTH = 10;
private DbContext Context { get; set; } //set during construction of my repository
public virtual IQueryable<TEntity> AllExcluding(string excludeProperties = "")
{
var propertiesToExclude = excludeProperties.Split(new[]
{
','
},
StringSplitOptions.RemoveEmptyEntries);
IQueryable<TEntity> initialQuery = Context.Set<TEntity>();
var elementType = initialQuery.ElementType;
var navigationPropertyPaths = new HashSet<string>();
var navigationPropertyNames = GetNavigationPropertyNames(elementType);
foreach (var propertyName in navigationPropertyNames)
{
if (!propertiesToExclude.Contains(propertyName))
{
ExtendNavigationPropertyPaths(navigationPropertyPaths, elementType, propertyName, propertyName, propertiesToExclude, 0);
}
}
return navigationPropertyPaths.Aggregate(initialQuery, (current, includeProperty) => current.Include(includeProperty));
}
private void ExtendNavigationPropertyPaths(ISet<string> navigationPropertyPaths,
Type parentType,
string propertyName,
string propertyPath,
ICollection<string> propertiesToExclude,
int expansionDepth)
{
if (expansionDepth > MAX_EXPANSION_DEPTH)
{
return;
}
var propertyInfo = parentType.GetProperty(propertyName);
var propertyType = propertyInfo.PropertyType;
var isEnumerable = typeof(IEnumerable).IsAssignableFrom(propertyType);
if (isEnumerable)
{
propertyType = propertyType.GenericTypeArguments[0];
}
var subNavigationPropertyNames = GetNavigationPropertyNames(propertyType);
var noSubNavigationPropertiesExist = !subNavigationPropertyNames.Any();
if (noSubNavigationPropertiesExist)
{
navigationPropertyPaths.Add(propertyPath);
return;
}
foreach (var subPropertyName in subNavigationPropertyNames)
{
if (propertiesToExclude.Contains(subPropertyName))
{
navigationPropertyPaths.Add(propertyPath);
continue;
}
var subPropertyPath = propertyPath + '.' + subPropertyName;
ExtendNavigationPropertyPaths(navigationPropertyPaths,
propertyType,
subPropertyName,
subPropertyPath,
propertiesToExclude,
expansionDepth + 1);
}
}
private ICollection<string> GetNavigationPropertyNames(Type elementType)
{
var objectContext = ((IObjectContextAdapter)Context).ObjectContext;
var entityContainer = objectContext.MetadataWorkspace.GetEntityContainer(objectContext.DefaultContainerName, DataSpace.CSpace);
var entitySet = entityContainer.EntitySets.FirstOrDefault(item => item.ElementType.Name.Equals(elementType.Name));
if (entitySet == null)
{
return new List<string>();
}
var entityType = entitySet.ElementType;
return entityType.NavigationProperties.Select(np => np.Name)
.ToList();
}

Add object to Entity Framework dynamically using Reflection

In the following code, the type of domainObject varies (but ends with DO, which I trim then to get the corresponding table name). Having the name of the table and its type, I want to update an existing object - its name is the same as the tableName due to the EF - in the database with the new property values from domainObject. Therefore, I have to find the POCO in the table with the same ID first to overwrite this. This is the code so far:
public void Update(object domainObject)
{
Type type = domainObject.GetType();
string tableName = type.Name.Substring(0, type.Name.Length - 2);
PropertyInfo tableProp = typeof(MyDbContext).GetProperty(tableName);
Type tableType = tableProp.PropertyType;
Type pocoType = tableType.GetGenericArguments()[0];
int id = (int)type.GetProperty("ID").GetValue(domainObject);
using (var context = new MyDbContext())
{
object table = tableProp.GetValue(context);
MethodInfo singleMethod = tableType.GetMethod("Single");
}
}
Usually, knowing the actual table and not just its type, I would now get the POCO via
var poco = context.TableName.Single(item => item.ID == id);
There's 2 problems here:
(1) Single is an extension method.
(2) I don't have an idea how to get the lambda expression in form of an object to pass it to the Invoke of Single.
Is there any way to do this at all with Reflection, or do I have to work around this? (For example, I could iterate through the items in table and check manually [which would load everything from the DB into memory and thus should be avoided], or maybe configure the EF to do some kind of 'override' whenever I just Add and object whose ID is already present if this is possible). Even supposing I could work around this, I'd still like to know a definitive answer to this question, since it's pretty interesting for me!

If you want to use reflection and to find given entity by ID then, if ID is primary key this is fairly simple as this is all you have to do:
object entity = context.Set(domainObject.GetType()).Find(id);
If your property is not primary key then you need to do it as follows:
ParameterExpression p = Expression.Parameter(domainObject.GetType());
Expression property = Expression.Property(p, "ID");
Expression c = Expression.Constant(id);
Expression body = Expression.Equal(property, c);
Expression exp = Expression.Lambda(body, new ParameterExpression []{ p });
MethodInfo singleMethod = typeof(Queryable).GetMethods()
.Single(m => m.Name == "Single" && m.GetParameters().Count() == 2)
.MakeGenericMethod(domainObject.GetType());
DbSet dbSet = context.Set(domainObject.GetType());
object entity = singleMethod.Invoke(null, new object[]{ dbSet, exp });
First with Expression class you build expression that will be passed to Single method (in your case this will be p => p.ID == id). Then you search proper Single method from Queryable class. The last thing is to invoke this method with proper parameters. This way you may do any linq queries with use of Reflection.

You simply need to make a generic method, with a type parameter that represents the type of your entity and use the corresponding DbSet.
public int Update<TEntity>(TEntity domainObject)
{
int id = domainObject.Id; // Needs interface !!
using (var context = new MyDbContext())
{
var objectInDb
= ctx.DbSet<TEntity>.Single(e => e.Id == id); // Needs interface !!
// Use ValueInjecter (not AutoMapper) to copy the properties
objectInDb.InjectFrom(domainObject); // needs ValueInjecter Nuget Package
context.SaveChanges();
}
return userId;
}
As you see in the code comments, your entities need to implement an interface so that you can access the Id property:
public interface IId
{
public int Id { get; set; }
}
And then you need to include the generic method in a generic class that has the corresponding type constraint:
public RepoClass<TEntity>
where TEntity : IId
{
// Define the generic method here
}
In this way you don't have to resort to Reflection.
If you're using some kind of T4 template,or whatever, to create your POCOs, make them partial classes, so that you can declare the interface in a separate file, like this:
public partial MyDomainClass : IId
{
}
In this wya, the interface won't be lost when you update your Db Context objects.
And finally, download an use ValueInjecter, for example using Nuget Package Manager, or running Install-Package ValueInjecter in the Nuget Package Manager console.
When you include using Omu.ValueInjecter; namespace in your code, you'll get an InjectFrom extension method on all objects, that allows to automatically copy all the properties from a source object (by matching their names). Don't use AutoMapper, or you'll have to solve other problems.
Alternatively, you can check that the object exists in the DB (for security) and use the original object, without copying the properties, i.e.
var updatedObject = ctx.Set<TEntity>().Attach(domainObject);
ctx.Entry(updatedObject).State = EntityState.Modified;
ctx.SaveChanges();
I prefer this solution, better than the previous one.

Calculation in EF Projection

Is there a way I can achieve the following?
// colourInfo.Discount = 75, but can change
// allPrice type has Part, Desc, Type
var a = allPricesForPgs.Where(x => x.PG == c && x.ColourCode == colourInfo.ColourCode).Select(y=> new AllPrice {Part=y.Part, Desc=y.Desc, Price=y.Price*(colourInfo.Discount/100)}));
I get the error : The entity or complex type 'Portal.AllPrice' cannot be constructed in a LINQ to Entities query.
It seems the EF cannot handle calculations, what are my options since I am getting a dynamic value from one table to do a calculation on another?

Sam1's comment is correct. You cannot project into another entity. Other options you have can be to create an anonymous type like so:
var a = allPricesForPgs
.Where(x => x.PG == c && x.ColourCode == colourInfo.ColourCode)
.Select(y=> new
{
Part=y.Part,
Desc=y.Desc,
Price=y.Price*(colourInfo.Discount/100)
}
));
Or to create a class that will hold the temporary data (such as a DTO).
Since it seems like all you need to do is have this information to modify some other entity, you should be able to do it with the anonymous type.
EDIT:
You could add a '.ToList()' right before the .Select(...). You'd essentially be using LINQ TO OBJECTS instead of LINQ TO ENTITIES, so if a lot of entities might match the allPricesForPgs.Where(...) statement, you should keep away from that.
But, if you want these as AllPrice's, why are they not added to the AllPrice DB? Are you keeping a separate list of some AllPrice's from Entity Framework and some AllPrice's from this list? This could get confusing and cause errors.
A final option would be to extend the class. All entities are declared PARTIAL. You can create another class like:
partial class AllPrice
{
Double DiscoutedPrice { get { Price * myDiscount/100; } }

Calling custom (formatting) method on LINQ to Entities

I'm using EF 4.1 and I'm trying to enumerate a company list for a grid. I have two options in the current project: select all companies from the DbContext (Entities) and load them into an object from a non-anonymous type (let's say EmpresaGrid) or select all companies into anonymous type objects with the same structure like Empresa (which is the entity I'm selecting from).
The first option (creating a model class for that) would require a little more work, but can be, eventually, more readable. Still, I'm not sure about that. The second option is what I'm using right now.
So, first question: it's better to create a model class only for displaying data or use anonymous type? Doing a direct select is out of question: a SELECT * is too big and that might make everything damn slow (I guess). So selection into another type creates a custom query with only the needed fields.
Using the second option (anonymous type), I have this code (simplified version):
public static IEnumerable<object> Grid()
{
Entities db = new Entities();
var empresas = db.Empresas
.Select(e => new
{
Cgc = e.Cgc, // PK
(...)
Address = new
{
AddressLine = e.EnderecoSede.AddressLine,
(...)
}
},
Contato = e.Contato,
(...)
})
.ToList();
return empresas;
}
The anonymous type I'm creating has around 40 lines of code, so it's kinda big, but it recreates part of the Empresa class struct (since the grid is waiting for a Empresa object). Anyway, I have a problem with the data format. For example, I would like to format the Cgc property using a custom string format. I have a public method for this, FormataCgc. This method receives a string and returns it formatted using some internal conditions.
So, my problem is how to that. For example, I have tried this:
var empresas = db.Empresas
.Select(e => new
{
Cgc = FormataCgc(e.Cgc),
}
But that doesn't work because FormataCgc cannot be translated into SQL (and I don't want to convert it). I also tried this:
var empresas = db.Empresas
.Select(e => new
{
(...)
}
.ToList();
foreach (var e in empresas) {
e.Cgc = FormataCgc(e.Cgc);
}
But it cannot be done since anonymous types have only read-only properties.
So, my second question is: how exactly can I do that? I need to change the data after selecting it, but using anonymous types? I've done a little research, and the best thing I've found was this: Calling a custom method in LINQ query. In that solution, Ladislav suggested doing a second select from the IEnumerable, but since the grid is excepting Empresa I cannot do that (I need to change or add properties, not encapsulate them).
I'm not sure if I was clear enough, but feel free to ask any questions. Also, the grid I'm currently using is a Telerik ASP.NET MVC Grid, which receives a IEnumerable (where T is a class) as model data and them iterates each object, doing its magic.

Since you're already converting this into an IEnumerable<T>, you can do the custom formatting as you stream the results in the client. Do your db.Select, and then convert to the appropriate format afterwards, ie:
var empresas = db.Empresas
.Select(e => new
{
(...)
})
.ToList();
foreach (var e in empresas) {
yield return new {
Cgc = FormataCgc(e.Cgc),
// Copy other properties here, as needed...
};
}
That being said, I'd personally recommend making a custom class, and not return an anonymous type. Your conversion would then be:
foreach (var e in empresas) {
yield return new YourClass(FormataCgc(e.Cgc), ...); // Construct as needed
}
This will dramatically improve the usability of this method, as you will have proper, named access to your properties from the caller of the method.

I think the solution to both of your questions is to create a model class. Sure it is a little bit more work up front, but it will allow you greater flexibility in the long run. Your custom model class can then handle the formatting for you.
public class EmpresaGridModel
{
public string Cgc { get; set; }
public string CgcFormatted
{
return FormataCgc(this.Cgc);
}
//properties for the other fields will have to be created as well obviously
}
Your telerik grid can then bind directly to the CgcFormatted property