We Keep Coding

Exclude a model property from loading in MVC5 EF6 [duplicate]

I'm using entity framework to connect with the database. I've one little problem:
I've one table which have one varbinary(MAX) column(with filestream).
I'm using SQL request to manage the "Data" part, but EF for the rest(metadata of the file).
I've one code which has to get all files id, filename, guid, modification date, ... of a file. This doesn't need at all the "Data" field.
Is there a way to retrieve a List but without this column filled?
Something like
context.Files.Where(f=>f.xyz).Exclude(f=>f.Data).ToList();
??
I know I can create anonymous objects, but I need to transmit the result to a method, so no anonymous methods. And I don't want to put this in a list of anonymous type, and then create a list of my non-anonymous type(File).
The goal is to avoid this:
using(RsSolutionsEntities context = new RsSolutionsEntities())
{
var file = context.Files
.Where(f => f.Id == idFile)
.Select(f => new {
f.Id, f.MimeType, f.Size, f.FileName, f.DataType,
f.DateModification, f.FileId
}).FirstOrDefault();
return new File() {
DataType = file.DataType, DateModification = file.DateModification,
FileId = file.FileId, FileName = file.FileName, Id = file.Id,
MimeType = file.MimeType, Size = file.Size
};
}
(I'm using here the anonymous type because otherwise you will get a NotSupportedException: The entity or complex type 'ProjectName.File' cannot be constructed in a LINQ to Entities query.)
(e.g. this code throw the previous exception:
File file2 = context.Files.Where(f => f.Id == idFile)
.Select(f => new File() {Id = f.Id, DataType = f.DataType}).FirstOrDefault();
and "File" is the type I get with a context.Files.ToList(). This is the good class:
using File = MyProjectNamespace.Common.Data.DataModel.File;
File is a known class of my EF datacontext:
public ObjectSet<File> Files
{
get { return _files ?? (_files = CreateObjectSet<File>("Files")); }
}
private ObjectSet<File> _files;

Is there a way to retrieve a List but without this column filled?
Not without projection which you want to avoid. If the column is mapped it is natural part of your entity. Entity without this column is not complete - it is different data set = projection.
I'm using here the anonymous type because otherwise you will get a
NotSupportedException: The entity or complex type 'ProjectName.File'
cannot be constructed in a LINQ to Entities query.
As exception says you cannot project to mapped entity. I mentioned reason above - projection make different data set and EF don't like "partial entities".
Error 16 Error 3023: Problem in mapping fragments starting at line
2717:Column Files.Data in table Files must be mapped: It has no
default value and is not nullable.
It is not enough to delete property from designer. You must open EDMX as XML and delete column from SSDL as well which will make your model very fragile (each update from database will put your column back). If you don't want to map the column you should use database view without the column and map the view instead of the table but you will not be able to insert data.
As a workaround to all your problems use table splitting and separate the problematic binary column to another entity with 1 : 1 relation to your main File entity.

I'd do something like this:
var result = from thing in dbContext.Things
select new Thing {
PropertyA = thing.PropertyA,
Another = thing.Another
// and so on, skipping the VarBinary(MAX) property
};
Where Thing is your entity that EF knows how to materialize. The resulting SQL statement shouldn't include the large column in its result set, since it's not needed in the query.
EDIT: From your edits, you get the error NotSupportedException: The entity or complex type 'ProjectName.File' cannot be constructed in a LINQ to Entities query. because you haven't mapped that class as an entity. You can't include objects in LINQ to Entities queries that EF doesn't know about and expect it to generate appropriate SQL statements.
You can map another type that excludes the VarBinary(MAX) column in its definition or use the code above.

you can do this:
var files = dbContext.Database.SqlQuery<File>("select FileId, DataType, MimeType from Files");
or this:
var files = objectContext.ExecuteStoreQuery<File>("select FileId, DataType, MimeType from Files");
depending on your version of EF

I had this requirement because I have a Document entity which has a Content field with the content of the file, i.e. some 100MB in size, and I have a Search function that I wanted to return the rest of the columns.
I chose to use projection:
IQueryable<Document> results = dbContext.Documents.Include(o => o.UploadedBy).Select(o => new {
Content = (string)null,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
FileName = o.FileName,
Id = o.Id,
// etc. even with related entities here like:
UploadedBy = o.UploadedBy
});
Then my WebApi controller passes this results object to a common Pagination function, which applies a .Skip, .Take and a .ToList.
This means that when the query gets executed, it doesn't access the Content column, so the 100MB data is not being touched, and the query is as fast as you'd want/expect it to be.
Next, I cast it back to my DTO class, which in this case is pretty much exactly the same as the entity class, so this might not be a step you need to implement, but it's follows my typical WebApi coding pattern, so:
var dtos = paginated.Select(o => new DocumentDTO
{
Content = o.Content,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
FileName = o.FileName,
Id = o.Id,
UploadedBy = o.UploadedBy == null ? null : ModelFactory.Create(o.UploadedBy)
});
Then I return the DTO list:
return Ok(dtos);
So it uses projection, which might not fit the original poster's requirements, but if you're using DTO classes, you're converting anyway. You could just as easily do the following to return them as your actual entities:
var dtos = paginated.Select(o => new Document
{
Content = o.Content,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
//...
Just a few extra steps but this is working nicely for me.

For EF Core 2
I implemented a solution like this:
var files = context.Files.AsNoTracking()
.IgnoreProperty(f => f.Report)
.ToList();
The base idea is to turn for example this query:
SELECT [f].[Id], [f].[Report], [f].[CreationDate]
FROM [File] AS [f]
into this:
SELECT [f].[Id], '' as [Report], [f].[CreationDate]
FROM [File] AS [f]
you can see the full source code in here:
https://github.com/aspnet/EntityFrameworkCore/issues/1387#issuecomment-495630292

I'd like to share my attempts to workaround this problem in case somebody else is in the same situation.
I started with what Jeremy Danyow suggested, which to me is the less painful option.
// You need to include all fields in the query, just make null the ones you don't want.
var results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName");
In my case, I needed a IQueryable<> result object so I added AsQueryable() at the end. This of course let me add calls to .Where, .Take, and the other commands we all know, and they worked fine. But there's a caveat:
The normal code (basically context.myEntity.AsQueryable()) returned a System.Data.Entity.DbSet<Data.DataModel.myEntity>, while this approach returned System.Linq.EnumerableQuery<Data.DataModel.myEntity>.
Apparently this means that my custom query gets executed "as is" as soon as needed and the filtering I added later is done afterwards and not in the database.
Therefore I tried to mimic Entity Framework's object by using the exact query EF creates, even with those [Extent1] aliases, but it didn't work. When analyzing the resulting object, its query ended like
FROM [dbo].[TableName] AS [Extent1].Where(c => ...
instead of the expected
FROM [dbo].[TableName] AS [Extent1] WHERE ([Extent1]...
Anyway, this works, and as long as the table is not huge, this method will be fast enough. Otherwise you have no option than to manually add the conditions by concatenating strings, like classic dynamic SQL. A very basic example in case you don't know what I'm talking about:
string query = "SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName";
if (parameterId.HasValue)
query += " WHERE Field1 = " + parameterId.Value.ToString();
var results = context.Database.SqlQuery<myEntity>(query);
In case your method sometimes needs this field, you can add a bool parameter and then do something like this:
IQueryable<myEntity> results;
if (excludeBigData)
results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName").AsQueryable();
else
results = context.myEntity.AsQueryable();
If anyone manages to make the Linq extensions work properly like if it was the original EF object, please comment so I can update the answer.

I'm using here the anonymous type because otherwise you will get a
NotSupportedException: The entity or complex type 'ProjectName.File'
cannot be constructed in a LINQ to Entities query.
var file = context.Files
.Where(f => f.Id == idFile)
.FirstOrDefault() // You need to exeucte the query if you want to reuse the type
.Select(f => new {
f.Id, f.MimeType, f.Size, f.FileName, f.DataType,
f.DateModification, f.FileId
}).FirstOrDefault();
And also its not a bad practice to de-normalize the table into further, i.e one with metadata and one with payload to avoid projection. Projection would work, the only issue is, need to edit any time a new column is added to the table.

I tried this:
From the edmx diagram (EF 6), I clicked the column I wanted to hide from EF and on their properties you can set their getter and setter to private. That way, for me it works.
I return some data which includes a User reference, so I wanted to hide the Password field even though it's encrypted and salted, I just didn't want it on my json, and I didn't want to do a:
Select(col => new {})
because that's a pain to create and maintain, especially for big tables with a lot of relationships.
The downside with this method is that if you ever regenerate your model, you would need to modify their getter and setter again.

Using Entity Framework Power Tools you can do the following in efpt.config.json:
"Tables": [
{
"ExcludedColumns": [
"FileData"
],
"Name": "[dbo].[Attachment]",
"ObjectType": 0
}
]

Add object to Entity Framework dynamically using Reflection

In the following code, the type of domainObject varies (but ends with DO, which I trim then to get the corresponding table name). Having the name of the table and its type, I want to update an existing object - its name is the same as the tableName due to the EF - in the database with the new property values from domainObject. Therefore, I have to find the POCO in the table with the same ID first to overwrite this. This is the code so far:
public void Update(object domainObject)
{
Type type = domainObject.GetType();
string tableName = type.Name.Substring(0, type.Name.Length - 2);
PropertyInfo tableProp = typeof(MyDbContext).GetProperty(tableName);
Type tableType = tableProp.PropertyType;
Type pocoType = tableType.GetGenericArguments()[0];
int id = (int)type.GetProperty("ID").GetValue(domainObject);
using (var context = new MyDbContext())
{
object table = tableProp.GetValue(context);
MethodInfo singleMethod = tableType.GetMethod("Single");
}
}
Usually, knowing the actual table and not just its type, I would now get the POCO via
var poco = context.TableName.Single(item => item.ID == id);
There's 2 problems here:
(1) Single is an extension method.
(2) I don't have an idea how to get the lambda expression in form of an object to pass it to the Invoke of Single.
Is there any way to do this at all with Reflection, or do I have to work around this? (For example, I could iterate through the items in table and check manually [which would load everything from the DB into memory and thus should be avoided], or maybe configure the EF to do some kind of 'override' whenever I just Add and object whose ID is already present if this is possible). Even supposing I could work around this, I'd still like to know a definitive answer to this question, since it's pretty interesting for me!

If you want to use reflection and to find given entity by ID then, if ID is primary key this is fairly simple as this is all you have to do:
object entity = context.Set(domainObject.GetType()).Find(id);
If your property is not primary key then you need to do it as follows:
ParameterExpression p = Expression.Parameter(domainObject.GetType());
Expression property = Expression.Property(p, "ID");
Expression c = Expression.Constant(id);
Expression body = Expression.Equal(property, c);
Expression exp = Expression.Lambda(body, new ParameterExpression []{ p });
MethodInfo singleMethod = typeof(Queryable).GetMethods()
.Single(m => m.Name == "Single" && m.GetParameters().Count() == 2)
.MakeGenericMethod(domainObject.GetType());
DbSet dbSet = context.Set(domainObject.GetType());
object entity = singleMethod.Invoke(null, new object[]{ dbSet, exp });
First with Expression class you build expression that will be passed to Single method (in your case this will be p => p.ID == id). Then you search proper Single method from Queryable class. The last thing is to invoke this method with proper parameters. This way you may do any linq queries with use of Reflection.

You simply need to make a generic method, with a type parameter that represents the type of your entity and use the corresponding DbSet.
public int Update<TEntity>(TEntity domainObject)
{
int id = domainObject.Id; // Needs interface !!
using (var context = new MyDbContext())
{
var objectInDb
= ctx.DbSet<TEntity>.Single(e => e.Id == id); // Needs interface !!
// Use ValueInjecter (not AutoMapper) to copy the properties
objectInDb.InjectFrom(domainObject); // needs ValueInjecter Nuget Package
context.SaveChanges();
}
return userId;
}
As you see in the code comments, your entities need to implement an interface so that you can access the Id property:
public interface IId
{
public int Id { get; set; }
}
And then you need to include the generic method in a generic class that has the corresponding type constraint:
public RepoClass<TEntity>
where TEntity : IId
{
// Define the generic method here
}
In this way you don't have to resort to Reflection.
If you're using some kind of T4 template,or whatever, to create your POCOs, make them partial classes, so that you can declare the interface in a separate file, like this:
public partial MyDomainClass : IId
{
}
In this wya, the interface won't be lost when you update your Db Context objects.
And finally, download an use ValueInjecter, for example using Nuget Package Manager, or running Install-Package ValueInjecter in the Nuget Package Manager console.
When you include using Omu.ValueInjecter; namespace in your code, you'll get an InjectFrom extension method on all objects, that allows to automatically copy all the properties from a source object (by matching their names). Don't use AutoMapper, or you'll have to solve other problems.
Alternatively, you can check that the object exists in the DB (for security) and use the original object, without copying the properties, i.e.
var updatedObject = ctx.Set<TEntity>().Attach(domainObject);
ctx.Entry(updatedObject).State = EntityState.Modified;
ctx.SaveChanges();
I prefer this solution, better than the previous one.

Calling custom (formatting) method on LINQ to Entities

I'm using EF 4.1 and I'm trying to enumerate a company list for a grid. I have two options in the current project: select all companies from the DbContext (Entities) and load them into an object from a non-anonymous type (let's say EmpresaGrid) or select all companies into anonymous type objects with the same structure like Empresa (which is the entity I'm selecting from).
The first option (creating a model class for that) would require a little more work, but can be, eventually, more readable. Still, I'm not sure about that. The second option is what I'm using right now.
So, first question: it's better to create a model class only for displaying data or use anonymous type? Doing a direct select is out of question: a SELECT * is too big and that might make everything damn slow (I guess). So selection into another type creates a custom query with only the needed fields.
Using the second option (anonymous type), I have this code (simplified version):
public static IEnumerable<object> Grid()
{
Entities db = new Entities();
var empresas = db.Empresas
.Select(e => new
{
Cgc = e.Cgc, // PK
(...)
Address = new
{
AddressLine = e.EnderecoSede.AddressLine,
(...)
}
},
Contato = e.Contato,
(...)
})
.ToList();
return empresas;
}
The anonymous type I'm creating has around 40 lines of code, so it's kinda big, but it recreates part of the Empresa class struct (since the grid is waiting for a Empresa object). Anyway, I have a problem with the data format. For example, I would like to format the Cgc property using a custom string format. I have a public method for this, FormataCgc. This method receives a string and returns it formatted using some internal conditions.
So, my problem is how to that. For example, I have tried this:
var empresas = db.Empresas
.Select(e => new
{
Cgc = FormataCgc(e.Cgc),
}
But that doesn't work because FormataCgc cannot be translated into SQL (and I don't want to convert it). I also tried this:
var empresas = db.Empresas
.Select(e => new
{
(...)
}
.ToList();
foreach (var e in empresas) {
e.Cgc = FormataCgc(e.Cgc);
}
But it cannot be done since anonymous types have only read-only properties.
So, my second question is: how exactly can I do that? I need to change the data after selecting it, but using anonymous types? I've done a little research, and the best thing I've found was this: Calling a custom method in LINQ query. In that solution, Ladislav suggested doing a second select from the IEnumerable, but since the grid is excepting Empresa I cannot do that (I need to change or add properties, not encapsulate them).
I'm not sure if I was clear enough, but feel free to ask any questions. Also, the grid I'm currently using is a Telerik ASP.NET MVC Grid, which receives a IEnumerable (where T is a class) as model data and them iterates each object, doing its magic.

Since you're already converting this into an IEnumerable<T>, you can do the custom formatting as you stream the results in the client. Do your db.Select, and then convert to the appropriate format afterwards, ie:
var empresas = db.Empresas
.Select(e => new
{
(...)
})
.ToList();
foreach (var e in empresas) {
yield return new {
Cgc = FormataCgc(e.Cgc),
// Copy other properties here, as needed...
};
}
That being said, I'd personally recommend making a custom class, and not return an anonymous type. Your conversion would then be:
foreach (var e in empresas) {
yield return new YourClass(FormataCgc(e.Cgc), ...); // Construct as needed
}
This will dramatically improve the usability of this method, as you will have proper, named access to your properties from the caller of the method.

I think the solution to both of your questions is to create a model class. Sure it is a little bit more work up front, but it will allow you greater flexibility in the long run. Your custom model class can then handle the formatting for you.
public class EmpresaGridModel
{
public string Cgc { get; set; }
public string CgcFormatted
{
return FormataCgc(this.Cgc);
}
//properties for the other fields will have to be created as well obviously
}
Your telerik grid can then bind directly to the CgcFormatted property

C#, Linq2SQL - tricks to fetch a ViewModel object with relation data?

I don't know Linq2Sql so well yet and I was wondering if there is a trick for this probably common MVVM scenario. I have Linq2Sql data context containing Domain models, but I am fetching data for my customized ViewModel object from it.
var query = from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
So i want my ViewModel to inculde both CurrencyId from domain object and the CurrencyText from related table to show it nicely in the View.
This code works great. It generates one DB call with join to fetch the CurrencyText. But the model is simplified, real one has many more fields. I want to make the code reusable because I have many different queries, that returns the same ViewModel. Now every minor change to OrderViewModel requires lots of maintainance.
So I moved the code to OrderViewModel itself as a constructor.
public OrderViewModel(Table_Order ord)
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
}
And call it like this.
var query = from ord in ctx.Table_Orders
select new OrderViewModel(ord);
The Problem: The join is gone DB query is no more optimised. Now I get 1+N calls to database to fetch CurrencyText for every line.
Any comments are welcome. Maybe I have missed different great approach.
This is how far i could get on my own, to get the code reusability. I created a function that does the job and has multiple parameters. Then I need to explicitly pass it everything that has crossed the line of entity.
var query = ctx.Table_Orders.Select(m =>
newOrderViewModel(m, m.Currency.CurrencyText));
The DB call is again optimized. But it still does not feel like I am there yet! What tricks do You know for this case?
EDIT : The final solution
Thanks to a hint by #Muhammad Adeel Zahid I arrived at this solution.
I created an extension for IQueryable
public static class Mappers
{
public static IEnumerable<OrderViewModel> OrderViewModels(this IQueryable<Table_Order> q)
{
return from ord in q
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
}
Now i can do this to get all list
var orders = ctx.Table_Order.OrderViewModels().ToList();
or this to get a single item, or anything in between with Where(x => ..)
var order = ctx.Table_Order
.Where(x => x.OrderId == id).OrderViewModels().SingleOrDefault();
And that completely solves this question. The SQL generated is perfect and the code to translate objects is reusable. Approach like this should work with both LINQ to SQL and LINQ to Entities. (Not tested with the latter) Thank You again #Muhammad Adeel Zahid

Whenever we query the database, we mostly require either enumeration of objects (more than one records in db) or we want a single entity (one record in db). you can write your mapping code in method that returns enumeration for whole table like
public IEnumerable<OrderViewModel> GetAllOrders()
{
return from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
Now you may want to filter these records and return another enumeration for example on currencyID
public IEnumerable<OrderViewModel> GetOrdersByCurrency(int CurrencyID)
{
return GetAllOrders().Where(x=>x.CurrencyId == CurrencyID);
}
Now you may also want to find single record out of all these view models
public OrderViewModel GetOrder(int OrderID)
{
return GetAllOrders().SingleOrDefault(x=>x.OrderId == OrderID);
}
The beauty of IEnumerable is that it keeps adding conditions to query and does not execute it until it is needed. so your whole table will not be loaded unless you really want it and you have kept your code in single place. Now if there are any changes in ViewModel Mapping or in query itself, it has to be done in GetAllOrders() method, rest of code will stay unchanged

You can avoid the N+1 queries problem by having Linq2SQL eagerly load the referenced entites you need to construct your viewmodels. This way you can build one list of objects (and some referenced objects) and use it to construct everything. Have a look at this blog post.
One word of warning though: This technique (setting LoadOptions for the Linq2SQL data context) can only be done once per data context. If you need to perform a second query with a different eager loading configuration, you must re-initalize your data context. I automated this with a simple wrapper class around my context.

Entity Framework 4.0 - Returning List of Model objects + Count of Children Per Object

I'm running into a common need in my project to return collections of my model objects, plus a count of certain types of children within each, but I don't know if it is possible or how to model a "TotalCount" property in a Model class and populate it as part of on single Entity Framework query, preferably using LINQ queries. Is it possible to do this whilst being able to use the Entity Framework .Include("Object") and .Skip() and .Take()? I'm new to the Entity Framework so I may be missing tons of obvious stuff that can allow this...
I would like to be able to paginate on the dynamically constructed count properties as well. I'm thinking that the most scalable approach would be to store the counts as separate database properties and then simply query the count properties. But for cases where there are small row counts that I'm dealing with, I'd rather do the counts dynamically.
In a model like this:
Table: Class
Table: Professor
Table: Attendee
Table: ClassComment
I'd like to return a list of Class objects in the form of List, but I would also like the counts of Attendees and Class comments to be determined in a single query (LINQ preferred) and set in two Class properties called AttendeeCount and ClassCommentCount.
I have this thus far:
var query = from u in context.Classes
orderby tl.Name
select u;
List<Class> topics = ((ObjectQuery<Class>)query)
.Include("ClassComments")
.Skip(startRecord).Take(recordsToReturn).ToList();
Any suggestions or alternative query approaches that can still allow the use of .Include() and pagination would be much much appreciated, in order to produce a single database query, if at all possible. Thank you for any suggestions!

Try this:
public class ClassViewModel {
public Class Class { get; set; }
public int AttendeeCount { get; set; }
public int ClassCommentCount { get; set; }
}
var viewModel = context.Classes.Select(clas =>
new ClassViewModel {
Class = clas,
AttendeeCount = clas.ClassAttendes.Count,
ClassCommentCount = clas.ClassComments.Count}
).OrderBy(model => model.ClassCommentCount).Skip(startRecord).Take(recordsToReturn).ToList();
You don't have to include comments to get count.

It will not work this way. The easiest approach is to use projection into anonymous (or custom) non entity type. I would try something like this:
var query = context.Classes
.Include("ClassComments") // Only add this if you want eager loading of all realted comments
.OrderBy(c => c.Name)
.Skip(startRecord)
.Take(recordsToReturn)
.Select(c => new
{
Class = c,
AttendeeCount = c.Attendees.Count(),
ClassCommentCount = c.ClassComments.Count() // Not needed because you are loading all Class comments so you can call Count on loaded collection
});
The problem in your requirement are AttendeeCount and ClassCommentCount properties. You can't easily add them to your model because there is no corresponding column in database (unless you define one and in such case you don't need to manually count records). You can define them in partial Class implementation but in such case you can't use them in Linq-to-entities query.
The only way to map this in EF is to use DB view and create special read only entity to represent it in your applicaiton or to use DefiningQuery which is custom SQL command defined in SSDL instead of DB table or view.