We Keep Coding

SelectMany converts query to Enumerable List. How to avoid it?

I have a MVC controller action that does a database query this way:
var marcaciones = db.Marcacion
where db is the database context in Entity Framework, and Marcacion is a database table. After that instruction, marcaciones type becomes
System.Data.Entity.DbSet`1[CasinosCloud.Models.Marcacion]
That allows to add any filter before framework actually executes the query in database.
So far, so good.
However, depending on certain condition, marcaciones variable is assigned in a different way.
The database model is such that marcaciones entity in database is a child of another entity. To get that marcaciones list, I can do this:
var marcaciones = trabajador.ServicioSupervisado.SelectMany(s => s.Marcacion).AsQueryable();
As you can infer from instruction above, trabajador is a parent database entity that have many ServicioSupervisado entities, which, in turn, can have many Marcacion entities.
Since marcaciones variable is the same as the marcaciones variable I showed before, I have to convert to Queryable.
After executing the above instruction, marcaciones type becomes;
{System.Linq.Enumerable+<SelectManyIterator>d__17`2
[CasinosCloud.Models.Servicio,CasinosCloud.Models.Marcacion]}
That mean query is actually converted to an Enumerable List.
All that works when no other filter is applied. When I add query filter I got problems with the second form. First, the whole web page is slower because all filters are applied in a memory list, not in the database, and second, I have problems with string comparisons, specially when I try to find a text in lowercase when in database is stored in uppercase. Of course, nothing is found in such a case.
I think the problem is reduced by solving the type issue. Why after calling SelectMany, the query is actually executed and converted to an Enumerable List? Is there a way to avoid this and all that to be executed in database? Maybe I should rewrite that instruction not using SelectMany. I tried by using db.Marcacion.Insersect() to do the intersection with this code, but the same problem occurs:
trabajador.ServicioSupervisado.SelectMany(s => s.Marcacion)
EDIT:
Query I want to execute in database takes the following form:
For the first way:
SELECT m.*
FROM Marcacion m
For the second way:
SELECT m.*
FROM Marcacion m
INNER JOIN Servicio s ON s.ServicioId = m.ServicioId
INNER JOIN Trabajador t ON t.TrabajadorId = s.TrabajadorId
WHERE t.TrabajadorId = 1069
EDIT 2:
For the second way, I tried with:
marcaciones = marcaciones.Where(m => trabajador.ServicioSupervisado.Any(s => s.ServicioId == m.ServicioId));
After that, when query is actually executed in database, this error happens:
System.NotSupportedException: 'Unable to create a constant value of type 'CasinosCloud.Models.Servicio'. Only primitive types or enumeration types are supported in this context.'

I solved it by writing the query this way:
var servicios = trabajador.ServicioSupervisado.Select(s => s.ServicioId);
marcaciones = marcaciones.Where(m => servicios.Any(s => s == m.ServicioId));
That way query is executed when I call ToList(), and works in both cases I told about, being very fast in both since query is run directly in database with all filters applied.
By seeing the database log, the final query executed by Entity Framework was:
SELECT
[Extent1].[MarcacionId] AS [MarcacionId],
[Extent1].[MonitorId] AS [MonitorId],
[Extent1].[TrabajadorId] AS [TrabajadorId],
[Extent1].[EmpresaId] AS [EmpresaId],
[Extent1].[ServicioId] AS [ServicioId],
[Extent1].[MarcacionFechaHora] AS [MarcacionFechaHora],
[Extent1].[MarcacionEntradaSalida] AS [MarcacionEntradaSalida],
[Extent1].[MarcacionChecksum] AS [MarcacionChecksum],
[Extent1].[MarcacionEquipo] AS [MarcacionEquipo],
[Extent1].[MarcacionEsManual] AS [MarcacionEsManual],
[Extent1].[MarcacionCreadoEn] AS [MarcacionCreadoEn],
[Extent1].[MarcacionActualizadoEn] AS [MarcacionActualizadoEn],
[Extent1].[MarcacionIndice] AS [MarcacionIndice]
FROM [dbo].[Marcacion] AS [Extent1]
INNER JOIN [dbo].[Trabajador] AS [Extent2] ON [Extent1].[TrabajadorId] = [Extent2].[TrabajadorId]
WHERE ( EXISTS (SELECT
1 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
WHERE 3 = [Extent1].[ServicioId]
)) AND ([Extent2].[TrabajadorNombres] + N' ' + [Extent2].[TrabajadorApellidos] LIKE #p__linq__0 ESCAPE N'~') AND ([Extent1].[MarcacionFechaHora] >= #p__linq__1) AND ([Extent1].[MarcacionFechaHora] <= #p__linq__2)

Does using AsNoTracking() make a difference when only returning fields?

So I've read a lot about using AsNoTracking() when performing a query in EF, specifically if it returns entities, as to not keep around references to things if you will not be updating.
But I've also read that AsNoTracking may also speed up the queries themselves as EF does not have to map each item queried to an entity in the map.
The question is, if my Linq query simply returns values from the columns/rows but not an entity type, does using AsNoTracking() make a difference to speed up the query? And if not obviously I shouldn't use it because it just clutters the code?
example 1 (I would expect to use AsNoTracking():
var result = (from p in context.Pogs
select p).AsNoTracking();
example 2 (My question... I'm thinking it doesn't make sense to use here, but I don't know the answer):
var result = (from p in context.Pogs
select p.Name); // assuming p.Name is a string or something
versus
var result = (from p in context.Pogs.AsNoTracking()
select p.Name);

No, it does not since the entities won't be loaded, as evidenced by examining context.Pogs.Local which won't contain the entities whose properties were retrieved through LINQ.
You can check the entities being tracked through DbContext.ChangeTracker. So if you retrieve the entries of the tracker for your Pogs DbSet through context.ChangeTracker.Entries<Pogs>() you'll see that for your first example there are entries tracking the corresponding entities, while for the second example there are none.

Joining a Many to Many table

I have a situation where I am providing a method to query for data in various ways. We provide the user with 4 different fitler criteria and they can mix and match as much as they want.
For example:
public Fruit GetFruit(Boolean isLocal, string storeName, string classificationType, string state);
This is simple when all of the attributes are on the table, but my issue arises as a result of complexity in the data model. Three of my attributes are simple, they are just joins, but i have one table that sits behind a one to many relationship. So in order to query for it I have to do a many to many join.
So lets say i am trying to determine all fruit a store offers. A store has a list of fruits and our classificationType sits behind a many relationship (FruitClassification)
alt text http://tinyurl.com/39q6ruj
The only successful way i have been able to query this in EF is by selecting all Fruits (by classification), and then selecting all stores that meet the filter criteria and then joining them.
You would think this query in ef would be functional:
var final = (
from s in Stores
join fc in FruitClassifications.Where(z=>z.Classifications.Code == classificationType && z.Classifications.Type.Code =="FRT").Select(x=>x.Fruit).Distinct()
on s.Fruits.Id equals f.Id
where s.Name=name && s.isLocal && s.State==state
select s
).ToList();
But it runs horrible(and looks the same when i profile it), Is there any way i can push this query down to the database? A better way to query?

I think this is what you want:
var final = (from s in Stores
where s.Name=name && s.isLocal && s.State==state
&& s.Fruits.Any(f =>
f.FruitClassifications.Any(fc => fc.Code == classificationType
&& fc.Type.Code == "FRT"))
select s).ToList();

http://learnentityframework.com/LearnEntityFramework/tutorials/many-to-many-relationships-in-the-entity-data-model/
this might help you. EF has the possibility to generate those relations with navigation properties from the designer, so you don't have to use the join.

Entity Framework SQL Query Execution

Using the Entity Framework, when one executes a query on lets say 2000 records requiring a groupby and some other calculations, does the query get executed on the server and only the results sent over to the client or is it all sent over to the client and then executed?
This using SQL Server.
I'm looking into this, as I'm going to be starting a project where there will be loads of queries required on a huge database and want to know if this will produce a significant load on the network, if using the Entity Framework.

I would think all database querying is done on the server side (where the database is!) and the results are passed over. However, in Linq you have what's known as Delayed Execution (lazily loaded) so your information isn't actually retrieved until you try to access it e.g. calling ToList() or accessing a property (related table).
You have the option to use the LoadWith to do eager loading if you require it.
So in terms of performance if you only really want to make 1 trip to the Database for your query (which has related tables) I would advise using the LoadWith options. However, it does really depend on the particular situation.

It's always executed on SQL Server. This also means sometimes you have to change this:
from q in ctx.Bar
where q.Id == new Guid(someString)
select q
to
Guid g = new Guid(someString);
from q in ctx.Bar
where q.Id == g
select q
This is because the constructor call cannot be translated to SQL.

Sql's groupby and linq's groupby return differently shaped results.
Sql's groupby returns keys and aggregates (no group members)
Linq's groupby returns keys and group members.
If you use those group members, they must be (re-)fetched by the grouping key. This can result in +1 database roundtrip per group.

well, i had the same question some time ago.
basically: your linq-statement is converted to a sql-statement. however: some groups will get translated, others not - depending on how you write your statement.
so yes - both is possible
example:
var a = (from entity in myTable where entity.Property == 1 select entity).ToList();
versus
var a = (from entity in myTable.ToList() where entity.Property == 1 select entity).ToList();

Select clause containing non-EF method calls

I'm having trouble building an Entity Framework LINQ query whose select clause contains method calls to non-EF objects.
The code below is part of an app used to transform data from one DBMS into a different schema on another DBMS. In the code below, Role is my custom class unrelated to the DBMS, and the other classes are all generated by Entity Framework from my DB schema:
// set up ObjectContext's for Old and new DB schemas
var New = new NewModel.NewEntities();
var Old = new OldModel.OldEntities();
// cache all Role names and IDs in the new-schema roles table into a dictionary
var newRoles = New.roles.ToDictionary(row => row.rolename, row => row.roleid);
// create a list or Role objects where Name is name in the old DB, while
// ID is the ID corresponding to that name in the new DB
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select new Role { Name = r.RoleName, ID = newRoles[r.RoleName] };
var list = roles.ToList();
But calling ToList gives me this NotSupportedException:
LINQ to Entities does not recognize
the method 'Int32
get_Item(System.String)' method, and
this method cannot be translated into
a store expression
Sounds like LINQ-to-Entities is barfing on my call to pull the value out of the dictionary given the name as a key. I admittedly don't understand enough about EF to know why this is a problem.
I'm using devart's dotConnect for PostgreSQL entity framework provider, although I assume at this point that this is not a DBMS-specific issue.
I know I can make it work by splitting up my query into two queries, like this:
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select r;
var roles2 = from r in roles.AsEnumerable()
select new Role { Name = r.RoleName, ID = newRoles[r.RoleName] };
var list = roles2.ToList();
But I was wondering if there was a more elegant and/or more efficient way to solve this problem, ideally without splitting it in two queries.
Anyway, my question is two parts:
First, can I transform this LINQ query into something that Entity Framework will accept, ideally without splitting into two pieces?
Second, I'd also love to understand a little about EF so I can understand why EF can't layer my custom .NET code on top of the DB access. My DBMS has no idea how to call a method on a Dictionary class, but why can't EF simply make those Dictionary method calls after it's already pulled data from the DB? Sure, if I wanted to compose multiple EF queries together and put custom .NET code in the middle, I'd expect that to fail, but in this case the .NET code is only at the end, so why is this a problem for EF? I assume the answer is something like "that feature didn't make it into EF 1.0" but I am looking for a bit more explanation about why this is hard enough to justify leaving it out of EF 1.0.

The problem is that in using Linq's delayed execution, you really have to decide where you want the processing and what data you want to traverse the pipe to your client application. In the first instance, Linq resolves the expression and pulls all of the role data as a precursor to
New.roles.ToDictionary(row => row.rolename, row => row.roleid);
At that point, the data moves from the DB into the client and is transformed into your dictionary. So far, so good.
The problem is that your second Linq expression is asking Linq to do the transform on the second DB using the dictionary on the DB to do so. In other words, it is trying to figure out a way to pass the entire dictionary structure to the DB so that it can select the correct ID value as part of the delayed execution of the query. I suspect that it would resolve just fine if you altered the second half to
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select r.RoleName;
var list = roles.ToDictionary(roleName => roleName, newRoles[roleName]);
That way, it resolves your select on the DB (selecting just the rolename) as a precursor to processing the ToDictionary call (which it should do on the client as you'd expect). This is essentially exactly what you are doing in your second example because AsEnumerable is pulling the data to the client before using it in the ToList call. You could as easily change it to something like
var roles = from rl in Old.userrolelinks
join r in Old.roles on rl.RoleID equals r.RoleID
where rl.UserID == userId
select r;
var list = roles.AsEnumerable().Select(r => new Role { Name = r.RoleName, ID = newRoles[r.RoleName] });
and it'd work out the same. The call to AsEnumerable() resolves the query, pulling the data to the client for use in the Select that follows it.
Note that I haven't tested this, but as far as I understand Entity Framework, that's my best explanation for what's going on under the hood.

Jacob is totally right.
You can not transform the desired query without splitting it in two parts, because Entity Framework is unable to translate the get_Item call into the SQL query.
The only way is to write the LINQ to Entities query and then write a LINQ to Objects query to its result, just as Jacob advised.
The problem is Entity-Framework-specific one, it does not arise from our implementation of the Entity Framework support.