We Keep Coding

Can't add calculated value to IQueryable

I'm running an EF statement where I need to calculate de deductibles. After long trying, I can't seem to add a custom function in a .Select() statement. Instead I'm trying to add the values after my .Select() statement.
The problem here is, in my CalculateDeductibles() I can't seem to add any values to item.Deductibles.
The GetDeductibles(item.RequestId) is a rather heavy funtion that does several extra queries, so I'm trying to prevent to convert my IQueryable to an IList object.
So there are actually 2 questions:
Can I have the GetDeductibles() function directly in my .Select() statement?
Can I somehow (with keeping an eye on performance) add the value after I did my .Select()
Code:
public IQueryable<ReinsuranceSlip> GetReinsuranceSlipsOverview(int userId, int companyId, string owner, string ownerCompany)
{
IQueryable<ReinsuranceSlip> model = null;
model = _context.Request
.Where(w => w.RequestGroup.ProgramData.MCContactId == userId)
.Select(x => new ReinsuranceSlip()
{
Id = x.Id,
RequestId = x.Id,
LocalPolicyNumber = x.LocalPolicyNumber,
BusinessLine = x.RequestGroup.ProgramData.BusinessLine.DisplayName,
BusinessLineId = x.RequestGroup.ProgramData.BusinessLine.Id,
ParentBroker = x.RequestGroup.ProgramData.Broker.Name,
LocalBroker = x.Broker.Name,
InceptionDate = x.InceptionDate,
RenewDate = x.RenewDate,
//Deductibles = CalculateDeductibles(x)
});
CalculateDeductibles(model);
return model;
}
private void CalculateDeductibles(IQueryable<ReinsuranceSlip> model)
{
//model.ForEach(m => m.Deductibles = GetDeductibles(m.RequestId));
foreach (var item in model)
{
item.Deductibles = GetDeductibles(item.RequestId);
}
}

Updated and Sorry for the first version of this answer. I didn't quite understand.
Answer 1: IQueryable is using to creating a complete SQL statement to call in SQL Server. So If you want to use IQueryable, your methods need to generate statements and return it. Your GetDetuctibles method get request Id argument but your queryable model object didn't collect any data from DB yet, and it didn't know x.Id value. Even more, your GetCarearDetuctiples get an argument so and with that argument generates a queryable object and after some calculations, it returns decimal. I mean yes you can use your methods in select statement but it's really complicated. You can use AsExpendable() LINQ method and re-write your methods return type Expression or Iqueryable.
For detailed info you should check. This:
Entity Navigation Property IQueryable cannot be translated into a store expression and this: http://www.albahari.com/nutshell/predicatebuilder.aspx
And you also should check this article to understand IQueryable interface: https://samueleresca.net/2015/03/the-difference-between-iqueryable-and-ienumerable/
Answer 2: You can use the IEnumerable interface instead IQueryable interface to achieve this. It will be easy to use in this case. You can make performance tests and improve your methods by time.
But if I were you, I'd consider using Stored Procedures for performance gain.

You'll have to understand the differences between an IEnumerable and an IQueryable.
An IEnumerable object holds everything to enumerate over the elements in the sequence that this object represents. You can ask for the first element, and once you've got it, you can repeatedly ask for the next element until there is no more next element.
An IQueryable works differently. An IQueryable holds an Expression and a Provider. The Expression is a generic description of what data should be selected. The Provider knows who has to execute the query (usually a database), and it knows how to translate the Expression into a format that the Provider understands.
There are two types of LINQ functions: the ones that return IQueryable<TResult> and the ones that return TResult. Functions form the first type do not execute the query, they will only change the expression. They use deferred execution. Functions of the second group will execute the query.
When the query must be executed, the Provider takes the Expression and tries to translate it into the format that the process that executes the query understand. If this process is a relational database management system this will usually be SQL.
This translation is the reason that you can't add your own functionality: the Expression must be translatable to SQL, and the only thing that your functions may do is call functions that will change the Expression to something that can be translated into SQL.
In fact, even entity framework does not support all LINQ functionalities. There is a list of Supported and Unsupported LINQ methods
Back to your questions
Can I have GetDeductibles directly in my query?
No you can't, unless you can make it thus simple that it will only change the Expression using only supporte LINQ methods. You'll have to write this in the format of an extension function. See extension methods demystified
Your GetDeductibles should have an IQueryable<TSource> as input, and return an IQueryable<TResult> as output:
static class QueryableExtensions
{
public static IQueryable<TResult> ToDeductibles<TSource, TResult, ...>(
this IQueryable<TSource> source,
... other input parameters, keySelectors, resultSelectors, etc)
{
IQueryable<TResult> result = source... // use only supported LINQ methods
return result;
}
}
If you really need to call other local functions, consider calling AsEnumerable just before calling the local functions. The advantage above ToList is that smart IQueryable providers, like the one in Entity Framework will not fetch all items but the items per page. So if you only need a few ones, you won't have transported all data to your local process. Make sure you throw away all data you don't need anymore before calling AsEnumerable, thus limiting the amount of transported data.
Can I somehow add the value after I did my .Select()
LINQ is meant to query data, not to change it. Before you can change the data you'll have to materialize it before changing it. In case of a database query, this means that you have a copy of the archived data, not the original. So if you make changes, you'll change the copies, not the originals.
When using entity framework, you'll have to fetch every item that you want to update / remove. Make sure you do not select values, but select the original items.
NOT:
var schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.Select(school = new
{
... // you get a copy of the values: fast, but not suitable for updates
})
.FirstOrDefault();
BUT:
School schoolToUpdate = schoolDbContext.Schools.Where(schoolId = 10)
.FirstOrDefault()
Now your DbContext has the original School in its ChangeTracker. If you change the SchoolToUpdate, and call SaveChanges, your SchoolToUpdate is compared with the original School, to check if the School must be updated.
If you want, you can bypass this mechanism, by Attaching a new School directly to the ChangeTracker, or call a Stored procedure.

Understanding System.Data.Linq.Table<T>

I'm looking at this blog:
If you scroll down a bit there's a section called The DataContext, where it says
Inside DataClasses1DataContext is property called Customers
public System.Data.Linq.Table<Customer> Customers
{
get
{
return this.GetTable<Customer>();
}
}
and then right below that:
If you write a LINQ query that retrieves customer records from the database, then you can access that data via this property. It is of type Table, where the internal Table class becomes in this case a collection of Customer records. One would typically access this property by writing code that looks like this:
DataClasses1DataContext db = new DataClasses1DataContext(ConnectionString);
var query = from c in db.Customers
select c;
Does this mean that at the point the property is referenced the entire Customer table is queried and returned?
if so, isn't this a bit inefficient if there's a lot of records in the table, especially if you just want to retrieve one customer?
Thanks

Does this mean that at the point the property is referenced the entire Customer table is queried and returned?
No - Table<T> implements IQueryable<T> which means that queries are deferred until the collection is enumerated.
Where, OrderBy, Select, and other clauses are "attached" to the query and converted to SQL when the query is either enumerated or converted to a linq-to-objects query via AsEnumerable().
Note that the article you reference is discussing Linq-to-SQL, but the principle is the same in Linq-to-Enitites (Entity Framework). Queries are build up using where, orderby, group by, select, etc. and converted to SQL when the query is executed.

Selecting rows in a DBSet with Entity Framework

I'm trying to get rows based on a WHERE clause in a DbSet object. I have this:
dbContext.Workers
I can get a list like this:
workers = m.Workers.Where(w => w.BranchId == curUser.BranchId).ToList<Worker>();
But as you can see, it returns a List<Worker> and I can't use methods like workers.Find(WorkerId) with it.
Basically, I'm trying to return a DBSet based on some filter I mean I want to use LINQ on a DBSet class. I want this because I need to use workers.Find(WorkerId) also maybe I will need to update this model. So, I will get a list based on where clause, I will change some values and will use dbContext.SaveChanges(). Is that possible?
Thanks

Where(...) returns an IQueryable, which you can manipulate BEFORE the query runs. ToList() will force execution of the query and put the objects into memory. If you want to 'find' an item AFTER the query then you could do something like this with your List:
workers.SingleOrDerfault(x => x.WorkerId == WorkerId);
If you have them all in memory like this, and make changes, then you will persist those changes with a call to .SaveChanges().
However, if you need to apply more filtering to your IQueryable BEFORE the query hits the database, then you'll want to manipulate the IQueryable BEFORE the call to ToList().

Should I be specifying IQueryable over List in my interfaces?

I'd like to know whether, when defining my interface, I should prefer IQueryable over List for groups of objects. Or perhaps IEnumerable is better, since IEnumerable types can be cast to IQueryable to be used with LINQ.
I was looking at a course online and it was dealing with EntityFramework, for which it is best to use IQueryable since LINQ uses it (okay there's more to it than that but probably not important right now).
The code below was used, and it got me thinking if I should be specifying IQueryable instead of List for my groups of objects.
namespace eManager.Domain
{
public interface IDepartmentDataSource
{
IQueryable<Employee> Employees { get; }
IQueryable<Department> Departments { get; }
}
If I were building an interface for a service that calls the repository to get Employees, I would usually specify
List <Employees>
but is this best practice? Would IQueryable give more flexibility to classes that implement my interface? What about the overhead of having to import LINQ if they don't need it (say they only want a List)? Should I use IEnumerable over both of these?

The IQueryable interface is in effect a query on Entity Framework that is not yet executed. When you call ToList(), First() or FirstOrDefault() etc. on IQueryable Entity Framework will construct the SQL query and query the database.
IEnumerable on the other hand is 'just' an enumerator on a collection. You can use it to filter the collection but you'd use LINQ to Objects. Entity Framework doesn't come into play here.
To answer your question: it depends. If you want the clients of your repository to be able to further customize the queries they can execute you should expose IQueryable, but if you want full control in your repository on how the database is queried you could use IEnumerable.
I prefer to use IEnumerable, because that doesn't leak the use of Entity Framework throughout the application. The repository is responsible for database access. It is also easier to make LINQ to EF optimizations, because the queries are only in the repository.

What I usually do is make the repositories return IQueryable. Then in the BL I specify either IEnumerable or IQueryble. It is important to know the main differences between IQueryble and IEnumerable.
Lets say you fetch data into IEnumerable
IEnumerable employees=this.repository.GetAll();
Now let's say this specific function require only employees with age over 21 and the others are not needed at all.
You would do:
employees.Where(a=>a.Age>21)
In this case the original query will be loaded in the memory and then the Where will be applied.
Now lets say you change the function to fetch the data into IQueryable
IQueryable employees=this.repository.GetAll();
employees.Where(a=>a.Age>21)
This time when you modify the query with the Where clause, the whole query will be executed in the database (if possible) and you will only get employees with age over 21 from the database.
In the IEnumerable case you will get all the employees from the database and then they will get filtered in memory to satisfy the where condition.
Use IEnumerable, IList or something else?
If you know what operations will be executed on the collection you can easily choose which interface to use. Basically if you will only iterate over the collection you would use IEnumerable. If you would do more operations you need to choose the appropriate interface. There are good videos of .NET collections in pluralsight.

NHibernate IQueryable collection as property of root

I have a root object that has a property that is a collection.
For example:
I have a Shelf object that has Books.
// Now
public class Shelf
{
public ICollection<Book> Books {get; set;}
}
// Want
public class Shelf
{
public IQueryable<Book> Books {get;set;}
}
What I want to accomplish is to return a collection that is IQueryable so that I can run paging and filtering off of the collection directly from the the parent.
var shelf = shelfRepository.Get(1);
var filtered = from book in shelf.Books
where book.Name == "The Great Gatsby"
select book;
I want to have that query executed specifically by NHibernate and not a get all to load a whole collection and then parse it in memory (which is what currently happens when I use ICollection).
The reasoning behind this is that my collection could be huge, tens of thousands of records, and a get all query could bash my database.
I would like to do this implicitly so that when NHibernate sees an IQueryable on my class it knows what to do.
I have looked at NHibernate's LINQ provider and currently I am making the decision to take large collections and split them into their own repository so that I can make explicit calls for filtering and paging.
LINQ To SQL offers something similar to what I'm talking about.

I've been trying to come up with a solution for a similar problem.
You can filter collections off an entity using ISession.FilterCollection. This creates an additional IQuery where you can count, page, add criteria, etc.
So, for example (my query in FilterCollection may be a little off, but you should get the idea):
ISession session = GetSession();
var shelf = session.Get<Shelf>(id);
var books = session.FilterCollection(shelf.Books, "where Name = :title").SetString("title", "The Great Gatsby").List<Book>();
There are a problem with that, however:
The consumer executing the code
needs to access
ISession.CreateFilter, or you need
to create a method on your
repository that takes in a property,
a query, and your query arguments
(as well as any paging or other
information). Not really the sexiest
thing on the planet.
It's not the LINQ you wanted.
Unfortunately, I don't think there's any way to get what you want out of the box with NHibernate. You could fake it, if you wanted to try, but they seem to fall flat to me:
Add a method or property that under the covers returns a LINQ to NHibernate IQueryable for this shelf:
public IQueryable<Book> FindBooks() {
return Resolver.Get<ISession>().Linq<Book>().Where(b => b.Shelf == this);
}
where someone might consume that like this:
var shelf = ShelfRepo.Get(id);
var books = (from book shelf.FindBooks()
where book.Title == "The Great Gatsby"
select book);
Yuck! You are bleeding your persistence needs through your domain model! Maybe you could make it a little less worse by having a repository emit IQueryable, which at runtime is actually LINQ to NHibernate:
public IQueryable<Book> FindBooks() {
return Resolver.Get<IRepository<Book>>().CreateQuery().Where(b => b.Shelf == this);
}
Still pretty blah.
Create your own custom collection type (and potentially an IQueryable implementation) that wraps a private field of the actual books, and map NHibernate to that field. However, it may be a difficult undertaking getting that working with ISession.CreateFilter. You have to consider "discovering" the current session, converting the LINQ expression into something you can use in CreateFilter, etc. Plus, your business logic is still dependent on NHibernate.
Nothing really satisfies at this point. Until NHibernate can do LINQ over a collection for you, it appears that you're better off just querying your Book repository normally as has already been suggested, even if it doesn't seem as sexy or optimal.

I tend to think like this:
Aggregate roots are boundaries of consistency, so if shelf needs to enforce some sort of consistency policies on the books it contains, then it should be an aggregate root.
And in such case it should hold a set/collection of books.
If you don't need to enforce consistency in any way from shelf to books, then I'd consider to remove the set/collection property and move those queries into a repository instead.
Also, since pagination and filtering most likely don't have anything to do with your domain logic, it is most likely for presentation.
Then I'd consider to make some special view for it instead of adding presentation facillities to my repositories.
e.g.
var result = Queries.FindBooksByShelf(shelfId,pageSize);
Such query could return projections and/or be optimized as plain SQL etc.
They are most likely specific for a certain view or report in your GUI.
This way your domain will focus on domain concepts only.

Perhaps you should give Nhibernate Linq a try. It allows you use the IQueryable and do things like:
Session.Linq<Book>().Where(b => b.Name == "The Great Gatsby");