We Keep Coding

LINQ using String Format

I am getting the error about LINQ to Entities does not recognize the method 'System.String Format but in the past I was able to do this when I have included .AsEnumerable() is there something different I need to do because of the GroupBy section?
select new PresentationLayer.Models.PanelMeeting
{
GroupId = pg.GroupId,
MeetingId = pmd.MeetingId,
GuidelineName = pmv.GuidelineName,
PanelDisclosuresAttendanceURL = string.Format("{0}?MeetingId={1}&GroupId=0",PanelDisclosureLink, pmd.MeetingId),
}).GroupBy(g => new
{
g.MeetingId,
g.GroupId
})
.AsEnumerable()
.SelectMany(grp => grp.AsEnumerable()).ToList(),

You have to be aware of the difference between an IEnumerable<...> and an IQueryable<...>.
IEnumerable
An object that implements IEnumerable<...> represents a sequence of similar items. You can ask for the first element of the sequence, and as long as you've got elements you can ask for the next element. IEnumerable objects are supposed to be executed within your own process. IEnumerable objects hold everything to enumerate the sequence.
At its lowest level, this is done using GetEnumerator() / MoveNext() / Current:
IEnumerable<Customer> customers = ...
IEnumerator<Customer> enumerator = customers.GetEnumerator();
while (enumerator.MoveNext())
{
// There is a next Customer
Customer customer = enumerator.Current;
ProcessCustomer(customer);
}
If you use foreach, then internally GetEnumerator / MoveNext / Current are called.
If you look closely to LINQ, you will see that there are two groups of LINQ methods. Those that return IEnumerable<TResult> and those that dont't return IEnumerable<...>
LINQ functions from the first group won't enumerate the query. They use deferred execution, or lazy execution. In the comments section of every LINQ method, you'll find this description.
The LINQ functions of the other group will execute the query. If you look at the reference source of extension class Enumerable, you'll see that they internally use foreach, or at lower level use GetEnumerator / MoveNext / Current
IQueryable
An object that implements IQueryable<...> seems like an IEnumerable. However, it represents the potential to fetch data for an Enumerable sequence. The data is usually provided by a different process.
For this, the IQueryable holds an Expression and a Provider. The Expression represents what must be fetched in some generic format. The Provider knows who will provide the data (usually a database management system) and how to communicate with this DBMS (usually SQL).
When you start enumerating the sequence, deep inside using GetEnumerator, the Expression is sent to the Provider, who will try to translate it into SQL. The data is fetched from the DBMS, and returned as an Enumerable object. The fetched data is accessed by repeatedly calling MoveNext / Current.
Because the database is not contacted until you start enumerating, you'll have to keep the connection to the database open until you've finished enumerating. You've probably made the following mistake once:
IQueryable<Customer> customers;
using (var dbContext = new OrderDbContext(...))
{
customers = dbContext.Customers.Where(customer => customer...);
}
var fetchedCustomers = customers.ToList();
Back to your question
In your query, you use string.Format(...). Your Provider doesn't know how to translate this method into SQL. Your Provider also doesn't know any of your local methods. In fact, there are even several standard LINQ methods that are not supported by LINQ to entities. See Supported and Unsupported LINQ methods.
How to solve the problem?
If you need to call unsupported methods, you can use AsEnumerable to fetch the data. All LINQ methods after AsEnumerable are executed by your own process. Hence you can call any of your own functions.
Database Management systems are extremely optimized in table handling. One of the slower parts of a database query is the transport of the selected data to your local process. Hence, let the DBMS do all selecting, try to transport as little data as possible.
So let your DBMS do your Where / (Group-)Join / Sum / FirstOrDefault / Any etc. String formatting can be done best by you.
In your String.Format you use PanelDisclosureLink and pmd.MeetingId. It will probably be faster if your process does the formatting. Alas you forgot to give us the beginning or your query.
I'm not sure where your PanelDisclosureLink comes from. Is it a local variable? If that is the case, then PanelDisclosuresAttendanceURL will be the same string for every item in your group. Is this intended?
var panelDisclosureLine = ...;
var result = dbContext... // probably some joining with Pgs, Pmds and Pmvs,
.Select(... => new
{
GroupId = pg.GroupId,
MeetingId = pmd.MeetingId,
GuidelineName = pmv.GuidelineName,
})
// make groups with same combinations of [MeetingId, GroupId]
.GroupBy(joinResult => new
{
MeetingId = joinResult.MeetingId,
GroupId = joinResult.GroupId,
},
// parameter resultSelector: use the Key, and all JoinResult items that have this key
// to make one new:
(key, joinResultItemsWithThisKey) => new
{
MeetingId = key.MeetingId,
GroupId = key.GroupId,
GuideLineNames = joinResultsItemsWithThisKey
.Select(joinResultItem => joinResultItem.GuideLineName)
.ToList(),
})
So by now the DBMS has transformed your join result into objects with
[MeetingId, GroupId] combinations and a list of all GuideLineNames that have belong to
this [MeetingId, GroupId] combination.
Now you can move it to your local process and use String.Format.
.AsEnumerable()
.SelectMany (fetchedItem => fetchedItem.GuideLineNames,
(fetchedItem, guideLineName) => PresentationLayer.Models.PanelMeeting
{
GroupId = fetchedItem.GroupId,
MeetingId = fetchedItem.MeetingId,
GuidelineName = guidelineName,
PanelDisclosuresAttendanceURL = string.Format("...",
PanelDisclosureLink,
fetchedItem.MeetingId);
Note: in my parameter choice plurals are collections; singulars are elements of these collections.
PanelDisclosuresAttendanceURL = string.Format("{0}?MeetingId={1}&GroupId=0",PanelDisclosureLink, pmd.MeetingId),
}).
.GroupBy

If you want to use string.Format you first have to get the data from the server.
You can just move the .GroupBy( ... ) and then the .AsEnumerable() call to the top, before select new PresentationLayer.Models.PanelMeeting { ... }. If you are not selecting too much data that way...

Create DateTime within LINQ query

I am getting the following error:
Only parameterless constructors and initializers are supported in LINQ
to Entities.
I am trying to query data from a result set, and part of the new result set returns a new date field.
var result = debts.Select(x => new
{
x.Snowball.User.Email,
x.SnowballID,
x.Description,
x.ID,
x.DueDayOfMonth,
Due = new DateTime(ExecutionDate.Year,
ExecutionDate.Month,
x.DueDayOfMonth)
}).ToList();
Is there a way I can create this deduced datetime?

I came here looking for the same thing, but on further research found another way using DbFunctions.CreateDateTime in System.Data.Entity. Using this method your query would look like this:
var result = debts.Select(x => new
{
x.Snowball.User.Email,
x.SnowballID,
x.Description,
x.ID,
x.DueDayOfMonth,
Due = DbFunctions.CreateDateTime(ExecutionDate.Year,
ExecutionDate.Month,
x.DueDayOfMonth, 0,0,0)
}).ToList();
EDIT: This is for EF6 and later, for earlier versions of EF use System.Data.Entity.Core.Objects.EntityFunctions instead of DbFunctions

LINQ to Entities has a boundary for the query.
The IQueryable result allows one or more filters and selections etc. to be added step by step if you wish.
While the query is being added to nothing has hit the database yet; only when you access the result set will the query be constructed by the LINQ Provider (a single Transact SQL Query for example), then run on the database, and its result returned.
In the case you have shown, you are trying to create a new object (Date - but could also be one of your own classes) in the database's result set.
So, for each row returned from the database query you are trying to create a column from an object (from your code base) different for each row (need to pass the parameters to the constructor).
Because the underlying provider cannot guarantee to be able to create arbitrary objects from an external source on the fly, this is not allowed (prevented by the LINQ Provider).
The query that will be run against the database is 'debts'. This will return all of the rows from the query unfiltered (as there is no WHERE clause).
The next query will be run in your code. In your code you can use parameterised constructors - and is now LINQ To Objects.
So the way to distinguish the separate stages of the query, and do what you want to do, is to place the ToList() at the end of the query that runs against the database so that the query is generated, run, and the results returned.
Then construct and run your next query (in your code) on those results.
var result = debts.ToList().Select(x => new
{
x.Snowball.User.Email,
x.SnowballID,
x.Description,
x.ID,
x.DueDayOfMonth,
Due = new DateTime(ExecutionDate.Year, ExecutionDate.Month, x.DueDayOfMonth)
});

You could create your own Date class with a parameter-less constructor and set it using property initializer syntax like
Due = new Date(){Year = ExecutionDate.Year, Month = ExecutionDate.Month, Day = x.DueDayOfMonth}
Then just define a cast from Date to DateTime.

You may try this:
var result = debts.Select(x =>
new
{
x.Snowball.User.Email, x.SnowballID, x.Description, x.ID,
x.DueDayOfMonth,
exY = ExecutionDate.Year,
exM = ExecutionDate.Month,
exD = x.DueDayOfMonth)
}).ToList()
.Select(a => new {Email, SnowballID, Description, ID, Due=new DateTime(exY, exM, exD)})
.ToList()

Code First EF6 - How to translate to a custom object thats not an entity

I have a Stored Proc that's returning 2 columns. The first is an ID column to an entity (UserId) the next is a count of how many UserIds(Count).
In Code, I would like to translate this into a Dictionary, where User is an EF entity.
var result = ((IObjectContextAdapter)dbContext).ObjectContext.Translate<Dictionary<User, int>>(reader).FirstOrDefault();
This returns an empty Dictionary and it appears translate isn't working the way I'm hoping it would.
I'm guessing since I'm only returning the UserId it's not able to materialize into the complete User entity.
Can someone provide an example of how to get the end result I'm looking for?

Look at the documentation:
The supplied DbDataReader must contain data that maps to the requested entity type.
So there are two things wrong in your code:
The type you're trying to materialize is a dictionary, not an entity type.
And if is was an entity type, the reader wouldn't contain data that maps to the requested entity type, i.e. rows with values that correspond with the entity's properties.
You have to get the users by a separate query, using the id values that were returned from the stored procedure. Something along these lines:
var sprocValues = GetStoredProcedureValues();
// Let's assume this returns a List of Tuple<int,int>
var usrIds = sprocValues.Select(t => t.Item1).ToArray();
var users = dbContext.Users.Where(u => usrIds.Contains(u.UserID)).ToArray();
var result = from u in users
join t in sprocValues on u.UserID equals t.Item1
select new { User = u, Count = t.Item2 };

How can you retrieve all records of certain subtype dynamically with Linq to Entities?

I am trying to get a list of all objects in the database of a specified type. I have done this before when the type was known at compile time, but now I am trying to pass a type into the method and have the method return all the records of that specified type, and I can't get it working. I have tried the following:
public IList<WritingObject> GetBasicObjectsByProject(int projectId, Type oType)
{
var results = from o in _objects.AsQueryable
where o.Project.Id == projectId
&& o.GetType() == oType
select o;
return results.ToList<WritingObject>();
}
This didn't work because Linq to Entities doesn't support the GetType() method. Next I tried
var results = from o in _objects.AsQueryable
where o.Project.Id == projectId
&& o is oType
select o;
This doesn't work because the compiler claims that oType is not of a known type. Using typeof(oType) produces the same error, as does performing OfType<oType>() on the IQueryable.
I'm running out of ideas to keep this dynamic without splitting it out into one method per sub-type. Does anyone have any ideas?

Something like: ?
var query = from o in _objects.AsQueryable()
where o.Project.Id == projectId
select o;
var ofType = typeof (Queryable).GetMethod("OfType").MakeGenericMethod(oType);
var list = (IQueryable)ofType.Invoke(
null, new object[] {query}).Cast<WritingObject>().ToList();
Note this will include other sub-sub-types too.

I think the issue here is that Linq-to-Entities is trying to translate o.GetType into SQL, which it obviously can't. After re-reading the question I realised that it's to do with linq to entities not being able to map a CLR type to a database type. Linq to Entities help talks about that:
The LINQ standard query operators that
deal with CLR type conversion and
testing are supported in the Entity
Framework. Only CLR types that map to
conceptual model types are supported
in LINQ to Entities. For a list of
conceptual model types, see Conceptual
Model Types.
The above means that it can only convert a handful of supported CLR types into appropriate EDM types as a part of query generation. If the type you are trying to use is not on that list, you will need to filter on type after retrieving the query result.
The solution to this problem might be inefficient - you will need to request all records matching projectId criteria and then filter them by type:
var results = (from o in _objects.AsQueryable
where o.Project.Id == projectId
select o).ToList(); //force query execution here
results = from o in results
where o.GetType() == oType
select o;
return results.ToList<WritingObject>();

Return IQueryable containing 2 Subclasses

Can you return IQueryable which is composed of two or more different subclasses ? Here's my attempt to show what I mean. It throws the error:
System.NotSupportedException: Types in
Union or Concat have members assigned
in different order..
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new OrderItemA {
// etc
} as OrderItem;
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new OrderItemB {
//etc
} as OrderItem;
return a.Concat<OrderItem>(b);

Try doing the concat on IEnumerable instead of IQueryable:
return a.AsEnumerable().Concat(b.AsEnumerable());
If you need an IQueryable result you could do this:
return a.AsEnumerable().Concat(b.AsEnumerable()).AsQueryable();
Doing this will force the concat to happen in-memory instead of in SQL, and any additional operations will also happen in-memory (LINQ To Objects).
However, unlike the .ToList() example, the execution should still be deferred (making your data lazy loaded).

My guess is that this is because you are using LINQ in an LINQ-to-SQL context.
So using Concat means that LINQ2SQL will need to join both query into a SQL UNION query which might be where the System.NotSupportedException originated from.
Can you try this:
return a.ToList().Concat<OrderItem>(b.ToList());
And see if it make any difference?
What the above does is that it executes the query twice and then concatenate them in-memory instead of hot-off-SQL as to avoid the query translation problem.
It might not be the ideal solution, but if this work, my assumption is probably correct, that it's a query translation problem:
More information about Union and Concat translation to SQL:
http://blog.benhall.me.uk/2007/08/linq-to-sql-difference-between-concat.html
http://msdn.microsoft.com/en-us/library/bb399342.aspx
http://msdn.microsoft.com/en-us/library/bb386979.aspx
Hope this helps.

Interestingly after reading your post and a bit of testing, I realized that what your actually doing does seem to work just fine for me given that the projection part you show as ellipsis in both of your queries match. You see, LINQ to SQL appears to construct the underlying projection for the SQL select command based off of the property assignment statements as opposed to the actual type being materialized so as long as both sides have the same number, type, and order (not sure about this) of member assignments the UNION query should be valid.
My solution that I've been working with is to create a property on my DataContext class which acts much like a SQL View in that it allows me to write a query (in my case a Union between two different tables) and then use that query as if it is itself like a table when composing read-only select statements.
public partial class MyDataContext
{
public IQueryable<MyView> MyView
{
get
{
var query1 =
from a in TableA
let default_ColumnFromB = (string)null
select new MyView()
{
ColumnFromA = a.ColumnFromA,
ColumnFromB = default_ColumnFromB,
ColumnSharedByAAndB = a.ColumnSharedByAAndB,
};
var query2 =
from a in TableB
let default_ColumnFromA = (decimal?)null
select new MyView()
{
ColumnFromA = default_ColumnFromA,
ColumnFromB = b.ColumnFromB,
ColumnSharedByAAndB = b.ColumnSharedByAAndB,
};
return query1.Union(query2);
}
}
}
public class MyView
{
public decimal? ColumnFromA { get; set; }
public string ColumnFromB { get; set; }
public int ColumnSharedByAAndB { get; set; }
}
Notice two key things:
First of all the projection formed by the queries which make up both halves of the Union have the same number, type, and order of columns. Now LINQ may require the order to be the same (not sure about this) but it is definitely true that SQL does for a UNION and we can be sure that LINQ will require at least the same type and number of columns and these "columns" are known by the member assignments and not from the properties of the type you are instantiating in your projection.
Secondly LINQ currently doesn't allow for multiple constants to be used within a projections for queries which formulate a Concat or Union and from my understanding this is mainly because these two separate queries are separately optimized before the Union operation is processed. Normally LINQ to SQL is smart enough to realize that if you have a constant value which is only being used in the projection, then why send it to SQL just to have it come right back the way it was instead of tacking it on as a post process after the raw data comes back from SQL Server. Unfortunately the problem here is that this is a case of LINQ to SQL being to smart for it's own good, as it optimizes each individual query too early in the process. The way I've found to work around this is to use the let keyword to form a range variable for each value in the projection which will be materialized by getting it's value from a constant. Somehow this tricks LINQ to SQL into carrying these constants through to the actual SQL command which keeps all expected columns in the resulting UNION. More on this technique can be found here.
Using this techinque I at least have something reusable so that no matter how complex or ugly the actual Union can get, especially with the range variables, that in your end queries you can write queries to these pseudo views such as MyView and deal with the complexity underneath.

Can you do the projection after the concat?
// construct the query
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new {
type = "A"
item = oi
}
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new {
type = "B"
item = oi
}
var temp = a.Concat<OrderItem>(b);
// create concrete types after concatenation
// to avoid inheritance issue
var result = from oi in temp
select (oi.type == "A"
? (new OrderItemA {
// OrderItemA projection
} as OrderItem)
: (new OrderItemB {
// OrderItemB projection
} as OrderItem)
);
return result
Not sure if the ternary operator works in LINQ2SQL in the above scenario but that might help avoid the inheritance issue.