I am trying to get the first and last values in a list. The query operator First() is supported but Last() and LastOrDefault() give an error. Am I using the Last() operator incorrectly?
var purchaseBills = db.PurchaseBills.OrderBy(p => p.BillID);
if (purchaseBills.Count() >0)
{
var firstBill = purchaseBills.First(); // This is supported
// Attempt 1
var lastBill = purchaseBills.Last(); // Not supported
// Attempt 2
var lastBill = purchaseBills.LastOrDefault(); // Not supported
//Attempt 3
var lastBill = purchaseBills.Reverse().First(); // Not supported
textBoxPurchaseBillFrom.Text = firstBill.BillNo.ToString();
textBoxPurchaseBillTo.Text = lastBill.BillNo.ToString();
}
Update:
--Errors--
Attempt 1: The query operator 'Last' is not supported.
Attempt 2: The query operator 'LastOrDefault' is not supported.
Attempt 3: The query operator 'Reverse' is not supported.
Instead of putting it into an own list by calling ToList() or ToArray() i would prefer to use AsEnumerable().
Additionally like the others you should try OrderByDescending()
Instead of Count() i would use Any().
either you switch your OrderBy to
.OrderByDescending(p => p.BillID)
(and use first) or you do something like
purchaseBills.ToArray().Last()
if this is not to expensive.
Last is not supported by the back-end DB. You should try other techniques:
Run your query using OrderByDescending so your requested item comes first.
Code your LINQ query as usual, but enforce Linq2Sql to render it to a CLR collection and then you'll have free access to everything locally, including Last. Example:
var bills = purchaseBills.ToList();
var last = bills.Last();
The problem is that there's no easy translation into SQL for Last or Reverse, so either convert it to something in memory (ToList, ToArray) if there aren't going to be too many records, or you could run the query a 2nd time, with OrderByDescending instead of OrderBy and use First.
I try not to use LastOrDefault() because sometime it doesn't work or support.
I'm sorting id by desc and then grab the first records.
.OrderByDescending(o=>o.id)
.FirstOrDefault(s => s.Name == Name)
It has something to do with the fact that the Last operator is trying to be sent to the SQL server which has no corresponding command. Once solution is to put a ToArray() or Tolist() at the end of your db call which makes that one line an explicit call to get the data (instead of lazing loading on each of the other lines).
Yet another way get last element without orderbydescending and load all entities:
var lastBill = purchaseBills
.Where(f => f.BillID == purchaseBills.Max(f2 => f2.BillID))
.FirstOrDefault();
You can convert your IEnumerable into a List using the ToList() method, which will ensure that all your attempts are successful, as shown:
var purchaseBills = db.PurchaseBills.OrderBy(p => p.BillID);
if (purchaseBills.Any())
{
var firstBill = purchaseBills.First(); // This is supported
// Attempt 1
var lastBill = purchaseBills.ToList().Last();
// Attempt 2
var lastBill = purchaseBills.ToList().LastOrDefault();
//NoLogic
var lastBill = purchaseBills.ToList().Reverse().First();
textBoxPurchaseBillFrom.Text = firstBill.BillNo.ToString();
textBoxPurchaseBillTo.Text = lastBill.BillNo.ToString();
}
Related
I have two collections, one is a list of image names, the second is a subset of that list. When a task has been completed its name is inserted into the second collection.
I need to retrieve a set of not yet completed image names from the first collection. I have achieved this successfully with:
var processedNames = processed.AsQueryable().Select(x => x.ImageName).ToArray();
foreach (var result in results.Where(x => !processedNames.Contains(x.ImageName))
However this brings a large list of strings back from the database and then sends it back to the database in a single document, which as well as being inefficient will break eventually.
So I tried to rewrite it so it's all performed server side with:
var results = from x in captures
join complete in processed.AsQueryable() on x.ImageName equals complete.ImageName into completed
where !completed.Any()
select x;
This fails with:
System.NotSupportedException: '$project or $group does not support {document}.'
I also tried using the non LINQ API:
var xs = capturesCollection.Aggregate()
.Lookup("Processed", "ImageName", "ImageName", #as: "CompletedCaptures")
.Match(x => x["CompletedCaptures"] == null)
.ToList();
This fails with:
MongoDB.Bson.BsonSerializationException: 'C# null values of type 'BsonValue' cannot be serialized using a serializer of type 'BsonValueSerializer'.'
How can I achieve this query completely server side with the C# driver? A pure LINQ solution is preferable for portability.
I worked out how to do it with the Aggregate API:
var results = capturesCollection.Aggregate()
.As<CaptureWithCompletions>()
.Lookup(processed, x => x.ImageName, x => x.ImageName, #as:(CaptureWithCompletions x) => x.CompletedCaptures)
.Match(x => !x.CompletedCaptures.Any())
//.Limit(2)
.ToList();
I am getting the error about LINQ to Entities does not recognize the method 'System.String Format but in the past I was able to do this when I have included .AsEnumerable() is there something different I need to do because of the GroupBy section?
select new PresentationLayer.Models.PanelMeeting
{
GroupId = pg.GroupId,
MeetingId = pmd.MeetingId,
GuidelineName = pmv.GuidelineName,
PanelDisclosuresAttendanceURL = string.Format("{0}?MeetingId={1}&GroupId=0",PanelDisclosureLink, pmd.MeetingId),
}).GroupBy(g => new
{
g.MeetingId,
g.GroupId
})
.AsEnumerable()
.SelectMany(grp => grp.AsEnumerable()).ToList(),
You have to be aware of the difference between an IEnumerable<...> and an IQueryable<...>.
IEnumerable
An object that implements IEnumerable<...> represents a sequence of similar items. You can ask for the first element of the sequence, and as long as you've got elements you can ask for the next element. IEnumerable objects are supposed to be executed within your own process. IEnumerable objects hold everything to enumerate the sequence.
At its lowest level, this is done using GetEnumerator() / MoveNext() / Current:
IEnumerable<Customer> customers = ...
IEnumerator<Customer> enumerator = customers.GetEnumerator();
while (enumerator.MoveNext())
{
// There is a next Customer
Customer customer = enumerator.Current;
ProcessCustomer(customer);
}
If you use foreach, then internally GetEnumerator / MoveNext / Current are called.
If you look closely to LINQ, you will see that there are two groups of LINQ methods. Those that return IEnumerable<TResult> and those that dont't return IEnumerable<...>
LINQ functions from the first group won't enumerate the query. They use deferred execution, or lazy execution. In the comments section of every LINQ method, you'll find this description.
The LINQ functions of the other group will execute the query. If you look at the reference source of extension class Enumerable, you'll see that they internally use foreach, or at lower level use GetEnumerator / MoveNext / Current
IQueryable
An object that implements IQueryable<...> seems like an IEnumerable. However, it represents the potential to fetch data for an Enumerable sequence. The data is usually provided by a different process.
For this, the IQueryable holds an Expression and a Provider. The Expression represents what must be fetched in some generic format. The Provider knows who will provide the data (usually a database management system) and how to communicate with this DBMS (usually SQL).
When you start enumerating the sequence, deep inside using GetEnumerator, the Expression is sent to the Provider, who will try to translate it into SQL. The data is fetched from the DBMS, and returned as an Enumerable object. The fetched data is accessed by repeatedly calling MoveNext / Current.
Because the database is not contacted until you start enumerating, you'll have to keep the connection to the database open until you've finished enumerating. You've probably made the following mistake once:
IQueryable<Customer> customers;
using (var dbContext = new OrderDbContext(...))
{
customers = dbContext.Customers.Where(customer => customer...);
}
var fetchedCustomers = customers.ToList();
Back to your question
In your query, you use string.Format(...). Your Provider doesn't know how to translate this method into SQL. Your Provider also doesn't know any of your local methods. In fact, there are even several standard LINQ methods that are not supported by LINQ to entities. See Supported and Unsupported LINQ methods.
How to solve the problem?
If you need to call unsupported methods, you can use AsEnumerable to fetch the data. All LINQ methods after AsEnumerable are executed by your own process. Hence you can call any of your own functions.
Database Management systems are extremely optimized in table handling. One of the slower parts of a database query is the transport of the selected data to your local process. Hence, let the DBMS do all selecting, try to transport as little data as possible.
So let your DBMS do your Where / (Group-)Join / Sum / FirstOrDefault / Any etc. String formatting can be done best by you.
In your String.Format you use PanelDisclosureLink and pmd.MeetingId. It will probably be faster if your process does the formatting. Alas you forgot to give us the beginning or your query.
I'm not sure where your PanelDisclosureLink comes from. Is it a local variable? If that is the case, then PanelDisclosuresAttendanceURL will be the same string for every item in your group. Is this intended?
var panelDisclosureLine = ...;
var result = dbContext... // probably some joining with Pgs, Pmds and Pmvs,
.Select(... => new
{
GroupId = pg.GroupId,
MeetingId = pmd.MeetingId,
GuidelineName = pmv.GuidelineName,
})
// make groups with same combinations of [MeetingId, GroupId]
.GroupBy(joinResult => new
{
MeetingId = joinResult.MeetingId,
GroupId = joinResult.GroupId,
},
// parameter resultSelector: use the Key, and all JoinResult items that have this key
// to make one new:
(key, joinResultItemsWithThisKey) => new
{
MeetingId = key.MeetingId,
GroupId = key.GroupId,
GuideLineNames = joinResultsItemsWithThisKey
.Select(joinResultItem => joinResultItem.GuideLineName)
.ToList(),
})
So by now the DBMS has transformed your join result into objects with
[MeetingId, GroupId] combinations and a list of all GuideLineNames that have belong to
this [MeetingId, GroupId] combination.
Now you can move it to your local process and use String.Format.
.AsEnumerable()
.SelectMany (fetchedItem => fetchedItem.GuideLineNames,
(fetchedItem, guideLineName) => PresentationLayer.Models.PanelMeeting
{
GroupId = fetchedItem.GroupId,
MeetingId = fetchedItem.MeetingId,
GuidelineName = guidelineName,
PanelDisclosuresAttendanceURL = string.Format("...",
PanelDisclosureLink,
fetchedItem.MeetingId);
Note: in my parameter choice plurals are collections; singulars are elements of these collections.
PanelDisclosuresAttendanceURL = string.Format("{0}?MeetingId={1}&GroupId=0",PanelDisclosureLink, pmd.MeetingId),
}).
.GroupBy
If you want to use string.Format you first have to get the data from the server.
You can just move the .GroupBy( ... ) and then the .AsEnumerable() call to the top, before select new PresentationLayer.Models.PanelMeeting { ... }. If you are not selecting too much data that way...
I'm trying to use LINQ-to-entities to query my DB, where I have 3 tables: Room, Conference, and Participant. Each room has many conferences, and each conference has many participants. For each room, I'm trying to get a count of its conferences, and a sum of all of the participants for all of the room's conferences. Here's my query:
var roomsData = context.Rooms
.GroupJoin(
context.Conferences
.GroupJoin(
context.Participants,
conf => conf.Id,
part => part.ConferenceId,
(conf, parts) => new { Conference = conf, ParticipantCount = parts.Count() }
),
rm => rm.Id,
data => data.Conference.RoomId,
(rm, confData) => new {
Room = rm,
ConferenceCount = confData.Count(),
ParticipantCount = confData.Sum(cd => cd.ParticipantCount)
}
);
When I try and turn this into a list, I get the error:
The cast to value type 'System.Int32' failed because the materialized value is null. Either the result type's generic parameter or the query must use a nullable type.
I can fix this by changing the Sum line to:
ParticipantCount = confData.Count() == 0 ? 0 : confData.Sum(cd => cd.ParticipantCount)
But the trouble is that this seems to generate a more complex query and add 100ms onto the query time. Is there a better way for me to tell EF that when it is summing ParticipantCount, an empty list for confData should just mean zero, rather than throwing an exception? The annoying thing is that this error only happens with EF; if I create an empty in-memory List<int> and call Sum() on that, it gives me zero, rather than throwing an exception!
You may use the null coalescing operator ?? as:
confData.Sum(cd => cd.ParticipantCount ?? 0)
I made it work by changing the Sum line to:
ParticipantCount = (int?)confData.Sum(cd => cd.ParticipantCount)
Confusingly, it seems that even though IntelliSense tells me that the int overload for Sum() is getting used, at runtime it is actually using the int? overload because the confData list might be empty. If I explicitly tell it the return type is int? it returns null for the empty list entries, and I can later null-coalesce the nulls to zero.
Use Enumerable.DefaultIfEmpty:
ParticipantCount = confData.DefaultIfEmpty().Sum(cd => cd.ParticipantCount)
Instead of trying to get EF to generate a SQL query that returns 0 instead of null, you change this as you process the query results on the client-side like this:
var results = from r in roomsData.AsEnumerable()
select new
{
r.Room,
r.ConferenceCount,
ParticipantCount = r.ParticipantCount ?? 0
};
The AsEnumerable() forces the SQL query to be evaluated and the subsequent query operators are client-side LINQ-to-Objects.
I have the following methods:
private IEnumerable<CTNTransactionsView> RetrieveCTNTransactionsNotInTLS() {
IQueryable<int> talismanIdCollection = this._cc.TLSTransactionView.Select(x => x.kSECSYSTrans);
return this._cc.CTNTransactionView
.Where(x => !talismanIdCollection.Contains(x.kSECSYSTrans));
}
public IEnumerable<CTNTransactionsView> RetrieveCTNTransactionsNotInTLSPast24Hours() {
DateTime previousDate = DateTime.Now.Date.AddDays(-1.0);
return this.RetrieveCTNTransactionsNotInTLS()
.Where(x => x.dSECSYSTimeStamp >= previousDate);
}
public IEnumerable<CTNTransactionsView> RetrieveCTNTransactionsNotInTLSPast24HoursVersionTwo() {
DateTime previousDate = DateTime.Now.Date.AddDays(-1.0);
IQueryable<int> talismanIdCollection = this._cc.TLSTransactionView
.Select(x => x.kSECSYSTrans);
return this._cc.CTNTransactionView
.Where(x => !talismanIdCollection.Contains(x.kSECSYSTrans))
.Where(x=> x.dSECSYSTimeStamp >= previousDate);
}
For some reason, the SQL output generated by Entity Framework 6 does not match the results.
The RetrieveCTNTransactionsNotInTLSPast24HoursVersionTwo() method will properly give a SQL Output Statement that has the following:
select ...... from ... where ... AND ([Extent1].[dSECSYSTimeStamp] >= #p__linq__0)}
The other one does not have the filter for the dSECSYSTimeStamp when I View the SQL Statement Output.
The methods I am comparing are the RetrieveCTNTransactionsNotInTLSPast24Hours() and the RetrieveCTNTransactionsNotInTLSPast24HoursVersionTwo().
I have compared the SQL using VS as well as attaching a Debug.Writeline() to the Database.Log in the context.
From debugging and looking at the SQL output, one seems to contain the date filter whereas the other doesn't and yet they both provide the correct result.
I have tried looking at the SQL (by breakpointing and seeing the output) from using the following:
System.Diagnostics.Debug.WriteLine("Running first method");
var result = this.repo.RetrieveCTNTransactionsNotInTLSPast24Hours();
var count = result.Count();
System.Diagnostics.Debug.WriteLine("Running Second method");
var resultTwo = this.repo.RetrieveCTNTransactionsNotInTLSPast24HoursVersionTwo();
var count2 = resultTwo.Count();
I am using EF 6.0.
Note: The results are the same as both do exactly the same thing and output the same result. However, I am curious and would like to understand why the SQL Generated isn't the same?
The issue is you are returning an IEnumerable from your method. If you do this, you force SQL to run the query (unfiltered) and then use C# to then run the second query. Alter your internal query to return an IQueryable. This will allow the unexecuted expression tree to be passed into the second query, which will then be evaluated when your run it.
i.e.
private IQueryable<CTNTransactionsView> RetrieveCTNTransactionsNotInTLS()
You should then get the same SQL.
I think, you just define a query expression, but not use it immediately, in this time, I will add .ToList() in the Linq expression's end, but you have to change method's return type into a right type like List<kSECSYSTrans> before this operation. I just seldom use the type IEnumerable.
I'm still very new to LINQ and PLINQ. I generally just use loops and List.BinarySearch in a lot of cases, but I'm trying to get out of that mindset where I can.
public class Staff
{
// ...
public bool Matches(string searchString)
{
// ...
}
}
Using "normal" LINQ - sorry, I'm unfamiliar with the terminology - I can do the following:
var matchedStaff = from s
in allStaff
where s.Matches(searchString)
select s;
But I'd like to do this in parallel:
var matchedStaff = allStaff.AsParallel().Select(s => s.Matches(searchString));
When I check the type of matchedStaff, it's a list of bools, which isn't what I want.
First of all, what am I doing wrong here, and secondly, how do I return a List<Staff> from this query?
public List<Staff> Search(string searchString)
{
return allStaff.AsParallel().Select(/* something */).AsEnumerable();
}
returns IEnumerable<type>, not List<type>.
For your first question, you should just replace Select with Where :
var matchedStaff = allStaff.AsParallel().Where(s => s.Matches(searchString));
Select is a projection operator, not a filtering one, that's why you are getting an IEnumerable<bool> corresponding to the projection of all your Staff objects from the input sequence to bools returned by your Matches method call.
I understand it can be counter intuitive for you not to use select at all as it seems you are more familiar with the "query syntax" where select keyword is mandatory which is not the case using the "lambda syntax" (or "fluent syntax" ... whatever the naming), but that's how it is ;)
Projections operators, such a Select, are taking as input an element from the sequence and transform/projects this element somehow to another type of element (here projecting to bool type). Whereas filtering operators, such as Where, are taking as input an element from the sequence and either output the element as such in the output sequence or are not outputing the element at all, based on a predicate.
As for your second question, AsEnumerable returns an IEnumerable as it's name indicates ;)
If you want to get a List<Staff> you should rather call ToList() (as it's name indicates ;)) :
return allStaff.AsParallel().Select(/* something */).ToList();
Hope this helps.
There is no need to abandon normal LINQ syntax to achieve parallelism. You can rewrite your original query:
var matchedStaff = from s in allStaff
where s.Matches(searchString)
select s;
The parallel LINQ (“PLINQ”) version would be:
var matchedStaff = from s in allStaff.AsParallel()
where s.Matches(searchString)
select s;
To understand where the bools are coming from, when you write the following:
var matchedStaff = allStaff.AsParallel().Select(s => s.Matches(searchString));
That is equivalent to the following query syntax:
var matchedStaff = from s in allStaff.AsParallel() select s.Matches(searchString);
As stated by darkey, if you want to use the C# syntax instead of the query syntax, you should use Where():
var matchedStaff = allStaff.AsParallel().Where(s => s.Matches(searchString));