We Keep Coding

DocumentClient takes too long when queried over a DateTime field

I have the following code to get the list of some objects from a DocumentDB database:
var document = this._client.CreateDocumentQuery<T>(UriFactory.CreateCollectionUri(dbName, collectionName), queryOptions)
.Where(r => r.pDate >= startDate && r.pDate <= endDate);
var result = document.ToList();
pDate is of type DateTime, and stored in the database as string with ISO8601 format.
The query takes unreasonably too long, like 4 to 5 minutes, to return the results back. When I trace the program it is that .ToList() where the program gets stuck. Oddly, the query quickly returns for some specific start and end dates.
The query also quickly comes back with some results if I put the filter on some fields other than pDate.
My settings are consistent with explanations in this document but I still get a very poor performance almost all the time except for those few exceptions.
I have tried several methods mentioned here and there to resolve the issue, but no luck so far. I appreciate any comment or solution to the problem.

It could be because the indexing that is applied on this particular field.
From this link
https://learn.microsoft.com/en-us/azure/cosmos-db/indexing-policies,
you would need an Range index for range or Order by Queries.
The default index policy that is applied to a collection is
"Hash for Strings and Range for Numbers"
Datetimes are stored as strings, so for running range comparison queries or order by queries you would need to set the indexing policy to "Range" for string datatypes with precision to -1.

DateTime comparison in LINQ not returning correct results

I have the following LINQ to query the database and retreive deleted products from a particular date.
return _myDbEntities.Log
.Where(p => p.Action.Equals("Deleted") &&
(p.ActionDate > fromDate))
.Select(p => new DeletedProduct()
{
ProductId = p.ProductId,
ActionDate = p.ActionDate
}).ToList();
However, the query is retreiving values like product.ActionDate.Value = {12/8/2016 11:41:00 AM} when the fromDate was fromDate = {12/8/2016 11:41:00 AM}
The query clearly says GREATER THAN. What is happening here?

There are fractions of a second to each of your properties. Most likely, your record wasn't created at an exact second, whereas any user-created time would be set as such.
Another possibility is the difference between datetime and datetime2 in SQL Server.

The DateTime type stores time at much higher precision than seconds. They could be differing at millisecond or even tick (100 nanoseconds) level.
If you want to compare on a higher level, try this:
(p.ActionDate.Ticks / 10000000) > (fromDate.Ticks / 10000000)
Where 10000000 is the number of ticks in a second. Since the /is an integer division that does truncate the fraction, you turn ticks into full seconds.
UPDATE:
It seems like you are using entity framework. The comparison above will possibly not work there. The solution is to run your original query against the database, do a ToList and then filter the results again in a LINQ2Objects query using the logic above.

Retrieve only the first row from a table in Entity Framework

Background:
Entity Framework 4, with SQL Server 2008
Problem:
I have a table Order. Each row has a column Timestamp.
The user can choose some time in past and I need to get the Order closest to the specified time, but that had occurred before the specified time. In other words, the last order before the specified time.
For example, if I have orders
2008-01-12
2009-04-17
2009-09-24
2010-11-02
2010-12-01
2011-05-16
and choose a date 2010-07-22, I should get the 2009-09-24 order, because that's the last order before the specified date.
var query = (from oData in db.OrderDatas
where oData.Timestamp <= userTime
orderby oData.Timestamp ascending
select oData).Last();
This is closest to what I am trying. However, I am not sure how exactly does the Last operator work when translated to SQL, if it's translated at all.
Question:
Will this query fetch all data (earlier than userTime) and then take the last element, or will it be translated so that only one element will be returned from the database? My table can hold very large number of rows (100000+) so performance is an issue here.
Also, how would one retrieve the closest time in the database (not necessarily the earlier time)? In the example of 2010-07-22, one would get 2010-11-02, because it is closer to the date specified than 2009-09-24.

In general, if you're concerned about how LINQ behaves, you should check what does happen with the SQL. If you haven't worked out how to see how your LINQ queries are turned into SQL, that should be the very next thing you do.
As you noted in your comment, Last() isn't supported by LINQ to SQL so the same may be true for EF. Fortunately, it's easy to use First() instead:
var query = (from oData in db.OrderDatas
where oData.Timestamp <= userTime
orderby oData.Timestamp descending
select oData).First();

Try using:
var query = (from oData in db.OrderDatas
where oData.Timestamp <= userTime
orderby oData.Timestamp descending
select oData).Take(1);
It's the equivalent of TOP 1

Question:
Will this query fetch all data (earlier than userTime) and then take
the last element, or will it be translated so that only one element
will be returned from the database? My table can hold very large
number of rows (100000+) so performance is an issue here.
In this case, using the first() approach, the query will be executed immediately and it will optimized in such a way that it will ony retrieve 1 record. Most probably a top(1) select. You really need to check the genereated sql with a sql profilihg tool or by using the log of the datacontext. Or you can use linqpad. linq-2-sql can lead to N+1 queries if not used the proper way. This behaviour is quite predictable but in the beginning you really have to be aware.

How to calculate total working time in LINQ?

I have a table which holds clocking in/out records for every user :
RecID User In/Out ClockInOutTime
8 1 IN 25/02/2011 09:36:44
9 1 OUT 25/02/2011 11:36:44
10 1 IN 25/02/2011 12:36:44
11 1 OUT 25/02/2011 17:36:44
12 1 IN 26/02/2011 00:00:00
13 1 OUT 26/02/2011 12:00:00
14 1 IN 26/02/2011 09:00:44
15 1 OUT 26/02/2011 12:36:44
Any ideas how I can work out the total time worked for every month using LINQ?
cheers

SELECT (SELECT SUM(DATEDIFF(SECOND,[ClockInOutTime], GETDATE()))
FROM [swp].[dbo].[Table_1] t1
WHERE [In/Out] = 'IN'
AND t1.[User] = t.[User]) -
Coalesce((SELECT SUM(DATEDIFF(SECOND,[ClockInOutTime], GETDATE()))
FROM [swp].[dbo].[Table_1] t2
WHERE [In/Out] = 'OUT'
AND t2.[User] = t.[User]),0)
FROM [swp].[dbo].[Table_1] t
GROUP BY [User]
SQL way to solve this, not the best, but works even when last event don't have OUT timestamp i.e when last session still hasn't been closed.

This is non-trivial to do in either Linq or SQL. There is no easy way to link each OUT record with the corresponding IN record in SQL.
You have two options:
Querying the data and calculating in code within a for loop.
Changing the table schema like: RecID, User, ClockInTime, ClockOutTime
Option 1 is easy to implement, but I would seriously consider option 2. How do you define in your business rules that each IN record must be followed by a corresponding OUT record (or be last record)?

TimeSpan output = new TimeSpan(0,0,0);
using (var enumerator = input.GetEnumerator())
{
while (enumerator.MoveNext())
{
var begin = enumerator.Current.ClockInOutTime;
if(!enumerator.MoveNext())
break;
var end = enumerator.Current.ClockInOutTime;
output += (end - begin);
}
}
Yes, it isn't LINQ but I wanted to offer a solution - secondly, if the dates aren't alternating (so after an IN is always an OUT) it'll break.

There is no solution that will only use linq. This is due to the fact that you need to introduce error handling as well (if a user forgets to sign out, usually there is a maximum time that will be applied then etc).
I would group the data by user, order it by date time and then run through it in a for each and do the calculation within the for each.

date difference in EF4

i need to get a difference of two dates ,one date from a table and one the current date, and the difference should be less than 9 0days. i need to use this as filter in where clause of the linq
i tried doing this
var list = from p in context.persons
where ((p.CreateDT).Subtract(DateTime.Now).Days < 90)
select p;
i get this excpetion :
LINQ to Entities does not recognize the method 'System.TimeSpan Subtract(System.DateTime)' method, and this method cannot be translated into a store expression.
I did research other articles but nothing helped..Any ideas

Trick here is that it can't translate all your fancy oo hoo-ha to plain old sql. Trick is to flip it on it's head:
var before = DateTime.Now.AddDays(-90);
var persons = context.Persons.Where(x => x.CreateDT > before);
EXPLANATION
Remember that everything in the WHERE bit of your LINQ statement must be translated from C# to SQL by the EF. It is very, very capable out of the box and handles most basic tasks, but it has no idea how to understand the most rudimentary method calls, such as DateTime.Subtract(). So the idea here is to let it do what it does best by precalculating a value and then passing that to the data tier.
The first line subtracts 90 days from the current time by adding negative 90 days. The second line passes it off to the database server.
The second line should translate to the SQL WHERE CreateDT > #BEFORETHIS

Update
It seems that EF doesn't support subtracting dates and returning a TimeSpan. Here's one way to solve the problem:
DateTime oldestDate = DateTime.Now.AddDays(-90);
var list = from p in context.persons
where p.CreateDT >= oldestDate
select p;
See this thread on Stackoverflow.

Try doing simply (p.CreateDate - DateTime.Now).Days < 90. Instead of calling DateTime.Subtract(). In some cases the operator overloads are implemented for Entity Framework even when the corresponding named methods are not.
If that doesn't work you could instead use ESQL or a stored procedure. As a final, dirty solution, you could call context.persons.ToList() and then call the DateTime.Subtract().