I want to fetch values from database with specific intervals in C# and need a single query for that.
This is how my Database looks like
Id SensorId Value CreatedOn
------------------------------------------------
1 8 33.5 15-11-2012 5:48 PM
2 5 49.2 15-11-2012 5:48 PM
3 8 33.2 15-11-2012 5:49 PM
4 5 48.5 15-11-2012 5:49 PM
5 8 31.8 15-11-2012 5:50 PM
6 5 42.5 15-11-2012 5:50 PM
7 8 36.5 15-11-2012 5:51 PM
8 5 46.5 15-11-2012 5:51 PM
9 8 39.2 15-11-2012 5:52 PM
10 5 44.4 15-11-2012 5:52 PM
11 8 36.5 15-11-2012 5:53 PM
12 5 46.5 15-11-2012 5:53 PM
13 8 39.2 15-11-2012 5:54 PM
14 5 44.4 15-11-2012 5:54 PM
.. .. ..... ...................
The interval is in minutes.
So, if the interval is 10 minutes, then we need the values at 5:48, 5:58, 6:08 and so on...
I tried doing it with a while loop but it is taking a lot of time as i shoot multiple queries to the database.
Is there any way of getting the data in a single query?
You can use datepart along with a modulus to get the matching rows (eg, #interval = 10, #offset = 8):
SELECT * FROM table
WHERE datepart(minute, CreatedOn) % #interval = #offset
Edit
Note that the above isn't a general solution of selecting by intervals. It will work across hours (and therefore across days) for intervals like 2, 3, 4, 5 ... any minute interval which divides into 60.
If you want to use a strange interval like 7 minutes, then you'd have to define a starting time for the interval and calculate the total minutes for each row, inclusive of hours/days. At that point you'd be best to create an indexed, computed column on the table, based on a user-defined function that calculates the interval in question.
Here is how you can do it, explanation is contained within comments in code:
/*We want 10-minute intervals starting
from minimum date to next day same time*/
DECLARE #startDateTime DATETIME = (
SELECT MIN(CreatedOn)
FROM #yourTable
)
DECLARE #endDateTime DATETIME = DATEADD(DAY, 1, #startDateTime)
DECLARE #startDateTimeTable TABLE (dt DATETIME)
INSERT #startDateTimeTable VALUES (#startDateTime)
/*Create a table that contains relevant datetimes (10-minute
intervals from starting date to end date)*/
;WITH a AS (
SELECT dt
FROM #startDateTimeTable
UNION ALL
SELECT DATEADD(MINUTE, 10, a.dt)
FROM a
JOIN #startDateTimeTable b ON a.dt <= #endDateTime
)
SELECT *
INTO #requiredDateTimes
FROM a
OPTION (MAXRECURSION 32767)
/*Now join data table to datetime table to
filter out only records with datetimes that we want*/
SELECT *
FROM #yourTable a
JOIN #requiredDateTimes b ON
a.CreatedOn = b.dt
Here is an SQL Fiddle
Any of the answers that recommend using modulus (%) are making several assumptions:
You will always have a reading on every sensor at the exact minute in question
You will never have more than one reading in a minute per sensor.
You will never have to deal with intervals smaller than a minute.
These are probably false assumptions, so you need a different approach. First, make a map of all of the time points you are querying over. Then take the last reading from each sensor on or before that point.
Here's a full unit test showing how it can be done in pure linq-to-objects. You may need some minor changes to the query to get it to work in linq-to-sql, but this is the right approach. I used the exact sample data you provided.
As an aside - I hope you are recording your CreatedOn dates in UTC, or you will have ambiguity of sensor readings during daylight savings time "fall-back" transitions. You need to record as DateTime in UTC, or using DateTimeOffset. Either are an appropriate representation of instantaneous time. A DateTime with .Kind of Local or Unspecified is only a valid representation of calendar time, which is not appropriate for sensor readings.
[TestClass]
public class LinqIntervalQueryTest
{
public class Item
{
public int Id { get; set; }
public int SensorId { get; set; }
public double Value { get; set; }
public DateTime CreatedOn { get; set; }
}
[TestMethod]
public void Test()
{
var data = new[]
{
new Item { Id = 1, SensorId = 8, Value = 33.5, CreatedOn = new DateTime(2012, 11, 15, 17, 48, 0, DateTimeKind.Utc) },
new Item { Id = 2, SensorId = 5, Value = 49.2, CreatedOn = new DateTime(2012, 11, 15, 17, 48, 0, DateTimeKind.Utc) },
new Item { Id = 3, SensorId = 8, Value = 33.2, CreatedOn = new DateTime(2012, 11, 15, 17, 49, 0, DateTimeKind.Utc) },
new Item { Id = 4, SensorId = 5, Value = 48.5, CreatedOn = new DateTime(2012, 11, 15, 17, 49, 0, DateTimeKind.Utc) },
new Item { Id = 5, SensorId = 8, Value = 31.8, CreatedOn = new DateTime(2012, 11, 15, 17, 50, 0, DateTimeKind.Utc) },
new Item { Id = 6, SensorId = 5, Value = 42.5, CreatedOn = new DateTime(2012, 11, 15, 17, 50, 0, DateTimeKind.Utc) },
new Item { Id = 7, SensorId = 8, Value = 36.5, CreatedOn = new DateTime(2012, 11, 15, 17, 51, 0, DateTimeKind.Utc) },
new Item { Id = 8, SensorId = 5, Value = 46.5, CreatedOn = new DateTime(2012, 11, 15, 17, 51, 0, DateTimeKind.Utc) },
new Item { Id = 9, SensorId = 8, Value = 39.2, CreatedOn = new DateTime(2012, 11, 15, 17, 52, 0, DateTimeKind.Utc) },
new Item { Id = 10, SensorId = 5, Value = 44.4, CreatedOn = new DateTime(2012, 11, 15, 17, 52, 0, DateTimeKind.Utc) },
new Item { Id = 11, SensorId = 8, Value = 36.5, CreatedOn = new DateTime(2012, 11, 15, 17, 53, 0, DateTimeKind.Utc) },
new Item { Id = 12, SensorId = 5, Value = 46.5, CreatedOn = new DateTime(2012, 11, 15, 17, 53, 0, DateTimeKind.Utc) },
new Item { Id = 13, SensorId = 8, Value = 39.2, CreatedOn = new DateTime(2012, 11, 15, 17, 54, 0, DateTimeKind.Utc) },
new Item { Id = 14, SensorId = 5, Value = 44.4, CreatedOn = new DateTime(2012, 11, 15, 17, 54, 0, DateTimeKind.Utc) },
};
var interval = TimeSpan.FromMinutes(3);
var startDate = data.First().CreatedOn;
var endDate = data.Last().CreatedOn;
var numberOfPoints = (int)((endDate - startDate + interval).Ticks / interval.Ticks);
var points = Enumerable.Range(0, numberOfPoints).Select(x => startDate.AddTicks(interval.Ticks * x));
var query = from item in data
group item by item.SensorId
into g
from point in points
let itemToUse = g.LastOrDefault(x => x.CreatedOn <= point)
orderby itemToUse.CreatedOn, g.Key
select new
{
itemToUse.CreatedOn,
itemToUse.Value,
SensorId = g.Key
};
var results = query.ToList();
Assert.AreEqual(6, results.Count);
Assert.AreEqual(data[1].CreatedOn, results[0].CreatedOn);
Assert.AreEqual(data[1].Value, results[0].Value);
Assert.AreEqual(data[1].SensorId, results[0].SensorId);
Assert.AreEqual(data[0].CreatedOn, results[1].CreatedOn);
Assert.AreEqual(data[0].Value, results[1].Value);
Assert.AreEqual(data[0].SensorId, results[1].SensorId);
Assert.AreEqual(data[7].CreatedOn, results[2].CreatedOn);
Assert.AreEqual(data[7].Value, results[2].Value);
Assert.AreEqual(data[7].SensorId, results[2].SensorId);
Assert.AreEqual(data[6].CreatedOn, results[3].CreatedOn);
Assert.AreEqual(data[6].Value, results[3].Value);
Assert.AreEqual(data[6].SensorId, results[3].SensorId);
Assert.AreEqual(data[13].CreatedOn, results[4].CreatedOn);
Assert.AreEqual(data[13].Value, results[4].Value);
Assert.AreEqual(data[13].SensorId, results[4].SensorId);
Assert.AreEqual(data[12].CreatedOn, results[5].CreatedOn);
Assert.AreEqual(data[12].Value, results[5].Value);
Assert.AreEqual(data[12].SensorId, results[5].SensorId);
}
}
Here's how you can do it in two calls to the database (untested):
int interval = 10;
DateTime firstDate = db.Items.Select(x => x.CreatedOn).Min();
var items = db.Items.Where(x => (x.CreatedOn - firstDate).TotalMinutes % interval == 0).ToList();
Related
I am keeping staff clock-in times daily.
Sometimes they work the next day without taking any break.
WHAT I NEED IS
Without a break,the Total Hrs of staff who finish at midnight.
For example
Staff
Start
End
Staff A
9 AM
3 PM
Staff A
3 PM
10 PM
Staff A
10 PM
12:00 AM
Staff B
3 AM
9 PM
Staff B
10 PM
12:00 AM
Staff C
3 AM
9 PM
Staff D
10 PM
12:00 AM
In the above table, both staffs finished at midnight.
Staff A worked 15hrs without any breaks.
But Staff B had a break between 9 PM and 10 Pm.
Expected result:
Staff
Total
Staff A
15
Staff B
2
Staff C
0 because he didn't finish at midnight.
Staff C
2
I tried for loop, but it failed because each staff has a different number of entries, codes got messy at the end.
I tried LINQ and joining the same table etc. but couldn't figure it out.
This website is my last option.
Thanks advance for help.
Okay, this answer will need a bunch more work if you need this for work segments that cross dates or multiple dates. This assumes all work segments are in the same day.
The following groups all segments by staff, then adds up each segment total minutes. Then only returns the total if the last segment ends at midnight.
This test code returns the following:
Staff A: 15
Staff B: 20
Staff C: 0
Staff D: 2
Here's the code:
void Main()
{
var segments = GetTestData();
DateTime midnight = DateTime.Now.AddDays(1).Date;
var results = segments
.GroupBy(s => s.Name)
.Select(grp =>
new {
Name = grp.Key,
Total = grp.Max(g => g.EndDate) == midnight
? grp.Sum(g => g.EndDate.Subtract(g.StartDate).TotalMinutes)
: 0
});
foreach (var result in results)
{
Console.WriteLine($"{result.Name}: {result.Total/60F:#,##0}");
}
}
class Segment
{
public string Name { get; set; }
public DateTime StartDate { get; set; }
public DateTime EndDate { get; set; }
}
Segment[] GetTestData()
{
return new[]
{
new Segment { Name = "Staff A", StartDate = new DateTime(2023, 1, 18, 9 , 0, 0), EndDate = new DateTime(2023, 1, 18, 15, 0, 0) },
new Segment { Name = "Staff A", StartDate = new DateTime(2023, 1, 18, 15, 0, 0), EndDate = new DateTime(2023, 1, 18, 22, 0, 0) },
new Segment { Name = "Staff A", StartDate = new DateTime(2023, 1, 18, 22, 0, 0), EndDate = new DateTime(2023, 1, 19, 0 , 0, 0) },
new Segment { Name = "Staff B", StartDate = new DateTime(2023, 1, 18, 3 , 0, 0), EndDate = new DateTime(2023, 1, 18, 21, 0, 0) },
new Segment { Name = "Staff B", StartDate = new DateTime(2023, 1, 18, 22, 0, 0), EndDate = new DateTime(2023, 1, 19, 0 , 0, 0) },
new Segment { Name = "Staff C", StartDate = new DateTime(2023, 1, 18, 3 , 0, 0), EndDate = new DateTime(2023, 1, 18, 21, 0, 0) },
new Segment { Name = "Staff D", StartDate = new DateTime(2023, 1, 18, 22, 0, 0), EndDate = new DateTime(2023, 1, 19, 0 , 0, 0) }
};
}
I have column ModificationDate stored in database as TimeStamp.
I'am using Oracle 12c database.
When i want to compare this date with other it looks like it lost precision during comparison, or somethink else happens....
Date value in database is '15/12/07 15:37:26,562000000'. And i'am using this queries:
DateTime date = new DateTime(2015, 12, 07, 15, 37, 26, 562);
var p1 = from p in entities.People
where p.ModificationDate > date
select p;
Console.WriteLine(p1.Count()); //Prints: 1
//Where condition: "Extent1"."MODIFICATIONDATE" > :p__linq__0
var p2 = from p in entities.People
where p.ModificationDate > new DateTime(2015, 12, 07, 15, 37, 26, 562)
select p;
Console.WriteLine(p2.Count()); //Prints: 0
//Where condition: "Extent1"."MODIFICATIONDATE" > TO_TIMESTAMP('2015-12-07 15:37:26.562', 'YYYY-MM-DD HH24:MI:SS.FF')
Why when i'am putting datetime as variable i'am getting incorrect result?
Is there any possibility to make it work correctly with datetime variable?
Edit:
After few tries i found much easier example:
DateTime date = new DateTime(2015, 12, 07, 15, 37, 26, 562);
var p3 = from p in entities.PEOPLE
where date == new DateTime(2015, 12, 07, 15, 37, 26, 562)
select p;
Console.WriteLine(p3.Count()); //Prints: 0
//Where condition: (TO_TIMESTAMP('2015-12-07 15:37:26.562', 'YYYY-MM-DD HH24:MI:SS.FF') = :p__linq__0) AND (:p__linq__0 IS NOT NULL)
It looks like DateTime variable have different precision than timestamp. Is there is a possiblily to tell linq query to treat DateTime variable as TIMESTAMP ?
Say I have the below set of data where I wish to group the Description column based on whether the dates for matching Descriptions are within a few seconds of each other.
The output I'm looking for is the Description and the minimum Date for that group. It is possible that the Description could be the same but the dates might be days different, in this case I would want two outputted rows.
In the case above take a look at the Description "TEST s" where I would want two outputted grouped rows
TEST s 2014-12-04 16:27:44.903
TEST s 2014-12-04 17:21:21.233
Is this possible using Linq?
Try this:
var q = from item in lstMyTable
group item by item.ItemDate.ToString("MM/dd/yyyy HH:mm") into ItemGroup
select new
{
Description = ItemGroup.First().Description,
Date = ItemGroup.OrderBy(x => x.ItemDate).First().ItemDate
};
The above linq query groups by date + hour + minutes, ignoring seconds. The OrderBy applied to ItemGroup ensures that we get the mininum date, as described in the OP.
With this input:
List<MyTable> lstMyTable = new List<MyTable>()
{
new MyTable() { Description = "TEST s", ItemDate = new DateTime(2014, 12, 4, 16, 27, 11) },
new MyTable() { Description = "TEST s", ItemDate = new DateTime(2014, 12, 4, 16, 27, 12) },
new MyTable() { Description = "TEST s", ItemDate = new DateTime(2014, 12, 4, 16, 27, 13) },
new MyTable() { Description = "TEST s", ItemDate = new DateTime(2014, 12, 4, 17, 21, 11) },
new MyTable() { Description = "TEST s", ItemDate = new DateTime(2014, 12, 4, 17, 21, 12) },
new MyTable() { Description = "TEST s", ItemDate = new DateTime(2014, 12, 4, 17, 21, 13) }
};
you get this result:
[0] = { Description = "TEST s", Date = {4/12/2014 4:27:11 pm} }
[1] = { Description = "TEST s", Date = {4/12/2014 5:21:11 pm} }
If you want to also group by description then simply substitue the group by clause with sth like this:
group item by new
{
a = item.ItemDate.ToString("MM/dd/yyyy HH:mm") ,
b = item.Description
} into ItemGroup
Finally, by converting the Datetime field to Ticks, you can fiddle with the required time distance between separate records, e.g.
group item by new
{
a = item.ItemDate.Ticks.ToString().Substring(0, item.ItemDate.Ticks.ToString().Length - 10), // instead of .ToString("MM/dd/yyyy HH:mm") ,
b = item.Description
} into ItemGroup
You can play around by varying the number of digits subtracted from item.ItemDate.Ticks, substituting e.g. 10 with 9 or 8, etc.
If you want left hand side to be a list then you can try this
var query = (from t in db.Table
group t by new {t.Description, t.Date }
into grp
select new
{
grp.Key.Description,
grp.Key.Date,
}).ToList();
I'm expecting the following sequence of time data in 30 minute interval from another system. I need to make this 30 minute pair in to one hour interval:
e.g.
Source Data:
Start:7:00
End: 7:30
Start:7:30
End: 8:00
Start:8:00
End: 8:30
Start:8:30
End: 9:00
Convert in to hour data like below:
Start:7:00
End: 8:00
Start:8:00
End: 9:00
If there's no match for 30 minute block this record should be ignored:
e.g
Start:7:00
End: 7:30
Start:7:30
End: 8:00
Start:8:00
End: 8:30
Start:8:30
End: 9:00
Start:9:00 <-- Ignore this
End: 9:30 <-- Ignore this
Can someone please suggest how this can be achieved?
The general idea for my solution is that you simply only take elements at indices:
0, 3, 4, 7, 8, 11, 12, ...
Or in other words - if index % 4 is 0 or 3.
Before doing that, we check if there are leftover dates at the end with count % 4.
If there are - we cut them.
This assumes that the dates will be all valid as the input you illustrated.
Here is my implementation:
public static IEnumerable<DateTime> CutTimes(IEnumerable<DateTime> source)
{
var count = source.Count();
var ignore = count % 4;
if (ignore != 0)
{
source = source.Take(count - ignore);
}
return source.Where((time, index) => index % 4 == 0 || index % 4 == 3);
}
Usage:
var times = new[]
{
new DateTime(2012, 5, 10, 7, 00, 0),
new DateTime(2012, 5, 10, 7, 30, 0),
new DateTime(2012, 5, 10, 7, 30, 0),
new DateTime(2012, 5, 10, 8, 00, 0),
new DateTime(2012, 5, 10, 8, 00, 0),
new DateTime(2012, 5, 10, 8, 30, 0),
new DateTime(2012, 5, 10, 8, 30, 0),
new DateTime(2012, 5, 10, 9, 00, 0),
new DateTime(2012, 5, 10, 9, 00, 0),
new DateTime(2012, 5, 10, 9, 30, 0)
};
var result = CutTimes(times);
foreach (var time in result)
{
Console.WriteLine(time);
}
Output:
10/05/2012 07:00:00
10/05/2012 08:00:00
10/05/2012 08:00:00
10/05/2012 09:00:00
I have an IEnumerable of an item class defined like this:
public class Item {
public DateTime Date { get; private set; }
public decimal? Value { get; private set; }
public Item(DateTime date, decimal? value) {
Date = date;
Value = value;
}
}
These items are in a specific time interval (5 minutes for exemple). I need to group them by the date, but changing the interval. For example, if the items are in the following order:
2010-08-24 00:05
2010-08-24 00:10
2010-08-24 00:15
2010-08-24 00:20
2010-08-24 00:25
2010-08-24 00:30
and I want to group them into a 15 minutes interval, the result should look like this:
2010-08-24 00:15
2010-08-24 00:30
The interval is provided by another class, but I can get the milliseconds that represent that interval (for example, Interval.FromMinutes(5).GetMilliseconds() should return 300000). The question is how can I write a grouping function that allows me to do something like this : data = items.GroupBy(t => GroupingFunction(t.DateTime, interval)) and obtain that result?
Update: the interval will not be necessarily in minutes. It could be in hours, minutes or even days.
Something like this ?
DateTime[] dateTimes = new[]
{
new DateTime(2010, 8, 24, 0, 5, 0),
new DateTime(2010, 8, 24, 0, 10, 0),
new DateTime(2010, 8, 24, 0, 15, 0),
new DateTime(2010, 8, 24, 0, 20, 0),
new DateTime(2010, 8, 24, 0, 25, 0),
new DateTime(2010, 8, 24, 0, 30, 0)
};
TimeSpan interval = new TimeSpan(0, 15, 0); // 15 minutes.
var groupedTimes = from dt in dateTimes
group dt by dt.Ticks/interval.Ticks
into g
select new {Begin = new DateTime(g.Key*interval.Ticks), Values = g.ToList()};
foreach (var value in groupedTimes)
{
Console.WriteLine(value.Begin);
Console.WriteLine("\t{0}", String.Join(", ", value.Values));
}
data = items.GroupBy(t => (int)(t.DateTime.Minutes/interval)))
Have you tried grouping by a modulus?
Untested stream-of-consciousness pseudocode follows:
data = items.GroupBy(t => t.TimeInterval % 15 == 0);