We Keep Coding

Linq group by month/year with empty months for past 12 months

How would I get this query to get the monthly count data for the past 12 months? I don't want to hard code the range, I want to use the current date DateTime.Now and get all the data for the past 12 months from that. I am trying to avoid adding a calendar table to the database and do this just using LINQ.
Some months might not have any data but I still need a count of 0 for those.
For example. If my data contains
Date Count
12/2/2013, 4
10/1/2014, 1
11/5/2014, 6
The results should be, using the current date of 11/9/2014
11/2013, 0
12/1013, 4
1/2014, 0
2/2014, 0
3/2014, 0
4/2014, 0
5/2014, 0
6/2014, 0
7/2014, 0
8/2014, 0
9/2014, 0
10/2014, 1
11/2014, 6
I can't get it to work. I think it's how I'm using Range but I'm not sure.
TimeSpan ts = new TimeSpan(365, 0, 0, 0);
DateTime yearAgo = DateTime.Now.Subtract(ts);
var changesPerYearAndMonth =
from year in Enumerable.Range(yearAgo.Year, 1)
from month in Enumerable.Range(1, 12)
let key = new { Year = year, Month = month }
join revision in list on key
equals new { revision.LocalTimeStamp.Year,
revision.LocalTimeStamp.Month } into g
select new { GroupCriteria = key, Count = g.Count() };
I have modified the answer from this this link as a starting point.
Linq: group by year and month, and manage empty months
I just found this article that is the same question but unanswered
Linq - group by datetime for previous 12 months - include empty months

To get the past twelve months, use
var now = DateTime.Now;
var months = Enumerable.Range(-12, 12)
.Select(x => new {
year = now.AddMonths(x).Year,
month = now.AddMonths(x).Month });
To be safe you should first move 'now' to the start of the month to avoid any end-of-month effects with AddMonth.
var now = DateTime.Now;
now = now.Date.AddDays(1-now.Day);
Complete example:-
var list = new [] {
new { LocalTimeStamp = DateTime.Parse("12/2/2013"), count = 4},
new { LocalTimeStamp = DateTime.Parse("10/1/2014"), count = 1 },
new { LocalTimeStamp = DateTime.Parse("11/5/2014"), count = 6}
};
var now = DateTime.Now;
now = now.Date.AddDays(1-now.Day);
var months = Enumerable.Range(-12, 13)
.Select(x => new {
year = now.AddMonths(x).Year,
month = now.AddMonths(x).Month });
var changesPerYearAndMonth =
months.GroupJoin(list,
m => new {month = m.month, year = m.year},
revision => new { month = revision.LocalTimeStamp.Month,
year = revision.LocalTimeStamp.Year},
(p, g) => new {month = p.month, year = p.year,
count = g.Sum(a => a.count)});
foreach (var change in changesPerYearAndMonth)
{
Console.WriteLine(change.month + " " + change.year +" " + change.count);
}

You don't need a 3-way join, you just need to filter your data before grouping.
1) Query expression syntax
// since your list item type was not posted, anyway same access as your LocalTimeStamp property
list = new List<DateTime>();
DateTime aYearAgo = DateTime.Now.AddYears(-1);
var dateslastYear = from date in list
where date > aYearAgo
group date by new { date.Year, date.Month } into g
select new { GroupCriteria = g.Key, Count = g.Count() };
2) Chained
dateslastYear = list.Where (d=>d>aYearAgo)
.GroupBy (date=>new{date.Year, date.Month });
3) If you want grouping by year/month pairs, including records of not existent entries, and also omitting those pairs that are older than a year occurring with the joined Enumerable.Range call:
var thisYearPairs = from m in Enumerable.Range(1, DateTime.Now.Month)
select new { Year = DateTime.Now.Year, Month = m };
var lastYearPairs = from m in Enumerable.Range(DateTime.Now.Month, 12 - DateTime.Now.Month + 1)
select new { Year = DateTime.Now.Year - 1, Month = m };
var ymOuter = from ym in thisYearPairs.Union(lastYearPairs)
join l in list on new { ym.Year, ym.Month } equals new { l.Year, l.Month } into oj
from p in oj.DefaultIfEmpty()
select new { a = ym, b = p == null ? DateTime.MinValue : p };
var ymGroup = from ym in ymOuter
group ym by ym into g
select new { GroupCriteria = g.Key.a, Count = g.Key.b == DateTime.MinValue ? 0 : g.Count() };

You are taking the range for the 12 months of last year only but you actually want the last twelve months.
You can do this using a Enumerable.Range and the AddMonths method:
var changesPerYearAndMonth =
from month in Enumerable.Range(0, 12)
let key = new { Year = DateTime.Now.AddMonths(-month).Year, Month = DateTime.Now.AddMonths(-month).Month }
join revision in list on key
equals new
{
revision.LocalTimeStamp.Year,
revision.LocalTimeStamp.Month
} into g
select new { GroupCriteria = key, Count = g.Count() };

public int YearDiff(DateTime a, DateTime b)
{
return (int) Math.Floor((a.Year + a.Month / 100.0 + a.Day / 10000.0) - (b.Year + b.Month / 100.0 + b.Day / 10000.0));
}

c# datatable add where condition on array of string

This is my code:
DateTime epoch = new DateTime(1970, 1, 1);
var result = (from row in InBoundtable.AsEnumerable()
group row by row.Field <string> ("Date") into grp
select new {
AbandonCalls = grp.Sum((r) => Double.Parse(r["AvgAbandonedCalls"].ToString())),
Date = ((DateTime.Parse(grp.Key)) - epoch).TotalMilliseconds
}).ToList();
where InBoundtable is a datatable.
now I have a string array campains
my question is there a way so in the select statement above I can make where the campain field, which is a string field, is either one of the values in the campains array?

You can use Enumerable.Contains("campain value"):
var query = from row in InBoundtable.AsEnumerable()
where campains.Contains(row.Field<string>("Campain"))
group row by row.Field <string> ("Date") into grp
select new {
AbandonCalls = grp.Sum(r => Double.Parse(r["AvgAbandonedCalls"].ToString())),
Date = ((DateTime.Parse(grp.Key)) - epoch).TotalMilliseconds
};
var result = query.ToList();

Using linq group by statement as a subquery

I have the following query which works fine:
var bands = new List<TimeBand>()
{
new TimeBand(){Region = 10,PeriodId = 5,StartDate = new DateTime(2013, 04, 01),EndDate = new DateTime(2014, 05, 31),DayName = "Friday",StartTime = "00:00",EndTime = "07:00"},
new TimeBand(){Region = 10,PeriodId = 5,StartDate = new DateTime(2013, 04, 01),EndDate = new DateTime(2013, 05, 31),DayName = "Friday",StartTime = "07:00",EndTime = "00:00"},
new TimeBand(){Region = 10,PeriodId = 4,StartDate = new DateTime(2013, 06, 01),EndDate = new DateTime(2013, 08, 31),DayName = "Saturday",StartTime = "20:00",EndTime = "00:00"}
};
var query = (from x in bands
group x by new {x.Region, x.DayName}
into grp
select new TimeBand()
{
Region = grp.Key.Region,
DayName = grp.Key.DayName,
StartDate = grp.Min(x => x.StartDate),
EndDate = grp.Max(x => x.EndDate)
}).ToList();
But as I group the results by Region and Dayname I am not getting the other columns in my result i.e StartTime and EndTime.
If this was a SQL query I would have used this grouped results in a subquery and get the other columns as well.
Is there any way of modifying this so I also get the properties which are not included in the group by statement.
Thanks

After grouping source items you have sequence of groups. How you will project these groups is up to you. Usually you select grouping keys and some aggregated values on each group (that is how SQL works). But you can select each group itself, or first item from each group, or some value from last group item:
from b in bands
group b by new { b.Region, b.DayName } into g
select new {
g.Key.Region,
g.Key.DayName,
StartDate = g.Min(x => x.StartDate),
EndDate = g.Max(x => x.EndDate),
AllBandsFromGroup = g,
FirstBand = g.First(),
LastBandPeriod = g.Last().Period
}

Just select out the whole group if you want the keys as well as all of the values for all of the items in the groups:
select grp;
The other option is to select out a sequence of all values of the columns that you want from a group:
select new TimeBand()
{
Region = grp.Key.Region,
DayName = grp.Key.DayName,
StartDates = grp.Select(x => x.StartDate),
EndDates = grp.Select(x => x.EndDate)
}
You'd only need to do this if you didn't want to pull down the data for some number of other columns. If you want all of the columns, just use the first option.

var query = (from x in bands
group x by new { x.Region, x.DayName }
into grp
select new
{
Region = grp.Key.Region,
DayName = grp.Key.DayName,
MinStartDate = grp.Min(x => x.StartDate),
AllStartDates = grp.Select(k => k.StartDate).ToList(),
EndDate = grp.Max(x => x.EndDate),
AllEndDates = grp.Select(k => k.EndDate).ToList(),
}).ToList();

It seems your are trying to find the minimum start date and time and maximum end date and time for each group. This might be much easier if you combine date and time into a single property.
Otherwise you will need a IComparer<TimeBand> for start date and time and another one for end date and time.
If you don't want to combine date and time you can write methods to get the combined values:
public DateTime GetStart()
{
int hour = int.Parse(StartTime.Substring(0, 2));
int minute = int.Parse(StartTime.Substring(3, 2));
return new DateTime(StartDate.Year, StartDate.Month, StartDate.Day, hour, minute, 0);
}
public DateTime GetEnd()
{
int hour = int.Parse(EndTime.Substring(0, 2));
int minute = int.Parse(EndTime.Substring(3, 2));
return new DateTime(EndDate.Year, EndDate.Month, EndDate.Day, hour, minute, 0);
}
Now you can group the bands and for each group find the minimum start and maximum end:
var query = from x in bands
group x by new
{
x.Region,
x.DayName
}
into grp
select new TimeBand()
{
Region = grp.Key.Region,
DayName = grp.Key.DayName,
StartDate = grp.Min(x => x.StartDate),
StartTime = grp.Min(x => x.GetStart()).ToShortTimeString(),
EndDate = grp.Max(x => x.EndDate),
EndTime = grp.Max(x => x.GetEnd()).ToShortTimeString(),
};
This is however not very elegant. I would prefer to combine date and time in the first place.

Group by date range , count and sort within each group LINQ

I have a collection of dates stored in my object. This is sample data. In real time, the dates will come from a service call and I will have no idea what dates and how many will be returned:
var ListHeader = new List<ListHeaderData>
{
new ListHeaderData
{
EntryDate = new DateTime(2013, 8, 26)
},
new ListHeaderData
{
EntryDate = new DateTime(2013, 9, 11)
},
new ListHeaderData
{
EntryDate = new DateTime(2013, 1, 1)
},
new ListHeaderData
{
EntryDate = new DateTime(2013, 9, 15)
},
new ListHeaderData
{
EntryDate = new DateTime(2013, 9, 17)
},
new ListHeaderData
{
EntryDate = new DateTime(2013, 9, 5)
},
};
I now need to group by date range like so:
Today (1) <- contains the date 9/17/2013 and count of 1
within 2 weeks (3) <- contains dates 9/15,9/11,9/5 and count of 3
More than 2 weeks (2) <- contains dates 8/26, 1/1 and count of 2
this is my LINQ statement which doesn't achieve what I need but i think i'm in the ballpark (be kind if I'm not):
var defaultGroups = from l in ListHeader
group l by l.EntryDate into g
orderby g.Min(x => x.EntryDate)
select new { GroupBy = g };
This groups by individual dates, so I have 6 groups with 1 date in each. How do I group by date range , count and sort within each group?

Introduce array, which contains ranges you want to group by. Here is two ranges - today (zero days) and 14 days (two weeks):
var today = DateTime.Today;
var ranges = new List<int?> { 0, 14 };
Now group your items by range it falls into. If there is no appropriate range (all dates more than two weeks) then default null range value will be used:
var defaultGroups =
from h in ListHeader
let daysFromToday = (int)(today - h.EntryDate).TotalDays
group h by ranges.FirstOrDefault(range => daysFromToday <= range) into g
orderby g.Min(x => x.EntryDate)
select g;
UPDATE: Adding custom ranges for grouping:
var ranges = new List<int?>();
ranges.Add(0); // today
ranges.Add(7*2); // two weeks
ranges.Add(DateTime.Today.Day); // within current month
ranges.Add(DateTime.Today.DayOfYear); // within current year
ranges.Sort();

How about doing this?
Introduce a new property for grouping and group by that.
class ListHeaderData
{
public DateTime EntryDate;
public int DateDifferenceFromToday
{
get
{
TimeSpan difference = DateTime.Today - EntryDate.Date;
if (difference.TotalDays == 0)//today
{
return 1;
}
else if (difference.TotalDays <= 14)//less than 2 weeks
{
return 2;
}
else
{
return 3;//something else
}
}
}
}
Edit: as #servy pointed in comments other developers may confuse of int using a enum will be more readable.
So, modified version of your class would look something like this
class ListHeaderData
{
public DateTime EntryDate;
public DateRange DateDifferenceFromToday
{
get
{
//I think for this version no comments needed names are self explanatory
TimeSpan difference = DateTime.Today - EntryDate.Date;
if (difference.TotalDays == 0)
{
return DateRange.Today;
}
else if (difference.TotalDays <= 14)
{
return DateRange.LessThanTwoWeeks;
}
else
{
return DateRange.MoreThanTwoWeeks;
}
}
}
}
enum DateRange
{
None = 0,
Today = 1,
LessThanTwoWeeks = 2,
MoreThanTwoWeeks = 3
}
and use it like this
var defaultGroups = from l in ListHeader
group l by l.DateDifferenceFromToday into g // <--Note group by DateDifferenceFromToday
orderby g.Min(x => x.EntryDate)
select new { GroupBy = g };

Do you specifically want to achieve the solution in this way? Also do you really want to introduce spurious properties into your class to meet these requirements?
These three lines would achieve your requirements and for large collections willbe more performant.
var todays = listHeader.Where(item => item.EntryDate == DateTime.Today);
var twoWeeks = listHeader.Where(item => item.EntryDate < DateTime.Today.AddDays(-1)
&& item.EntryDate >= DateTime.Today.AddDays(-14));
var later = listHeader.Where(item => item.EntryDate < DateTime.Today.AddDays(-14));
also you then get the flexibility of different groupings without impacting your class.
[Edit: in response to ordering query]
Making use of the Enum supplied above you can apply the Union clause and OrderBy clause Linq extension methods as follows:
var ord = todays.Select(item => new {Group = DateRange.Today, item.EntryDate})
.Union(
twoWeeks.Select(item => new {Group = DateRange.LessThanTwoWeeks, item.EntryDate}))
.Union(
later.Select(item => new {Group = DateRange.MoreThanTwoWeeks, item.EntryDate}))
.OrderBy(item => item.Group);
Note that I'm adding the Grouping via a Linq Select and anonymous class to dynamically push a Group property again not effecting the original class. This produces the following output based on the original post:
Group EntryDate
Today 17/09/2013 00:00:00
LessThanTwoWeeks 11/09/2013 00:00:00
LessThanTwoWeeks 15/09/2013 00:00:00
LessThanTwoWeeks 05/09/2013 00:00:00
MoreThanTwoWeeks 26/08/2013 00:00:00
MoreThanTwoWeeks 01/01/2013 00:00:00
and to get grouped date ranges with count:
var ord = todays.Select(item => new {Group = DateRange.Today, Count=todays.Count()})
.Union(
twoWeeks.Select(item => new {Group = DateRange.LessThanTwoWeeks, Count=twoWeeks.Count()}))
.Union(
later.Select(item => new {Group = DateRange.MoreThanTwoWeeks, Count=later.Count()}))
.OrderBy(item => item.Group);
Output is:
Group Count
Today 1
LessThanTwoWeeks 3
MoreThanTwoWeeks 2

I suppose this depends on how heavily you plan on using this. I had/have a lot of reports to generate so I created a model IncrementDateRange with StartTime, EndTime and TimeIncrement as an enum.
The time increment handler has a lot of switch based functions spits out a list of times between the Start and End range based on hour/day/week/month/quarter/year etc.
Then you get your list of IncrementDateRange and in linq something like either:
TotalsList = times.Select(t => new RetailSalesTotalsListItem()
{
IncrementDateRange = t,
Total = storeSales.Where(s => s.DatePlaced >= t.StartTime && s.DatePlaced <= t.EndTime).Sum(s => s.Subtotal),
})
or
TotalsList = storeSales.GroupBy(g => g.IncrementDateRange.StartTime).Select(gg => new RetailSalesTotalsListItem()
{
IncrementDateRange = times.First(t => t.StartTime == gg.Key),
Total = gg.Sum(rs => rs.Subtotal),
}).ToList(),

Find max and min DateTime in Linq group

I'm trying to find the max and min DateTimes from a CSV import.
I have this to import the data from the temp DataTable:
var tsHead = from h in dt.AsEnumerable()
select new
{
Index = h.Field<string>("INDEX"),
TimeSheetCategory = h.Field<string>("FN"),
Date = DdateConvert(h.Field<string>("Date")),
EmployeeNo = h.Field<string>("EMPLOYEE"),
Factory = h.Field<string>("FACTORY"),
StartTime = DdateConvert(h.Field<string>("START_TIME")), //min
FinishTime = DdateConvert(h.Field<string>("FINISH_TIME")), //max
};
Which works fine. I then want to group the data and show the Start time and finish time which is the min / max of the respective fields.
So far I have this:
var tsHeadg = from h in tsHead
group h by h.Index into g //Pull out the unique indexes
let f = g.FirstOrDefault() where f != null
select new
{
f.Index,
f.TimeSheetCategory,
f.Date,
f.EmployeeNo,
f.Factory,
g.Min(c => c).StartTime, //Min starttime should be timesheet start time
g.Max(c => c).FinishTime, //Max finishtime should be timesheet finish time
};
With the thinking that g.Min and g.Max would give me the lowest and highest DateTime for each timesheet (grouped by index)
This doesn't work however... Whats the best way of finding the highest and lowest value of DateTimes within a group?

Try using this
var tsHeadg =
(from h in tsHead
group h by h.Index into g //Pull out the unique indexes
let f = g.FirstOrDefault()
where f != null
select new
{
f.Index,
f.TimeSheetCategory,
f.Date,
f.EmployeeNo,
f.Factory,
MinDate = g.Min(c => c.StartTime),
MaxDate = g.Max(c => c.FinishTime),
});