We Keep Coding

Search query IQueryable<> MVC

I Have a problem with my search query that it doesnt select any values from the database when i iam using IQueryable<> in MVC.
My code look like this:
IQueryable<Invoice> res =
(from c in table
join i in tableX on c.attr equals i.Id
where c.attr== merchant
select i);
if (buyer > 0)
{
res = res.Where(i => i.Buyer.Msisdn == buyer);
}
if (send)
{
res =res.Where(i => i.Status == InvoiceStatus.Sent);
}
if (paid)
{
res= res.Where(i => i.Status == InvoiceStatus.Paid);
}
if (notBilled)
{
res = res.Where(i => i.Status == InvoiceStatus.Open);
}
if (startDate <= endDate)
{
res = res.Where(i => i.DateCreated >= startDate && i.DateCreated <= endDate);
}
return res.ToList();
if i dont set res = res.Where() and instead just have res.where() the query is selecting the values from the database. Can someone please make me understand why that is. I thought you need to store the query result in a variable.

The code you've posted looks to be the appropriate way to implement IQueryable.
res = res.Where(...)
basically tacks on additional where clause information until the query is executed at res.ToList();.
calling res.Where doesn't actually make changes to the res query.
You're likely just limiting your where clause too much and eliminating all records from the query.
Have you tried profiling the query to determine what is being queried?
I can tell you that if more than one of send, paid, or notbilled is true, that would immediately not allow any results to be returned from the query as they're all checking against the Status column - which could not possibly have more than one value.
EDIT
I don't know if this will help, but here's a fiddle going through some of the intricacies of IQueryable: https://dotnetfiddle.net/d70XKA
Here's the code from the fiddle:
public class Program
{
public static void Main()
{
Thingy t = new Thingy();
// Note execution is deferred until enumeration (in this case Count())
var allData = t.GetData();
Console.WriteLine("All Data count: {0}", allData.Count());
// Select only valid records from data set (should be 2)
var isValid = t.GetData();
isValid = isValid.Where(w => w.IsValid);
Console.WriteLine("IsValid count: {0}", isValid.Count());
// select only records with an ID greater than 1 (should be 2)
var gt1 = t.GetData();
gt1 = gt1.Where(w => w.Id > 1);
Console.WriteLine("gt 1 count: {0}", gt1.Count());
// Here we're combining in a single statement, IsValid and gt 1 (should be 1)
var isValidAndIdGt1 = t.GetData();
isValidAndIdGt1 = isValidAndIdGt1.Where(w => w.IsValid && w.Id > 1);
Console.WriteLine("IsValid and gt 1 count: {0}", isValidAndIdGt1.Count());
// This is the same query as the one directly above, just broken up (could perhaps be some if logic in there to determine if to add the second Where
// Note this is how you're doing it in your question (and it's perfectly valid (should be 1)
var isValidAndIdGt1Appended = t.GetData();
isValidAndIdGt1Appended = isValidAndIdGt1Appended.Where(w => w.IsValid);
isValidAndIdGt1Appended = isValidAndIdGt1Appended.Where(w => w.Id > 1);
Console.WriteLine("IsValid and gt 1 count w/ appended where: {0}", isValidAndIdGt1Appended.Count());
// This is the same query as the one directly above, but note we are executing the query twice
var isValidAndIdGt1AppendedTwice = t.GetData();
isValidAndIdGt1AppendedTwice = isValidAndIdGt1AppendedTwice.Where(w => w.IsValid);
Console.WriteLine("IsValid and gt 1 count w/ appended where executing twice: {0}", isValidAndIdGt1AppendedTwice.Count()); // 2 results are valid
isValidAndIdGt1AppendedTwice = isValidAndIdGt1AppendedTwice.Where(w => w.Id > 1);
Console.WriteLine("IsValid and gt 1 count w/ appended where executing twice: {0}", isValidAndIdGt1AppendedTwice.Count()); // 1 result is both valid and id gt 1
// This is one of the things you were asking about - note that without assigning the additional Where criteria to the Iqueryable, you do not get the results of the where clause, but the original query - in this case there are no appended where conditions on the t.GetData() call, so you get the full result set.
var notReallyValid = t.GetData();
notReallyValid.Where(w => w.Name == "this name definitly does not exist");
Console.WriteLine("where clause not correctly appended count: {0}", notReallyValid.Count());
// vs
var validUse = t.GetData();
validUse = validUse.Where(w => w.Name == "this name definitly does not exist");
Console.WriteLine("valid use count: {0}", validUse.Count());
}
}
public class Thingy
{
private List<Foo> _testData = new List<Foo>()
{
new Foo()
{
Id = 1,
Name = "Alpha",
Created = new DateTime(2015, 1, 1),
IsValid = true
},
new Foo()
{
Id = 2,
Name = "Beta",
Created = new DateTime(2015, 2, 1),
IsValid = false
},
new Foo()
{
Id = 3,
Name = "Gamma",
Created = new DateTime(2015, 3, 1),
IsValid = true
},
};
public IQueryable<Foo> GetData()
{
return _testData.AsQueryable();
}
public void PrintData(IEnumerable<Foo> data)
{
// Note calling this will enumerate the data for IQueryable
foreach (Foo f in data)
{
Console.WriteLine(string.Format("id: {0}, name: {1}, created: {2}, isValid: {3}", f.Id, f.Name, f.Created, f.IsValid));
}
}
}
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
public DateTime Created { get; set; }
public bool IsValid { get; set; }
}
Having said all of that, there is something in your where clause that is filtering out your expected data. As you can see from the example above res = res.Where(...) is very different from res.Where(...) - the former is the correct approach. The latter is simply omitting all where clauses from your statement completely, then when ToList() is called you're getting the full result set as no Where conditions have been added (save where c.attr== merchant from the original var creation)

IQueryable objects don't actually contain data until you either do a ToList(). Until then they're just queries. So what you've done in you assignment is replaced the query with ... I don't know what. What you should do is something like this:
IQueryable Results = res.Where(i => i.Status == InvoiceStatus.Paid); //(or whatever)
return (Results.ToList());

Group on Two Columns and Create Two Separate Counts

Question
We have a table of StudentId and LectureId, and we want to know two things.
CountStudentId How many times each StudentId occurs.
CountStudentIdLectureId How many times each StudentId + LectureId pair occurs.
(2) is done in below. (1) is not.
In other words, how can we count two different groups in a single query?
Another way of thinking about this, would be to count the StudentId + LectureId group and also sum that count for each StudentId.
What we've tried
The following query groups on StudentId + LectureId. It does count how many times the StudentId + LectureId group occurs. It doesn't count how many times the StudentId group occurs. That's what we also want.
var query = joinTable
.GroupBy(jt => new { jt.StudentId, jt.LectureId } )
.Select(g => new {
StudentId = g.Key.StudentId,
LectureId = g.Key.LectureId,
CountStudentId = -1, // Count all StudentId (i.e. 10)?
CountStudentIdLectureId = g.Count()
});
This is the result we're currently receiving. In each row, the -1 value should be 10 (because we seeded the JoinTable with ten of each StudentId) and we haven't achieved that.
Results we want to achieve
...but with 10 instead of -1 in each case.**
StudentId LectureId CountStudentId CountStudentLectureId
0 0 -1 3
0 1 -1 3
0 2 -1 3
0 3 -1 1
1 0 -1 2
1 1 -1 3
1 2 -1 3
1 3 -1 2
In those results, we need CountStudentId to be 10 not -1 (the latter is just a placeholder for now.)
That's the expected result, because each StudentId occurs 10 times and because the sum of CountStudentLectureId for each StudentId is 10, which is just two ways of saying the same thing.
Full demo code
This is the full Fiddle code for reference.
using System;
using System.Linq;
using System.Collections.Generic;
public static class Program
{
public static void Main()
{
var joinTable = SeedJoinTable();
var query = joinTable
.GroupBy(jt => new { jt.StudentId, jt.LectureId } )
.Select(g => new {
StudentId = g.Key.StudentId,
LectureId = g.Key.LectureId,
CountStudentId = -1, // Count all StudentId (i.e. 10)?
CountStudentIdLectureId = g.Count()
});
// this is just the printing of the results
Console.WriteLine(
"StudentId".PadRight(15) +
"LectureId".PadRight(15) +
"CountStudentId".PadRight(17) +
"CountStudentLectureId".PadRight(15));
foreach(var x in query)
{
Console.WriteLine(string.Format("{0}{1}{2}{3}",
x.StudentId.ToString().PadRight(15),
x.LectureId.ToString().PadRight(15),
x.CountStudentId.ToString().PadRight(17),
x.CountStudentIdLectureId.ToString().PadRight(15)));
}
}
public static List<JoinTable> SeedJoinTable()
{
var list = new List<JoinTable>();
var studentId = 0;
var lectureId = 0;
// insert 20 records
for(int i = 0; i < 20; ++i)
{
if(i != 0)
{
if(i % 10 == 0)
{
// 10 of each studentId
++studentId;
lectureId = 0;
}
if(i % 3 == 0)
{
// 3 of each lectureId per student
++lectureId;
}
}
list.Add(new JoinTable() {
StudentId = studentId,
LectureId = lectureId
});
}
return list;
}
public class JoinTable
{
public int StudentId { get; set; }
public int LectureId { get; set; }
}
}

Here is a working DotNotFiddle that produces the results that you want to achieve.
You will want to group by StudentId and set the value to LectureId. This allows you to get the count of both studentId and studentIdLectureId pairs.
var query = joinTable
.GroupBy(jt => jt.StudentId, jt => jt.LectureId)
.Select(x =>
new {
StudentId = x.Key,
CountStudentId = x.Count(),
LectureIds = x.GroupBy(y => y),
});
This does alter how you will loop through the final list, but will provide you the same data with the same amount of loops:
foreach(var x in query)
{
foreach(var lectureId in x.LectureIds)
{
Console.WriteLine(string.Format("{0}{1}{2}{3}",
x.StudentId.ToString().PadRight(15),
lectureId.Key.ToString().PadRight(15),
x.CountStudentId.ToString().PadRight(17),
lectureId.Count().ToString().PadRight(15)));
}
}
If you want to include anything with the lectureId (lecture name, professor, etc.) you can do so like this:
var query = joinTable
.GroupBy(jt => jt.StudentId, jt => new {LectureId = jt.LectureId, ProfessorId = jt.ProfessorId})
.Select(x =>
new {
StudentId = x.Key,
CountStudentId = x.Count(),
LectureIds = x.GroupBy(y => y),
});

How to group datatable by unknown column names and calculate sum of one field?

I have a table like this:
Name Age Gender
Sasha 12 W
Sasha 20 W
Sasha 21 M
Bob 21 M
And I want to group by multiple fields, for example [Name] and [Gender] and sum this by field [Age]. The columns are unknown at compile time because the user can select them.
So, in in this example i want this:
Name Age Gender
Sasha 32 W
Sasha 21 M
Bob 21 M
But I can't do it by LINQ, because I don't know the columns at compile time.
Thanks for answers!

You can use an anonymous type if you want to group by multiple columns.
var ageSumsPerNameAndGender = table.AsEnumerable()
.GroupBy(row => new { Name = row.Field<string>("Name"), Gender = row.Field<string>("Gender") })
.Select(group => new
{
Name = group.Key.Name,
Gender = group.Key.Gender,
SumOfAge = group.Sum(row => row.Field<int>("Age"))
});
If you want to ouput this you could use a foreach-loop:
Console.WriteLine("Name Age Gender");
foreach(var x in ageSumPerNamegenders)
Console.WriteLine("{0} {1} {2}", x.Name, x.SumOfAge, x.Gender);
According to your comments it seems that you actually don't know the columns because the user specifies them. Then it's more difficult and error-prone.
One way is to provide a custom IEqualityComparer<T> for multiple fields. This should work:
public class MultiFieldComparer : IEqualityComparer<IEnumerable<object>>
{
public bool Equals(IEnumerable<object> x, IEnumerable<object> y)
{
if(x == null || y == null) return false;
return x.SequenceEqual(y);
}
public int GetHashCode(IEnumerable<object> objects)
{
if(objects == null) return 0;
unchecked
{
int hash = 17;
foreach(object obj in objects)
hash = hash * 23 + (obj == null ? 0 : obj.GetHashCode());
return hash;
}
}
}
Now you can use an instance of this comparer for Enumerable.GroupBy (and many other LINQ methods). Here is a working example:
List<string> columnNames = new List<string> { "Name", "Gender" };
var columnsToGroupBy = table.Columns.Cast<DataColumn>()
.Where(c => columnNames.Contains(c.ColumnName, StringComparer.InvariantCultureIgnoreCase))
.ToArray();
var comparer = new MultiFieldComparer();
var summed = table.AsEnumerable()
.GroupBy(row => columnsToGroupBy.Select(c => row[c]), comparer)
.Select(group => new
{
AllFields = group.Key,
Sum = group.Sum(row => row.IsNull("Age") ? 0 : decimal.Parse(row["Age"].ToString()))
});
foreach (var x in summed)
{
Console.WriteLine("{0} Sum: {1}", string.Join(" ", x.AllFields), x.Sum);
}
As you can see i've hardcoded "Age" as sum-column. It must be a numeric column, so you have to ensure that. You could also let the user provide it. But again, it must be parsable to decimal otherwise it doesn't work.

Linq: find similar objects from two different lists

I've got two separate lists of custom objects. In these two separate lists, there may be some objects that are identical between the two lists, with the exception of one field ("id"). I'd like to know a smart way to query these two lists to find this overlap. I've attached some code to help clarify. Any suggestions would be appreciated.
namespace ConsoleApplication1
{
class userObj
{
public int id;
public DateTime BirthDate;
public string FirstName;
public string LastName;
}
class Program
{
static void Main(string[] args)
{
List<userObj> list1 = new List<userObj>();
list1.Add(new userObj()
{
BirthDate=DateTime.Parse("1/1/2000"),
FirstName="John",
LastName="Smith",
id=0
});
list1.Add(new userObj()
{
BirthDate = DateTime.Parse("2/2/2000"),
FirstName = "Jane",
LastName = "Doe",
id = 1
});
list1.Add(new userObj()
{
BirthDate = DateTime.Parse("3/3/2000"),
FirstName = "Sam",
LastName = "Smith",
id = 2
});
List<userObj> list2 = new List<userObj>();
list2.Add(new userObj()
{
BirthDate = DateTime.Parse("1/1/2000"),
FirstName = "John",
LastName = "Smith",
id = 3
});
list2.Add(new userObj()
{
BirthDate = DateTime.Parse("2/2/2000"),
FirstName = "Jane",
LastName = "Doe",
id = 4
});
List<int> similarObjectsFromTwoLists = null;
//Would like this equal to the overlap. It could be the IDs on either side that have a "buddy" on the other side: (3,4) or (0,1) in the above case.
}
}
}

I don't know why you want a List<int>, i assume this is what you want:
var intersectingUser = from l1 in list1
join l2 in list2
on new { l1.FirstName, l1.LastName, l1.BirthDate }
equals new { l2.FirstName, l2.LastName, l2.BirthDate }
select new { ID1 = l1.id, ID2 = l2.id };
foreach (var bothIDs in intersectingUser)
{
Console.WriteLine("ID in List1: {0} ID in List2: {1}",
bothIDs.ID1, bothIDs.ID2);
}
Output:
ID in List1: 0 ID in List2: 3
ID in List1: 1 ID in List2: 4

You can implement your own IEqualityComparer<T> for your userObj class and use that to run a comparison between the two lists. This will be the most performant approach.
public class NameAndBirthdayComparer : IEqualityComparer<userObj>
{
public bool Equals(userObj x, userObj y)
{
return x.FirstName == y.FirstName && x.LastName == y.LastName && x.BirthDate == y.BirthDate;
}
public int GetHashCode(userObj obj)
{
unchecked
{
var hash = (int)2166136261;
hash = hash * 16777619 ^ obj.FirstName.GetHashCode();
hash = hash * 16777619 ^ obj.LastName.GetHashCode();
hash = hash * 16777619 ^ obj.BirthDate.GetHashCode();
return hash;
}
}
}
You can use this comparer like this:
list1.Intersect(list2, new NameAndBirthdayComparer()).Select(obj => obj.id).ToList();

You could simply join the lists on those 3 properties:
var result = from l1 in list1
join l2 in list2
on new {l1.BirthDate, l1.FirstName, l1.LastName}
equals new {l2.BirthDate, l2.FirstName, l2.LastName}
select new
{
fname = l1.FirstName,
name = l1.LastName,
bday = l1.BirthDate
};
Instead of doing a simple join on just one property (column), two anonymous objects are created new { prop1, prop2, ..., propN}, on which the join is executed.
In your case we are taking all properties, except the Id, which you want to be ignored and voila:
Output:
And Tim beat me to it by a minute

var similarObjectsFromTwoLists = list1.Where(x =>
list2.Exists(y => y.BirthDate == x.BirthDate && y.FirstName == x.FirstName && y.LastName == x.LastName)
).ToList();
This is shorter, but for large list is more efficient "Intersect" or "Join":
var similarObjectsFromTwoLists =
list1.Join(list2, x => x.GetHashCode(), y => y.GetHashCode(), (x, y) => x).ToList();
(suposing GetHashCode() is defined for userObj)

var query = list1.Join (list2,
obj => new {FirstName=obj.FirstName,LastName=obj.LastName, BirthDate=obj.BirthDate},
innObj => new {FirstName=innObj.FirstName, LastName=innObj.LastName, BirthDate=innObj.BirthDate},
(obj, userObj) => (new {List1Id = obj.id, List2Id = userObj.id}));
foreach (var item in query)
{
Console.WriteLine(item.List1Id + " " + item.List2Id);
}

Select Single Element from Jagged Array

I'm working on a problem that's making my brain melt although I don't think it should be this hard. My example is long so I'll try to keep my question short!
I have an Array object that contains some elements that are also Arrays. For example:
customerAddresses = new customer_address[]
{
new // address #1
{
customer_id = 6676979,
customer_address_seq = 1,
customer_address_match_codes = new []
{
new
{
customer_address_seq = 1,
customer_id = 6676979,
customer_match_code_id = 5
}
}
},
new // address #2
{
customer_id = 6677070,
customer_address_seq = 1,
customer_address_match_codes = new []
{
new
{
customer_address_seq = 1,
customer_id = 6677070,
customer_match_code_id = 4
},
new
{
customer_address_seq = 1,
customer_id = 6677070,
customer_match_code_id = 5
},
new
{
customer_address_seq = 1,
customer_id = 6677070,
customer_match_code_id = 3
}
}
},
new // address #3
{
customer_id = 6677070,
customer_address_seq = 2,
customer_address_match_code = new []
{
new
{
customer_address_seq = 2,
customer_id = 6677070,
customer_match_code_id = 4
},
new
{
customer_address_seq = 2,
customer_id = 6677070,
customer_match_code_id = 5
}
}
}
};
As you can see, the Array contains a number of address records, with one record per combination of customer_id and customer_address_seq. What I'm trying to do is find the best matching customer_address according to the following rules:
There must be customer_match_code_id equal to 4 and there must be one equal to 5
If there is a customer_match_code_id equal to 3, then consider that customer_address a stronger match.
According to the above rules, the 2nd customer_address element is the "best match". However, the last bit of complexity in this problem is that there could be multiple "best matches". How I need to handle that situation is by taking the customer_address record with the minimum customer_id and minimum customer_address_seq.
I was thinking that using LINQ would be my best bet, but I'm not experienced enough with it, so I just keep spinning my wheels.

Had to make a change to your class so that you are actually assigning your one collection to something:
customer_address_match_codes = new customer_address_match_code[]
{
new
{
customer_address_seq = 1,
customer_id = 6676979,
customer_match_code_id = 5
}
}
And then here is the LINQ that I've tested and does what you specify:
var result = (from c in customerAddresses
let isMatch = c.customer_address_match_codes
.Where (cu => cu.customer_match_code_id == 4).Any () &&
c.customer_address_match_codes
.Where (cu => cu.customer_match_code_id == 5).Any ()
let betterMatch = isMatch && c.customer_address_match_codes
.Where (cu => cu.customer_match_code_id == 3).Any () ? 1 : 0
where isMatch == true
orderby betterMatch descending, c.customer_id, c.customer_address_seq
select c)
.FirstOrDefault ();
I've worked up an example using your data with anonymous types here: http://ideone.com/wyteM

Not tested and not the same names but this should get you going
customer cb = null;
customer[] cs = new customer[] {new customer()};
foreach (customer c in cs.OrderBy(x => x.id).ThenBy(y => y.seq))
{
if(c.addrs.Any(x => x.num == "5"))
{
if(c.addrs.Any(x => x.num == "3"))
{
if (cb == null) cb = c;
if (c.addrs.Any(x => x.num == "2"))
{
cb = c;
break;
}
}
}
}

This sounds like a job for LINQ
var bestMatch = (from address in DATA
where address.customer_address_match_code.Any(
x => x.customer_match_code_id == 4)
where address.customer_address_match_code.Any(
x => x.customer_match_code_id == 5)
select address).OrderBy(
x => x.customer_address_match_code.Where(
y => y.customer_match_code_id >= 3)
.OrderBy(y => y.customer_match_code_id)
.First()
.customer_match_code_id).FirstOrDefault();
My theory is this: Select addresses that have both a customer_match_code_id == 4 and a customer_match_code_id == 5. Then sort them by the the lowest customer_match_code_id they have that are at least 3, and then take the very first one. If there are a customer_match_code_id that equals 3 then that one is selected, if not, some else is selected. If nothing matches both 4 and 5 then null is returned.
Untested.

Seems quite straight forward in LINQ:
var query =
from ca in customerAddresses
where ca.customer_address_match_codes.Any(
mc => mc.customer_match_code_id == 4)
where ca.customer_address_match_codes.Any(
mc => mc.customer_match_code_id == 5)
orderby ca.customer_id
orderby ca.customer_address_seq
orderby ca.customer_address_match_codes.Any(
mc => mc.customer_match_code_id == 3) descending
select ca;
var result = query.Take(1);
How does that look?