We Keep Coding

Combine values from two columns in a data table to a single list

I am trying to build a list of integers from a data table by combining the data in two of the columns in the data table and getting a distinct list of values.
To get the data table.. i am calling a stored procedure (which i don't have the ability to edit)
After I call my method to return the data table that's returned by the stored procedure..
I want to look at two of the columns in the data table and get a list of values that are in these columns.
The data table returned by the stored procedure looks like this:
I want to get a list of the values in columns MailerKey and BillToKey.
What is the best way to do this? Can I use Linq to do this?
So far I've tried doing:
using(DataTable dt = getCustomers())
{
List<int> MailerKeys = dt.AsEnumerable().Select(x => x.Field<int>("MailerKey")).Distinct().ToList();
List<int> BillToKeys = dt.AsEnumerable().Select(x => x.Field<int>("BillToKey")).Distinct().ToList();
}
But how can I now combine the values?
By combine I mean, to get a list that contains the distinct values from both columns.. so eg:
I should get back:
275
58
250
50
59
99
55

You could:
create a HashSet<int> and add both lists to it;
add one list to the other and use Distinct from System.Linq.
For example:
var a = new[] { 1, 2 }; // Could be any IEnumerale<int>
var b = new[] { 2, 3 };
var hash = new HashSet<int>(a);
hash.UnionWith(b);
Console.WriteLine(string.Join(",", hash)); // Prints 1,2,3

You can use Linq to achieve the "combine" step.
First concatenate your lists using Concat, then keep only unique values using Distinct.
using(DataTable dt = getCustomers())
{
var mailerKeys = dt.AsEnumerable().Select(x => x.Field<int>("MailerKey")).Distinct().ToArray();
var billToKeys = dt.AsEnumerable().Select(x => x.Field<int>("BillToKey")).Distinct().ToArray();
var combinedDistinctKeys = mailerKeys.Concat(billToKeys).Distinct();
}
Notes:
'combinedDistinctKeys' is not yet enumerated, this would occur using ToArray or ToList for example;
I preferred to use ToArray for the source lists, since I know I will not need List<> features.
For more information about the Concat method from Linq, see: https://learn.microsoft.com/en-us/dotnet/api/system.linq.enumerable.concat?view=net-5.0

Is there a way in linq wherin i can insert a row(from dictionary) in datatable using the list of column names c#?

I have a List<Dictionary<string,string>> something like this:
[0] key1 val,key2 val,key3 val
[1] key1 val,key2 val,key3 val
[2] key1 val,key2 val,key3 val
And i have a list of column names in the same order as columns in the datatable.
I want to filter only those keys which are there inside the list from the dictionary and also insert it in the proper order.
I'm able to filter the required keys to be inserted but then how do i insert it in the proper order in linq.
var colList = new List<string>() { "key3", "key1"};
dict.ForEach(p => jsonDataTable.Rows.Add(p.Where(q=>colList.Contains(q.key)).Select(r => r.Value).ToArray()));
I cannot do like this because number of columns will vary and also the method must work when we pass any list of column names:
foreach(var item in dict)
jsonDatatable.Rows.Add(item[colList[0]], item[colList[1]]);
Please suggest some ways.

LINQ will never ever change the input sources. You can only extract data from it.
Divide problems in subproblems
The only way to change the input sources is by using the extracted data to update your sources. Make sure that before you update the source you have materialized your query (= ToList() etc)
You can divide your problem into subproblems:
Convert the table into a sequence of columns in the correct order
convert the sequence of columns into a sequence of column names (still in the correct order)
use the column names and the dictionary to fetch the requested data.
By separating your problem into these steps, you prepare your solution for reusability. If in future you change your table to a DataGridView, or a table in an entity framework database, or a CSV file, or maybe even JSON, you can reuse the latter steps. If in future you need to use the column names for something else, you can still use the earlier steps.
To be able to use the code in a LINQ-like way, my advice would be to create extension method. If you are unfamiliar with extension methods, read Extension Methods Demystified
You will be more familiar with the layout of your table (System.Data.DataTable? Windows.Forms.DataGridView? DataGrid in Windows.Controls?) and your columns, so you'll have to create the first ones yourself. In the example I use MyTable and MyColumn; replace them with your own Table and Column classes.
public static IEnumerable<MyColumn> ToColumns(this MyTable)
{
// TODO: return the columns of the table
}
public static IEnumerable<string> ToColumnNames(this IEnumerable<MyColumn> columns)
{
return columns.Select(column => ...);
}
If the column name is just a property of the column, I wouldn't bother creating the second procedure. However, the nice thing is that it hides where you get the name from. So to be future-changes-proof, maybe create the method anyway.
You said these columns were sorted. If you want to be able to use ThenBy(...) consider returning an IOrderedEnumerable<MyColumn>. If you won't sort the sorted result, I wouldn't bother.
Usage:
MyTable table = ...
IEnumerable<string> columnNames = table.ToColumns().ToColumnNames();
or:
IEnumerable<string> columnNames = table.ToColumns()
.Select(column => column.Name);
The third subproblem is the interesting one.
Join and GroupJoin
In LINQ whenever you have two tables and you want to use a property of the elements in one table to match them with the properties of another table, consider to use (Group-)Join.
If you only want items of the first table that match exactly one item of the other table, use Join: "Get Customer with his Address", "Get Product with its Supplier". "Book with its Author"
On the other hand, if you expect that one item of the first table matches zero or more items from the other table, use GroupJoin: "Schools, each with their Students", "Customers, each with their Orders", "Authors, each with their Books"
Some people still think in database terms. They tend to use some kind of Left Outer Join to fetch "Schools with their Students". The disadvantage of this is that if a School has 2000 Students, then the same data of the School is transferred 2000 times, once for every Student. GroupJoin will transfer the data of the School only once, and the data of every Student only once.
Back to your question
In your problem: every column name is the key of exactly one item in the Dictionary.
What do you want to do with column names without keys? If you want to discard them, use Join. If you still want to use the column names that have nothing in the Dictionary, use GroupJoin.
IEnumerable<string> columNames = ...
var result = columnNames.Join(myDictionary,
columName => columName, // from every columName take the columnName,
dictionaryItem => dictionaryItem.Key, // from every dictionary keyValuePair take the key
// parameter resultSelector: from every columnName and its matching dictionary keyValuePair
// make one new object:
(columnName, keyValuePair) => new
{
// Select the properties that you want:
Name = columnName,
// take the whole dictionary value:
Value = keyValuePair.Value,
// or select only the properties that you plan to use:
Address = new
{
Street = keyValuePair.Street,
City = keyValuePair.City,
PostCode = keyValuePair.Value.PostCode
...
},
});
If you use this more often: consider to create an extension method for this.
Note: the order of the result of a Join is not specified, so you'll have to Sort after the Order
Usage:
Table myTable = ...
var result = myTable.ToColumns()
.Select(column => column.Name)
.Join(...)
.Sort(joinResult => joinResult.Name)
.ToList();

Instead of filtering on the List<Dictionary<string, string>>, filter on the colList so that you will get in the same order and only if the colList is available in the List<Dictionary<string, string>>
This is as per my understanding, please comment if you need the result in any other way.
var dictAllValues = dict.SelectMany(x => x.Select(y => y.Value)).ToList();
// Now you can filter the colList using the above values
var filteredList = colList.Where(x => dictAllValues.Contains(x));
// or you can directly add to final list as below
jsonDataTable.Rows.AddRange(colList.Where(x => dictAllValues.Contains(x)).ToList());

Determine if an element in a list contains the same value

I have a List containing a number of Guid's.
List<Guid> recordIds = new List<Guid>;
I need to verify if the Guid's in this list are all identical.
So instead of iterating the whole list I was thinking about using some sort of:
var IdsAreIdentical = recordIds.TrueForAll(x => x == DontKnowWhatToPutHere);
My issue is that I am not really shure about the usage. Maybe somebody can put me in the right direction.

If you want to verify if all the id are the same, you could verify that all the values are the same as the first one :
bool allIdentical = recordIds.TrueForAll(i => i.Equals(recordIds.FirstOrDefault());
Another variant would be to verify the number of distinct values that you have. If the result is 1, the ids are all identicals.
var allIdentical = list.Distinct().Count() == 1;

Optional Random ordered nested grouping in linq

I am really in a position where I can't think of answer regarding optional grouping in linq.
Basically,I want to generate report which comes from a screen having filters.
These filters(mostly grouped) are optional and can be rearranged.It's something like
Filters: 1.Clients Projects Tasks Duration
or
2.Projects Clients Tasks Duration
or
3.Task Duration etc.
with all possible combinations.
Then data should look like
1.ClientA
ProjectA
TaskA
26hrs 45mins
TaskB
43hrs 23mins
ProjectB
TaskX......
2.ProjectA
ClientA
TaskA
26hrs 45mins...
3.TaskA
26hrs 45mins
TaskB
6hrs 35mins
I have data.But unable to write logic which is generalized.
I am thinking with some enum which will hold filters (viewmodel)selected like
enum.Client,enum.Project... and
if (clientGroupChecked) then
foreach(var clientGroup in list){
//group list by client here
if(projectGroupChecked) then
foreach(var projectGroup in clientGroup){
//group list by project here
}
}
I know,it's wrong.This way I have to put logic for all the combinations possible.
Couldn't think of anything else.I want it really to be generalized because it may have more filters added in future and I don't want to change entire logic just for extra filters(Of course,I want to add new filter group somewhere in the logic.But I want it to be more easy to maintain also.
Edited:#sschimmel :My point is grouping can be shuffled(for this I have buttons[selected -->green and unselected-->gray and these buttons are movable for grouping].So when writing linq logic,how can I know on what criteria I have to group in particular way? For ex: I have columns A B C D E F.In this, I can just choose to group by A or by A B or B A or ACB....etc. with all possible combinations.How to achieve this?I don't want if else check because we have many possibilities.If one more filter is added,it would have many more possibilities. That's why I am thinking for need of general approach to do this.
Edit 2:
Please find attachment and how I am trying below.
//for the following ,I need some way of writing properly passing right values
var reportGroupingCP = (from t in TaskEntries
group t by new { clientId,projectId } into g
select new
{
ClientId = g.Key.clientId,
ProjectId = g.Key.projectId,
Result = (Type)g //What could be T
}).ToList();
var reportGroupingCE = (from t in TaskEntries
group t by new { clientId,employeeId } into g
select new
{
ClientId = g.Key.clientId,
EmployeeId = g.Key.employeeId,
Result = (Type)g //What could be T
}).ToList();
//Can't use above if there is filter only for client.similarly for other cases/I don't want to write for each one.I need way to do this dynamically.May be by passing enum or adding ids to some class or something else
Filter 1
Filter 2

If I understood your question correctly, you wan't to do group your data dynamically on multiple properties.
The easiest solution would be to use Dynamic LINQ which lets you create queries from strings you can easily compose from user inputs.
// userSelections is created dynamically from what the user selected.
var userSelections = new[] { "clientId", "projectId" };
var propertiesToGroupBy = string.Join(", ", userSelections);
var groupedData = data.GroupBy("new(" + propertiesToGroupBy + ")");
It's not type safe nor checked during compile time, but fairly easy to use and solves your problem. It's also documented nicely.
I tried to come up with a solution that dynamically combines Expression<Func<TaskEntry, object>>s but got stuck on creating the Expression that ìnstantiates the anonymous type you would use inside the GroupBy(new { ... }). Because the number of selected properties to group by is not known during compile time, it's not possible to create the anonymous type.

Use Linq to get values

I am using Json.net
I have created a JArray and parsing the id values from it like so -
JArray userData = JArray.Parse(response.Content);
foreach (JObject content in userData.Children<JObject>())
{
items.Add(content["id"].ToString());
}
However I was trying to also do this usig Linq -
var items = userData.Where(x => x["id"].ToString() != null).ToList();
This seems to be a faster way to do this, however the problem I am having is that using the first method only adds the id values as I wanted, the Linq option puts the entire data set into items when the condition is met.
How an I change my condition so that it only extracts the id values?

It seems like you actually want LINQ Select.
It allows you to get items by projecting each element of a sequence into a new form.
var items = userData.Select(x => x["id"].ToString()).ToList();