We Keep Coding

How to get distinct values of each property independently

I have large yielded collection and I would like to get distinct values of each property independently:
IEnumerable<MyClass> collection = ...;
var prop1Values = collection.Select(i => i.Prop1).Distinct();
var prop2Values = collection.Select(i => i.Prop2).Distinct();
var prop3Values = collection.Select(i => i.Prop3).Distinct();
How to get it without enumerating the collection multiple times? Looking for most intuitive solution :)

You can try do it in a single foreach with a help of HashSet<T>s:
//TODO: put the right types for TypeOfProp1, TypeOfProp2, TypeOfProp3
var prop1Values = new HashSet<TypeOfProp1>();
var prop2Values = new HashSet<TypeOfProp2>();
var prop3Values = new HashSet<TypeOfProp3>();
foreach (var item in collection) {
prop1Values.Add(item.Prop1);
prop2Values.Add(item.Prop2);
prop3Values.Add(item.Prop3);
}

Saving group of items from a list

What I am trying to accomplish is to group my list by supplier id. Iterate over that list calculate stuff etc and when finished pass that list on to save and clear the list out and start over again. The error is the : "collection was modified enumeration operation may not execute. c#". Now from googling it I'm just suming then pass that group off to save and clear the list rinse and repeat. I thought the clear would empty the list and allow for reuse.
Error - "collection was modified enumeration operation may not execute. c#"
group is grouping the list that is passed in to this method.
I created the count to see how many groups I had(6).
ProcessSpecialOrders receives a grouped list, po and the total.
What happens here is this will process through the first group once. Save to the DB then it will throw the error. If I refresh my page or check the DB I can see grop one has saved.
List<xxxxx> specialOrderList = new List<xxxxx>();
var group = from supplier in list group supplier by supplier.SupplierId;
var count = group.Count();
foreach (var grp in group.ToList())
{
specialOrderList.Clear();
foreach (var g in grp.ToList())
{
var orderItemList = new MOrderItem();
if (g.ItemType == "SD")
{
xxx.Id = 0;
xxx.ItemId = g.Id;
xxx.OnHandQty = g.OnHand;
xxx.ItemNo = g.ItemNumber;
xxx.ThisQty = 0;
xxx.NetCost = itemPrice.Where(x => x.ItemId == g.Id).Sum(l => (decimal?)l.NetCost * 1) ?? 0.0M;
xxx.ExtendedCost = 123.99M;
xxx.OnHandWhenBuildQty = 0;
xxx.OnOrderQty = matcoOrderItem.Where(x => x.ItemId == g.Id).Sum(x => (int?)x.ThisQty) ?? 0;
specialOrderList.Add(orderItemList);
}
}
ProcessSpecialOrders(specialOrderList, poNo, specialOrderTotal);
}

how do I make this LINQ query faster?

modelData has 100,000 items in the list.
I am doing 2 "Selects" within 2 loops.
Could it be structured differently - as it take a long time - 10 mins
public class ModelData
{
public string name;
public DateTime DT;
public int real;
public int trade;
public int position;
public int dayPnl;
}
List<ModelData> modelData;
var dates = modelData.Select(x => x.DT.Date).Distinct();
var names = modelData.Select(x => x.name).Distinct();
foreach (var aDate in dates)
{
var dateRealTrades = modelData.Select(x => x)
.Where(x => x.DT.Date.Equals(aDate) && x.real.Equals(1));
foreach (var aName in names)
{
var namesRealTrades = dateRealTrades.Select(x => x)
.Where(x => x.name.Equals(aName));
// DO MY PROCESSING
}
}

I believe what you want can be achieved with two queries using group by. One to create a lookup by the date and the other to give you the name-date grouped items.
var data = modelData.Where(x => x.real.Equals(1))
.GroupBy(x => new { x.DT.Date, x.name });
var byDate = modelData.Where(x => x.real.Equals(1))
.ToLookup(x => x.DT.Date);
foreach(var item in data)
{
var aDate = item.Key.Date;
var aName = item.Key.name;
var namesRealTrades = item.ToList();
var dateRealTrades = byDate[aDate].ToList();
// DO MY PROCESSING
}
The first query will give you items grouped by the name and date to iterate over and the second will give you a lookup to get all the items associated with a given date. The second uses a lookup so that the list is iterated once and gives you fast access to the resulting list of items.
This should greatly reduce the number of times you iterate over modelData from what you currently have.

You could rewrite your for loop like this:
foreach (var namesRealTrades in names.Select(aName => dateRealTrades.Where(x => x.name.Equals(aName))))
{
//DO STUFF
}
Depending on your data this could reduce the number of queries you have to make

Did you try to compile your query as suggested on MSDN WebSite?
When you have an application that executes structurally similar
queries many times, you can often increase performance by compiling
the query one time and executing it several times with different
parameters. For example, an application might have to retrieve all the
customers who are in a particular city, where the city is specified at
runtime by the user in a form. LINQ to SQL supports the use of
compiled queries for this purpose.
https://msdn.microsoft.com/en-us/library/bb399335(v=vs.110).aspx

A couple of things:
use .ToList() to calculate a sequence once, so you can keep it for later.
use .GroupBy() to avoid re-searching modelData for things you have already found.
// Collections of models having the same Date or Name.
var dates = modelData.GroupBy(x => x.DT.Date);
var names = modelData.GroupBy(x => x.Name);
foreach (var modelsWithDate in dates)
{
var aDate = modelsWithDate.Key;
var dateRealTrades = modelsWithDate.Where(x => x.real == 1).ToList();
foreach (var modelsWithName in names)
{
var aName = modelsWithName.Key;
var namesRealTrades = modelsWithName.ToList();
// DO MY PROCESSING
}
}

There are two ways the code is ineffective.
names has deffered evaluation. Every time You iterate over it, it has to go though the whole data to find all the distinct names again. You should save the result.
You find distinct values from collection and then You go through collection again for every distinct value and look fot its occurences. You should use grouping.
the rewritten code can look like this
var dates = modelData.GroupBy(x => x.DT.Date);
var names = modelData.Select(x => x.name).Distinct().ToArray();
foreach (var date in dates)
{
var dateRealTrades = date.Where(x => x.real.Equals(1)).ToArray();
var namesRealTradesLookup = dateRealTrades.ToLookup(x => x.name);
foreach (var aName in names)
{
var namesRealTrades = namesRealTradesLookup[aName];
// DO MY PROCESSING
// var aDate = date.Key;
}
}
In case You are not interestested in date/name combination with no real trade, it can be done in much more straightforward way
var realModelData = modelData.Where(x => x.real.Equals(1));
foreach (var dateRealTrades in realModelData.ToLookup(x => x.DT.Date))
{
foreach (var namesRealTrades in dateRealTrades.ToLookup(x => x.name))
{
// DO MY PROCESSING
//var aDate = dateRealTrades.Key;
//var aName = namesRealTrades.Key;
//foreach(var trade in namesRealTrades) { ...
//foreach(var trade in dateRealTrades) { ...
}
}

Removing duplicates from a sorted list c#

I have a list of details about a large number of files. This list contains the file ID, last modified date and the file path. The problem is there are duplicates of the files which are older versions and sometimes have different file paths. I want to only store the newest version of a file regardless of file path. So I created a loop that iterates through the ordered list, checks to see if the ID is unique and if it is, it gets stored in a new unique list.
var ordered = list.OrderBy(x => x.ID).ThenByDescending(x => x.LastModifiedDate);
List<Item> unique = new List<Item>();
string curAssetId = null;
foreach (Item result in ordered)
{
if (!result.ID.Equals(curAssetId))
{
unique.Add(result);
curAssetId = result.ID;
}
}
However this is still allowing duplicates into the DB and I can't figure out why this code isn't working as expected. By duplicates I mean, the files have the same ID but different file paths, which like I said before shouldn't be an issue. I just want the latest version regardless of pathway. Can anyone else see what the issue is? Thanks
var ordered = listOfItems.OrderBy(x => x.AssetID).ThenByDescending(x => x.LastModifiedDate);
List<Item> uniqueItems = new List<Item>();
foreach (Item result in ordered)
{
if (!uniqueItems.Any(x => x.AssetID.Equals(result.AssetID)))
{
uniqueItems.Add(result);
}
}
this is what I have now and it is still allowing duplicates

This is because , you are not searching entire list to check whether the id is unique or not
List<Item> unique = new List<Item>();
string curAssetId = null; // here is the problem
foreach (Item result in ordered)
{
if (!result.ID.Equals(curAssetId)) // here you only compare the last value.
{
unique.Add(result);
curAssetId = result.ID; // You are only assign the current ID value and
}
}
to solve this , change the following
if (!result.ID.Equals(curAssetId)) // here you only compare the last value.
{
unique.Add(result);
curAssetId = result.ID; // You are only assign the current ID value and
}
to
if (!unique.Any(x=>x.ID.Equals(result.ID)))
{
unique.Add(result);
}

I don't know if this code is just simplified, but have you considered grouping on ID, sorting on LastModifiedDate, then just taking the first from each group?
Something like:
var unique = list.GroupBy(i => i.ID).Select(x => x.OrderByDescending(y => y.LastModifiedDate).First());

var ordered = list.OrderBy(x => x.ID).ThenByDescending(x => x.LastModifiedDate).Distinct() ??

For this purpose you have to create your own EquityComparer and after that you could use linq's Distinct method. Enumerable.Distinct at msdn
Also I think you could stay with your current code but you have to modify it in such a way (as a sample):
var ordered = list.OrderByDescending(x => x.LastModifiedDate);
var unique = new List<Item>();
foreach (Item result in ordered)
{
if (unique.Any(x => x.ID == result.ID))
continue;
unique.Add(result);
}

List<Item> p = new List<Item>();
var x = p.Select(c => new Item
{
AssetID = c.AssetID,
LastModifiedDate = c.LastModifiedDate.Date
}).OrderBy(y => y.id).ThenByDescending(c => c.LastModifiedDate).Distinct();

Assigning the result of group by to items in a list with linq

I have an
IEnumerable<typeA> result;
from this result I need to get sum group by some id.
So I have the query
var groupeddata = from data in result
group data by data.Title
into grouped
select new { intid= grouped.Key,
expsum= grouped.Sum(x=>x.expnum)};
now this expsum I need to assign to the items of result where typeA.id is same as intid. Now how to do this assignment?

The simplest approach would probably be to use a dictionary:
var sumDictionary = query.ToDictionary(pair => pair.intid, pair => pair.expsum);
foreach (var item in result)
{
// We don't know which property you actually want to assign to
item.Sum = sumDictionary[item.id];
}