We Keep Coding

Determine duplicates based off minimum N characters from smaller comparing string

I have two lists, both containing models that share a common field, ID(String value). I am comparing the ID's for duplication.
I currently have a LINQ statement in place to determine the duplicated ID values, which stores them into a list of strings:
List<string> duplicateRecords = testData.TestRecords.GroupBy(aa => aa.ID).Where(x => x.Count() > 1).Select(y => y.Key).ToList();
And a second LINQ statement that maps a List of respected models based off the duplicated ID LINQ result:
List<Model> modelRecords = testData.Models.Where(x => duplicateRecords.Any(y => x.ID == y)).ToList();
These two LINQ statements do exactly what I expected them to do which is great. But now there is a recent request to determine duplicate ID's based off of their minimum N characters during a comparison. This minimum N comparison must happen for the last N characters in a string.
EX)
ID1: 123 == ID2: 123
ID1: 0123 == ID2: 123
ID1: 123 == ID2: 0123
ID1: 1230 != ID2: 123
ID1: 123 != ID2: 1230
ID1: 122110123 == ID2: 123
Hopefully those examples give some insight into the problem I am trying to solve. This could be done using foreach loops but I have come to experience the code becomes very messy and unmanageable on complex list query's.
So my question is this: How can I use the last N characters of the smaller of the two comparing strings to determine duplicates using LINQ?
Note: I am also very open to more elegant ways of solving this problem, would really appreciate excluding any for or foreach solutions.

I assume that when the input contains 123 and 0123 you want the result to have both of them
var input = new List<Model>()
{
new Model {ID = "123"},
new Model {ID = "0123"},
new Model {ID = "1230"},
new Model {ID = "12"},
new Model {ID = "122110123"}
};
var result = input.Where(x => input.Any(y => y != x && (y.ID.EndsWith(x.ID) || x.ID.EndsWith(y.ID)))).ToList();
\\this will return 123, 0123 and 122110123
If you want to check agains existing duplicateRecords list then this should work:
List<Model> modelRecords = testData.Models.Where(x => duplicateRecords.Any(y => x.ID.EndsWith(y) || y.EndsWith(x.ID))).ToList();

In order to efficiently find the duplicates you need to sort the IDs by length so you can minimize the comparisons necessary. (The sort adds some overhead, but greatly decreases the comparisons that must be done - in my test where 9 IDs have and 3 are duplicates of 8 values, it is 15 comparisons sorted versus 42 unsorted.) Once you have them sorted by length, just compare each one to the ones that are equal to or longer (in case of complete duplicates) to find which short IDs need to be kept, marking any matches so you can skip them and then find all the Models that end with the found matches.
Create the List of IDs ordered by their length:
var orderedIDs = testData.TestRecords.Select(tr => tr.ID).OrderBy(id => id.Length).ToList();
I don't think there is any way to do this efficiently with LINQ, but a nested for loop skipping previous matches optimizes the search for duplicates.
First, variables to keep track of IDsand whichID`s have already matched:
var dupRecordSubIDs = new List<string>();
var alreadyMatched = new bool[testData.TestRecords.Count];
Now loop through the IDs and save the shorter matching IDs:
// foreach ID in length order
for (int n1 = 0; n1 < testData.TestRecords.Count-1; ++n1) {
// skip the ones that already matched a shorter ID
if (!alreadyMatched[n1]) {
// remember if the shorter ID was alrady added
var added_n1 = false;
// compare the ID to all greater than or equal length IDs
for (int n2 = n1 + 1; n2 < testData.TestRecords.Count; ++n2) {
// if not previously matched, see if we have a new match
if (!alreadyMatched[n2] && orderedIDs[n2].EndsWith(orderedIDs[n1])) {
// only add the shorter ID once for new matches
if (!added_n1) {
dupRecordSubIDs.Add(orderedIDs[n1]);
added_n1 = true;
}
// remember which longer IDs are already matched
alreadyMatched[n2] = true;
}
}
}
}
Now find all the Models that match one of the IDs with a duplicate:
var modelRecords = testData.Models.Where(m => dupRecordSubIDs.Any(d => m.ID.EndsWith(d))).ToList();

I assume ID is string. If so, you can do this :
string match = "123";
var duplicate = list.Where(x=> x.Substring(x.Length - match.Length) == match).ToList();

If I understand your question correctly, it looks to be just a matter of chopping off the last N characters in each ID property while grouping.
Something like this:
using System;
using System.Linq;
public class TestRecord
{
public string ID { get; set; }
}
public class TestModel
{
public string ID { get; set; }
}
public class Program
{
public static void Main()
{
var N = 3; // This is where you define the desired N length
var rand = new Random();
var testRecords = new TestRecord[]
{
new TestRecord {ID = "123"},
new TestRecord {ID = "0123"},
new TestRecord {ID = "1230"},
new TestRecord {ID = "122110123"},
};
var testModels = new TestModel[]
{
new TestModel {ID = "123"},
new TestModel {ID = "0123"},
new TestModel {ID = "1230"},
new TestModel {ID = "122110123"},
};
bool SortEm(string a, string b) => a.Length < b.Length ? b.EndsWith(a) : a.EndsWith(b);
var models = testRecords
.Where(record => testRecords.Any(target => record.ID != target.ID && SortEm(target.ID, record.ID)))
.ToDictionary(
key => key,
key => testModels.Where(testModel => SortEm(key.ID, testModel.ID)).ToArray());
foreach (var kvp in models)
{
System.Console.WriteLine($"For duplicate key ({kvp.Key.ID}) found models: \r\n\t{string.Join("\r\n\t", kvp.Value.Select(x => x.ID))}");
}
}
}

Delete records with multiple ids based on condition

Below is my class :
public partial class Ads
{
public int Id { get; set; }
public int RegionId { get; set; }
public string Name { get; set; }
public int Group { get; set; }
}
Records :
Id Name Group
1 abc 1
2 xyz 1
3 lmn 1
4 xxx 2
5 ppp 2
6 ttt 3
7 ggg 3
Now I want to remove all records/only that record with particular id of same group for some ids.
Code :
public void Delete(int[] ids,bool flag = false)
{
using (var context = new MyEntities())
{
context.Ads.RemoveRange(
context.Ads.Where(t => (flag ?
(context.Ads.Any(x => ids.Contains(x.Id) && x.Group == t.Group)) : false)));
context.SaveChanges();
}
}
What I am trying to do is something like below :
If flag is false with ids=3,5 then
I want to delete only records with Id=3,5
Else if flag is true with ids=3,5 then
I want to delete records with Id=3,5 but all other records too of the group to which ids=3,5 belong to.
Here id=3 belongs to group 1 so I want to delete all records of group1 i.e id=1,2 like wise ids=5 belongs to
group 2 so I want to delete all records of group 2 i.e id=4.
Expected output for this last case(flag=true) :
Id Name Group
6 ttt 3
7 ggg 3
But I think that I haven't done this is a proper way, and there is some source of improvement in the query.
Note : ids[] will always contains ids from different group and that too highest ids from different group.
How can I to improve my query for both the cases(flag=true and false)?

What about
var removeRecs=context.Ads.where(t => ids.contains(t.id))
if(flag)
removeRecs.AddRange(context.Ads.where(t=> removeRecs.Any(r =>t.groupId==r.Id)))
Ads.RemoveRange(removeRecs);

Do not make it too hard for your self, not everything must/can be done in the where statement of a query. Also a general rule of thumb in a loop try to factor out all the constant values and checks. So try this:
public static void Delete(int[] ids, bool flag = false)
{
using (var context = new MyEntities())
{
var query = context.Ads.AsQueryable();
query = flag
? query.Where(x => context.Ads
.Where(i => ids.Contains(i.Id))
.Select(i => i.Group)
.Contains(x.Group))
: query.Where(x => ids.Contains(x.Id));
context.Ads.RemoveRange(query);
context.SaveChanges();
}
}

public void Delete(int[] ids, bool flag = false)
{
using (var context = new MyEntities())
{
var items = context.Ads.Where(x => ids.Any(a => x.Id == a));
if (!flag)
{
//flag=false --> delete items with Id in ids[]
context.Ads.RemoveRange(items);
}
else
{
var groups = items.GroupBy(a => a.Group).Select(a => a.Key);
//flag=true --> delete all items in selected groups
context.Ads.RemoveRange(context.Ads.Where(x => groups.Any(a => x.Group == a)));
}
context.SaveChanges();
}

You should separate your tasks...
if (flag)
{
groupIds = db.Ads.Where(x => ids.Contains(x.Id)).Select(x => x.Group).ToList();
db.Ads.RemoveRange(db.Ads.Where(x => groupIds.Contains(x.Group)).ToList());
}
else
{
db.Ads.RemoveRange(db.Ads.Where(x => ids.Contains(x.Id)).ToList());
}

To me it looks like you have two different deletes here.
In first case you are only deleting the ads with given ID and this is pretty straight forward.
In second case you are deleting the ads with given ID and all other ads that contain the group of the recently deleted Ads. So in this case instead of deleting the ads with given Id first why not actualy get distinct groups for these ID-s and than just delete the groups.
EDIT
You can do it like this.
using (var context = new TestEntities())
{
if (!flag)
context.Ads.RemoveRange(context.Ads.Where(a => ids.Contains(a.Id)));
else
context.Ads.RemoveRange(context.Ads.Where(a => context.Ads.Where(g => ids.Contains(g.Id)).Select(x => x.Group).Distinct().Contains(a.Group)));
context.SaveChanges();
}
For the more complicated case I am trying to get distinct groups for given id-s. So for ID-s 3 and 5 I am selecting the groups and than I am doing distinct on the groups since it might happen that the id-s have the same group. Than I am fetching all the ads that have these groups. So for passed values of 3 and 5 I would get groups 1 and 2 which I would than use to get all the ads that have that group. That in turn would yield id-s 1, 2, 3, 4, and 5 which I would than delete.
EDIT 2
If the complexity of second Linq query bothers you than write a SQL query.
context.Database.ExecuteSqlCommand(
"DELETE Ads WHERE Group IN (SELECT Group FROM Ads WHERE Id IN(#p1, #p2))", new SqlParameter("#p1", ids[0]), new SqlParameter("#p2", ids[1]));
This should be extra performant rather than rely on EF which will delete it one by one.

Paging List of Objects using LINQ

I have a list of objects List<Parent> Parents.
The Parent class has a List<Child> Children.
Until now, I have applied paging to Parents using LINQ:
List<Parent> PageX = Parents.Skip(PageIndex * PageSize).Take(PageSize);
For instance, if PageSize=2, I have the following result:
--------------- Page 1 ----------------------
Parent 1
Child 1
Child 2
Child 3
Parent 2
Child 1
Child 2
--------------- Page 2 ----------------------
Parent 3
Child 1
Parent 4
Child 1
Child 2
What I want to achieve is the following:
--------------- Page 1 ----------------------
Parent 1
Child 1
Child 2
--------------- Page 2 ----------------------
Child 3
Parent 2
Child 1
--------------- Page 3 ----------------------
Child 2
Parent 3
Child 1
How can I achieve this?

You could use SelectMany:
var page = parents.SelectMany(p => p.Children)
.Skip(PageIndex * PageSize).Take(PageSize);
See a working fiddle here.
Update:
I did some research as this did interest me also. Using the following assumptions:
you are using EF
you have only one level of this parent->children relationship
your entities have all a Position property
you should be able to execute the following against the DB to get the items for a page with the correct order "in one query":
var pageItems = db.Parents.SelectMany(p => p.Children).Include(c => c.Parent)
.OrderBy(c => c.Parent.Position).ThenBy(c => c.Position)
.Skip(PageIndex * PageSize).Take(PageSize);
I did not test this code, as I have no DB here at the moment to test it, so please report back, if you can test it, if the assumtions are correct for your case.

I am presuming that both Parent and Child have a common property (say, Name) which is used to identify them?
In that case, you would have to flatten the hierarchy to include both parents and children info at the same list level.
So, presuming something like:
class Parent
{
public string Name { get; set; }
private readonly List<Child> _children = new List<Child>();
public List<Child> Children
{
get { return _children; }
}
}
class Child
{
public string Name { get; set; }
}
You would flatten it to a single IEnumerable<string> using:
var flattened = Parents
.Select(p => new [] { p.Name }.Concat(p.Children.Select(c => c.Name)))
.SelectMany(x => x);
And then you page it the way you did so far:
var results = flattened.Skip(PageIndex * PageSize).Take(PageSize);
foreach (var x in results)
Console.WriteLine(x);
It would probably be cooler if your Parent and Child both inherited from the same class or interface.

LINQ and how to return a list of a specific type

I have 2 tables (Document and DocumentClass) that have the following columns:
DocumentClass: DocClassID, Name, ParentID
Document: DocID, Name, DocClassID
The DocumentClass table contains parent and child records and the relationship between a parent and a child is the ParentID column. A Document record is linked with a child record in the DocumentClass by the DocClassID foreign key.
Parent records in DocumentClass have ParentID = 0 and Child records in the DocumentClass have ParentID != 0
I want to retrieve a childs Name and that child's Parents Name from the DocumentClass table.
I Have managed to create a function that does that for me. I send in a list of Document ids, find those DocumentClass records (the child records) that the document is linked to and then find the parent to those child records. Then I put that information I want into a Child class.
public List<Child> GetDocClassInfo(List<int> docIds)
{
var result = from dc in _context.DocClasses
from d in _context.Documents
where dc.DocClassID == d.DocClassID
where dc.DocClassID != 0
where docIds.Contains(d.DocID)
select new
{
children = from p in _context.DocClasses
where dc.ParentID == p.DocClassID
select new Child
{
ChildId = dc.DocClassID,
ChildDocClassName = dc.DocClassName,
ParentId = p.DocClassID,
ParentDocClassName = p.DocClassName
}
};
return result.ToList();
}
My problem is that I want to have a List to return from the function, but the compiler doesn't like this at all. I get an error saying that
Cannot implicitly convert type System.Collections.Generic.List``<AnonymousType#1> to System.Collection.Generic.List<Child>.
How can I write that LINQ query to return a List?
Best regards,
OKB

result is a list of anonymous types, each with a member (children) that is an enumerable set of Child records. You should be able to use SelectMany here:
var list = (from item in result
from child in item.children
select child).ToList();
or (identical):
var list = result.SelectMany(item => item.children).ToList();

You are returning an anonymous type when you say select new { children ...

var result = (from dc in _context.DocClasses
join d in _context.Documents
on dc.DocClassID equals d.DocClassID
where dc.DocClassID != 0 && docIds.Contains(d.DocID)
let children = from p in _context.DocClasses
where dc.ParentID == p.DocClassID
select new Child {
ChildId = dc.DocClassID,
ChildDocClassName = dc.DocClassName,
ParentId = p.DocClassID,
ParentDocClassName = p.DocClassName
}
select children).SelectMany(c=>c).ToList();

C#: Creating objects that does not exist in a different list

I have two lists of two different kinds of objects representing data rows from two sql queries. The first list contains data, and the second contains more detailed data. So as an example:
List1: List2:
1 Alice 1 15
2 Bob 1 19
3 Carol 2 5
4 Dave 2 7
2 20
4 16
I want to insert rows into List2 so that everyone in List1 has at least one row in List2. So when no rows exist in List2 for a certain person, I want to insert a single one with a default value. In the example case I would have to insert one row for Carol, so I would end up with:
List1: List2:
1 Alice 1 15
2 Bob 1 19
3 Carol 2 5
4 Dave 2 7
2 20
3 0
4 16
Does anyone have a clever, clean and efficient way of doing this?
I know that to join these tables together into one I would have to use an Outer Join, for example like in this Outer Join Sample. But I don't want a new result set. I just want those missing rows to be inserted into List2.
Note: Yes, I know the question\title is kind of... blah... but I don't know how to formulate it better. Someone please fix it if you can.
Note 2: I can not use SQL. I can not insert those rows in the original table. I am reporting on data, which means I do not touch any of the data. I just read it out. The data is to be used in a master-detail report, and my issue is that when no details exist for a certain master row, then you end up with just an empty space. Which is not good. So I want to insert rows with sensible info so that the user can see that there was nothing to show here.

Assuming your lists are sorted by the Key value like in your example (in this case an integer), something like this should work:
int i = 0;
foreach (var item in List1)
{
// Skip any items in List2 that don't exist in List1
// (not sure this is needed in your case), or that we've
// already passed in List1
while (List2[i].Key < item.Key)
i++;
if (List2[i].Key > item.Key)
{
// Create new item in List2
List2.Add(new List2Item(item.Key, 0));
}
}
// TODO: resort List2
Depending on how many items you expect to be missing, you might want to Insert into List2 instead, eliminating the need for the resorting. If you expect a lot of items to be missing however, this method will be faster. Alternatively, you could use a linked list for List2.
Note that this will fail if there are duplicate Key entries in List1. You'd need to check for that seperately to prevent multiple new items from being created in List2.

var lst1 = new List<int>() { 1, 2, 3, 4 };
var lst2 = new List<int>() { 1, 1, 2, 2, 2, 4 };
lst2.AddRange(lst1.Except(lst2));

LINQ: From the example you gave in the link, just change the code from:
foreach (var i in q) {
Console.WriteLine("Customer: {0} Order Number: {1}",
i.Name.PadRight(11, ' '), i.OrderNumber);
}
to
foreach (var i in q) {
if (i.OrderNumber == "(no orders)")
order.Add(new Order {Key = i.ID /* add your default values here*/});
}
Of course you can save some lines here as well in the code before.

OK, here is goes:
1. Create a type to represent an item from your lists:
struct ListType
{
public object Id;
public object Name;
}
or, of course you can build in another way that suits you better.
Create your List2 as an IEnumerable< ListType > from your LINQ query
I assume List1 has the same structure as List2 with an Id and Name field (you could use the same ListType type for the list items)
With the assumptions above, here is the code to solve the initial problem :
List newLst2 = list2.ToList();
Array.ForEach(list1.ToArray(), list1It =>
{
var isInList2 = from list2it in newLst2.ToArray()
where (string)list2it.Id == list1It.Id
select list2it;
if (isInList2.Count() == 0)
newLst2.Add(new ListType { Id = list1It.Id, Name = list1It.Name });
});
Comments: for each element in List1 make a query in List2 and check if the Id exists. If it does not exist, add a new item.
There probably are more efficient ways of doing this but this should be able to get you started.

Here is a solution using LINQ.
public class List1
{
public int ID { get; set; }
public string Person { get; set; }
}
public class List2
{
public int ID { get; set; }
public int Value { get; set; }
}
var lList1 = new List<List1>
{
new List1 {ID = 1, Person = "Alice"},
new List1 {ID = 2, Person = "Bob"},
new List1 {ID = 3, Person = "Carol"},
new List1 {ID = 4, Person = "Dave"}
};
var lList2 = new List<List2>
{
new List2 {ID = 1, Value = 15},
new List2 {ID = 1, Value = 19},
new List2 {ID = 2, Value = 5},
new List2 {ID = 2, Value = 7},
new List2 {ID = 2, Value = 20},
new List2 {ID = 4, Value = 16}
};
var lOutput = lList1.SelectMany(pArg =>
lList2.Where(pArg1 => pArg1.ID == pArg.ID)
.DefaultIfEmpty(new List2 { ID = pArg.ID, Value = 0})
.Select(pArg1 => pArg1));

Uh... It seems like it would be straightforward to just use Contains, no?
foreach (Key key in List1.Keys)
{
if (!List2.Keys.Contains(key)) List2.Add(key, "0");
}
This would have no issues with duplicate keys in List1.

LINQ implementation
public class Master
{
public int ID;
}
public class Detail
{
public int ID;
public int Count;
}
public static void AddMissingDetails(IEnumerable<Master> masters, List<Detail> details)
{
AddMissingDetails(masters, details, x => new Detail
{
ID = x,
Count = 0
});
}
public static void AddMissingDetails(IEnumerable<Master> masters, List<Detail> details, Func<int, Detail> createDefault)
{
details.AddRange(
masters
.Select(x => x.ID)
.Except(details.Select(x => x.ID).Distinct())
.Select(createDefault));
}

YOu may not like my solution. But i would like to thank you for this post.
It gave me a chance to do some useful stuff using linq.
I am using Extension method and Linq to add the missing items in your target list (List2).
I am not sure if you are working on 3.0/3.5 framework, if you do, then this solution would work for you and it is also "a clever, clean and efficient way of doing this" :).
public static void MergeLists() {
var listOne=new List<List1> {
new List1 {ID=1, Person="Alice"},
new List1 {ID=2, Person="Bob"},
new List1 {ID=3, Person="Carol"},
new List1 {ID=4, Person="Dave"},
new List1 {ID=5, Person="Dave2"},
new List1 {ID=6, Person="Dave3"},
};
var listTwo=new List<List2> {
new List2 {ID=1, Value=15},
new List2 {ID=1, Value=19},
new List2 {ID=2, Value=5},
new List2 {ID=2, Value=7},
new List2 {ID=2, Value=20},
new List2 {ID=4, Value=16}
};
var listTwoWithAddedItems=listOne.AddMissingItems(listTwo, (item1, item2) => item1.ID==item2.ID,
item2 => new List2 { ID=item2.ID, Value=-1 }).ToList();//For this value, you can put whatever default value you want to set for the missing items.
Console.Read();
}
public static class AmbyExtends {
public static List<Target> AddMissingItems<Source, Target>(this IEnumerable<Source> source, List<Target> target, Func<Source, Target, bool> selector, Func<Source, Target> creator) {
foreach(var item in source) {
if(!target.Any(x=>selector(item,x))) {
target.Add(creator(item));
}
}
return target;
}
}

INSERT INTO List2(ID, Value)
SELECT ID, 0
FROM List1
WHERE NOT EXISTS (SELECT NULL FROM List2 WHERE List2.ID=List1.ID)

Using SQL on the database, the solution would be:
INSERT INTO List2 (ID)
SELECT l1.ID
FROM List1 l1
LEFT JOIN List2 l2
ON l1.ID = l2.ID
WHERE l2.ID IS NULL
Here the assumption is that the other columns in List2 table are either NOT NULL or have a DEFAULT value constraint.