We Keep Coding

How to find the placement of a List within another List?

I am working with two lists. The first contains a large sequence of strings. The second contains a smaller list of strings. I need to find where the second list exists in the first list.
I worked with enumeration, and due to the large size of the data, this is very slow, I was hoping for a faster way.
List<string> first = new List<string>() { "AAA","BBB","CCC","DDD","EEE","FFF" };
List<string> second = new List<string>() { "CCC","DDD","EEE" };
int x = SomeMagic(first,second);
And I would need x to = 2.

Ok, here is my variant with old-good-for-each-loop:
private int SomeMagic(IEnumerable<string> source, IEnumerable<string> target)
{
/* Some obvious checks for `source` and `target` lenght / nullity are ommited */
// searched pattern
var pattern = target.ToArray();
// candidates in form `candidate index` -> `checked length`
var candidates = new Dictionary<int, int>();
// iteration index
var index = 0;
// so, lets the magic begin
foreach (var value in source)
{
// check candidates
foreach (var candidate in candidates.Keys.ToArray()) // <- we are going to change this collection
{
var checkedLength = candidates[candidate];
if (value == pattern[checkedLength]) // <- here `checkedLength` is used in sense `nextPositionToCheck`
{
// candidate has match next value
checkedLength += 1;
// check if we are done here
if (checkedLength == pattern.Length) return candidate; // <- exit point
candidates[candidate] = checkedLength;
}
else
// candidate has failed
candidates.Remove(candidate);
}
// check for new candidate
if (value == pattern[0])
candidates.Add(index, 1);
index++;
}
// we did everything we could
return -1;
}
We use dictionary of candidates to handle situations like:
var first = new List<string> { "AAA","BBB","CCC","CCC","CCC","CCC","EEE","FFF" };
var second = new List<string> { "CCC","CCC","CCC","EEE" };

If you are willing to use MoreLinq then consider using Window:
var windows = first.Window(second.Count);
var result = windows
.Select((subset, index) => new { subset, index = (int?)index })
.Where(z => Enumerable.SequenceEqual(second, z.subset))
.Select(z => z.index)
.FirstOrDefault();
Console.WriteLine(result);
Console.ReadLine();
Window will allow you to look at 'slices' of the data in chunks (based on the length of your second list). Then SequenceEqual can be used to see if the slice is equal to second. If it is, the index can be returned. If it doesn't find a match, null will be returned.

Implemented SomeMagic method as below, this will return -1 if no match found, else it will return the index of start element in first list.
private int SomeMagic(List<string> first, List<string> second)
{
if (first.Count < second.Count)
{
return -1;
}
for (int i = 0; i <= first.Count - second.Count; i++)
{
List<string> partialFirst = first.GetRange(i, second.Count);
if (Enumerable.SequenceEqual(partialFirst, second))
return i;
}
return -1;
}

you can use intersect extension method using the namepace System.Linq
var CommonList = Listfirst.Intersect(Listsecond)

Inserting value in List of values of a Key in Dictionary

I have a rowsDictionary that its keys point to a list of EmployeeSummary classes.
In those EmployeeSummary classes we also have a string property of Delivery_System
I am looping through this in this way but now stuck in the part that I want to have a deliverySystemFinder dictioanry that its keys are combinedKey as below and the value for each key is a list of distinct delivery_system values
//rowsDictionary is a Dictionary<string, List<EmployeeSummary>>
Dictionary<string, List<string>> deliverySystemFinder = new Dictionary<string, List<string>>();
foreach (string key in rowsDictionary.Keys)
{
List<EmployeeSummary> empList = rowsDictionary[key];
foreach (EmployeeSummary emp in empList)
{
string combinedKey = emp.LastName.Trim().ToUpper() + emp.FirstName.Trim().ToUpper();
string delivery_system = emp.Delivery_System;
// so now I should go and
//A) does deliverySystemFinder have this combinedKey? if not add it.
//B) Does combinedKey in the list of its values already have the value for delivery_system? if it does not then add it
}
}

This would work, for start:
foreach (string key in rowsDictionary.Keys)
{
List<EmployeeSummary> empList = rowsDictionary[key];
foreach (EmployeeSummary emp in empList)
{
string combinedKey = emp.LastName.Trim().ToUpper() +
emp.FirstName.Trim().ToUpper();
string delivery_system = emp.Delivery_System;
List<string> systems = null;
// check if the dictionary contains the list
if (!deliverySystemFinder.TryGetValue(combinedKey, out systems))
{
// if not, create it and add it
systems = new List<string>();
deliverySystemFinder[combinedKey] = systems;
}
// check if the list contains the value and add it
if (!systems.Contains(delivery_system))
systems.Add(delivery_system);
}
}
Now, a couple of remarks:
It doesn't make sense to iterate through Keys, and then do a lookup in each iteration. You can directly iterate KeyValuePairs using a foreach loop.
Using concatenated strings as unique keys often fails. In this case, what happens if you have users { LastName="Some", FirstName="Body" } and { LastName="So", FirstName="Mebody" } in your list?
Checking if a List contains a value is a O(n) operation. You would greatly improve performance if you used a HashSet<string> instead.
Finally, the simplest way to achieve what you're trying to do is to ditch those loops and simply use:
// returns a Dictionary<EmployeeSummary, List<string>>
// which maps each distinct EmployeeSummary into a list of
// distinct delivery systems
var groupByEmployee = rowsDictionary
.SelectMany(kvp => kvp.Value)
.GroupBy(s => s, new EmployeeSummaryEqualityComparer())
.ToDictionary(
s => s.Key,
s => s.Select(x => x.Delivery_System).Distinct().ToList());
With EmployeeSummaryEqualityComparer defined something like:
class EmployeeSummaryEqualityComparer : IEqualityComparer<EmployeeSummary>
{
public bool Equals(EmployeeSummary x, EmployeeSummary y)
{
if (object.ReferenceEquals(x, null))
return object.ReferenceEquals(y, null);
return
x.FirstName == y.FirstName &&
x.LastName == y.LastName &&
... (depending on what constitutes 'equal' for you)
}
public int GetHashCode(EmployeeSummary x)
{
unchecked
{
var h = 31; // null checks might not be necessary?
h = h * 7 + (x.FirstName != null ? x.FirstName.GetHashCode() : 0);
h = h * 7 + (x.LastName != null ? x.LastName.GetHashCode() : 0);
... other properties similarly ...
return h;
}
}
}
If you really think that using the string key will work in all your cases, you can do it without the custom equality comparer:
// returns a Dictionary<string, List<string>>
var groupByEmployee = rowsDictionary
.SelectMany(kvp => kvp.Value)
.GroupBy(s => s.LastName.ToUpper() + s.FirstName.ToUpper())
.ToDictionary(
s => s.Key,
s => s.Select(x => x.Delivery_System).Distinct().ToList());

Grouping by an unknown initial prefix

Say I have the following array of strings as an input:
foo-139875913
foo-aeuefhaiu
foo-95hw9ghes
barbazabejgoiagjaegioea
barbaz8gs98ghsgh9es8h
9a8efa098fea0
barbaza98fyae9fghaefag
bazfa90eufa0e9u
bazgeajga8ugae89u
bazguea9guae
aifeaufhiuafhe
There are 3 different prefixes used here, "foo-", "barbaz" and "baz" - however these prefixes are not known ahead of time (they could be something completely different).
How could you establish what the different common prefixes are so that they could then be grouped by? This is made a bit tricky since in the data I've provided there's two that start with "bazg" and one that starts "bazf" where of course "baz" is the prefix.
What I've tried so far is sorting them into alphabetical order, and then looping through them in order and counting how many characters in a row are identical to the previous. If the number is different or when 0 characters are identical, it starts a new group. The problem with this is it falls over at the "bazg" and "bazf" problem I mentioned earlier and separates those into two different groups (one with just one element in it)
Edit: Alright, let's throw a few more rules in:
Longer potential groups should generally be preferred over shorter ones, unless there is a closely matching group of less than X characters difference in length. (So where X is 2, baz would be preferred over bazg)
A group must have at least Y elements in it or not be a group at all
It's okay to simply throw away elements that don't match any of the 'groups' to within the rules above.
To clarify the first rule in relation to the second, if X was 0 and Y was 2, then the two 'bazg' entries would be in a group, and the 'bazf' would be thrown away because its on its own.

Well, here's a quick hack, probably O(something_bad):
IEnumerable<Tuple<String, IEnumerable<string>>> GuessGroups(IEnumerable<string> source, int minNameLength=0, int minGroupSize=1)
{
// TODO: error checking
return InnerGuessGroups(new Stack<string>(source.OrderByDescending(x => x)), minNameLength, minGroupSize);
}
IEnumerable<Tuple<String, IEnumerable<string>>> InnerGuessGroups(Stack<string> source, int minNameLength, int minGroupSize)
{
if(source.Any())
{
var tuple = ExtractTuple(GetBestGroup(source, minNameLength), source);
if (tuple.Item2.Count() >= minGroupSize)
yield return tuple;
foreach (var element in GuessGroups(source, minNameLength, minGroupSize))
yield return element;
}
}
Tuple<String, IEnumerable<string>> ExtractTuple(string prefix, Stack<string> source)
{
return Tuple.Create(prefix, PopWithPrefix(prefix, source).ToList().AsEnumerable());
}
IEnumerable<string> PopWithPrefix(string prefix, Stack<string> source)
{
while (source.Any() && source.Peek().StartsWith(prefix))
yield return source.Pop();
}
string GetBestGroup(IEnumerable<string> source, int minNameLength)
{
var s = new Stack<string>(source);
var counter = new DictionaryWithDefault<string, int>(0);
while(s.Any())
{
var g = GetCommonPrefix(s);
if(!string.IsNullOrEmpty(g) && g.Length >= minNameLength)
counter[g]++;
s.Pop();
}
return counter.OrderBy(c => c.Value).Last().Key;
}
string GetCommonPrefix(IEnumerable<string> coll)
{
return (from len in Enumerable.Range(0, coll.Min(s => s.Length)).Reverse()
let possibleMatch = coll.First().Substring(0, len)
where coll.All(f => f.StartsWith(possibleMatch))
select possibleMatch).FirstOrDefault();
}
public class DictionaryWithDefault<TKey, TValue> : Dictionary<TKey, TValue>
{
TValue _default;
public TValue DefaultValue {
get { return _default; }
set { _default = value; }
}
public DictionaryWithDefault() : base() { }
public DictionaryWithDefault(TValue defaultValue) : base() {
_default = defaultValue;
}
public new TValue this[TKey key]
{
get { return base.ContainsKey(key) ? base[key] : _default; }
set { base[key] = value; }
}
}
Example usage:
string[] input = {
"foo-139875913",
"foo-aeuefhaiu",
"foo-95hw9ghes",
"barbazabejgoiagjaegioea",
"barbaz8gs98ghsgh9es8h",
"barbaza98fyae9fghaefag",
"bazfa90eufa0e9u",
"bazgeajga8ugae89u",
"bazguea9guae",
"9a8efa098fea0",
"aifeaufhiuafhe"
};
GuessGroups(input, 3, 2).Dump();

Ok, well as discussed, the problem wasn't initially well defined, but here is how I'd go about it.
Create a tree T
Parse the list, for each element:
for each letter in that element
if a branch labeled with that letter exists then
Increment the counter on that branch
Descend that branch
else
Create a branch labelled with that letter
Set its counter to 1
Descend that branch
This gives you a tree where each of the leaves represents a word in your input. Each of the non-leaf nodes has a counter representing how many leaves are (eventually) attached to that node. Now you need a formula to weight the length of the prefix (the depth of the node) against the size of the prefix group. For now:
S = (a * d) + (b * q) // d = depth, q = quantity, a, b coefficients you'll tweak to get desired behaviour
So now you can iterate over each of the non-leaf node and assign them a score S. Then, to work out your groups you would
For each non-leaf node
Assign score S
Insertion sort the node in to a list, so the head is the highest scoring node
Starting at the root of the tree, traverse the nodes
If the node is the highest scoring node in the list
Mark it as a prefix
Remove all nodes from the list that are a descendant of it
Pop itself off the front of the list
Return up the tree
This should give you a list of prefixes. The last part feels like some clever data structures or algorithms could speed it up (the last part of removing all the children feels particularly weak, but if you input size is small, I guess speed isn't too important).

I'm wondering if your requirements aren't off. It seems as if you are looking for a specific grouping size as opposed to specific key size requirements. I have below a program that will, based on a specified group size, break up the strings into the largest possible groups up too, and including the group size specified. So if you specify a group size of 5, then it will group items on the smallest key possible to make a group of size 5. In your example it would group foo- as f since there is no need to make a more complex key as an identifier.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ConsoleApplication2
{
class Program
{
/// <remarks><c>true</c> in returned dictionary key are groups over <paramref name="maxGroupSize"/></remarks>
public static Dictionary<bool,Dictionary<string, List<string>>> Split(int maxGroupSize, int keySize, IEnumerable<string> items)
{
var smallItems = from item in items
where item.Length < keySize
select item;
var largeItems = from item in items
where keySize < item.Length
select item;
var largeItemsq = (from item in largeItems
let key = item.Substring(0, keySize)
group item by key into x
select new { Key = x.Key, Items = x.ToList() } into aGrouping
group aGrouping by aGrouping.Items.Count() > maxGroupSize into x2
select x2).ToDictionary(a => a.Key, a => a.ToDictionary(a_ => a_.Key, a_ => a_.Items));
if (smallItems.Any())
{
var smallestLength = items.Aggregate(int.MaxValue, (acc, item) => Math.Min(acc, item.Length));
var smallItemsq = (from item in smallItems
let key = item.Substring(0, smallestLength)
group item by key into x
select new { Key = x.Key, Items = x.ToList() } into aGrouping
group aGrouping by aGrouping.Items.Count() > maxGroupSize into x2
select x2).ToDictionary(a => a.Key, a => a.ToDictionary(a_ => a_.Key, a_ => a_.Items));
return Combine(smallItemsq, largeItemsq);
}
return largeItemsq;
}
static Dictionary<bool, Dictionary<string,List<string>>> Combine(Dictionary<bool, Dictionary<string,List<string>>> a, Dictionary<bool, Dictionary<string,List<string>>> b) {
var x = new Dictionary<bool,Dictionary<string,List<string>>> {
{ true, null },
{ false, null }
};
foreach(var condition in new bool[] { true, false }) {
var hasA = a.ContainsKey(condition);
var hasB = b.ContainsKey(condition);
x[condition] = hasA && hasB ? a[condition].Concat(b[condition]).ToDictionary(c => c.Key, c => c.Value)
: hasA ? a[condition]
: hasB ? b[condition]
: new Dictionary<string, List<string>>();
}
return x;
}
public static Dictionary<string, List<string>> Group(int maxGroupSize, IEnumerable<string> items, int keySize)
{
var toReturn = new Dictionary<string, List<string>>();
var both = Split(maxGroupSize, keySize, items);
if (both.ContainsKey(false))
foreach (var key in both[false].Keys)
toReturn.Add(key, both[false][key]);
if (both.ContainsKey(true))
{
var keySize_ = keySize + 1;
var xs = from needsFix in both[true]
select needsFix;
foreach (var x in xs)
{
var fixedGroup = Group(maxGroupSize, x.Value, keySize_);
toReturn = toReturn.Concat(fixedGroup).ToDictionary(a => a.Key, a => a.Value);
}
}
return toReturn;
}
static Random rand = new Random(unchecked((int)DateTime.Now.Ticks));
const string allowedChars = "aaabbbbccccc"; // "aAbBcCdDeEfFgGhHiIjJkKlLmMnNoOpPqQrRsStTuUvVwWxXyYzZ";
static readonly int maxAllowed = allowedChars.Length - 1;
static IEnumerable<string> GenerateText()
{
var list = new List<string>();
for (int i = 0; i < 100; i++)
{
var stringLength = rand.Next(3,25);
var chars = new List<char>(stringLength);
for (int j = stringLength; j > 0; j--)
chars.Add(allowedChars[rand.Next(0, maxAllowed)]);
var newString = chars.Aggregate(new StringBuilder(), (acc, item) => acc.Append(item)).ToString();
list.Add(newString);
}
return list;
}
static void Main(string[] args)
{
// runs 1000 times over autogenerated groups of sample text.
for (int i = 0; i < 1000; i++)
{
var s = GenerateText();
Go(s);
}
Console.WriteLine();
Console.WriteLine("DONE");
Console.ReadLine();
}
static void Go(IEnumerable<string> items)
{
var dict = Group(3, items, 1);
foreach (var key in dict.Keys)
{
Console.WriteLine(key);
foreach (var item in dict[key])
Console.WriteLine("\t{0}", item);
}
}
}
}

Elegant way to check if a list contains an object where one property is the same, and replace only if the date of another property is later

I have a class as follows :
Object1{
int id;
DateTime time;
}
I have a list of Object1. I want to cycle through another list of Object1, search for an Object1 with the same ID and replace it in the first list if the time value is later than the time value in the list. If the item is not in the first list, then add it.
I'm sure there is an elegant way to do this, perhaps using linq? :
List<Object1> listOfNewestItems = new List<Object1>();
List<Object1> listToCycleThrough = MethodToReturnList();
foreach(Object1 object in listToCycleThrough){
if(listOfNewestItems.Contains(//object1 with same id as object))
{
//check date, replace if time property is > existing time property
} else {
listOfNewestItems.Add(object)
}
Obviously this is very messy (and that's without even doing the check of properties which is messier again...), is there a cleaner way to do this?

var finalList = list1.Concat(list2)
.GroupBy(x => x.id)
.Select(x => x.OrderByDescending(y=>y.time).First())
.ToList();
here is the full code to test
public class Object1
{
public int id;
public DateTime time;
}
List<Object1> list1 = new List<Object1>()
{
new Object1(){id=1,time=new DateTime(1991,1,1)},
new Object1(){id=2,time=new DateTime(1992,1,1)}
};
List<Object1> list2 = new List<Object1>()
{
new Object1(){id=1,time=new DateTime(2001,1,1)},
new Object1(){id=3,time=new DateTime(1993,1,1)}
};
and OUTPUT:
1 01.01.2001
2 01.01.1992
3 01.01.1993

This is how to check:
foreach(var object in listToCycleThrough)
{
var currentObject = listOfNewestItems
.SingleOrDefault(obj => obj.Id == object.Id);
if(currentObject != null)
{
if (currentObject.Time < object.Time)
currentObject.Time = object.Time
}
else
listOfNewestItems.Add(object)
}
But if you have large data, would be suggested to use Dictionary in newest list, time to look up will be O(1) instead of O(n)

You can use LINQ. Enumerable.Except to get the set difference(the newest), and join to find the newer objects.
var listOfNewestIDs = listOfNewestItems.Select(o => o.id);
var listToCycleIDs = listToCycleThrough.Select(o => o.id);
var newestIDs = listOfNewestIDs.Except(listToCycleIDs);
var newestObjects = from obj in listOfNewestItems
join objID in newestIDs on obj.id equals objID
select obj;
var updateObjects = from newObj in listOfNewestItems
join oldObj in listToCycleThrough on newObj.id equals oldObj.id
where newObj.time > oldObj.time
select new { oldObj, newObj };
foreach (var updObject in updateObjects)
updObject.oldObj.time = updObject.newObj.time;
listToCycleThrough.AddRange(newestObjects);
Note that you need to add using System.Linq;.
Here's a demo: http://ideone.com/2ASli

I'd create a Dictionary to lookup the index for an Id and use that
var newItems = new List<Object1> { ...
IList<Object1> itemsToUpdate = ...
var lookup = itemsToUpdate.
Select((i, o) => new { Key = o.id, Value = i }).
ToDictionary(i => i.Key, i => i.Value);
foreach (var newItem in newitems)
{
if (lookup.ContainsKey(newitem.ID))
{
var i = lookup[newItem.Id];
if (newItem.time > itemsToUpdate[i].time)
{
itemsToUpdate[i] = newItem;
}
}
else
{
itemsToUpdate.Add(newItem)
}
}
That way, you wouldn't need to reenumerate the list for each new item, you'd benefit for the hash lookup performance.
This should work however many times an Id is repeated in the list of new items.

Merge and Update Two Lists in C#

I have two List<T> objects:
For example:
List 1:
ID, Value where Id is populated and value is blank and it contains say IDs from 1 to 10.
1,""
2,""
...
10,""
List 2:
ID, Value and other attributes all filled with values but this list is a subset of List 1 in terms of IDs. (e.g only 3 items)
2,67
4,90
5,98
What I want is a merged list 1, but with updated values. Does anyone have any good extension method which will do this or any elegent code to perform this operation. The final list should be:
ID, Value
1,""
2,67 //value from list 2
3,""
4,90
5,98
6,""
...
10,""

use linq: list1=list2.Union(list1);

I would probably use a dictionary rather than a list:
// sample data
var original = new Dictionary<int, int?>();
for (int i = 1; i <= 10; i++)
{
original.Add(i, null);
}
var updated = new Dictionary<int, int>();
updated.Add(2, 67);
updated.Add(4, 90);
updated.Add(5, 98);
updated.Add(11, 20); // add
// merge
foreach (var pair in updated)
{
original[pair.Key] = pair.Value;
}
// show results
foreach (var pair in original.OrderBy(x => x.Key))
{
Console.WriteLine(pair.Key + ": " + pair.Value);
}
If you are talking about properties of an object, it will be trickier, but still doable.

This is O(m*n) but should do the job for arbitrary lists
foreach (var record in List1)
{
var other = List2.FirstOrDefault(x => x.Key == record.Key);
if(other != null) record.Value = other.Value;
}
If the lists are guaranteed ordered, then it could be brought down to O(n) at the cost of more code. The algortihm would be
Current items start as head of each list
While items remain in both lists
If the current item of list1 has lower key than list2 advance to next in list1
else if the current item of list2 has lower key than list1 advance to next in list2
else copy value from current list2 item into list1 item and advance both lists.

If you have both lists sorted by ID, you can use a variation of the classical merge algorithm:
int pos = 0;
foreach (var e in list2) {
pos = list1.FindIndex(pos, x => x.Id==e.Id);
list1[pos].Value = e.Value;
}
Note that this also requires list2 to be a strict subset of list1 in terms of ID (i.e. list1 really contains all ids of list2)
Of course you can also wrap this in an extension method
public static void UpdateWith<T>(this List<T> list1, List<T> list2)
where T:SomeIdValueSupertype {
int pos = 0;
foreach (var e in list2) {
pos = list1.FindIndex(pos, x => x.Id==e.Id);
list1[pos].Value = e.Value;
}
}

private void btnSearch_Click(object sender, EventArgs e)
{
String searchBy = cmbSearchBy.Text.ToString();
String searchFor = txtSearchFor.Text.Trim();
var List3 = (from row in JobTitleDB.jobList
where (row.JID.ToString()+row.JobTitleName.ToString().ToLower()).Contains(searchFor.ToLower())
select row).ToList();
if (searchBy == "All")
{
dgJobTitles.DataSource = null;
//dgJobTitles.DataSource = List1;
//dgJobTitles.DataSource = List2;
//dgJobTitles.DataSource = List1.Concat(List2);
//dgJobTitles.DataSource = List1.Union(List2);
dgJobTitles.DataSource = List3;
//dgJobTitles.DataSource=List1.AddRange(List2);
}
}

Dictionary<int, string> List1 = new Dictionary<int, string>();
List1.Add(1,"");
List1.Add(2,"");
List1.Add(3,"");
List1.Add(4,"");
List1.Add(5,"");
List1.Add(6,"");
Dictionary<int, string> List2 = new Dictionary<int, string>();
List2.Add(2, "two");
List2.Add(4, "four");
List2.Add(6, "six");
var Result = List1.Select(x => new KeyValuePair<int, string>(x.Key, List2.ContainsKey(x.Key) ? List2[x.Key] : x.Value)).ToList();