We Keep Coding

fastest starts with search algorithm

I need to implement a search algorithm which only searches from the start of the string rather than anywhere within the string.
I am new to algorithms but from what I can see it seems as though they go through the string and find any occurrence.
I have a collection of strings (over 1 million) which need to be searched everytime the user types a keystroke.
EDIT:
This will be an incremental search. I currently have it implemented with the following code and my searches are coming back ranging between 300-700ms from over 1 million possible strings. The collection isnt ordered but there is no reason it couldnt be.
private ICollection<string> SearchCities(string searchString) {
return _cityDataSource.AsParallel().Where(x => x.ToLower().StartsWith(searchString)).ToArray();
}

I've adapted the code from this article from Visual Studio Magazine that implements a Trie.
The following program demonstrates how to use a Trie to do fast prefix searching.
In order to run this program, you will need a text file called "words.txt" with a large list of words. You can download one from Github here.
After you compile the program, copy the "words.txt" file into the same folder as the executable.
When you run the program, type a prefix (such as prefix ;)) and press return, and it will list all the words beginning with that prefix.
This should be a very fast lookup - see the Visual Studio Magazine article for more details!
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
namespace ConsoleApp1
{
class Program
{
static void Main()
{
var trie = new Trie();
trie.InsertRange(File.ReadLines("words.txt"));
Console.WriteLine("Type a prefix and press return.");
while (true)
{
string prefix = Console.ReadLine();
if (string.IsNullOrEmpty(prefix))
continue;
var node = trie.Prefix(prefix);
if (node.Depth == prefix.Length)
{
foreach (var suffix in suffixes(node))
Console.WriteLine(prefix + suffix);
}
else
{
Console.WriteLine("Prefix not found.");
}
Console.WriteLine();
}
}
static IEnumerable<string> suffixes(Node parent)
{
var sb = new StringBuilder();
return suffixes(parent, sb).Select(suffix => suffix.TrimEnd('$'));
}
static IEnumerable<string> suffixes(Node parent, StringBuilder current)
{
if (parent.IsLeaf())
{
yield return current.ToString();
}
else
{
foreach (var child in parent.Children)
{
current.Append(child.Value);
foreach (var value in suffixes(child, current))
yield return value;
--current.Length;
}
}
}
}
public class Node
{
public char Value { get; set; }
public List<Node> Children { get; set; }
public Node Parent { get; set; }
public int Depth { get; set; }
public Node(char value, int depth, Node parent)
{
Value = value;
Children = new List<Node>();
Depth = depth;
Parent = parent;
}
public bool IsLeaf()
{
return Children.Count == 0;
}
public Node FindChildNode(char c)
{
return Children.FirstOrDefault(child => child.Value == c);
}
public void DeleteChildNode(char c)
{
for (var i = 0; i < Children.Count; i++)
if (Children[i].Value == c)
Children.RemoveAt(i);
}
}
public class Trie
{
readonly Node _root;
public Trie()
{
_root = new Node('^', 0, null);
}
public Node Prefix(string s)
{
var currentNode = _root;
var result = currentNode;
foreach (var c in s)
{
currentNode = currentNode.FindChildNode(c);
if (currentNode == null)
break;
result = currentNode;
}
return result;
}
public bool Search(string s)
{
var prefix = Prefix(s);
return prefix.Depth == s.Length && prefix.FindChildNode('$') != null;
}
public void InsertRange(IEnumerable<string> items)
{
foreach (string item in items)
Insert(item);
}
public void Insert(string s)
{
var commonPrefix = Prefix(s);
var current = commonPrefix;
for (var i = current.Depth; i < s.Length; i++)
{
var newNode = new Node(s[i], current.Depth + 1, current);
current.Children.Add(newNode);
current = newNode;
}
current.Children.Add(new Node('$', current.Depth + 1, current));
}
public void Delete(string s)
{
if (!Search(s))
return;
var node = Prefix(s).FindChildNode('$');
while (node.IsLeaf())
{
var parent = node.Parent;
parent.DeleteChildNode(node.Value);
node = parent;
}
}
}
}

A couple of thoughts:
First, your million strings need to be ordered, so that you can "seek" to the first matching string and return strings until you no longer have a match...in order (seek via C# List<string>.BinarySearch, perhaps). That's how you touch the least number of strings possible.
Second, you should probably not try to hit the string list until there's a pause in input of at least 500 ms (give or take).
Third, your queries into the vastness should be async and cancelable, because it's certainly going to be the case that one effort will be superseded by the next keystroke.
Finally, any subsequent query should first check that the new search string is an append of the most recent search string...so that you can begin your subsequent seek from the last seek (saving lots of time).

I suggest using linq.
string x = "searchterm";
List<string> y = new List<string>();
List<string> Matches = y.Where(xo => xo.StartsWith(x)).ToList();
Where x is your keystroke search text term, y is your collection of strings to search, and Matches is the matches from your collection.
I tested this with the first 1 million prime numbers, here is the code adapted from above:
Stopwatch SW = new Stopwatch();
SW.Start();
string x = "2";
List<string> y = System.IO.File.ReadAllText("primes1.txt").Split(' ').ToList();
y.RemoveAll(xo => xo == " " || xo == "" || xo == "\r\r\n");
List <string> Matches = y.Where(xo => xo.StartsWith(x)).ToList();
SW.Stop();
Console.WriteLine("matches: " + Matches.Count);
Console.WriteLine("time taken: " + SW.Elapsed.TotalSeconds);
Console.Read();
Result is:
matches: 77025
time taken: 0.4240604
Of course this is testing against numbers and I don't know whether linq converts the values before, or if numbers make any difference.

How to compare two csv files by 2 columns?

I have 2 csv files
1.csv
spain;russia;japan
italy;russia;france
2.csv
spain;russia;japan
india;iran;pakistan
I read both files and add data to lists
var lst1= File.ReadAllLines("1.csv").ToList();
var lst2= File.ReadAllLines("2.csv").ToList();
Then I find all unique strings from both lists and add it to result lists
var rezList = lst1.Except(lst2).Union(lst2.Except(lst1)).ToList();
rezlist contains this data
[0] = "italy;russia;france"
[1] = "india;iran;pakistan"
At now I want to compare, make except and union by second and third column in all rows.
1.csv
spain;russia;japan
italy;russia;france
2.csv
spain;russia;japan
india;iran;pakistan
I think I need to split all rows by symbol ';' and make all 3 operations (except, distinct and union) but cannot understand how.
rezlist must contains
india;iran;pakistan
I added class
class StringLengthEqualityComparer : IEqualityComparer<string>
{
public bool Equals(string x, string y)
{
...
}
public int GetHashCode(string obj)
{
...
}
}
StringLengthEqualityComparer stringLengthComparer = new StringLengthEqualityComparer();
var rezList = lst1.Except(lst2,stringLengthComparer ).Union(lst2.Except(lst1,stringLengthComparer),stringLengthComparer).ToList();

Your question is not very clear: for instance, is india;iran;pakistan the desired result primarily because russia is at element[1]? Isn't it also included because element [2] pakistan does not match france and japan? Even though thats unclear, I assume the desired result comes from either situation.
Then there is this: find all unique string from both lists which changes the nature dramatically. So, I take it that the desired results are because "iran" appears in column[1] no where else in column[1] in either file and even if it did, that row would still be unique due to "pakistan" in col[2].
Also note that a data sample of 2 leaves room for a fair amount of error.
Trying to do it in one step makes it very confusing. Since eliminating dupes found in 1.CSV is pretty easy, do it first:
// parse "1.CSV"
List<string[]> lst1 = File.ReadAllLines(#"C:\Temp\1.csv").
Select(line => line.Split(';')).
ToList();
// parse "2.CSV"
List<string[]> lst2 = File.ReadAllLines(#"C:\Temp\2.csv").
Select(line => line.Split(';')).
ToList();
// extracting once speeds things up in the next step
// and leaves open the possibility of iterating in a method
List<List<string>> tgts = new List<List<string>>();
tgts.Add(lst1.Select(z => z[1]).Distinct().ToList());
tgts.Add(lst1.Select(z => z[2]).Distinct().ToList());
var tmpLst = lst2.Where(x => !tgts[0].Contains(x[1]) ||
!tgts[1].Contains(x[2])).
ToList();
That results in the items which are not in 1.CSV (no matching text in Col[1] nor Col[2]). If that is really all you need, you are done.
Getting unique rows within 2.CSV is trickier because you have to actually count the number of times each Col[1] item occurs to see if it is unique; then repeat for Col[2]. This uses GroupBy:
var unique = tmpLst.
GroupBy(g => g[1], (key, values) =>
new GroupItem(key,
values.ToArray()[0],
values.Count())
).Where(q => q.Count == 1).
GroupBy(g => g.Data[2], (key, values) => new
{
Item = string.Join(";", values.ToArray()[0]),
Count = values.Count()
}
).Where(q => q.Count == 1).Select(s => s.Item).
ToList();
The GroupItem class is trivial:
class GroupItem
{
public string Item { set; get; } // debug aide
public string[] Data { set; get; }
public int Count { set; get; }
public GroupItem(string n, string[] d, int c)
{
Item = n;
Data = d;
Count = c;
}
public override string ToString()
{
return string.Join(";", Data);
}
}
It starts with tmpList, gets the rows with a unique element at [1]. It uses a class for storage since at this point we need the array data for further review.
The second GroupBy acts on those results, this time looking at col[2]. Finally, it selects the joined string data.
Results
Using 50,000 random items in File1 (1.3 MB), 15,000 in File2 (390 kb). There were no naturally occurring unique items, so I manually made 8 unique in 2.CSV and copied 2 of them into 1.CSV. The copies in 1.CSV should eliminate 2 if the 8 unique rows in 2.CSV making the expected result 6 unique rows:
NepalX and ItalyX were the repeats in both files and they correctly eliminated each other.
With each step it is scanning and working with less and less data, which seems to make it pretty fast for 65,000 rows / 130,000 data elements.

your GetHashCode()-Method in EqualityComparer are buggy. Fixed version:
public int GetHashCode(string obj)
{
return obj.Split(';')[1].GetHashCode();
}
now the result are correct:
// one result: "india;iran;pakistan"
btw. "StringLengthEqualityComparer"is not a good name ;-)

private void GetUnion(List<string> lst1, List<string> lst2)
{
List<string> lstUnion = new List<string>();
foreach (string value in lst1)
{
string valueColumn1 = value.Split(';')[0];
string valueColumn2 = value.Split(';')[1];
string valueColumn3 = value.Split(';')[2];
string result = lst2.FirstOrDefault(s => s.Contains(";" + valueColumn2 + ";" + valueColumn3));
if (result != null)
{
if (!lstUnion.Contains(result))
{
lstUnion.Add(result);
}
}
}
}

class Program
{
static void Main(string[] args)
{
var lst1 = File.ReadLines(#"D:\test\1.csv").Select(x => new StringWrapper(x)).ToList();
var lst2 = File.ReadLines(#"D:\test\2.csv").Select(x => new StringWrapper(x));
var set = new HashSet<StringWrapper>(lst1);
set.SymmetricExceptWith(lst2);
foreach (var x in set)
{
Console.WriteLine(x.Value);
}
}
}
struct StringWrapper : IEquatable<StringWrapper>
{
public string Value { get; }
private readonly string _comparand0;
private readonly string _comparand14;
public StringWrapper(string value)
{
Value = value;
var split = value.Split(';');
_comparand0 = split[0];
_comparand14 = split[14];
}
public bool Equals(StringWrapper other)
{
return string.Equals(_comparand0, other._comparand0, StringComparison.OrdinalIgnoreCase)
&& string.Equals(_comparand14, other._comparand14, StringComparison.OrdinalIgnoreCase);
}
public override bool Equals(object obj)
{
if (ReferenceEquals(null, obj)) return false;
return obj is StringWrapper && Equals((StringWrapper) obj);
}
public override int GetHashCode()
{
unchecked
{
return ((_comparand0 != null ? StringComparer.OrdinalIgnoreCase.GetHashCode(_comparand0) : 0)*397)
^ (_comparand14 != null ? StringComparer.OrdinalIgnoreCase.GetHashCode(_comparand14) : 0);
}
}
}

How to compare sequential elements in a foreach loop in C#

In a foreach loop I want to compare an element with the previous element that was read. How can I do that? What is the syntax for addressing a previous element in a foreach loop?

You don't have that option built in with a foreach loop.
You can either switch to a for loop or use a variable.
Suppose you iterate through a list of objects, these are your options:
object prev = null;
foreach(var current in myListOfObjects)
{
if(current == prev)
{
// do stuff
}
// don't forget the next row!
prev = current;
}
or
for(var i = 1; i < myListOfObjects.count, i++) // Note: starting from 1 to avoid another condition inside the loop.
{
if(myListOfObjects[i] == myListOfObjects[i-1])
{
// do stuff
}
}

Everything is better with Bluetooth extension methods:
public static class EnumerableExtensions
{
public struct CurrentAndPrevious<T>
{
public T Current { get; private set; }
public T Previous { get; private set; }
public CurrentAndPrevious(T current, T previous) : this()
{
Previous = previous;
Current = current;
}
}
public static IEnumerable<CurrentAndPrevious<T>> WithPrevious<T>(this IEnumerable<T> enumerable)
{
var previous = default(T);
using(var enumerator = enumerable.GetEnumerator())
{
while(enumerator.MoveNext())
{
yield return new CurrentAndPrevious<T>(enumerator.Current, previous);
previous = enumerator.Current;
}
}
}
}
var items = new[] { 1, 2, 3, 4, 5 };
foreach(var item in items.WithPrevious())
{
Console.WriteLine(item.Previous + " " + item.Current);
}
You might need to tweak this depending on how you want first and last elements handled.

You can loop over a bit modified source instead of initial, say ListOfMyObjects:
MyObject prior = default(MyObject);
var source = ListOfMyObjects
.Select(item => {
var result = new {
Current = item,
Prior = prior,
};
prior = item; // side effect, not a good practice
return result;
});
So you can loop
foreach(var item in source) {
if (item.Prior == item.Current) {
...
}
}

A foreach itself has no syntax 'for addressing a previous element'. There are two options, depending on the characteristics of the collection and also the notion of a 'previous' element in respect of the first one. The following the examples are a little bit simplistic, but you should be able to choose the right path and fine-tune the details.
Option 1: Use a temporary variable
Works well if there's no cheap (performance-wise) way to index elements in the sequence, and you are OK with 'pretending' there's an empty (null, or default(T)) item before the very first item.
T previous = default(T); // corresponds to null for reference types
foreach (T item in sequence)
{
… work with previous and item here…
// the current 'item' is the new 'previous' for the next iteration
previous = item;
}
Note that if T is a value type, your would be actually copying the values themselves.
Option 2: Use a for loop and indexing
Works well if there is a cheap (performance-wise) way to index individual elements directly. List<T> and arrays are good examples here.
// indexing from 1, i.e. from the second item in the sequence
for (int i = 1; i < sequence.Count; i++)
{
var previous = sequence[i-1]; // this is obviously the previous item
var current = sequence[i]; // this is obviously the current item
}

Similar to using a temp variable, however this solution moves the scope of the temp variable inside the loop
var collection = new List<int>() { 1, 2, 3, 4, 5 };
foreach (var item in collection)
{
var currentIndex = collection.IndexOf(item);
if (currentIndex > 0 && currentIndex < collection.Count)
{
var previousItem = collection[currentIndex - 1];
}
}

As mentioned by Pham X, one easy way to do this would be a temp variable.
ObjectType temp_object = null;
foreach(var entry in ListOfObjects)
{
if(temp_object==null)
{
//this is the first time through...
temp_object=entry;
}
else
{
//it's anything after the first loop
if(entry==temp_object) Console.WriteLine("There is a match between two entries.");
else temp_object=entry;
}
}

Logic to decipher consecutive date ranges

Say I have a class Slots as follows
public class Slots
{
//Other properties
.
.
.
public DateTime StartTime{get;set;}
public DateTime EndTime{ge;set;}
//Methods
}
And I have List<Slots>, what is the most efficient way to determine consecutive Slotgroups?
Consecutive slots are defined as any group of slots which start the next day relative to the previous slot, there is no overlap.
If there is a day gap(no slots on that day in the list) then it should be considered start of another slot group.
public List<List<Slots>> GetSlotGroups(List<SLots> slots)
{
//return list of slot groups according to logic described above
}

This kind of operation is best achieved with GetEnumerator.
The following code expects the list to be sorted.
IEnumerable<IEnumerable<Slot>> ToGroups(IEnumerable<Slot> slots)
{
using (var ie = slots.GetEnumerator())
{
var range = new List<Slot>();
while (ie.MoveNext())
{
if (range.Count > 0)
{
if (ie.Current.Start > range[range.Count - 1].End)
{
yield return range;
range = new List<Slot>{ie.Current};
continue;
}
}
range.Add(ie.Current);
}
yield return range;
}
}

It's simple. Sort by StartTime, then iterate over the sorted set, if the current item is not consecutive to the previous one, add a new group and make it current. Then simply add the item to the current group.
public static List<List<Slots>> GetSlotGroups(List<Slots> slots)
{
var slotGroups = new List<List<Slots>>();
using (var e = slots.OrderBy(slot => slot.StartTime).GetEnumerator())
{
List<Slots> currentGroup = null;
Slots lastSlot = null;
while (e.MoveNext())
{
var currentSlot = e.Current;
if (lastSlot == null || currentSlot.StartTime.Date.Subtract(lastSlot.EndTime.Date).Days > 1)
slotGroups.Add(currentGroup = new List<Slots>());
currentGroup.Add(currentSlot);
lastSlot = currentSlot;
}
}
return slotGroups;
}

Here's the code I came up with. I don't know if it's the most efficient, but it's readable and reasonably fast.
public static List<List<Slots>> GetGroups(List<Slots> slots)
{
List<List<Slots>> groups = new List<List<Slots>>();
DateTime? nextDate = null;
List<Slots> currentGroup = null;
foreach (var slot in slots.OrderBy(x => x.StartDate))
{
//first time through nextDate and currentGroup are null
//this condition matches the first time through or any time there is a gap in dates
if (nextDate == null || nextDate.Value < slot.StartDate)
{
if (currentGroup != null)
{
//if currentGroups isn't null then we have a completed group
groups.Add(currentGroup);
}
//start a new group
currentGroup = new List<Slots>();
}
nextDate = slot.EndDate.AddDays(1);
currentGroup.Add(slot);
}
//if there are no items in the collection currentGroup will still be null, otherwise currentGroup has the last group in it still. We finished iterating before finding a gap in dates
if (currentGroup != null)
{
groups.Add(currentGroup);
}
return groups;
}
This code keeps track of the next date in the range by adding one to the end date of the previous slot. As we go from slot to slot we append to a temporary list named currentGroup. When our next date is less than the start date of the current slot, we add the current group to the results list named groups and create a new list for our current group. At the end we presumably have some slots in currentGroup for the last group, so we have to add that one as well.

The best answer was based on George's code and Roberts comment, but modified to consider having more than one slot on a given day.
protected IEnumerable<IEnumerable<ExamCalendar>>ToBlocks(IEnumerable<ExamCalendar> slots)
{
using (var ie = slots.OrderBy(slot => slot.StartDate).GetEnumerator())
{
var block = new List<ExamCalendar>();
while (ie.MoveNext())
{
if (block.Count > 0)
{
if (ie.Current.StartDate.Date != block[block.Count - 1].StartDate.Date && ie.Current.StartDate.Date != block[block.Count - 1].EndDate.AddDays(1).Date)
{
yield return block;
block = new List<ExamCalendar> { ie.Current };
continue;
}
}
block.Add(ie.Current);
}
yield return block;
}
}

Consolidation of an ObservableCollection based on keys

I'm just starting out so forgive me if I don't use the correct terminology. I'm trying to consolidate an ObservableCollection> by looping through and comparing one key to all the other keys in the collection. If they are the same it should then compare the matching keys values.
I don't have enough rep to post a pic.
private void CombineUDAs(ObservableCollection<Tuple<object, object>> UDAs)
{
foreach (var item in UDAs)
{
}
}

You can do this like so:
public void CombineUDAs( ObservableCollection<Tuple<object, object>> UDAs )
{
foreach ( var item in UDAs )
foreach ( var innerItem in UDAs.Where( innerItem => innerItem != innerItem && item.Item1 == innerItem.Item1 ) )
Console.WriteLine( "Value is same: {0}", item.Item2 == innerItem.Item2 );
}
Loop over each item
For each item search in the collection for items with the same “key”
Check if the “values” are equals

I'm a little rusty on my c# so my syntax is probably off, and you can probably do this more cleanly, but here's a rough idea...
Note that the inner for loop starts at the outer object index so you aren't looping over duplicate objects. Might increase performance.
public void CombineUDAs( ObservableCollection<Tuple<object, object>> UDAs )
{
for(outer=0; outer<UDAs.Count; outer++)
for (inner = outer; inner<UDAs.Count; inner++)
if(outer != inner && (UDAs[inner].item1 == UDAs[outer].item1) && (UDAs[inner].item2 == UDAs[outer].item2))
//Add to collection
}
It may be easier to just add the elements that are duplicates to a new collection. If you're navigating through the new collection frequently it may save performance depending on the size of the collection.
If you want to make the other objects blank you'll just have to inverse the if statement as needed. Probably something like:
if(outer != inner && (UDAs[inner].item1 != UDAs[outer].item1) || (UDAs[inner].item2 != UDAs[outer].item2))

Got it working the way I wanted with the help of a coworker here's the resulting code. Enumerator is the selected objects. I still need to go back to tighten the code up but the functionality is there.
_udaTuple = new ObservableCollection<Tuple<object, object>>();
var tempDictionary = new Dictionary<object, object>();
foreach (var item in Enumerator)
{
var modelObject = item as TSM.ModelObject;
if (modelObject != null)
{
var tempHash = new Hashtable();
modelObject.GetAllUserProperties(ref tempHash);
foreach (DictionaryEntry dictionaryEntry in tempHash)
{
if (tempDictionary.ContainsKey(dictionaryEntry.Key))
{
if (tempDictionary[dictionaryEntry.Key] is string && dictionaryEntry.Value is string)
{
if ((string)tempDictionary[dictionaryEntry.Key]!=(string)dictionaryEntry.Value)
{
tempDictionary[dictionaryEntry.Key] = "Values Do Not Match";
}
}
else if (tempDictionary[dictionaryEntry.Key] is double && dictionaryEntry.Value is double)
{
if ((double)tempDictionary[dictionaryEntry.Key] != (double)dictionaryEntry.Value)
{
tempDictionary[dictionaryEntry.Key] = "Values Do Not Match";
}
}
else if (tempDictionary[dictionaryEntry.Key] is int && dictionaryEntry.Value is int)
{
if ((int)tempDictionary[dictionaryEntry.Key] != (int)dictionaryEntry.Value)
{
tempDictionary[dictionaryEntry.Key] = "Values Do Not Match";
}
}
}
else
{
tempDictionary.Add(dictionaryEntry.Key, dictionaryEntry.Value);
}
}
}
}
foreach (var item in tempDictionary)
{
_udaTuple.Add(new Tuple<object, object>(item.Key, item.Value));
}