I find that my program is searching through lots of lengthy strings (20,000+) trying to find a particular unique phrase.
What is the most efficent method for doing this in C#?
Below is the current code which works like this:
The search begins at startPos because the target area is somewhat removed from the start
It loops through the string, at each step it checks if the substring from that point starts with the startMatchString, which is an indicator that the start of the target string has been found. (The length of the target string varys).
From here it creates a new substring (chopping off the 11 characters that mark the start of the target string) and searches for the endMatchString
I already know that this is a horribly complex and possibly very inefficent algorithm.
What is a better way to accomplish the same result?
string result = string.Empty;
for (int i = startPos; i <= response.Length - 1; i++)
{
if (response.Substring(i).StartsWith(startMatchString))
{
string result = response.Substring(i).Substring(11);
for (int j = 0; j <= result.Length - 1; j++)
{
if (result.Substring(j).StartsWith(endMatchString))
{
return result.Remove(j)
}
}
}
}
return result;
You can use String.IndexOf, but make sure you use StringComparison.Ordinal or it may be one order of magnitude slower.
private string Search2(int startPos, string startMatchString, string endMatchString, string response) {
int startMarch = response.IndexOf(startMatchString, startPos, StringComparison.Ordinal);
if (startMarch != -1) {
startMarch += startMatchString.Length;
int endMatch = response.IndexOf(endMatchString, startMarch, StringComparison.Ordinal);
if (endMatch != -1) { return response.Substring(startMarch, endMatch - startMarch); }
}
return string.Empty;
}
Searching 1000 times a string at about the 40% of a 183 KB file took about 270 milliseconds. Without StringComparison.Ordinal it took about 2000 milliseconds.
Searching 1 time with your method took over 60 seconds as it creates a new string (O(n)) each iteration, making your method O(n^2).
There are a whole bunch of algorithms,
boyer and moore
Sunday
Knuth-Morris-Pratt
Rabin-Karp
I would recommend to use the simplified Boyer-Moore, called Boyer–Moore–Horspool.
The C-code appears at the wikipedia.
For the java code look at
http://www.fmi.uni-sofia.bg/fmi/logic/vboutchkova/sources/BoyerMoore_java.html
A nice article about these is available under
http://www.ibm.com/developerworks/java/library/j-text-searching.html
If you want to use built-in stuff go for regular expressions.
It depends on what you're trying to find in the string. If you're looking for a specific sequence IndexOf/Contains are fast, but if you're looking for wild card patterns Regex is optimized for this kind of search.
I would try to use a Regular Expression instead of rolling my own string search algorithm. You can precompile the regular expression to make it run faster.
For very long strings you cannot beat the boyer-moore search algorithm. It is more complex than I might try to explain here, but The CodeProject site has a pretty good article on it.
You could use a regex; it’s optimized for this kind of searching and manipulation.
You could also try IndexOf ...
string result = string.Empty;
if (startPos >= response.Length)
return result;
int startingIndex = response.IndexOf(startMatchString, startPos);
int rightOfStartIndex = startingIndex + startMatchString.Length;
if (startingIndex > -1 && rightOfStartIndex < response.Length)
{
int endingIndex = response.IndexOf(endMatchString, rightOfStartIndex);
if (endingIndex > -1)
result = response.Substring(rightOfStartIndex, endingIndex - rightOfStartIndex);
}
return result;
Here's an example using IndexOf (beware: written from the top of my head, didn't test it):
int skip = 11;
int start = response.IndexOf(startMatchString, startPos);
if (start >= 0)
{
int end = response.IndexOf(startMatchString, start + skip);
if (end >= 0)
return response.Substring(start + skip, end - start - skip);
else
return response.Substring(start + skip);
}
return string.Empty;
As said before regex is your friend.
You might want to look at RegularExpressions.Group.
This way you can name part of the matched resultset.
Here is an example
Related
This might be a very basic question. I need to write a code which works similar as string replace algorithm.
static string stringReplace(string s, string stringOld, string stringNew)
{
string newWord = "";
int oldMax = stringOld.Length;
int index = 0;
for (int i = 0; i < s.Length; i++)
{
if (index != oldMax && s[i] == stringOld[index])
{
if (stringOld[index] < stringNew[index])
{
newWord = newWord + stringNew[index];
index++;
}
else
{
newWord = newWord + stringNew[index];
}
}
else
{
newWord = newWord + s[i];
}
}
return newWord;
}
Since it's 3am the code above is probably bugged. When the new word is shorter than the old one, it goes wrong. Same as when it's longer. When the index variable is equal for both stringOld and stringNew, it will do the swap. I think... Please don't post "use string.Replace(), I have to write that algorithm myself...
I don't know what you're trying to do with your code, but the problem is not a small one.
Think logically about what you are trying to do.
It is a two step process:
Find the starting index of stringOld in s.
If found replace stringOld with stringNew.
Step 1:
There are many rather complex (and elegant) efficient string search algorithms, you can search for them online or look at popular 'Introduction to Algorithms' by Cormen, Leiserson, Rivest & Stein, but the naive approach involves two loops and is pretty simple. It is also described in that book (and online.)
Step 2:
If a match is found at index i; simply copy characters 0 to i-1 of s to newWord, followed by newString and then the rest of the characters in s starting at index i + oldString.Length.
I am looking for an elegant way, preferably a short linq expression, to count how many alphanumeric chars a given string contains.
The 'boring' way I do it now is this:
int num = 0;
for (int i = 0; i < password.Length; i++)
{
if (!char.IsLetterOrDigit(password, i))
{
num++;
}
}
if (num < MinRequiredNonAlphanumericCharacters)
return false;
This is rather short already, but I am sure with some linq magic this can be done in an even shorter, equally understandable expression, right?
Here's the quick and dirty LINQ way to get the letter & digit count:
password.Count(char.IsLetterOrDigit)
This is more of a direct copy of what you're doing:
password.Count(c => !char.IsLetterOrDigit(c))
int num = password.Where((t, i) => !char.IsLetterOrDigit(password, i)).Count();
if (num < MinRequiredNonAlphanumericCharacters)
return false;
You could do it with one line of Regex. Whether that's understandable is a moot point.
num = new Regex(#"\w", RegexOptions.IgnoreCase).Matches(input).Count
You can use Take() to ensure that you don't inspect more letters than necessary:
int minCount = password.Where(x => !char.IsLetterOrDigit(x)).Take(MinRequired).Count();
if (minCount < MinRequired) { // No good }
The idea is that we only keep checking until you've hit the minimum number. At that point, we can stop, because we know we have an admissible password. Take() takes as many as it can, and no more, so if there aren't enough, it will just return a number less than you've requested.
Sorry for such a basic question regarding lists, but do we have this feature in C#?
e.g. imagine this Python List:
a = ['a','b,'c']
print a[0:1]
>>>>['a','b']
Is there something like this in C#? I currently have the necessity to test some object properties in pairs. edit: pairs are always of two :P
Imagine a larger (python) list:
a = ['a','a','b','c','d','d']
I need to test for example if a[0] = a[1], and if a[1] = a[2] etc.
How this can be done in C#?
Oh, and a last question: what is the tag (here) i can use to mark some parts of my post as code?
You can use LINQ to create a lazily-evaluated copy of a segment of a list. What you can't do without extra code (as far as I'm aware) is take a "view" on an arbitrary IList<T>. There's no particular reason why this shouldn't be feasible, however. You'd probably want it to be a fixed size (i.e. prohibit changes via Add/Remove) and you could also make it optionally read-only - basically you'd just proxy various calls on to the original list.
Sounds like it might be quite useful, and pretty easy to code... let me know if you'd like me to do this.
Out of interest, does a Python slice genuinely represent a view, or does it take a copy? If you change the contents of the original list later, does that change the contents of the slice? If you really want a copy, the the LINQ solutions using Skip/Take/ToList are absolutely fine. I do like the idea of a cheap view onto a collection though...
I've been looking for something like Python-Slicing in C# with no luck.
I finally wrote the following string extensions to mimic the python slicing:
static class StringExtensions
{
public static string Slice(this string input, string option)
{
var opts = option.Trim().Split(':').Select(s => s.Length > 0 ? (int?)int.Parse(s) : null).ToArray();
if (opts.Length == 1)
return input[opts[0].Value].ToString(); // only one index
if (opts.Length == 2)
return Slice(input, opts[0], opts[1], 1); // start and end
if (opts.Length == 3)
return Slice(input, opts[0], opts[1], opts[2]); // start, end and step
throw new NotImplementedException();
}
public static string Slice(this string input, int? start, int? end, int? step)
{
int len = input.Length;
if (!step.HasValue)
step = 1;
if (!start.HasValue)
start = (step.Value > 0) ? 0 : len-1;
else if (start < 0)
start += len;
if (!end.HasValue)
end = (step.Value > 0) ? len : -1;
else if (end < 0)
end += len;
string s = "";
if (step < 0)
for (int i = start.Value; i > end.Value && i >= 0; i+=step.Value)
s += input[i];
else
for (int i = start.Value; i < end.Value && i < len; i+=step.Value)
s += input[i];
return s;
}
}
Examples of how to use it:
"Hello".Slice("::-1"); // returns "olleH"
"Hello".Slice("2:-1"); // returns "ll"
I'm doing some work with strings, and I have a scenario where I need to determine if a string (usually a small one < 10 characters) contains repeated characters.
`ABCDE` // does not contain repeats
`AABCD` // does contain repeats, ie A is repeated
I can loop through the string.ToCharArray() and test each character against every other character in the char[], but I feel like I am missing something obvious.... maybe I just need coffee. Can anyone help?
EDIT:
The string will be sorted, so order is not important so ABCDA => AABCD
The frequency of repeats is also important, so I need to know if the repeat is pair or triplet etc.
If the string is sorted, you could just remember each character in turn and check to make sure the next character is never identical to the last character.
Other than that, for strings under ten characters, just testing each character against all the rest is probably as fast or faster than most other things. A bit vector, as suggested by another commenter, may be faster (helps if you have a small set of legal characters.)
Bonus: here's a slick LINQ solution to implement Jon's functionality:
int longestRun =
s.Select((c, i) => s.Substring(i).TakeWhile(x => x == c).Count()).Max();
So, OK, it's not very fast! You got a problem with that?!
:-)
If the string is short, then just looping and testing may well be the simplest and most efficient way. I mean you could create a hash set (in whatever platform you're using) and iterate through the characters, failing if the character is already in the set and adding it to the set otherwise - but that's only likely to provide any benefit when the strings are longer.
EDIT: Now that we know it's sorted, mquander's answer is the best one IMO. Here's an implementation:
public static bool IsSortedNoRepeats(string text)
{
if (text.Length == 0)
{
return true;
}
char current = text[0];
for (int i=1; i < text.Length; i++)
{
char next = text[i];
if (next <= current)
{
return false;
}
current = next;
}
return true;
}
A shorter alternative if you don't mind repeating the indexer use:
public static bool IsSortedNoRepeats(string text)
{
for (int i=1; i < text.Length; i++)
{
if (text[i] <= text[i-1])
{
return false;
}
}
return true;
}
EDIT: Okay, with the "frequency" side, I'll turn the problem round a bit. I'm still going to assume that the string is sorted, so what we want to know is the length of the longest run. When there are no repeats, the longest run length will be 0 (for an empty string) or 1 (for a non-empty string). Otherwise, it'll be 2 or more.
First a string-specific version:
public static int LongestRun(string text)
{
if (text.Length == 0)
{
return 0;
}
char current = text[0];
int currentRun = 1;
int bestRun = 0;
for (int i=1; i < text.Length; i++)
{
if (current != text[i])
{
bestRun = Math.Max(currentRun, bestRun);
currentRun = 0;
current = text[i];
}
currentRun++;
}
// It's possible that the final run is the best one
return Math.Max(currentRun, bestRun);
}
Now we can also do this as a general extension method on IEnumerable<T>:
public static int LongestRun(this IEnumerable<T> source)
{
bool first = true;
T current = default(T);
int currentRun = 0;
int bestRun = 0;
foreach (T element in source)
{
if (first || !EqualityComparer<T>.Default(element, current))
{
first = false;
bestRun = Math.Max(currentRun, bestRun);
currentRun = 0;
current = element;
}
}
// It's possible that the final run is the best one
return Math.Max(currentRun, bestRun);
}
Then you can call "AABCD".LongestRun() for example.
This will tell you very quickly if a string contains duplicates:
bool containsDups = "ABCDEA".Length != s.Distinct().Count();
It just checks the number of distinct characters against the original length. If they're different, you've got duplicates...
Edit: I guess this doesn't take care of the frequency of dups you noted in your edit though... but some other suggestions here already take care of that, so I won't post the code as I note a number of them already give you a reasonably elegant solution. I particularly like Joe's implementation using LINQ extensions.
Since you're using 3.5, you could do this in one LINQ query:
var results = stringInput
.ToCharArray() // not actually needed, I've left it here to show what's actually happening
.GroupBy(c=>c)
.Where(g=>g.Count()>1)
.Select(g=>new {Letter=g.First(),Count=g.Count()})
;
For each character that appears more than once in the input, this will give you the character and the count of occurances.
I think the easiest way to achieve that is to use this simple regex
bool foundMatch = false;
foundMatch = Regex.IsMatch(yourString, #"(\w)\1");
If you need more information about the match (start, length etc)
Match match = null;
string testString = "ABCDE AABCD";
match = Regex.Match(testString, #"(\w)\1+?");
if (match.Success)
{
string matchText = match.Value; // AA
int matchIndnex = match.Index; // 6
int matchLength = match.Length; // 2
}
How about something like:
string strString = "AA BRA KA DABRA";
var grp = from c in strString.ToCharArray()
group c by c into m
select new { Key = m.Key, Count = m.Count() };
foreach (var item in grp)
{
Console.WriteLine(
string.Format("Character:{0} Appears {1} times",
item.Key.ToString(), item.Count));
}
Update Now, you'd need an array of counters to maintain a count.
Keep a bit array, with one bit representing a unique character. Turn the bit on when you encounter a character, and run over the string once. A mapping of the bit array index and the character set is upto you to decide. Break if you see that a particular bit is on already.
/(.).*\1/
(or whatever the equivalent is in your regex library's syntax)
Not the most efficient, since it will probably backtrack to every character in the string and then scan forward again. And I don't usually advocate regular expressions. But if you want brevity...
I started looking for some info on the net and I got to the following solution.
string input = "aaaaabbcbbbcccddefgg";
char[] chars = input.ToCharArray();
Dictionary<char, int> dictionary = new Dictionary<char,int>();
foreach (char c in chars)
{
if (!dictionary.ContainsKey(c))
{
dictionary[c] = 1; //
}
else
{
dictionary[c]++;
}
}
foreach (KeyValuePair<char, int> combo in dictionary)
{
if (combo.Value > 1) //If the vale of the key is greater than 1 it means the letter is repeated
{
Console.WriteLine("Letter " + combo.Key + " " + "is repeated " + combo.Value.ToString() + " times");
}
}
I hope it helps, I had a job interview in which the interviewer asked me to solve this and I understand it is a common question.
When there is no order to work on you could use a dictionary to keep the counts:
String input = "AABCD";
var result = new Dictionary<Char, int>(26);
var chars = input.ToCharArray();
foreach (var c in chars)
{
if (!result.ContainsKey(c))
{
result[c] = 0; // initialize the counter in the result
}
result[c]++;
}
foreach (var charCombo in result)
{
Console.WriteLine("{0}: {1}",charCombo.Key, charCombo.Value);
}
The hash solution Jon was describing is probably the best. You could use a HybridDictionary since that works well with small and large data sets. Where the letter is the key and the value is the frequency. (Update the frequency every time the add fails or the HybridDictionary returns true for .Contains(key))
I have a regex to find the nth occurrence of a character in a string, here's the code:
public static int NthIndexOf(this string target, string value, int n)
{
Match m = Regex.Match(target, "((" + value + ").*?){" + n + "}");
if (m.Success)
{
return m.Groups[2].Captures[n - 1].Index;
}
else
{
return -1;
}
}
Now, I have 1594 entries in this string, with 1593 semicolons. If I write:
tempstring.NthIndexOf(";", 1593)
The answer comes back immediately and correctly. If I give it anything over 1594 it hangs. Does anyone know how to fix this?
Test Case
string holder = "test;test2;test3";
string test = "";
for (int i = 0; i < 600; i++)
{
test += holder;
}
int index = test.NthIndexOf(";", 2000);
This takes a very long time. Change 600 to 6 and it is very fast. Make 2000 to 1700 and it is very fast as well.
Why is my regular expression so slow?
If you're really only looking for character repetitions, and not string repetitions, then you should be able to replace you method with something simple like
public static int NthIndexOf(this string target, char testChar, int n)
{
int count = 0;
for(int i=0; i<target.Length; i++)
{
if(target[i] == testChar)
{
count++;
if(count == n) return i;
}
}
return -1;
}
and use that. It should have far fewer limitations.
As for why your original regex is going slow, here's what I suspect:
For your fast case, it's working because it can find a match on it's first pass through (with each group matching exactly one character)
For the slow case is because it can't find a match (and won't ever find one, because there aren't enough semicolons to satisfy the regex), but it recursively tries every possible way to break up the string (which is a really big operation)
Try to use a more distinct and efficient regular expression:
"^(?:[^" + value + "]*" + value + "){" + (n - 1) + "}([^" + value + "]*)
This will build the following regular expression for tempstring.NthIndexOf(";", 1593):
^(?:[^;]*;){1592}([^;]*)
But this will only work for single characters as separator.
Another approach would be to step through each character and count the occurences of the character you were looking for.