We Keep Coding

Is there a way to prevent index out of bounds error here?

I am new here. I'm trying a C# problem on Leetcode about Longest Common Prefix. I know this problem has been solved MANY times. I'm just having a hard time understanding why my code doesn't work under certain conditions. When the input ["flower","flow","flight"] is put in, it works just fine and get the output "fl". However, when I get to the input ["ab", "a"], I suddenly get an IndexOutOfRangeException. Anyone know why? Here's my code:
public class Solution {
public string LongestCommonPrefix(string[] strs) {
StringBuilder sb = new StringBuilder();
int h = 1;
int count = 0;
if (strs.Length == 1){
return strs[0];
}
for (int i = 0; i<strs[0].Length; i++)
{
if ((strs[0])[i].Equals((strs[h])[i])){
count++;
if (h<strs.Length-1){
h++;
}
}
}
for (int j = 0; j < count; j++)
{
sb.Append((strs[0])[j]);
}
return sb.ToString();
}
}

You can debug by making a simple console app in Visual Studio. I added a Console.Writeline(strs[0] + strs[h]) to your code to demonstrate the logic as is. Running your code in debug, you can step through and see where the exception gets thrown:
Notice that the console prints the same thing each time, and that the strs[h] is only one character long. When you increment the i to 1, you're trying to access the second element of a one element array.

When the input ["flower","flow","flight"] is put in, it works just fine and get the output "fl". However, when I get to the input ["ab", "a"], I suddenly get an IndexOutOfRangeException. Anyone know why?
Because the algorithm as written loops until i has reached the length of the first string, not the shortest string. This is fine for flower/flow/flight because all these strings will find the common prefix "fl" before i comes close to running off the end of the shortest string; i is 2 by the time the algorithm determines that the o in flower is different to the i in flight and hence the code doesn't go out of bounds.
In the ab/a case, i is 1 and the algorithm hasn't yet determined the prefixes have diverged, you attempt to access the nonexistent index 1 on string "a" and get a crash
As comments have pointed out you also have a problem on your h logic; h should be cycled starting from 1 in a loop of its own inside the i loop. As it is now h just increments up to the number of strings-1 and stays there so in essence you end up only looking for prefixes in the first and last strings. If your strings were flower/flight/float/flowing/flowers you'd declare a prefix of "flower" because flight/float/flowing would only be considered when i was 0, 1 and 2 respectively
I suggest you write out the algorithm in comments, in the language you think in then have a go at turning it into c#

Why is my memory running out when file gets to big?

So the error I am getting is this.
An unhandled exception of type 'System.OutOfMemoryException' occurred
in mscorlib.dll
I have never encountered this error before and I've looked it up on Google.
I do have a 64-Bit system.
I do have 16GB of RAM.
Some people said that I need to set the platform target to x64 in my project properties but wont that make it so only 64-bit systems will be able to run this application?
public static string RC4(string input, string key)
{
StringBuilder result = new StringBuilder();
int x, y, j = 0;
int[] box = new int[256];
for (int i = 0; i < 256; i++)
{
box[i] = i;
}
for (int i = 0; i < 256; i++)
{
j = (key[i % key.Length] + box[i] + j) % 256;
x = box[i];
box[i] = box[j];
box[j] = x;
}
for (int i = 0; i < input.Length; i++)
{
y = i % 256;
j = (box[y] + j) % 256;
x = box[y];
box[y] = box[j];
box[j] = x;
result.Append((char)(input[i] ^ box[(box[y] + box[j]) % 256]));
}
return result.ToString(); //This would be the line throwing me the error.
}
Because every second im appending new text from a keyboard hook to a textfile. So lets say I type abc the first second its going to append that, now if I type def the next second its going to append that. This is all happening inside of a timer tick so its really straight forward.
Whenever the textfile reaches around 350 000KB it throws me that error.

There is a limit to how big single objects are allowed to be, even on x64. The actual limit doesn't matter and depends on configuration, but the upshot is that when processing large volumes of data you should read through the data in pieces (usually via Stream), processing chunks at a time. Never try to hold the entire thing in memory at once. This applies equally to input and output.
Now, if you managed to load the entire thing into input, then you got lucky; but StringBuilder is intentionally oversized, so that it doesn't have to keep allocating all the time. You might be able to "fix" your code by telling StringBuilder the correct amount of characters you need in the constructor, but that is only a temporary hack that will let you to process slightly larger data. The real fix is to not attempt to process huge data in a single chunk.

Video rate image construction from binary data performance

First things first:
I have a git repo over here that holds the code of my current efforts and an example data set
Background
The example data set holds a bunch of records in Int32 format. Each record is composed of several bit fields that basically hold info on events where an event is either:
The detection of a photon
The arrival of a synchronizing signal
Each Int32 record can be treated like following C-style struct:
struct {
unsigned TimeTag :16;
unsigned Channel :12;
unsigned Route :2;
unsigned Valid :1;
unsigned Reserved :1; } TTTRrecord;
Whether we are dealing with a photon record or a sync event, time
tag will always hold the time of the event relative to the start of
the experiment (macro-time).
If a record is a photon, valid == 1.
If a record is a sync signal or something else, valid == 0.
If a record is a sync signal, sync type = channel & 7 will give either a value indicating start of frame or end of scan line in a frame.
The last relevant bit of info is that Timetag is 16 bit and thus obviously limited. If the Timetag counter rolls over, the rollover counter is incremented. This rollover (overflow) count can easily be obtained from channel overflow = Channel & 2048.
My Goal
These records come in from a high speed scanning microscope and I would like to use these records to reconstruct images from the recorded photon data, preferably at 60 FPS.
To do so, I obviously have all the info:
I can look over all available data, find all overflows, which allows me to reconstruct the sequential macro time for each record (photon or sync).
I also know when the frame started and when each line composing the frame ended (and thus also how many lines there are).
Therefore, to reconstruct a bitmap of size noOfLines * noOfLines I can process the bulk array of records line by line where each time I basically make a "histogram" of the photon events with edges at the time boundary of each pixel in the line.
Put another way, if I know Tstart and Tend of a line, and I know the number of pixels I want to spread my photons over, I can walk through all records of the line and check if the macro time of my photons falls within the time boundary of the current pixel. If so, I add one to the value of that pixel.
This approach works, current code in the repo gives me the image I expect but it is too slow (several tens of ms to calculate a frame).
What I tried already:
The magic happens in the function int[] Renderline (see repo).
public static int[] RenderlineV(int[] someRecords, int pixelduration, int pixelCount)
{
// Will hold the pixels obviously
int[] linePixels = new int[pixelCount];
// Calculate everything (sync, overflow, ...) from the raw records
int[] timeTag = someRecords.Select(x => Convert.ToInt32(x & 65535)).ToArray();
int[] channel = someRecords.Select(x => Convert.ToInt32((x >> 16) & 4095)).ToArray();
int[] valid = someRecords.Select(x => Convert.ToInt32((x >> 30) & 1)).ToArray();
int[] overflow = channel.Select(x => (x & 2048) >> 11).ToArray();
int[] absTime = new int[overflow.Length];
absTime[0] = 0;
Buffer.BlockCopy(overflow, 0, absTime, 4, (overflow.Length - 1) * 4);
absTime = absTime.Cumsum(0, (prev, next) => prev * 65536 + next).Zip(timeTag, (o, tt) => o + tt).ToArray();
long lineStartTime = absTime[0];
int tempIdx = 0;
for (int j = 0; j < linePixels.Length; j++)
{
int count = 0;
for (int i = tempIdx; i < someRecords.Length; i++)
{
if (valid[i] == 1 && lineStartTime + (j + 1) * pixelduration >= absTime[i])
{
count++;
}
}
// Avoid checking records in the raw data that were already binned to a pixel.
linePixels[j] = count;
tempIdx += count;
}
return linePixels;
}
Treating photon records in my data set as an array of structs and addressing members of my struct in an iteration was a bad idea. I could increase speed significantly (2X) by dumping all bitfields into an array and addressing these. This version of the render function is already in the repo.
I also realised I could improve the loop speed by making sure I refer to the .Length property of the array I am running through as this supposedly eliminates bounds checking.
The major speed loss is in the inner loop of this nested set of loops:
for (int j = 0; j < linePixels.Length; j++)
{
int count = 0;
lineStartTime += pixelduration;
for (int i = tempIdx; i < absTime.Length; i++)
{
//if (lineStartTime + (j + 1) * pixelduration >= absTime[i] && valid[i] == 1)
// Seems quicker to calculate the boundary before...
//if (valid[i] == 1 && lineStartTime >= absTime[i] )
// Quicker still...
if (lineStartTime > absTime[i] && valid[i] == 1)
{
// Slow... looking into linePixels[] each iteration is a bad idea.
//linePixels[j]++;
count++;
}
}
// Doing it here is faster.
linePixels[j] = count;
tempIdx += count;
}
Rendering 400 lines like this in a for loop takes roughly 150 ms in a VM (I do not have a dedicated Windows machine right now and I run a Mac myself, I know I know...).
I just installed Win10CTP on a 6 core machine and replacing the normal loops by Parallel.For() increases the speed by almost exactly 6X.
Oddly enough, the non-parallel for loop runs almost at the same speed in the VM or the physical 6 core machine...
Regardless, I cannot imagine that this function cannot be made quicker. I would first like to eke out every bit of efficiency from the line render before I start thinking about other things.
I would like to optimise the function that generates the line to the maximum.
Outlook
Until now, my programming dealt with rather trivial things so I lack some experience but things I think I might consider:
Matlab is/seems very efficient with vectored operations. Could I achieve similar things in C#, i.e. by using Microsoft.Bcl.Simd? Is my case suited for something like this? Would I see gains even in my VM or should I definitely move to real HW?
Could I gain from pointer arithmetic/unsafe code to run through my arrays?
...
Any help would be greatly, greatly appreciated.
I apologize beforehand for the quality of the code in the repo, I am still in the quick and dirty testing stage... Nonetheless, criticism is welcomed if it is constructive :)
Update
As some mentioned, absTime is ordered already. Therefore, once a record is hit that is no longer in the current pixel or bin, there is no need to continue the inner loop.
5X speed gain by adding a break...
for (int i = tempIdx; i < absTime.Length; i++)
{
//if (lineStartTime + (j + 1) * pixelduration >= absTime[i] && valid[i] == 1)
// Seems quicker to calculate the boundary before...
//if (valid[i] == 1 && lineStartTime >= absTime[i] )
// Quicker still...
if (lineStartTime > absTime[i] && valid[i] == 1)
{
// Slow... looking into linePixels[] each iteration is a bad idea.
//linePixels[j]++;
count++;
}
else
{
break;
}
}

C# Char permutation with repetition on large set of chars

Hello Im trying to get all possible combinations with repetitions of given char array.
Char array consists of alphabet letters(only lower) and I need to generate strings with length of 30 or more chars.
I tried with method of many for-loops,but when I try to get all combinations of char in char array with length of string more then 5 I get out of Memory Exception.
So I created similar Method that takes only first 200000 strings,then next 2000000 and so on this was proven sucessfull but only with smaller length strings.
This was my method with length of 7 chars:
public static int Progress = 0;
public static ArrayList CreateRngUrl7()
{
ArrayList AllCombos = new ArrayList();
int passed = 0;
int Too = Progress + 200000;
char[] alpha = "ABCDEFGHIJKLMNOPQRSTUVWXYZ".ToLower().ToCharArray();
for (int i = 0; i < alpha.Length; i++)
for (int i1 = 0; i1 < alpha.Length; i1++)
for (int i2 = 0; i2 < alpha.Length; i2++)
for (int i3 = 0; i3 < alpha.Length; i3++)
for (int i4 = 0; i4 < alpha.Length; i4++)
for (int i5 = 0; i5 < alpha.Length; i5++)
for (int i6 = 0; i6 < alpha.Length; i6++)
{
if (passed > (Too - 200000) && passed < Too)
{
string word = new string(new char[] { alpha[i], alpha[i1], alpha[i2], alpha[i3], alpha[i4], alpha[i5],alpha[i6] });
AllCombos.Add(word);
}
passed++;
}
if (Too >= passed)
{
MessageBox.Show("All combinations of RNG7 were returned");
}
Progress = Too;
return AllCombos;
}
I tried adding 30 for-loops with in way described above so i Would get strings with lenghts of 30 but application just hangs.Is there any better way to do this? All answers would be much appreciated. Thank you in advance!
Can someone please just post method how it is done with larger legth strings I just want to see an example? I don't have to store that data,I just need to compare it with something and release it from memory. I used alphabet for example I don't need whole alphabet.Question was not how long it would take or how much combinations would it be!!!!!

You get an OutOfMemoryException because inside the loop you allocate a string and store it in an ArrayList. The strings have to stay in memory until the ArrayList is garbage collected and your loop creates more strings than you will be able to store.
If you simply want to check the string for a condition you should put the check inside the loop:
for ( ... some crazy loop ...) {
var word = ... create word ...
if (!WordPassesTest(word)) {
Console.WriteLine(word + " failed test.");
return false;
}
}
return true;
Then you only need storage for a single word. Of course, if the loop is crazy enough, it will not terminate before the end of the universe as we know it.
If you need to execute many nested but similar loops you can use recursion to simplify the code. Here is an example that is not incredible efficient, but at least it is simple:
Char[] chars = "ABCD".ToCharArray();
IEnumerable<String> GenerateStrings(Int32 length) {
if (length == 0) {
yield return String.Empty;
yield break;
}
var strings = chars.SelectMany(c => GenerateStrings(length - 1), (c, s) => c + s);
foreach (var str in strings)
yield return str;
}
Calling GenerateStrings(3) will generate all strings of length 3 using lazy evaluation (so no additional storage is required for the strings).
Building on top of an IEnumerable generating your strings you can create primites to buffer and process buffers of strings. An easy solution is to using Reactive Extensions for .NET. Here you already have a Buffer primitive:
GenerateStrings(3)
.ToObservable()
.Buffer(10)
.Subscribe(list => ... ship the list to another computer and process it ...);
The lambda in Subscribe will be called with a List<String> with at most 10 strings (the parameter provided in the call to Buffer).
Unless you have an infinte number of computers you will still have to pull the computers from a pool and only recycle them back to the pool when they have finished the computation.
It should be obvious from the comments on this question that you will not be able to process 26^30 strings even if you have multiple computers at your disposal.

I don't have time right now to write some code but essentially if you are running out of RAM use disk. I'm thinking along the lines of one thread running an algorithm to find the combinations and another persisting the results to disk and releasing the RAM.

Generating a dataset with few unique values

Note: This is part 2 of a 2 part question.
Part 1 here
I'm wanting to more about sorting algorithms and what better way to do than then to code! So I figure I need some data to work with.
My approach to creating some "standard" data will be as follows: create a set number of items, not sure how large to make it but I want to have fun and make my computer groan a little bit :D
Once I have that list, I'll push it into a text file and just read off that to run my algorithms against. I should have a total of 4 text files filled with the same data but just sorted differently to run my algorithms against (see below).
Correct me if I'm wrong but I believe I need 4 different types of scenarios to profile my algorithms.
Randomly sorted data (for this I'm going to use the knuth shuffle)
Reversed data (easy enough)
Nearly sorted (not sure how to implement this)
Few unique (once again not sure how to approach this)
This question is for generating a list with a few unique items of data.
Which approach is best to generate a dataset with a few unique items.

Answering my own question here. Don't know if this is the best but it works.
public static int[] FewUnique(int uniqueCount, int returnSize)
{
Random r = _random;
int[] values = new int[uniqueCount];
for (int i = 0; i < uniqueCount; i++)
{
values[i] = i;
}
int[] array = new int[returnSize];
for (int i = 0; i < returnSize; i++)
{
array[i] = values[r.Next(0, values.Count())];
}
return array;
}

It might be worth having a look at NBuilder. It's a framework designed to generate objects for testing with and sounds like just what you need.
You could deal with the "few unique" items with some code like this:
var products = Builder<YourClass>.CreateListOfSize(1000)
.WhereAll().AreConstructedWith("some fixed value")
.WhereRandom(20).AreConstructedWith("some other fixed value")
.Build();
There's plenty of other variations you can use as well to get the data like you want it. Have a look at some of the samples on the site for more ideas.

http://pages.cs.wisc.edu/~bart/fuzz/
Is all about fuzz testing which focuses on semi random data. It should be straight forward to adapt this approach to your problem

I guess your solution is ok. I would only modify it slighly:
public static int[] FewUnique(int uniqueCount, int low, int high, int returnSize)
{
Random r = _random;
int[] values = new int[uniqueCount];
for (int i = 0; i < uniqueCount; i++)
{
values[i] = r.Next(low, high);
}
int[] array = new int[returnSize];
for (int i = 0; i < returnSize; i++)
{
array[i] = values[r.Next(0, values.Count())];
}
return array;
}
For some algorithms this might make a difference.