Related
I'm trying to use the Span2D type to "roll" entries in a 2d array, but it's not working as expected.
By rolling I mean the following - given an array such as:
{
{ 1, 1, 1, 1, 1 },
{ 2, 2, 2, 2, 2 },
{ 0, 0, 0, 0, 0 },
}
I would like to copy the first two rows down one row, so the top row can be repopulated. After the roll operation the array should look like this:
{
{ 1, 1, 1, 1, 1 },
{ 1, 1, 1, 1, 1 },
{ 2, 2, 2, 2, 2 },
}
The Span2d CopyTo method seems perfect for this - I create a Slice of the top two rows, and a slice of the bottom two rows, copy the first slice to the second slice. But instead of the expected result above, I get:
{
{ 1, 1, 1, 1, 1 },
{ 1, 1, 1, 1, 1 },
{ 1, 1, 1, 1, 1 },
}
Here's a runnable class that shows the problem:
public class SpanTest
{
public static void Main()
{
int[,] array =
{
{ 1, 2, 3, 4, 5 },
{ 0, 0, 0, 0, 0 },
{ 0, 0, 0, 0, 0 },
};
var h = array.GetLength(0) - 1;
var w = array.GetLength(1);
Console.WriteLine($"slice height:{h} width: {w}\n-----------");
Span2D<int> span = array;
Console.WriteLine($"{span.ToStringMatrix()}-----------");
var sourceSlice = span.Slice(0, 0, h, w);
Console.WriteLine($"{sourceSlice.ToStringMatrix()}-----------");
var targetSlice = span.Slice(1, 0, h, w);
Console.WriteLine($"{targetSlice.ToStringMatrix()}-----------");
sourceSlice.CopyTo(targetSlice);
Console.WriteLine($"{span.ToStringMatrix()}-----------");
}
}
with a helper for printing the Span2Ds:
public static class Utils
{
public static string ToStringMatrix<T>(this Span2D<T> arr)
{
var sb = new StringBuilder();
for (var i = 0; i < arr.Height; i++)
{
for (var j = 0; j < arr.Width; j++)
{
sb.Append($"{arr[i, j]} ");
}
sb.Append(Environment.NewLine);
}
return sb.ToString();
}
}
How can I make the copy operation behave as expected? Thanks
Well, the answer is quite obvious actually - at least it was when it occurred to me at 5am this morning!
Span2D wraps an array, copying to itself alters the backing array during the copy process. By the time the second row is copied, it already contains the contents of the first row. And hence, the first row gets propagated throughout the 2d array.
This question already has answers here:
What is an IndexOutOfRangeException / ArgumentOutOfRangeException and how do I fix it?
(5 answers)
Closed 1 year ago.
This post was edited and submitted for review 1 year ago and failed to reopen the post:
Original close reason(s) were not resolved
I was working on KNN from Accord.NET and I faced this error for some reason when I need to test model.
but this error message didn't help at all (Index was outside the bounds of the array) because this error happen in the library itself.
simple code with random data:
using Accord.MachineLearning;
double[][] inputs =
{
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 5 ,1},
new double[] { 16, 2 ,0}, new double[] { 4, 15 ,1},
};
int[] outputs =
{
0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 9
};
var knn = new KNearestNeighbors(k: 15);
knn.Learn(inputs, outputs);
//test
var t = new double[] { 16, 2, 0 };
int answer = knn.Decide(t);
and here the exception:
but I found way around and I share solution with you below :
after many days and after implementing this simple sample I found that output array should have continues range values (encoded : 0,1,2,3,....) .
so here 9 is why that bug happen here 🙂
int[] outputs =
{
0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 2
};
I'm dealing with the BigInteger class with numbers in the order of 2 raised to the power 10,000,000.
The BigInteger Log function is now the most expensive function in my algorithm and I am desperately looking for an alternative.
Since I only need the integral part of the log, I came across this answer which seems brilliant in terms of speed but for some reason I am not getting accurate values. I do not care about the decimal part but I do need to get an accurate integral part whether the value is floored or ceiled as long as I know which.
Here is the function I implemented:
public static double LogBase2 (System.Numerics.BigInteger number)
{
return (LogBase2(number.ToByteArray()));
}
public static double LogBase2 (byte [] bytes)
{
// Corrected based on [ronalchn's] answer.
return (System.Math.Log(bytes [bytes.Length - 1], 2) + ((bytes.Length - 1) * 8));
}
The values are now incredibly accurate except for corner cases. The values 7 to 7.99999, 15 to 15.9999, 23 to 23.9999 31 to 31.9999, etc. return -Infinity. The numbers seem to revolve around byte boundaries. Any idea what's going on here?
Example:
LogBase2( 1081210289) = 30.009999999993600 != 30.000000000000000
LogBase2( 1088730701) = 30.019999999613300 != 30.000000000000000
LogBase2( 2132649894) = 30.989999999389400 != 30.988684686772200
LogBase2( 2147483648) = 31.000000000000000 != -Infinity
LogBase2( 2162420578) = 31.009999999993600 != -Infinity
LogBase2( 4235837212) = 31.979999999984800 != -Infinity
LogBase2( 4265299789) = 31.989999999727700 != -Infinity
LogBase2( 4294967296) = 32.000000000000000 != 32.000000000000000
LogBase2( 4324841156) = 32.009999999993600 != 32.000000000000000
LogBase2( 545958373094) = 38.989999999997200 != 38.988684686772200
LogBase2( 549755813887) = 38.999999999997400 != 38.988684686772200
LogBase2( 553579667970) = 39.009999999998800 != -Infinity
LogBase2( 557430119061) = 39.019999999998900 != -Infinity
LogBase2( 561307352157) = 39.029999999998300 != -Infinity
LogBase2( 565211553542) = 39.039999999997900 != -Infinity
LogBase2( 569142910795) = 39.049999999997200 != -Infinity
LogBase2( 1084374326282) = 39.979999999998100 != -Infinity
LogBase2( 1091916746189) = 39.989999999998500 != -Infinity
LogBase2( 1099511627775) = 39.999999999998700 != -Infinity
Try this:
public static int LogBase2(byte[] bytes)
{
if (bytes[bytes.Length - 1] >= 128) return -1; // -ve bigint (invalid - cannot take log of -ve number)
int log = 0;
while ((bytes[bytes.Length - 1]>>log)>0) log++;
return log + bytes.Length*8-9;
}
The reason for the most significant byte being 0 is because the BigInteger is a signed integer. When the most significant bit of the high-order byte is 1, an extra byte is tacked on to represent the sign bit of 0 for positive integers.
Also changed from using the System.Math.Log function because if you only want the rounded value, it is much faster to use bit operations.
If you have Microsoft Solver Foundation (download at http://msdn.microsoft.com/en-us/devlabs/hh145003.aspx), then you can use the BitCount() function:
public static double LogBase2(Microsoft.SolverFoundation.Common.BigInteger number)
{
return number.BitCount;
}
Or you can use the java library. Add a reference to the vjslib library (found in the .NET tab - this is the J# implementation of the java library).
You can now add "using java.math" in your code.
java.math.BigInteger has a bitLength() function
BigInteger bi = new BigInteger(128);
int log = bi.Log2();
public static class BigIntegerExtensions
{
static int[] PreCalc = new int[] { 8, 7, 6, 6, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1};
public static int Log2(this BigInteger bi)
{
byte[] buf = bi.ToByteArray();
int len = buf.Length;
return len * 8 - PreCalc[buf[len - 1]] - 1;
}
}
Years late but maybe this will help someone else...
.Net Core 3 added the .GetBitLength() that is basically log2. (but just one increment too high) Since it is built-in to .net I think this is about as fast as we can get.
// Create some example number
BigInteger myNum= (BigInteger)32;
// Get the Log2
BigInteger myLog2 = myNum.GetBitLength() - 1;
https://dotnetfiddle.net/7ggy4D
There are many implementations of bit counting out there but in my case, I need to test if an arbitrarily large number contains at most two set bits.
I wrote the following function that does the job and seems to be quite fast but I wanted to find out if it can be further optimized for C#. This function gets called in a loop a few million times.
public static byte [] BitCountLookupArray = new byte []
{
0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4, 1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5, 1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5, 2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6, 1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5, 2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6, 2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6, 3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7, 1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5, 2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6, 2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6, 3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7, 2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6, 3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7, 3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7, 4, 5, 5, 6, 5, 6, 6, 7, 5, 6, 6, 7, 6, 7, 7, 8
};
// The parameter [number] will NEVER be negative.
public static bool HasSetBitCountOfLessThenThree (System.Numerics.BigInteger number)
{
int sum = 0;
byte [] bytes = null;
bytes = number.ToByteArray();
for (int i=0; i < bytes.Length; i++)
{
sum += BitCountLookupArray [bytes [i]];
}
return (sum < 3);
}
IMPORTANT: The argument [number] sent to the function will NEVER be negative.
Some points I thought of were:
Making the function static. Done.
Using a static lookup array. Done.
Using pointers instead of array indexes since the number of bytes often crosses 100,000. Not sure how much this would help.
Forcing an inline function which sadly cannot be guaranteed in .NET.
Open to other suggestions.
This way you can optimise it further
for (int i=0; i < bytes.Length; i++)
{
sum += BitCountLookupArray [bytes [i]];
if(sum >= 3)
{
return false // This will stop the execution of unnecessary lines
// as we need to know whether sum is less than 3 or not.
}
}
return true;
Since you only need to know whether you have fewer than 3 set bits, I would suggest this:
// remove two bits
number &= number - 1;
number &= number - 1;
// if number != 0, then there were 3 or more bits set
return number.IsZero;
Of course Rain's method works too, and I'm not sure which strategy will be faster.
Alternative:
//remove one bit
number &= number - 1;
// if the number of bits left is 0 or 1, there were < 3 bits set
return number.IsZero || number.IsPowerOfTwo;
It's probably faster to test first, and remove the bit later:
return number.IsZero || // zero bits?
number.IsPowerOfTwo || // one bit?
(number & (number - 1)).IsPowerOfTwo; // two bits?
The most obvious optimisation is to drop out of the loop as soon as sum == 3, since any further matches past that point are immaterial.
There's also no need to set bytes twice; simply use byte [] bytes = number.ToByteArray();, but the benifit here is miniscule.
I have 2 byte arrays:
Dim A() As Byte = {1, 2, 3, 4, 5, 6, 7, 8, 9}
Dim B() As Byte = {5, 6, 7}
Now I want to find the occurance of the full B in A. I tried Array.IndexOf(A, B) with no luck. Is there a simple way to search an array by array without the need to use any loops?
It should find the index (position) of 5,6,7 in the same order as in B().
If A() contains {1,2,3,4,7,6,5,9} it should return false or -1 because they are not in the same order.
The following Linq statement will give an IEnumerable<int> containing the positions of b in a (or an empty set if none occur):
Enumerable
.Range( 0, 1 + a.Length - b.Length )
.Where( i => a.Skip(i).Take(b.Length).SequenceEqual(b) );
I have no idea how to translate to VB.NET.
This might work, but it's C# and uses a loop:
private static int[] GetIndicesOf(byte[] needle, byte[] haystack)
{
int[] foundIndices = new int[needle.Length];
int found = 0;
for (int i = 0; i < haystack.Length; i++)
{
if (needle[found] == haystack[i])
{
foundIndices[found++] = i;
if (found == needle.Length)
return foundIndices;
}
else
{
i -= found; // Re-evaluate from the start of the found sentence + 1
found = 0; // Gap found, reset, maybe later in the haystack another occurrance of needle[0] is found
continue;
}
}
return null;
}
Tested with input:
Byte[] haystack = { 5, 6, 7, 8, 9, 0, 5, 6, 7 };
Byte[] needle = { 5, 6, 7 };
// Returns {0, 1, 2}
Byte[] haystack = { 5, 6, 0, 8, 9, 0, 5, 6, 7 };
Byte[] needle = { 5, 6, 7 };
// Returns {6, 7, 8}
Byte[] haystack = { 5, 6, 0, 7, 9, 0, 5, 6, 8 };
Byte[] needle = { 5, 6, 7 };
// Returns null
Byte[] haystack = { 1, 2, 1, 2, 2 };
Byte[] needle = { 1, 2, 2 };
// Returns {2, 3, 4}
Byte[] haystack = { 1, 2, 1, 2, 1, 2, 3 };
Byte[] needle = { 1, 2, 1, 2, 3 };
// Returns {2, 3, 4, 5, 6}
Byte[] haystack = { 1, 1, 1, 1, 2 };
Byte[] needle = { 1, 2 };
// Returns {3, 4}
But the Linq implementation of #spender looks nicer. :-P
How about creating a method that:
Concatinates the elements of the searched list to one string
Concatinates the elements of the list to search for to one string
Looks in the first string for the precense of the second string
Like so:
public bool IsSubSetOf(IList<int> list1, IList<int> list2){
var string1 = string.Join("", list1);
var string2 = string.Join("", list2);
return string1.Contains(string2);
}
Not tested...
An efficient way of solving this problem in general is the KMP algorithm. Quick googling suggest that a .NET implementation may be found here. It's implementational pseudocode is availible from Wikipedia.
An inefficient, but harmlessly easy to code way is presented in one of the links above as follows:
int[] T = new[]{1, 2, 3, 4, 5};
int[] P = new[]{3, 4};
for (int i = 0; i != T.Length; i++)
{
int j = 0
for (;i+j != T.Length && j != P.Length && T[i+j]==P[j]; j++);
if (j == P.Length) return i;
}
My take would be:
public static int Search<T>(T[] space, T[] searched) {
foreach (var e in Array.FindAll(space, e => e.Equals(searched[0]))) {
var idx = Array.IndexOf(space, e);
if (space.ArraySkip(idx).Take(searched.Length).SequenceEqual(searched))
return idx;
}
return -1;
}
public static class Linqy {
public static IEnumerable<T> ArraySkip<T>(this T[] array, int index) {
for (int i = index; i < array.Length; i++) {
yield return array[i];
}
}
}
As always, it depends on your data whether this is "good enough" or you will have to resort to more complex yet efficient algorithms. I introduced an arrayskip as the Linq skip does indeed only assume the IEnumerable interface and would enumerate up to the index.