So it turns out all arrays are not created equal. Multi-dimensional arrays can have non-zero lower bounds. See for example Excel PIA's Range.Value property object[,] rectData = myRange.Value;
I need to convert these data into a jagged array. My first try below smells of complexity. Any suggestions to optimize it? It needs to handle the general case where lower bounds may not be zero.
I have this ex method:
public static T[][] AsJagged<T>( this T[,] rect )
{
int row1 = rect.GetLowerBound(0);
int rowN = rect.GetUpperBound(0);
int col1 = rect.GetLowerBound(1);
int colN = rect.GetUpperBound(1);
int height = rowN - row1 + 1;
int width = colN - col1 + 1;
T[][] jagged = new T[height][];
int k = 0;
int l;
for ( int i = row1; i < row1 + height; i++ )
{
l = 0;
T[] temp = new T[width];
for ( int j = col1; j < col1 + width; j++ )
temp[l++] = rect[i, j];
jagged[k++] = temp;
}
return jagged;
}
Used like this:
public void Foo()
{
int[,] iRect1 = { { 1, 1, 1, 1 }, { 1, 1, 1, 1 }, { 1, 1, 1, 1 }, { 1, 1, 1, 1 }, { 1, 1, 1, 1 }, { 1, 1, 1, 1 }, { 1, 1, 1, 1 }, { 1, 1, 1, 1 } };
int[][] iJagged1 = iRect1.AsJagged();
int[] lengths = { 3, 5 };
int[] lowerBounds = { 7, 8 };
int[,] iRect2 = (int[,])Array.CreateInstance(typeof(int), lengths, lowerBounds);
int[][] iJagged2 = iRect2.AsJagged();
}
Curious if Buffer.BlockCopy() would work or be faster?
Edit: AsJagged needs to handle reference types.
Edit: Found bug in AsJagged(). Added int l; and added col1 + width to inner loop.
A view caveats/assumptions up front:
You seem to use only int as your data type (or at least seem to be OK with using Buffer.BlockCopy which would imply you can life with primitive types in general).
For the test data you show, I don't think there will be much different using any somewhat sane approach.
Having that said, the following implementation (which needs to be specialized for a specific primitive type (here int) because it uses fixed) is around 10 times faster than the approach using the inner loop:
unsafe public static int[][] AsJagged2(int[,] rect)
{
int row1 = rect.GetLowerBound(0);
int rowN = rect.GetUpperBound(0);
int col1 = rect.GetLowerBound(1);
int colN = rect.GetUpperBound(1);
int height = rowN - row1 + 1;
int width = colN - col1 + 1;
int[][] jagged = new int[height][];
int k = 0;
for (int i = row1; i < row1 + height; i++)
{
int[] temp = new int[width];
fixed (int *dest = temp, src = &rect[i, col1])
{
MoveMemory(dest, src, rowN * sizeof(int));
}
jagged[k++] = temp;
}
return jagged;
}
[DllImport("kernel32.dll", EntryPoint = "RtlMoveMemory")]
unsafe internal static extern void MoveMemory(void* dest, void* src, int length);
Using the following "test code":
static void Main(string[] args)
{
Random rand = new Random();
int[,] data = new int[100,1000];
for (int i = 0; i < data.GetLength(0); i++)
{
for (int j = 0; j < data.GetLength(1); j++)
{
data[i, j] = rand.Next(0, 1000);
}
}
Stopwatch sw = Stopwatch.StartNew();
for (int i = 0; i < 100; i++)
{
int[][] dataJagged = AsJagged(data);
}
Console.WriteLine("AsJagged: " + sw.Elapsed);
sw = Stopwatch.StartNew();
for (int i = 0; i < 100; i++)
{
int[][] dataJagged2 = AsJagged2(data);
}
Console.WriteLine("AsJagged2: " + sw.Elapsed);
}
Where AsJagged (the first case) is your original function, I get the following output:
AsJagged: 00:00:00.9504376
AsJagged2: 00:00:00.0860492
So there is indeed a faster way of doing it, however depending on the size of the test data, the number of times you actually perform this operation, and your willingness to allow unsafe and P/Invoke code, you're probably not going to need it.
Having that said, we were using large matrixes of double (say 7000x10000 elements) where it indeed did make a huge difference.
Update: about using Buffer.BlockCopy
I might overlook some Marshal or other trick, but I don't think using Buffer.BlockCopy is possible here. This is due to the fact that it requires both the source and destination array to, well, be an Array.
In our example, the destination is an array (e.g. int[] temp = ...) however the source is not. While we "know" that for two dimensional arrays of primitive types the layout is such, that each "row" (i.e. first dimension) is an array of the type in memory, there is no safe (as in unsafe) way to get that array without the overhead of copying it first. So we basically need to use a function that simply deals with memory and doesn't care about the actual content of it - like MoveMemory. BTW, the internal implementation of Buffer.BlockCopy does something similar.
Your complexity is O(N*M) N - number of rows, M - number of columns. That's the best you can get when copying N*M values...
Buffer.BlockCopy might be faster than your inner for loop, but I wouldn't be surprised if the compiler knows how to handle this code properly and you won't gain any further speed. You should test it to make sure.
You may be able to achieve better performance by not copying the data at all (at the potential expense of slightly slower lookups). If you create an 'array row' class, that holds your rect and a row number, and provides an indexer that accesses the correct column, you can create an array of such rows, and save yourself the copying altogether.
The complexity of creating such an array of 'array rows' is O(N).
EDIT: An ArrayRow class, just because it bugs me...
The ArrayRow could look something like this:
class ArrayRow<T>
{
private T[,] _source;
private int _row;
public ArrayRow(T[,] rect, int row)
{
_source = rect;
_row = row;
}
public T this[int col] { get { return _source[_row, col]; } }
}
Now you create an array of ArrayRows, you don't copy anything at all, and the optimizer has a good chance of optimizing accessing an entire row in sequence.
Related
When I run this code the array has a new size after, is there anything wrong or bad about it ?
static int[] ExpandArray(int[] input, int add_size)
{
for (int i = 0; i < add_size; i++)
{
int[] temp = input;
input = new int[input.Length + 1];
for (var j = 0; j < temp.Length; j++)
{
input[j] = temp[j];
}
}
return input;
}
static void Main(string[] args)
{
int[] ovride = new int[3] { 1, 2, 3 };
ovride = ExpandArray(ovride, 10);
ovride = ExpandArray(ovride, 10);
Console.WriteLine(ovride.Length);
}
is there anything wrong or bad about it ?
This isn't code review, but:
Yes. You should not resize arrays. This involves a new allocation and a copy of all elements. As does Array.Resize(), by the way.
Hey, there is a method that already does this: Array.Resize(). Don't reinvent the wheel.
You definitely should not do the resize in a loop.
So to clean up the code a little:
static int[] ExpandArray(int[] input, int sizeToAdd)
{
// Create a new array with the desired size
var ouput = new int[input.Length + sizeToAdd];
// Copy all elements from input to output
for (int i = 0; i < input.Length; i++)
{
output[i] = input[i];
}
// Return the new array, having the remaining
// items set to their default (0 for int)
return output;
}
You'd actually want input to be updatable by ref, and then end with input = output.
Ultimately, just use a List<int>, as that allows for more efficient resizing, and does so automatically when necessary.
You can use Array.Resize which:
Changes the number of elements of a one-dimensional array to the specified new size.
int[] ovride = new int[3] { 1, 2, 3 };
Array.Resize(ref ovride, ovride.Length + 10);
Array.Resize(ref ovride, ovride.Length + 10);
Console.WriteLine(ovride.Length); // prints 23
But if you expect collection size changes List can be a more suitable option for your goal.
I would like to create a nested loop where the nesting depth determined by the size on an array. For example I have an array of integers and I would like to check all tuples where . Is there a way to do this simply in c# (or any other language)?
The only easy idea I had was to just multiply the numbers and do a for loop up to that number but unfortunately the product reaches the int limit. Also in this case I have no way to add extra conditions on each level.
Here's a small example and the product approach:
int[] a = new int[]{2, 3, 2}; //we have an array like this. In the description above it is a_1, a_2, a_3
void f(int[] i) //I have a function that works on an array input
{...}
bool check(int[] i) //I have a checker function, with an array parameter too
{...}
//And I would like to perform this function on all these arrays
//f({0, 0, 0})
//f({0, 0, 1})
//010
//011
//020
//021
//100
//101
//110
//111
//120
//f({1, 2, 1})
//But if let's say check({1, 0}) is false then don't perform the f function on {1, 0, 0} and {1, 0, 1}
//A simple implementation if we know the size of the array a is the following
for(int i1=0;i1<a[0];i1++)
{
if(!check({i1}))
continue;
for(int i2=0;i2<a[1];i2++)
{
if(!check({i1, i2}))
continue;
for(int i3=0;i3<a[2];i3++)
{
if(!check({i1, i2, i3}))
continue;
f({i1, i2, i3});
}
}
}
//But this obviously fails as we have no idea apriori of the size of the array a
//An alternative I have is the following:
int prod = 1;
foreach(int x in a)
{
prod *= x;
}
for(int c=0;c<prod;c++)
{
int d=c;
int[] i = new int[a.Length];
for(int l=0;l<a.Length;l++)
{
i[l]=d%(a[l]);
d /= a[l];
}
f(i);
}
//But the problem with this implementation is that in my case prod is larger than the int limit. Also this loops through all the incorrect cases too, where the check function can highly reduce the number of cases to calculate.
I've managed to solve the problem. The idea is that increasing the i value by one is really easy and we can check the condition that we don't overstep the a values and don't violate the check function easily. Below is a code that works
int[] a = ...;
void f(int[] i){...}
bool check(int[] i){...}
int n = a.Length;
int[] i = new int[n];
while (true) //keep increasing the coordinates of i, while we can
{
for (int l = 0; l < n; l++)
{
int[] il = copyFirst(i, l);
while(!check(il)) //check for all first few coordinates if it is correct, skip if incorrect.
//There is a way to improve this even further, as we don't have to check the first few
// coords if it was correct before, so should only care about the recently changed section
{
i = increase(a, i, l);
if (i == null)
{
return;
}
else
{
il = copyFirst(i, l);
}
}
}
f(i);
i = increase(a, i, n-1);
if (i == null) return;
}
int[] copyFirst(int[] i, int l) //this is just a small helper function to copy the first few elements of i, to use in the check
{
int[] ret = new int[l];
for (int k = 0; k < l; k++)
{
ret[k] = i[k];
}
return ret;
}
int[] increase(int[] a, int[] i, int l) //this results in the increased vector and in null if we've reached the end
{
for (int k = l; k >= 0; k--)
{
i[k] = i[k] + 1;
if (i[k] >= a[k])
{
i[k] = 0;
}
else
{
return i;
}
}
return null;
}
Suppose you have an array like:
double[,] rectArray = new double[10,3];
Now you want the fouth row as a double[] array of 3 elements without doing:
double[] fourthRow = new double[]{rectArray[3,0],
rectArray[3,1],
rectArray[3,2]};
Is it possible someway? Even using a Marshal.Something approach?
Thanks!
You can use Buffer.BlockCopy method:
const int d1 = 10;
const int d2 = 3;
const int doubleSize = 8;
double[,] rectArray = new double[d1, d2];
double[] target = new double[d2];
int rowToGet = 3;
Buffer.BlockCopy(rectArray, doubleSize * d2 * rowToGet, target, 0, doubleSize * d2);
LINQ to the rescue:
var s = rectArray.Cast<double>().Skip(9).Take(3).ToArray();
Explanation: Casting a multi-dimensional array flattens it to a single-dimensional array. After that all we need to do is skip to the element we want (the 4th element in the 2-D array resolves to Skip(9)...) and take 3 elements from it).
Why not make a generic extension method?
public static T[] GetRow<T>(this T[,] input2DArray, int row) where T : IComparable
{
var width = input2DArray.GetLength(0);
var height = input2DArray.GetLength(1);
if (row >= height)
throw new IndexOutOfRangeException("Row Index Out of Range");
// Ensures the row requested is within the range of the 2-d array
var returnRow = new T[width];
for(var i = 0; i < width; i++)
returnRow[i] = input2DArray[i, row];
return returnRow;
}
Like this all you have to code is:
array2D = new double[,];
// ... fill array here
var row = array2D.GetRow(4) // Implies getting 5th row of the 2-D Array
This is useful if you're trying to chain methods after obtaining a row and could be helpful with LINQ commands as well.
You probably want to use a jagged array. That is not an array of 10 by 3 but instead an array of arrays.
Something like :
double[][] rectArray;
....
double [] rowArray = rectArray[3];
There are lots of places to learn more about jagged arrays. For example Dynamically created jagged rectangular array
If you must use a rectangular array and just want to simplify the syntax, you can use a method to get the row like so:
double[] fourthRow = GetRow(rectArray, 3);
public static T[] GetRow<T>(T[,] matrix, int row)
{
var columns = matrix.GetLength(1);
var array = new T[columns];
for (int i = 0; i < columns; ++i)
array[i] = matrix[row, i];
return array;
}
Although this is an old thread, an addition to Joseph Sturtevants answer may be useful. His function crashes in case the matrix's first column is not zero, but another integer.
This is e.g. always the case in case of retrieving data from Excel, like
object[,] objects = null;
Excel.Range range = worksheet.get_Range("A1", "C5");
objects = range.Cells.Value; //columns start at 1, not at 0
The GetRow function could be modified like this:
public static T[] GetRow<T>(T[,] matrix, int row, int firstColumn)
{
var columns = matrix.GetLength(1);
var array = new T[columns];
for (int i = firstColumn; i < firstColumn + columns; ++i)
array[i-firstColumn] = matrix[row, i];
return array;
}
I have an array in c# that is 1-based (generated from a call to get_Value for an Excel Range
I get a 2D array for example
object[,] ExcelData = (object[,]) MySheet.UsedRange.get_Value(Excel.XlRangeValueDataType.xlRangeValueDefault);
this appears as an array for example ExcelData[1..20,1..5]
is there any way to tell the compiler to rebase this so that I do not need to add 1 to loop counters the whole time?
List<string> RowHeadings = new List<string>();
string [,] Results = new string[MaxRows, 1]
for (int Row = 0; Row < MaxRows; Row++) {
if (ExcelData[Row+1, 1] != null)
RowHeadings.Add(ExcelData[Row+1, 1]);
...
...
Results[Row, 0] = ExcelData[Row+1, 1];
& other stuff in here that requires a 0-based Row
}
It makes things less readable since when creating an array for writing the array will be zero based.
Why not just change your index?
List<string> RowHeadings = new List<string>();
for (int Row = 1; Row <= MaxRows; Row++) {
if (ExcelData[Row, 1] != null)
RowHeadings.Add(ExcelData[Row, 1]);
}
Edit: Here is an extension method that would create a new, zero-based array from your original one (basically it just creates a new array that is one element smaller and copies to that new array all elements but the first element that you are currently skipping anyhow):
public static T[] ToZeroBasedArray<T>(this T[] array)
{
int len = array.Length - 1;
T[] newArray = new T[len];
Array.Copy(array, 1, newArray, 0, len);
return newArray;
}
That being said you need to consider if the penalty (however slight) of creating a new array is worth improving the readability of the code. I am not making a judgment (it very well may be worth it) I am just making sure you don't run with this code if it will hurt the performance of your application.
Create a wrapper for the ExcelData array with a this[,] indexer and do rebasing logic there. Something like:
class ExcelDataWrapper
{
private object[,] _excelData;
public ExcelDataWrapper(object[,] excelData)
{
_excelData = excelData;
}
public object this[int x, int y]
{
return _excelData[x+1, y+1];
}
}
Since you need Row to remain as-is (based on your comments), you could just introduce another loop variable:
List<string> RowHeadings = new List<string>();
string [,] Results = new string[MaxRows, 1]
for (int Row = 0, SrcRow = 1; SrcRow <= MaxRows; Row++, SrcRow++) {
if (ExcelData[SrcRow, 1] != null)
RowHeadings.Add(ExcelData[SrcRow, 1]);
...
...
Results[Row, 0] = ExcelData[SrcRow, 1];
}
Why not use:
for (int Row = 1; Row <= MaxRows; Row++) {
Or is there something I'm missing?
EDIT: as it turns out that something is missing, I would use another counter (starting at 0) for that purpose, and use a 1 based Row index for the array. It's not good practice to use the index for another use than the index in the target array.
Is changing the loop counter too hard for you?
for (int Row = 1; Row <= MaxRows; Row++)
If the counter's range is right, you don't have to add 1 to anything inside the loop so you don't lose readability. Keep it simple.
I agree that working with base-1 arrays from .NET can be a hassle. It is also potentially error-prone, as you have to mentally make a shift each time you use it, as well as correctly remember which situations will be base 1 and which will be base 0.
The most performant approach is to simply make these mental shifts and index appropriately, using base-1 or base-0 as required.
I personally prefer to convert the two dimensional base-1 arrays to two dimensional base-0 arrays. This, unfortunately, requires the performance hit of copying over the array to a new array, as there is no way to re-base an array in place.
Here's an extension method that can do this for the 2D arrays returned by Excel:
public static TResult[,] CloneBase0<TSource, TResult>(
this TSource[,] sourceArray)
{
If (sourceArray == null)
{
throw new ArgumentNullException(
"The 'sourceArray' is null, which is invalid.");
}
int numRows = sourceArray.GetLength(0);
int numColumns = sourceArray.GetLength(1);
TResult[,] resultArray = new TResult[numRows, numColumns];
int lb1 = sourceArray.GetLowerBound(0);
int lb2 = sourceArray.GetLowerBound(1);
for (int r = 0; r < numRows; r++)
{
for (int c = 0; c < numColumns; c++)
{
resultArray[r, c] = sourceArray[lb1 + r, lb2 + c];
}
}
return resultArray;
}
And then you can use it like this:
object[,] array2DBase1 = (object[,]) MySheet.UsedRange.get_Value(Type.Missing);
object[,] array2DBase0 = array2DBase1.CloneBase0();
for (int row = 0; row < array2DBase0.GetLength(0); row++)
{
for (int column = 0; column < array2DBase0.GetLength(1); column++)
{
// Your code goes here...
}
}
For massively sized arrays, you might not want to do this, but I find that, in general, it really cleans up your code (and mind-set) to make this conversion, and then always work in base-0.
Hope this helps...
Mike
For 1 based arrays and Excel range operations as well as UDF (SharePoint) functions I use this utility function
public static object[,] ToObjectArray(this Object Range)
{
Type type = Range.GetType();
if (type.IsArray && type.Name == "Object[,]")
{
var sourceArray = Range as Object[,];
int lb1 = sourceArray.GetLowerBound(0);
int lb2 = sourceArray.GetLowerBound(1);
if (lb1 == 0 && lb2 == 0)
{
return sourceArray;
}
else
{
int numRows = sourceArray.GetLength(0);
int numColumns = sourceArray.GetLength(1);
var resultArray = new Object[numRows, numColumns];
for (int r = 0; r < numRows; r++)
{
for (int c = 0; c < numColumns; c++)
{
resultArray[r, c] = sourceArray[lb1 + r, lb2 + c];
}
}
return resultArray;
}
}
else if (type.IsCOMObject)
{
// Get the Value2 property from the object.
Object value = type.InvokeMember("Value2",
System.Reflection.BindingFlags.Instance |
System.Reflection.BindingFlags.Public |
System.Reflection.BindingFlags.GetProperty,
null,
Range,
null);
if (value == null)
value = string.Empty;
if (value is string)
return new object[,] { { value } };
else if (value is double)
return new object[,] { { value } };
else
{
object[,] range = (object[,])value;
int rows = range.GetLength(0);
int columns = range.GetLength(1);
object[,] param = new object[rows, columns];
Array.Copy(range, param, rows * columns);
return param;
}
}
else
throw new ArgumentException("Not A Excel Range Com Object");
}
Usage
public object[,] RemoveZeros(object range)
{
return this.RemoveZeros(range.ToObjectArray());
}
[ComVisible(false)]
[UdfMethod(IsVolatile = false)]
public object[,] RemoveZeros(Object[,] range)
{...}
The first function is com visible and will accept an excel range or a chained call from another function (the chained call will return a 1 based object array), the second call is UDF enabled for Excel Services in SharePoint. All of the logic is in the second function. In this example we are just reformatting a range to replace zero with string.empty.
You could use a 3rd party Excel compatible component such as SpreadsheetGear for .NET which has .NET friendly APIs - including 0 based indexing for APIs such as IRange[int rowIndex, int colIndex].
Such components will also be much faster than the Excel API in almost all cases.
Disclaimer: I own SpreadsheetGear LLC
I was shocked to find out today that C# does not support dynamic sized arrays. How then does a VB.NET developer used to using ReDim Preserve deal with this in C#?
At the beginning of the function I am not sure of the upper bound of the array. This depends on the rows returned from the database.
VB.NET doesn't have the idea of dynamically sized arrays, either - the CLR doesn't support it.
The equivalent of "Redim Preserve" is Array.Resize<T> - but you must be aware that if there are other references to the original array, they won't be changed at all. For example:
using System;
class Foo
{
static void Main()
{
string[] x = new string[10];
string[] y = x;
Array.Resize(ref x, 20);
Console.WriteLine(x.Length); // Prints out 20
Console.WriteLine(y.Length); // Still prints out 10
}
}
Proof that this is the equivalent of Redim Preserve:
Imports System
Class Foo
Shared Sub Main()
Dim x(9) as String
Dim y as String() = x
Redim Preserve x(19)
Console.WriteLine(x.Length)
Console.WriteLine(y.Length)
End Sub
End Class
The two programs are equivalent.
If you truly want a dynamically sized collection, you should use List<T> (or something similar). There are various issues with using arrays directly - see Eric Lippert's blog post for details. That's not to say you should always avoid them, by any means - but you need to know what you're dealing with.
Use ArrayLists or Generics instead
Use a List<T>. It will dynamically size as needed.
You really shouldn't be using ReDim, it can be very expensive. I prefer List(Of T), but there are many options in this area.
That said, you had a question and here is your answer.
x = (int[]) Utils.CopyArray((Array) x, new int[10]);
I couldn't help but notice that none of the above answers approach the concept of multidimensional arrays. That being said, here's an example. The array in question is predefined as x.
int[,] temp = new int[newRows, newCols];
int minRows = Math.Min(newRows, x.GetUpperBound(0) + 1);
int minCols = Math.Min(newCols, x.GetUpperBound(1) + 1);
for (int i = 0; i < minRows ; ++i)
for (int j = 0; j < minCols; ++j)
temp[i, j] = x[i, j];
x = temp;
Just for fun, here's one way to use generics in order to redim/extend a unidimensional array (add one more "row") :
static T[] Redim<T>(T[] arr, bool preserved)
{
int arrLength = arr.Length;
T[] arrRedimed = new T[arrLength + 1];
if (preserved)
{
for (int i = 0; i < arrLength; i++)
{
arrRedimed[i] = arr[i];
}
}
return arrRedimed;
}
And one to add n rows (though this doesn't prevent user from undersizing the array, which will throw an error in the for loop) :
static T[] Redim<T>(T[] arr, bool preserved, int nbRows)
{
T[] arrRedimed = new T[nbRows];
if (preserved)
{
for (int i = 0; i < arr.Length; i++)
{
arrRedimed[i] = arr[i];
}
}
return arrRedimed;
}
I'm sure you get the idea.
For a multidimensional array (two dimensions), here's one possibility:
static T[,] Redim<T>(T[,] arr, bool preserved)
{
int Ubound0 = arr.GetUpperBound(0);
int Ubound1 = arr.GetUpperBound(1);
T[,] arrRedimed = new T[Ubound0 + 1, Ubound1];
if (preserved)
{
for (int j = 0; j < Ubound1; j++)
{
for (int i = 0; i < Ubound0; i++)
{
arrRedimed[i, j] = arr[i, j];
}
}
}
return arrRedimed;
}
In your program, use this with or even without the type specified, the compiler will recognize it :
int[] myArr = new int[10];
myArr = Redim<int>(myArr, true);
or
int[] myArr = new int[10];
myArr = Redim(myArr, true);
Not sure if all this is really relevant though. =D
Please feel free to correct me or improve my code. ;)
Even though it's a long time ago it might help someone looking for a simple solution - I found something great in another forum:
//from Applied Microsoft.NET framework Programming - Jeffrey Richter
public static Array RedimPreserve(Array origArray, Int32 desiredSize)
{
System.Type t = origArray.GetType().GetElementType();
Array newArray = Array.CreateInstance(t, desiredSize);
Array.Copy(origArray, 0, newArray, 0, Math.Min(origArray.Length, desiredSize));
return newArray;
}
Source: https://social.msdn.microsoft.com/Forums/en-US/6759816b-d525-4752-a3c8-9eb5f4a5b194/redim-in-c?forum=csharplanguage