I would like to get a byte[] from a float[] as quickly as possible, without looping through the whole array (via a cast, probably). Unsafe code is fine. Thanks!
I am looking for a byte array 4 time longer than the float array (the dimension of the byte array will be 4 times that of the float array, since each float is composed of 4 bytes). I'll pass this to a BinaryWriter.
EDIT:
To those critics screaming "premature optimization":
I have benchmarked this using ANTS profiler before I optimized. There was a significant speed increase because the file has a write-through cache and the float array is exactly sized to match the sector size on the disk. The binary writer wraps a file handle created with pinvoke'd win32 API. The optimization occurs since this lessens the number of function calls.
And, with regard to memory, this application creates massive caches which use plenty of memory. I can allocate the byte buffer once and re-use it many times--the double memory usage in this particular instance amounts to a roundoff error in the overall memory consumption of the app.
So I guess the lesson here is not to make premature assumptions ;)
There is a dirty fast (not unsafe code) way of doing this:
[StructLayout(LayoutKind.Explicit)]
struct BytetoDoubleConverter
{
[FieldOffset(0)]
public Byte[] Bytes;
[FieldOffset(0)]
public Double[] Doubles;
}
//...
static Double Sum(byte[] data)
{
BytetoDoubleConverter convert = new BytetoDoubleConverter { Bytes = data };
Double result = 0;
for (int i = 0; i < convert.Doubles.Length / sizeof(Double); i++)
{
result += convert.Doubles[i];
}
return result;
}
This will work, but I'm not sure of the support on Mono or newer versions of the CLR. The only strange thing is that the array.Length is the bytes length. This can be explained because it looks at the array length stored with the array, and because this array was a byte array that length will still be in byte length. The indexer does think about the Double being eight bytes large so no calculation is necessary there.
I've looked for it some more, and it's actually described on MSDN, How to: Create a C/C++ Union by Using Attributes (C# and Visual Basic), so chances are this will be supported in future versions. I am not sure about Mono though.
Premature optimization is the root of all evil! #Vlad's suggestion to iterate over each float is a much more reasonable answer than switching to a byte[]. Take the following table of runtimes for increasing numbers of elements (average of 50 runs):
Elements BinaryWriter(float) BinaryWriter(byte[])
-----------------------------------------------------------
10 8.72ms 8.76ms
100 8.94ms 8.82ms
1000 10.32ms 9.06ms
10000 32.56ms 10.34ms
100000 213.28ms 739.90ms
1000000 1955.92ms 10668.56ms
There is little difference between the two for small numbers of elements. Once you get into the huge number of elements range, the time spent copying from the float[] to the byte[] far outweighs the benefits.
So go with what is simple:
float[] data = new float[...];
foreach(float value in data)
{
writer.Write(value);
}
There is a way which avoids memory copying and iteration.
You can use a really ugly hack to temporary change your array to another type using (unsafe) memory manipulation.
I tested this hack in both 32 & 64 bit OS, so it should be portable.
The source + sample usage is maintained at https://gist.github.com/1050703 , but for your convenience I'll paste it here as well:
public static unsafe class FastArraySerializer
{
[StructLayout(LayoutKind.Explicit)]
private struct Union
{
[FieldOffset(0)] public byte[] bytes;
[FieldOffset(0)] public float[] floats;
}
[StructLayout(LayoutKind.Sequential, Pack = 1)]
private struct ArrayHeader
{
public UIntPtr type;
public UIntPtr length;
}
private static readonly UIntPtr BYTE_ARRAY_TYPE;
private static readonly UIntPtr FLOAT_ARRAY_TYPE;
static FastArraySerializer()
{
fixed (void* pBytes = new byte[1])
fixed (void* pFloats = new float[1])
{
BYTE_ARRAY_TYPE = getHeader(pBytes)->type;
FLOAT_ARRAY_TYPE = getHeader(pFloats)->type;
}
}
public static void AsByteArray(this float[] floats, Action<byte[]> action)
{
if (floats.handleNullOrEmptyArray(action))
return;
var union = new Union {floats = floats};
union.floats.toByteArray();
try
{
action(union.bytes);
}
finally
{
union.bytes.toFloatArray();
}
}
public static void AsFloatArray(this byte[] bytes, Action<float[]> action)
{
if (bytes.handleNullOrEmptyArray(action))
return;
var union = new Union {bytes = bytes};
union.bytes.toFloatArray();
try
{
action(union.floats);
}
finally
{
union.floats.toByteArray();
}
}
public static bool handleNullOrEmptyArray<TSrc,TDst>(this TSrc[] array, Action<TDst[]> action)
{
if (array == null)
{
action(null);
return true;
}
if (array.Length == 0)
{
action(new TDst[0]);
return true;
}
return false;
}
private static ArrayHeader* getHeader(void* pBytes)
{
return (ArrayHeader*)pBytes - 1;
}
private static void toFloatArray(this byte[] bytes)
{
fixed (void* pArray = bytes)
{
var pHeader = getHeader(pArray);
pHeader->type = FLOAT_ARRAY_TYPE;
pHeader->length = (UIntPtr)(bytes.Length / sizeof(float));
}
}
private static void toByteArray(this float[] floats)
{
fixed(void* pArray = floats)
{
var pHeader = getHeader(pArray);
pHeader->type = BYTE_ARRAY_TYPE;
pHeader->length = (UIntPtr)(floats.Length * sizeof(float));
}
}
}
And the usage is:
var floats = new float[] {0, 1, 0, 1};
floats.AsByteArray(bytes =>
{
foreach (var b in bytes)
{
Console.WriteLine(b);
}
});
If you do not want any conversion to happen, I would suggest Buffer.BlockCopy().
public static void BlockCopy(
Array src,
int srcOffset,
Array dst,
int dstOffset,
int count
)
For example:
float[] floatArray = new float[1000];
byte[] byteArray = new byte[floatArray.Length * 4];
Buffer.BlockCopy(floatArray, 0, byteArray, 0, byteArray.Length);
You're better-off letting the BinaryWriter do this for you. There's going to be iteration over your entire set of data regardless of which method you use, so there's no point in playing with bytes.
Although you can obtain a byte* pointer using unsafe and fixed, you cannot convert the byte* to byte[] in order for the writer to accept it as a parameter without performing data copy. Which you do not want to do as it will double your memory footprint and add an extra iteration over the inevitable iteration that needs to be performed in order to output the data to disk.
Instead, you are still better off iterating over the array of floats and writing each float to the writer individually, using the Write(double) method. It will still be fast because of buffering inside the writer. See sixlettervariables's numbers.
Using the new Span<> in .Net Core 2.1 or later...
byte[] byteArray2 = MemoryMarshal.Cast<float, byte>(floatArray).ToArray();
Or, if Span can be used instead, then a direct reinterpret cast can be done: (very fast - zero copying)
Span<byte> byteArray3 = MemoryMarshal.Cast<float, byte>(floatArray);
// with span we can get a byte, set a byte, iterate, and more.
byte someByte = byteSpan[2];
byteSpan[2] = 33;
I did some crude benchmarks. The time taken for each is in the comments. [release/no debugger/x64]
float[] floatArray = new float[100];
for (int i = 0; i < 100; i++) floatArray[i] = i * 7.7777f;
Stopwatch start = Stopwatch.StartNew();
for (int j = 0; j < 100; j++)
{
start.Restart();
for (int k = 0; k < 1000; k++)
{
Span<byte> byteSpan = MemoryMarshal.Cast<float, byte>(floatArray);
}
long timeTaken1 = start.ElapsedTicks; ////// 0 ticks //////
start.Restart();
for (int k = 0; k < 1000; k++)
{
byte[] byteArray2 = MemoryMarshal.Cast<float, byte>(floatArray).ToArray();
}
long timeTaken2 = start.ElapsedTicks; ////// 26 ticks //////
start.Restart();
for (int k = 0; k < 1000; k++)
{
byte[] byteArray = new byte[sizeof(float) * floatArray.Length];
for (int i = 0; i < floatArray.Length; i++)
BitConverter.GetBytes(floatArray[i]).CopyTo(byteArray, i * sizeof(float));
}
long timeTaken3 = start.ElapsedTicks; ////// 1310 ticks //////
start.Restart();
for (int k = 0; k < 1000; k++)
{
byte[] byteArray = new byte[sizeof(float) * floatArray.Length];
Buffer.BlockCopy(floatArray, 0, byteArray, 0, byteArray.Length);
}
long timeTaken4 = start.ElapsedTicks; ////// 33 ticks //////
start.Restart();
for (int k = 0; k < 1000; k++)
{
byte[] byteArray = new byte[sizeof(float) * floatArray.Length];
MemoryStream memStream = new MemoryStream();
BinaryWriter writer = new BinaryWriter(memStream);
foreach (float value in floatArray)
writer.Write(value);
writer.Close();
}
long timeTaken5 = start.ElapsedTicks; ////// 1080 ticks //////
Console.WriteLine($"{timeTaken1/10,6} {timeTaken2 / 10,6} {timeTaken3 / 10,6} {timeTaken4 / 10,6} {timeTaken5 / 10,6} ");
}
We have a class called LudicrousSpeedSerialization and it contains the following unsafe method:
static public byte[] ConvertFloatsToBytes(float[] data)
{
int n = data.Length;
byte[] ret = new byte[n * sizeof(float)];
if (n == 0) return ret;
unsafe
{
fixed (byte* pByteArray = &ret[0])
{
float* pFloatArray = (float*)pByteArray;
for (int i = 0; i < n; i++)
{
pFloatArray[i] = data[i];
}
}
}
return ret;
}
Although it basically does do a for loop behind the scenes, it does do the job in one line
byte[] byteArray = floatArray.Select(
f=>System.BitConverter.GetBytes(f)).Aggregate(
(bytes, f) => {List<byte> temp = bytes.ToList(); temp.AddRange(f); return temp.ToArray(); });
Related
As my goal is to out perform the List<T>
i am testing arrays and found few starting points to get on testing
i have tested this before trying to capture bitmaps off screen,
and tests proved the usage is suffice.
my question is what data types could use this Copy() code except for byte[]
say i want a data storage unit to take the advantage of unmanaged / unsafe
public unsafe struct NusT
{
public unsafe int vi;
public unsafe bool vb;
}
instead of populating a list
i initialise the struct as follows : 1)
NusT n;
n.vi= 90;
n.vb=true
i have tested this after testing the folowing: 2)
NusT n = new NusT(){vi=90, vb=true};
this test was after testing :3)
NusT n = new NusT("90", true);
i think both last had same results but the first one is blazing fast, as i do not create an object so
NusT n-> instructions- 1
n.vi=90 -> instructions- 1
n.vb=true -> instructions- 1
now i minimized what i could and this started at the begining with a class:
whitch was even worse than 2 & 3 above as it also uses properties
class bigAndSlow
{
public int a { get; private set;}
public bool b { get; private set;}
public string c { get; private set;}
public bigAndSlow(int .. ,boo .. , string.. )
{
initialise ...
}
}
so now when the final decision is
public unsafe struct NusT
{
public unsafe int vi;
public unsafe bool vb;
}
how can i implement this blazingly fast data unit to use Copy() on
NusT[] NustyArr;
static unsafe void Copy(byte[] src, int srcIndex,
byte[] dst, int dstIndex, int count)
{
if (src == null || srcIndex < 0 ||
dst == null || dstIndex < 0 || count < 0)
{
throw new ArgumentException();
}
int srcLen = src.Length;
int dstLen = dst.Length;
if (srcLen - srcIndex < count ||
dstLen - dstIndex < count)
{
throw new ArgumentException();
}
// The following fixed statement pins the location of
// the src and dst objects in memory so that they will
// not be moved by garbage collection.
fixed (byte* pSrc = src, pDst = dst)
{
byte* ps = pSrc;
byte* pd = pDst;
// Loop over the count in blocks of 4 bytes, copying an
// integer (4 bytes) at a time:
for (int n = 0; n < count / 4; n++)
{
*((int*)pd) = *((int*)ps);
pd += 4;
ps += 4;
}
// Complete the copy by moving any bytes that weren't
// moved in blocks of 4:
for (int n = 0; n < count % 4; n++)
{
*pd = *ps;
pd++;
ps++;
}
}
}
static void Main(string[] args)
{
byte[] a = new byte[100];
byte[] b = new byte[100];
for (int i = 0; i < 100; ++i)
a[i] = (byte)i;
Copy(a, 0, b, 0, 100);
Console.WriteLine("The first 10 elements are:");
for (int i = 0; i < 10; ++i)
Console.Write(b[i] + " ");
Console.WriteLine("\n");
}
Yes, you can do this with any blittable type. The blittable types are primitive types (integer and float types, but not bool), one-dimensional arrays of blittable types and structures containing fields of blittable types only.
The structure NusT is not blittable because it contains bool field. Just change it to byte and you will get a blittable structure for which you can obtain a pointer.
Here is the code that works for any type:
static unsafe void UnsafeCopy<T>(T[] src, int srcIndex, T[] dst, int dstIndex, int count) where T : struct
{
if (src == null || srcIndex < 0 || dst == null || dstIndex < 0 || count < 0 || srcIndex + count > src.Length || dstIndex + count > dst.Length)
{
throw new ArgumentException();
}
int elem_size = Marshal.SizeOf(typeof(T));
GCHandle gch1 = GCHandle.Alloc(src, GCHandleType.Pinned);
GCHandle gch2 = GCHandle.Alloc(dst, GCHandleType.Pinned);
byte* ps = (byte*)gch1.AddrOfPinnedObject().ToPointer() + srcIndex * elem_size;
byte* pd = (byte*)gch2.AddrOfPinnedObject().ToPointer() + dstIndex * elem_size;
int len = count * elem_size;
try
{
// Loop over the count in blocks of 4 bytes, copying an
// integer (4 bytes) at a time:
for (int n = 0; n < len / 4; n++)
{
*((int*)pd) = *((int*)ps);
pd += 4;
ps += 4;
}
// Complete the copy by moving any bytes that weren't
// moved in blocks of 4:
for (int n = 0; n < len % 4; n++)
{
*pd = *ps;
pd++;
ps++;
}
}
finally
{
gch1.Free();
gch2.Free();
}
}
But I strongly advice you to use Array.Copy. It is already the most efficient way to copy arrays. See the benchmarks of copying array of 1M elements below:
byte[] Array.Copy: 57,491 us
byte[] FastCopy: 138,198 us
byte[] JustCopy: 792,399 us
byte[] UnsafeCopy: 138,575 us
byte[] MemCpy: 57,667 us
NusT[] Array.Copy: 1,197 ms
NusT[] JustCopy: 1,843 ms
NusT[] UnsafeCopy: 1,550 ms
NusT[] MemCpy: 1,208 ms
FastCopy is your copy function, UnsafeCopy is my templated function, JustCopy is a simple implementation for (int i = 0; i < src.Length; i++) dst[i] = src[i];. MemCpy is PInvoke call of msvcrt memcpy function.
The verdict is: using pointers in C# for performance improvement is a bad practice. JIT does not optimize the unsafe code. The best solution is to move performance critical code to native DLLs.
I am working with Unity 4.5, grabbing images as bytes arrays (each byte represent a channel, taking 4 bytes per pixel (rgba) and displaying them on a texture converting the array to a Color32 array, using this loop:
img = new Color32[byteArray.Length / nChannels]; //nChannels being 4
for (int i=0; i< img.Length; i++) {
img[i].r = byteArray[i*nChannels];
img[i].g = byteArray[i*nChannels+1];
img[i].b = byteArray[i*nChannels+2];
img[i].a = byteArray[i*nChannels+3];
}
Then, it is applied to the texture using:
tex.SetPixels32(img);
However, this slows down the application significantly (this loop is executed on every single frame), and I would like to know if there is any other way to speed up the copying process. I've found some people (Fast copy of Color32[] array to byte[] array) using the Marshal.Copy functions in order to do the reverse process (Color32 to byte array), but I have not been able to make it work to copy a byte array to a Color32 array. Does anybody know a faster way?
Thank you in advance!
Yes, Marshal.Copy is the way to go. I've answered a similar question here.
Here's a generic method to copy from struct[] to byte[] and vice versa
private static byte[] ToByteArray<T>(T[] source) where T : struct
{
GCHandle handle = GCHandle.Alloc(source, GCHandleType.Pinned);
try
{
IntPtr pointer = handle.AddrOfPinnedObject();
byte[] destination = new byte[source.Length * Marshal.SizeOf(typeof(T))];
Marshal.Copy(pointer, destination, 0, destination.Length);
return destination;
}
finally
{
if (handle.IsAllocated)
handle.Free();
}
}
private static T[] FromByteArray<T>(byte[] source) where T : struct
{
T[] destination = new T[source.Length / Marshal.SizeOf(typeof(T))];
GCHandle handle = GCHandle.Alloc(destination, GCHandleType.Pinned);
try
{
IntPtr pointer = handle.AddrOfPinnedObject();
Marshal.Copy(source, 0, pointer, source.Length);
return destination;
}
finally
{
if (handle.IsAllocated)
handle.Free();
}
}
Use it as:
[StructLayout(LayoutKind.Sequential)]
public struct Demo
{
public double X;
public double Y;
}
private static void Main()
{
Demo[] array = new Demo[2];
array[0] = new Demo { X = 5.6, Y = 6.6 };
array[1] = new Demo { X = 7.6, Y = 8.6 };
byte[] bytes = ToByteArray(array);
Demo[] array2 = FromByteArray<Demo>(bytes);
}
This code requires unsafe switch but should be fast. I think you should benchmark these answers...
var bytes = new byte[] { 1, 2, 3, 4 };
var colors = MemCopyUtils.ByteArrayToColor32Array(bytes);
public class MemCopyUtils
{
unsafe delegate void MemCpyDelegate(byte* dst, byte* src, int len);
static MemCpyDelegate MemCpy;
static MemCopyUtils()
{
InitMemCpy();
}
static void InitMemCpy()
{
var mi = typeof(Buffer).GetMethod(
name: "Memcpy",
bindingAttr: BindingFlags.NonPublic | BindingFlags.Static,
binder: null,
types: new Type[] { typeof(byte*), typeof(byte*), typeof(int) },
modifiers: null);
MemCpy = (MemCpyDelegate)Delegate.CreateDelegate(typeof(MemCpyDelegate), mi);
}
public unsafe static Color32[] ByteArrayToColor32Array(byte[] bytes)
{
Color32[] colors = new Color32[bytes.Length / sizeof(Color32)];
fixed (void* tempC = &colors[0])
fixed (byte* pBytes = bytes)
{
byte* pColors = (byte*)tempC;
MemCpy(pColors, pBytes, bytes.Length);
}
return colors;
}
}
Using Parallel.For may give you a significant performance increase.
img = new Color32[byteArray.Length / nChannels]; //nChannels being 4
Parallel.For(0, img.Length, i =>
{
img[i].r = byteArray[i*nChannels];
img[i].g = byteArray[i*nChannels+1];
img[i].b = byteArray[i*nChannels+2];
img[i].a = byteArray[i*nChannels+3];
});
Example on MSDN
I haven't profiled it, but using fixed to ensure your memory doesn't get moved around and to remove bounds checks on array accesses might provide some benefit:
img = new Color32[byteArray.Length / nChannels]; //nChannels being 4
fixed (byte* ba = byteArray)
{
fixed (Color32* c = img)
{
byte* byteArrayPtr = ba;
Color32* colorPtr = c;
for (int i = 0; i < img.Length; i++)
{
(*colorPtr).r = *byteArrayPtr++;
(*colorPtr).g = *byteArrayPtr++;
(*colorPtr).b = *byteArrayPtr++;
(*colorPtr).a = *byteArrayPtr++;
colorPtr++;
}
}
}
It might not provide much more benefit on 64-bit systems - I believe that the bounds checking is is more highly optimized. Also, this is an unsafe operation, so take care.
public Color32[] GetColorArray(byte[] myByte)
{
if (myByte.Length % 1 != 0)
throw new Exception("Must have an even length");
var colors = new Color32[myByte.Length / nChannels];
for (var i = 0; i < myByte.Length; i += nChannels)
{
colors[i / nChannels] = new Color32(
(byte)(myByte[i] & 0xF8),
(byte)(((myByte[i] & 7) << 5) | ((myByte[i + 1] & 0xE0) >> 3)),
(byte)((myByte[i + 1] & 0x1F) << 3),
(byte)1);
}
return colors;
}
Worked about 30-50 times faster than just i++. The "extras" is just styling. This code is doing, in one "line", in the for loop, what you're declaring in 4 lines plus it is much quicker. Cheers :)
Referenced + Referenced code: Here
I am writing a live-video imaging application and need to speed up this method. It's currently taking about 10ms to execute and I'd like to get it down to 2-3ms.
I've tried both Array.Copy and Buffer.BlockCopy and they both take ~30ms which is 3x longer than the manual copy.
One thought was to somehow copy 4 bytes as an integer and then paste them as an integer, thereby reducing 4 lines of code to one line of code. However, I'm not sure how to do that.
Another thought was to somehow use pointers and unsafe code to do this, but I'm not sure how to do that either.
All help is much appreciated. Thank you!
EDIT: Array sizes are: inputBuffer[327680], lookupTable[16384], outputBuffer[1310720]
public byte[] ApplyLookupTableToBuffer(byte[] lookupTable, ushort[] inputBuffer)
{
System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
sw.Start();
// Precalculate and initialize the variables
int lookupTableLength = lookupTable.Length;
int bufferLength = inputBuffer.Length;
byte[] outputBuffer = new byte[bufferLength * 4];
int outIndex = 0;
int curPixelValue = 0;
// For each pixel in the input buffer...
for (int curPixel = 0; curPixel < bufferLength; curPixel++)
{
outIndex = curPixel * 4; // Calculate the corresponding index in the output buffer
curPixelValue = inputBuffer[curPixel] * 4; // Retrieve the pixel value and multiply by 4 since the lookup table has 4 values (blue/green/red/alpha) for each pixel value
// If the multiplied pixel value falls within the lookup table...
if ((curPixelValue + 3) < lookupTableLength)
{
// Copy the lookup table value associated with the value of the current input buffer location to the output buffer
outputBuffer[outIndex + 0] = lookupTable[curPixelValue + 0];
outputBuffer[outIndex + 1] = lookupTable[curPixelValue + 1];
outputBuffer[outIndex + 2] = lookupTable[curPixelValue + 2];
outputBuffer[outIndex + 3] = lookupTable[curPixelValue + 3];
//System.Buffer.BlockCopy(lookupTable, curPixelValue, outputBuffer, outIndex, 4); // Takes 2-10x longer than just copying the values manually
//Array.Copy(lookupTable, curPixelValue, outputBuffer, outIndex, 4); // Takes 2-10x longer than just copying the values manually
}
}
Debug.WriteLine("ApplyLookupTableToBuffer(ms): " + sw.Elapsed.TotalMilliseconds.ToString("N2"));
return outputBuffer;
}
EDIT: I've updated the method keeping the same variable names so others can see how the code would translate based on HABJAN's solution below.
public byte[] ApplyLookupTableToBufferV2(byte[] lookupTable, ushort[] inputBuffer)
{
System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
sw.Start();
// Precalculate and initialize the variables
int lookupTableLength = lookupTable.Length;
int bufferLength = inputBuffer.Length;
byte[] outputBuffer = new byte[bufferLength * 4];
//int outIndex = 0;
int curPixelValue = 0;
unsafe
{
fixed (byte* pointerToOutputBuffer = &outputBuffer[0])
fixed (byte* pointerToLookupTable = &lookupTable[0])
{
// Cast to integer pointers since groups of 4 bytes get copied at once
uint* lookupTablePointer = (uint*)pointerToLookupTable;
uint* outputBufferPointer = (uint*)pointerToOutputBuffer;
// For each pixel in the input buffer...
for (int curPixel = 0; curPixel < bufferLength; curPixel++)
{
// No need to multiply by 4 on the following 2 lines since the pointers are for integers, not bytes
// outIndex = curPixel; // This line is commented since we can use curPixel instead of outIndex
curPixelValue = inputBuffer[curPixel]; // Retrieve the pixel value
if ((curPixelValue + 3) < lookupTableLength)
{
outputBufferPointer[curPixel] = lookupTablePointer[curPixelValue];
}
}
}
}
Debug.WriteLine("2 ApplyLookupTableToBuffer(ms): " + sw.Elapsed.TotalMilliseconds.ToString("N2"));
return outputBuffer;
}
I did some tests, and I managed to achieve max speed by turning my code into unsafe along with using the RtlMoveMemory API. I figured out that Buffer.BlockCopy and Array.Copy were much slower than direct RtlMoveMemory usage.
So, at the end you will end up with something like this:
fixed(byte* ptrOutput= &outputBufferBuffer[0])
{
MoveMemory(ptrOutput, ptrInput, 4);
}
[DllImport("Kernel32.dll", EntryPoint = "RtlMoveMemory", SetLastError = false)]
private static unsafe extern void MoveMemory(void* dest, void* src, int size);
EDIT:
Ok, now once when I figured out your logic and when I did some tests, I managed to speed up your method for almost up to 50%. Since you need to copy a small data blocks (always 4 bytes), yes, you were right, RtlMoveMemory wont help here and it's better to copy data as integer. Here is the final solution I came up with:
public static byte[] ApplyLookupTableToBufferV2(byte[] lookupTable, ushort[] inputBuffer)
{
int lookupTableLength = lookupTable.Length;
int bufferLength = inputBuffer.Length;
byte[] outputBuffer = new byte[bufferLength * 4];
int outIndex = 0, curPixelValue = 0;
unsafe
{
fixed (byte* ptrOutput = &outputBuffer[0])
fixed (byte* ptrLookup = &lookupTable[0])
{
uint* lkp = (uint*)ptrLookup;
uint* opt = (uint*)ptrOutput;
for (int index = 0; index < bufferLength; index++)
{
outIndex = index;
curPixelValue = inputBuffer[index];
if ((curPixelValue + 3) < lookupTableLength)
{
opt[outIndex] = lkp[curPixelValue];
}
}
}
}
return outputBuffer;
}
I renamed your method to ApplyLookupTableToBufferV1.
And here are my test result:
int tc1 = Environment.TickCount;
for (int i = 0; i < 200; i++)
{
byte[] a = ApplyLookupTableToBufferV1(lt, ib);
}
tc1 = Environment.TickCount - tc1;
Console.WriteLine("V1: " + tc1.ToString() + "ms");
Result - V1: 998 ms
int tc2 = Environment.TickCount;
for (int i = 0; i < 200; i++)
{
byte[] a = ApplyLookupTableToBufferV2(lt, ib);
}
tc2 = Environment.TickCount - tc2;
Console.WriteLine("V2: " + tc2.ToString() + "ms");
Result - V2: 473 ms
is it possible to somehow cast the type of a pointer created by the fixed() statement?
This is the situation:
I have an array of byte, which i would like to iterate through, however i would like the values to be treated as int, thus having an int* instead of a byte*.
Here's some exemplary code:
byte[] rawdata = new byte[1024];
fixed(int* ptr = rawdata) //this fails with an implicit cast error
{
for(int i = idx; i < rawdata.Length; i++)
{
//do some work here
}
}
Can this be done without having to do the cast inside the iteration?
byte[] rawdata = new byte[1024];
fixed(byte* bptr = rawdata)
{
int* ptr=(int*)bptr;
for(int i = idx; i < rawdata.Length; i++)
{
//do some work here
}
}
I believe you have to go via a byte*. For example:
using System;
class Test
{
unsafe static void Main()
{
byte[] rawData = new byte[1024];
rawData[0] = 1;
rawData[1] = 2;
fixed (byte* bytePtr = rawData)
{
int* intPtr = (int*) bytePtr;
Console.WriteLine(intPtr[0]); // Prints 513 on my box
}
}
}
Note that when iterating, you should use rawData.Length / 4, not rawData.Length if you're treating your byte array as a sequence of 32-bit values.
I found a - seemingly - more elegant and for some reason also faster way of doing this:
byte[] rawData = new byte[1024];
GCHandle rawDataHandle = GCHandle.Alloc(rawData, GCHandleType.Pinned);
int* iPtr = (int*)rawDataHandle.AddrOfPinnedObject().ToPointer();
int length = rawData.Length / sizeof (int);
for (int idx = 0; idx < length; idx++, iPtr++)
{
(*iPtr) = idx;
Console.WriteLine("Value of integer at pointer position: {0}", (*iPtr));
}
rawDataHandle.Free();
This way the only thing i need to do - apart from setting the correct iteration length - is increment the pointer. I compared the code with the one using the fixed statement, and this one is slightly faster.
How to write bits (not bytes) to a file with c#, .net? I'm preety stuck with it.
Edit: i'm looking for a different way that just writing every 8 bits as a byte
The smallest amount of data you can write at one time is a byte.
If you need to write individual bit-values. (Like for instance a binary format that requires a 1 bit flag, a 3 bit integer and a 4 bit integer); you would need to buffer the individual values in memory and write to the file when you have a whole byte to write. (For performance, it makes sense to buffer more and write larger chunks to the file).
Accumulate the bits in a buffer (a single byte can qualify as a "buffer")
When adding a bit, left-shift the buffer and put the new bit in the lowest position using OR
Once the buffer is full, append it to the file
I've made something like this to emulate a BitsWriter.
private BitArray bitBuffer = new BitArray(new byte[65536]);
private int bitCount = 0;
// Write one int. In my code, this is a byte
public void write(int b)
{
BitArray bA = new BitArray((byte)b);
int[] pattern = new int[8];
writeBitArray(bA);
}
// Write one bit. In my code, this is a binary value, and the amount of times
public void write(int b, int len)
{
int[] pattern = new int[len];
BitArray bA = new BitArray(len);
for (int i = 0; i < len; i++)
{
bA.Set(i, (b == 1));
}
writeBitArray(bA);
}
private void writeBitArray(BitArray bA)
{
for (int i = 0; i < bA.Length; i++)
{
bitBuffer.Set(bitCount + i, bA[i]);
bitCount++;
}
if (bitCount % 8 == 0)
{
BitArray bitBufferWithLength = new BitArray(new byte[bitCount / 8]);
byte[] res = new byte[bitBuffer.Count / 8];
for (int i = 0; i < bitCount; i++)
{
bitBufferWithLength.Set(i, (bitBuffer[i]));
}
bitBuffer.CopyTo(res, 0);
bitCount = 0;
base.BaseStream.Write(res, 0, res.Length);
}
}
You will have to use bitshifts or binary arithmetic, as you can only write one byte at a time, not individual bits.