I've 4 bytes of data and need an 8 bytes array for a security operation. I should produce these 8 bytes form the 4 bytes byte array and this should be reproducible.
I was thinking of using exact byte array and adding 4 extra bytes and fill them with AND, OR, XOR... of the initial array in a known sequence. I'm not sure if it's a good idea. I just need an 8 byte array from this 4 bytes and the operation should be reproducible (same 8 bytes with same given 4 bytes). Please give an example in C#
Why not just pad the existing 4 bytes with another 4 bytes of zeroes? Or repeat the original 4 bytes. For example:
static byte[] Pad(byte[] input)
{
// Alternatively use Array.Resize
byte[] output = new byte[input.Length + 4];
Buffer.BlockCopy(input, 0, output, 0, input.Length);
return output;
}
static byte[] Repeat(byte[] input)
{
byte[] output = new byte[input.Length * 2];
Buffer.BlockCopy(input, 0, output, 0, input.Length);
Buffer.BlockCopy(input, 0, output, input.Length, input.Length);
return output;
}
Both of these fulfil your original criteria, I believe... but I suspect you're looking for something else. If that's the case, you need to be explicit about what you need.
EDIT: As I've said in the comments, you're basically not adding any real security here - padding will make that clearer, IMO. On the other hand, if you do want some security-through-obscurity, you could find a random number generator that allows seeding, and use that as a starting point. For example:
// Don't use this - see below. Just the concept...
int seed = BitConverter.ToInt32(input, 0); // TODO: Cope with endianness
Random rng = new Random(seed);
byte[] output = new byte[8];
Buffer.BlockCopy(input, 0, output, 0, 4);
for (int i = 4; i < 8; i++) {
output[i] = (byte) rng.Next(256);
}
Now, the reason I've got the comment above is that you probably need an algorithm which is guaranteed not to change between versions of .NET. Find code to something like the Mersenne Twister, for exmaple.
There are multiple methods of doing padding for block ciphers.
This Wikipedia article covers some of the more accepted solutions: http://en.wikipedia.org/wiki/Padding_(cryptography)#Padding_methods
Barring any other considerations, I would use PKCS#7 padding.
How about
bytes.Concat(bytes)
I would be very careful. If a security operation requires 64 bits worth of data it is probably because it requires that much data. If you create your 64 bits from 32 bits with a known reproducible formula you will still only have 32 bits worth of data.
If the security is not affected by the data you have you can just fill the the remaining four bytes with ones or zeros. But you should really try to get 8 bytes of "real" data.
Related
So I've been trying to add bits of a value to a MemoryStream but the issue is I have no idea how. I've seen that it's used for performance when it comes to networking.
I know I want a function that takes the bit value and how many bits it takes to store that value. So for instance, to store the value 3 I would need to allocate 2 bits 0000 0000 0000 0011. I would essentially pack the bits into a byte array and then add that byte array to the MemoryStream
var ms = new MemoryStream();
ms.WriteByte(1);
ms.WriteByte(1);
ms.WriteByte(1);
ms.WriteByte(1);
ms.WriteByte(1);
WriteBits(2, 3);
WriteBits(1, 1);
void WriteBits(int numbBits, int value)
{
/* Convert the "value" to a byte or bytes and add it to the MemoryStream */
}
How do I properly implement this?
Java Example
public void writeBits(int numBits, int value) {
int bytePos = bitPosition >> 3;
int bitOffset = 8 - (bitPosition & 7);
bitPosition += numBits;
for (; numBits > bitOffset; bitOffset = 8) {
buffer[bytePos] &= ~bitMaskOut[bitOffset]; // mask out the desired area
buffer[bytePos++] |= (value >> (numBits - bitOffset))
& bitMaskOut[bitOffset];
numBits -= bitOffset;
}
if (numBits == bitOffset) {
buffer[bytePos] &= ~bitMaskOut[bitOffset];
buffer[bytePos] |= value & bitMaskOut[bitOffset];
} else {
buffer[bytePos] &= ~(bitMaskOut[numBits] << (bitOffset - numBits));
buffer[bytePos] |= (value & bitMaskOut[numBits]) << (bitOffset - numBits);
}
}
So I've been trying to add bits of a value to a MemoryStream
You don't, MemoryStream only handles bytes.
So for instance, to store the value 3 I would need to allocate 2 bits
This would only be true if the range of values you want to store is [0, 3]. If you want the possibility of storing any larger value you need more bits.
How do I properly implement this?
you would need to implement your own bit-stream. The java example looks like it has a byte[] buffer, and a bitPosition. You would need to implement this. The bif-fiddeling code looks like it should work just about the same in c#. Once you have a byte[] it is trivial to write this out to whatever stream you want, and usually possible to send directly over the network.
I've seen that it's used for performance when it comes to networking
I think there is a significant misunderstanding here. While you could manually manipulate individual bits, in most cases it would just be a waste of (development) time.
In general, a better way to get good performance is to use existing, well optimized and designed libraries. And there are a variety of serialization libraries that converts objects to byte-streams for you. An example would be protobuf (.net), this actually encodes numbers with a variable number of bytes.
If you still need smaller data it is usually more efficient to use some form of compression. The old classic deflate usually gives good compromise between size and performance, while algorithms like lz4 prioritizes speed over compression ratio.
I had exactly the same problem and wrote an entire BitStream library which can handle any reads and writes of an arbitrary number of bits to a MemoryStream (and any other stream, too). The library is open-source, MIT-licensed and fast (https://github.com/martinweihrauch/BitStream).
Writing bits to a MemoryStream.
These are the steps to write a value to a certain number of bits to a specific position in the MemoryStream:
Have a Stream available, e. g. a MemoryStream(), to which you want to write.
Connect this Stream to a new Bitstream
using SharpBitStream;
uint[] testDataUnsigned = { 5, 62, 17, 50, 33 };
var ms = new MemoryStream();
var bs = new BitStream(ms);
Now, you can start writing to the BitStream like this:
foreach(var bits in testDataUnsigned)
{
bs.WriteUnsigned(6, (ulong)bits);
}
Writing can be done as above by only providing the bitlength and the value, but you of course also have full controll of exactly where to write the bits like so:
bs.WriteUnsigned(3, 2, 4, 5);
// Overloaded signature of WriteUnsigned:
// public void WriteUnsigned(long offsetByteStream, int offsetBit, int bitLength, ulong value)
// For signed numbers (e. g. -17), use
// bs.WriteSigned(3, 2, 4, -5);
This means, you can control that you write to the 4th byte (3, because starting at 0) in the underlying byte Stream,
starting from the the 3rd (=2) position of the byte with a length of 6 bits and the value 5 (=0b0101);
Reading works similarly:
Just read the next 6 bits, wherever your byte and bit position is (e. g. for loops, etc):
ulong number = bs.ReadUnsigned(6);
// For Signed, use
// long number = bs.ReadSigned(6);
Read a specific position, in this example read 4 bits from 3rd byte in Stream (2= 3rd position), starting with bit #0:
ulong number = bs.ReadUnsigned(2, 0, 4);
// For signed, use
// long number = bs.ReadSigned(2, 0, 4);
Note: The bit offset is always counting from 0 from the left-most position.
I am using a Microsoft example for interprocess communication. In the example there are two methods for reading/writing a string to/from a stream. The code sends the length of the string being streamed in the data. I need similar code, but need to make some modifications. An explanation of the highlighted lines would be helpful.
In WriteString(), they taken the length of the byte array being written and divide it by 256. The opposite is done in ReadString(), but an explanation as to why 256 is used would be great. Then, it writes another byte by taking the length and & it with 255. I don't understand the reasoning for this either. I'm thinking it shifts the value, but I don't really understand why this is needed. And then in ReadString() it does a += to the length by reading a byte. An explanation of this would be really helpful. I'm new to streaming and just want to understand exactly what is happening and why.
public string ReadString()
{
int len = 0;
// the next two lines
len = ioStream.ReadByte() * 256;
len += ioStream.ReadByte();
byte[] inBuffer = new byte[len];
ioStream.Read(inBuffer, 0, len);
return streamEncoding.GetString(inBuffer);
}
public int WriteString(string outString)
{
byte[] outBuffer = streamEncoding.GetBytes(outString);
int len = outBuffer.Length;
if (len > UInt16.MaxValue)
{
len = (int)UInt16.MaxValue;
}
// the next to lines
ioStream.WriteByte((byte)(len / 256));
ioStream.WriteByte((byte)(len & 255));
ioStream.Write(outBuffer, 0, len);
ioStream.Flush();
return outBuffer.Length + 2;
}
This code is bad, find a new tutorial.
The 256 stuff is there to convert the low 2 bytes of the length integer to bytes in order to serialize/deserialize them. This is not how it's normally done. Use BinaryReader/Writer or code not based on multiplication but on binary and and shift.
Dividing by 256 is equivalent to x >> 8. Also this only works with positive integers. x & 255 is used to take the lowest byte. This could simply be (byte)x. Sometimes people write x % 256 for this which not idiomatic and has problems with signedness.
Good code would be new byte[] { (byte)(x >> 8), (byte)(x >> 0) } and x = bytes[1] << 8 | bytes[0]. Much simpler, faster and idiomatic. I like writing >> 0 which does nothing for the sake of symmetry. It's optimized away. This might seem ridiculous for 16 bit ints but with longer ints there are 4 or 8 components and having one of them slightly off seems like needless inconsistency.
ioStream.Read(inBuffer, 0, len);
is a bug because it assumes the read completes in one chunk. Need a loop or again BinaryReader.
if (len > UInt16.MaxValue)
{
len = (int)UInt16.MaxValue;
}
I'm going to use this opportunity to warn anyone who reads this: Microsoft .NET sample code often is of extremely poor quality. Read it with great skepticism.
I am a newbie with a question regarding reading specific bytes out of an array in C#. I received an array response that is 128 bytes long and I am looking to read and store the first 4 bytes of the array. How do I do so?
I've read a number of posts about shifting bytes but was a bit confused and I am looking to get going in the right direction.
Use one of the Array.Copy overloads:
byte[] bytes = new byte[4];
Array.Copy(originalArray, bytes, bytes.Length);
Use Array.Copy method:
// your original array
var sourceArray = new byte[128];
// create a new one with the size you need
var firstBytes = new byte[4];
// copy the required number of bytes.
Array.Copy(sourceArray, 0, firstBytes, 0, firstBytes.Length);
If you do not need the rest of the data in the original array, you can also resize it, truncating everything after the first 4 bytes:
Array.Resize(ref sourceArray, 4);
Read the first 4 bytes in 4 different variables:
byte first = array[0],
second = array[1],
third = array[2],
forth = array[4];
Read them as one 32-bit long integer:
int result =
first << 24 ||
second << 16 ||
third << 8 ||
forth;
The x << n operator shifts the bits of number x, n times to the left. You are effectively moving the bytes of first 24 positions to the left, thus first will be stored in bits with indices 24 to 31 like that:
first 00000000 00000000 00000000
Next, move second 16 positions to the left, thus storing it in bits 16 to 23 and so on:
first second 00000000 00000000
Wikipedia provides a good summary of bitwise operations if you need more details.
Example from a network stream.
byte[] bytesRead = 0;
using (NetworkStream stream = client.GetStream())
{
byte[] length = new byte[4];
bytesRead = stream.Read(length, 0, 4);
}
bytesRead contains the first 4 bytes stored in the networkstream.
You could use the Take method. Although, this is only available if you are using .NET 3.5 or later. You will need to have using System.Linq; at the top of the .cs file.
var bytes = originalArray.Take(4).ToArray();
I want to convert a given byte array to an int. Then I want to reverse the process. That is I want to get back the original byte array from that int.
I thought something like this would have worked:
byte[] myBytes = { 0, 0, 0, 32 };
if (BitConverter.IsLittleEndian)
Array.Reverse(myBytes);
int i = BitConverter.ToInt32(myBytes, 0);
Console.WriteLine("int: {0}", i); // Output: 32
byte[] newBytes = BitConverter.GetBytes(i);
Console.WriteLine("byte array: " + BitConverter.ToString(newBytes));
// Outputs: 20-00-00-00
So it doesn't give me back the original byte array. What am I doing wrong?
You're reversing the bytes with Array.Reverse for no obvious reason - given that you're using BitConverter for both conversion to an int and from an int, you don't need the Array.Reverse call at all.
If you want to treat the byte array as big-endian and you're faced with a little-endian BitConverter, you have to reverse the array in both cases, not just one. Basically you should regard BitConverter as providing a reversible conversion, but it may not be the exact conversion you want.
You might want to use my EndianBitConverter in my MiscUtil project if you want to specify endianness for this kind of conversion.
i have exponent whats 3 bytes long. Now i need it to be 4 bytes. I found somewhere that i could pad at the leading octets.. but i have no idea to do that.. So can anybody help me out?
Example Input: exponent what i have right now is 65537, int bytes its then 01 00 01.
Assuming you just want to pad it with zeroes, create a new four byte array and copy the existing one into it:
byte[] newArray = new byte[4];
// Copy bytes 0, 1, 2 of oldArray into 1, 2, 3 of newArray.
Array.Copy(oldArray, 0, newArray, 1, 3);
(You can do this manually with three assignments as well; that's potentially simpler for this situation, but doesn't scale well (in terms of code) to larger sizes.)
Change the "1" to "0" if you find you need the padding at the end instead of the start... or use Array.Resize in that case.
Not entirely sure of the meaning, but it sounds like you just want:
byte[] first = /* 3 bytes */
byte[] second = new byte[4];
// since 3 bytes, we'll do this manually; note second[0] is 0 already
second[1] = first[0];
second[2] = first[1];
second[3] = first[2];
Of course, if you are actually dealing with an int it is already padded on the left with 0, to 4 bytes.