what's the best way to write the binary representation of an int array (Int32[]) to a Stream?
Stream.Write only accepts byte[] as source and I would like to avoid converting/copying the array to an byte[] (array but instead streaming directly from the 'original location').
In a more system-oriented language (a.k.a. C++) I would simply cast the int array to a byte* but as far as I understood this isn't possible with C# (and moreover, casting byte* to byte[] wouldn't work out either way)
Thanks
Martin
PS: Actually, I would also like to stream single int values. Does using BinaryConverter.GetBytes() create a new byte array? In this case I extend my question to how to efficiently stream single int values ...
The simplest option would be to use BinaryWriter wrapping your output stream, and call Write(int) for each of your int values. If that doesn't use the right endianness for you, you could use EndianBinaryWriter from my MiscUtil library.
I don't know of anything built-in to do this more efficiently... I'd hope that the buffering within the stream would take care of it for the most part.
System.Array and System.Int32 both have the SerializableAttribute and so both support default serialization in a retrievable format.
http://msdn.microsoft.com/en-us/library/system.serializableattribute.aspx
There is sample code for Binary output and readback here:
http://msdn.microsoft.com/en-us/library/aa904194(VS.71).aspx
Related
I have a float array :
float[] samples32array
I need to convert it into a binary file so I can read it in matlab.
Is there any way to do that?
It's simple. First, you should use FileStream and create a file. Then, you can use BinaryWriter, which can write any C# datatype into an underlaying stream, such as a FileStream.
using (FileStream file = File.Create(path))
{
using (BinaryWriter writer = new BinaryWriter(file))
{
foreach (float value in samples32array)
{
writer.Write(value);
}
}
}
Since the constructor of BinaryWriter accepts the basic type Stream, any stream type can be used. It works for file streams as well as NetworkStream or a MemoryStream etc. It's a very generic class.
And please avoid converting the float[] into a byte[] beforehand as it will allocate memory and this is bad if your array is big (don't know if that's the case for you).
You can use BinaryWriter to write the data to a file very easily:
foreach (var value in samples32array)
{
writer.Write(value);
}
Now BinaryWriter is guaranteed to use little-endian format, so in your Matlab call, you should specify a machinefmt value of l to explicitly read it in little-endian format too.
This SO answer shows a way to convert a float array into a byte array. Then you can use File.WriteAllBytes() method to write it out to a file. How MatLab reads it, though, will be the issue.
I found some documentation for MatLab for the fread command. It looks like is has some arguments that will allow you to define the precision of the read. You may be able to use "float" as the precision value. Though, the is a bit of an educated guess as I am not very familiar with MatLab.
What is the safest way to guarantee that the following operation will be performed correctly:
When I read in 4 bytes as a uint32, I will write it out to a text file.
Later I will open this text file, read the number I wrote out previously, and then convert it back into the 4 bytes for use in other processing.
There is the BitConverter class to help you convert between primitive types and bytes.
Since you are storing this as a string, there isn't a whole lot to this. Obviously there is no issue converting the number into a string using .ToString(). So the only question I assume is how to go back in a reliable fashion. The solution is to use uint.Parse. i.e.:
var s = "12343632423432";
uint i = uint.Parse(s);
(PS: BitConverter is not helpful for conversion from strings)
I'm trying to create a class to manage the opening of a certain file. I would one of the properties to be a byte array of the file, but I don't know how big the file is going to be. I tried declaring the byte array as :
public byte[] file;
...but it won't allow me to set it the ways I've tried. br is my BinaryReader:
file = br.ReadBytes(br.BaseStream.Length);
br.Read(file,0,br.BaseStream.Length);
Neither way works. I assume it's because I have not initialized my byte array, but I don't want to give it a size if I don't know the size. Any ideas?
edit: Alright, I think it's because the Binary Reader's BaseStream length is a long, but its readers take int32 counts. If I cast the 64s into 32s, is it possible I will lose bytes in larger files?
I had no problems reading a file stream:
byte[] file;
var br = new BinaryReader(new FileStream("c:\\Intel\\index.html", FileMode.Open));
file = br.ReadBytes((int)br.BaseStream.Length);
Your code doesn't compile because the Length property of BaseStream is of type long but you are trying to use it as an int. Implicit casting which might lead to data loss is not allowed so you have to cast it to int explicitly.
Update
Just bear in mind that the code above aims to highlight your original problem and should not be used as it is. Ideally, you would use a buffer to read the stream in chunks. Have a look at this question and the solution suggested by Jon Skeet
You can't create unknown sized array.
byte []file=new byte[br.BaseStream.Length];
PS: You should have to repeatedly read chunks of bytes for larger files.
BinaryReader.ReadBytes returns a byte[]. There is no need to initialize a byte array because that method already does so internally and returns the complete array to you.
If you're looking to read all the bytes from a file, there's a handy method in the File class:
http://msdn.microsoft.com/en-us/library/system.io.file.readallbytes.aspx
I have a large byte array with mostly 0's but some values that I need to process. If this was C++ or unsafe C# I would use a 32bit pointer and only if the current 32bit were not 0, I would look at the individual bytes. This enables much faster scanning through the all 0 blocks. Unfortunately this must be safe C# :-)
I could use an uint array instead of a byte array and then manipulate the individual bytes but it makes what I'm doing much more messy than I like. I'm looking for something simpler, like the pointer example (I miss pointers sigh)
Thanks!
If the code must be safe, and you don't want to use a larger type and "shift", them you'll have to iterate each byte.
(edit) If the data is sufficiently sparse, you could use a dictionary to store the non-zero values; then finding the non-zeros is trivial (and enormous by sparse arrays become cheap).
I'd follow what this guy said:
Using SSE in c# is it possible?
Basically, write a little bit of C/C++, possibly using SSE, to implement the scanning part efficiently, and call it from C#.
You can access the characters
string.ToCharArray()
Or you can access the raw byte[]
Text.Encoding.UTF8Encoding.GetBytes(stringvalue)
Ultimately, what I think you'd need here is
MemoryStream stream;
stream.Write(...)
then you will be able to directly hadnle the memory's buffer
There is also UnmanagedMemoryStream but I'm not sure whether it'd use unsafe calls inside
You can use the BitConverter class:
byte[] byteArray = GetByteArray(); // or whatever
for (int i = 0; i < b.Length; I += 2)
{
uint x = BitConverter.ToUInt32(byteArray, i);
// do what you want with x
}
Another option is to create a MemoryStream from the byte array, and then use a BinaryReader to read 32-bit values from it.
I have a BigInteger serialized to a file by a Java program using the writeObject method from ObjectOutputStream.
Can I deserialize it in C#? I tried using the java.math and java.io classes of vjslib, but I get an exception:
InvalidClassException
the class does not match the class of the persisted object for cl = java.lang.Number : __SUID = -8742448824652078965, getSUID(cl) = 3166984097235214156
Any ideas?
Do you have control over the serialization step from Java?
If so, I would suggest serializing a byte array, either as binary, or base64, and reading the byte array from the serialized structure.
Then you can pass the byte array to the System.Numerics.BigInteger constructor.
If you don't mind ugly hacks: I'd say the easiest (albeit not most efficient) way would be to just write it out as an ASCII String on the Java side, and parse that string on the C# side, instead of using binary de/serialization.
I suggest you don't use serialization for this, since the two versions of BigInteger are not compatible - they have different version ids.
You should write the object out in some other way, probably using the byte array from BigInteger.toByteArray
Reading this this question about serialization might also be insightful for you