I'm trying to create a class to manage the opening of a certain file. I would one of the properties to be a byte array of the file, but I don't know how big the file is going to be. I tried declaring the byte array as :
public byte[] file;
...but it won't allow me to set it the ways I've tried. br is my BinaryReader:
file = br.ReadBytes(br.BaseStream.Length);
br.Read(file,0,br.BaseStream.Length);
Neither way works. I assume it's because I have not initialized my byte array, but I don't want to give it a size if I don't know the size. Any ideas?
edit: Alright, I think it's because the Binary Reader's BaseStream length is a long, but its readers take int32 counts. If I cast the 64s into 32s, is it possible I will lose bytes in larger files?
I had no problems reading a file stream:
byte[] file;
var br = new BinaryReader(new FileStream("c:\\Intel\\index.html", FileMode.Open));
file = br.ReadBytes((int)br.BaseStream.Length);
Your code doesn't compile because the Length property of BaseStream is of type long but you are trying to use it as an int. Implicit casting which might lead to data loss is not allowed so you have to cast it to int explicitly.
Update
Just bear in mind that the code above aims to highlight your original problem and should not be used as it is. Ideally, you would use a buffer to read the stream in chunks. Have a look at this question and the solution suggested by Jon Skeet
You can't create unknown sized array.
byte []file=new byte[br.BaseStream.Length];
PS: You should have to repeatedly read chunks of bytes for larger files.
BinaryReader.ReadBytes returns a byte[]. There is no need to initialize a byte array because that method already does so internally and returns the complete array to you.
If you're looking to read all the bytes from a file, there's a handy method in the File class:
http://msdn.microsoft.com/en-us/library/system.io.file.readallbytes.aspx
Related
I have a small problem with crc checking in c#, im needing to read a file which contains the crc value in the last 8 bytes, how im doing it now is like
using filestream with filemode open
calculate stream length minus 8 bytes
stream.read(buffer,0,streamlength minus 8 bytes)
crc computehash passed in buffer
this leaves the remaining 8bytes which I compare against the crcvalue
the problem ive got is that it works ok for small files, but obviously I get a system out of memory exception for bigger files, I know computehash will take a stream but its either pass in full stream which means I cant get the remaining bytes.
Is there a better way of doing this?
kindest regards
Providing a code snippet will prove to be a great help for us that are trying to help you. While I understand what you are saying I can never be sure that I do get what you did without reading your code.
Not being sure what you want to do with the file I suggest you also look at the MemoryStream class. One quick advantage of a MemoryStream is that there is no need to create temporary buffers and files in an application meaning that you could actually save on memory.
You can apply your current method in a similar fashion to MemoryStream and see if that works.
Info on MemoryStream: http://msdn.microsoft.com/en-us/library/system.io.memorystream%28v=vs.110%29.aspx
I'm new to programming in general (My understanding of programming concepts is still growing.). So this question is about learning, so please provide enough info for me to learn but not so much that I can't, thank you.
(I would also like input on how to make the code reusable with in the project.)
The goal of the project I'm working on consists of:
Read binary file.
I have known offsets I need to read to find a particular chunk of data from within this file.
First offset is first 4 bytes(Offset for end of my chunk).
Second offset is 16 bytes from end of file. I read for 4 bytes.(Gives size of chunk in hex).
Third offset is the 4 bytes following previous, read for 4 bytes(Offset for start of chunk in hex).
Locate parts in the chunk to modify by searching ASCII text as well as offsets.
Now I have the start offset, end offset and size of my chunk.
This should allow me to read bytes from file into a byte array and know the size of the array ahead of time.
(Questions: 1. Is knowing the size important? Other than verification. 2. Is reading part of a file into a byte array in order to change bytes and overwrite that part of the file the best method?)
So far I have managed to read the offsets from the file using BinaryReader on a MemoryStream. I then locate the chunk of data I need and read that into a byte array.
I'm stuck in several ways:
What are the best practices for binary Reading / Writing?
What's the best storage convention for the data that is read?
When I need to modify bytes how do I go about that.
Should I be using FileStream?
Since you want to both read and write, it makes sense to use the FileStream class directly (using FileMode.Open and FileAccess.ReadWrite). See FileStream on MSDN for a good overall example.
You do need to know the number of bytes that you are going to be reading from the stream. See the FileStream.Read documentation.
Fundamentally, you have to read the bytes into memory at some point if you're going to use and later modify their contents. So you will have to make an in-memory copy (using the Read method is the right way to go if you're reading a variable-length chunk at a time).
As for best practices, always dispose your streams when you're done; e.g.:
using (var stream = File.Open(FILE_NAME, FileMode.Open, FileAccess.ReadWrite))
{
//Do work with the FileStream here.
}
If you're going to do a large amount of work, you should be doing the work asynchronously. (Let us know if that's the case.)
And, of course, check the FileStream.Read documentation and also the FileStream.Write documentation before using those methods.
Reading bytes is best done by pre-allocating an in-memory array of bytes with the length that you're going to read, then reading those bytes. The following will read the chunk of bytes that you're interested in, let you do work on it, and then replace the original contents (assuming the length of the chunk hasn't changed):
EDIT: I've added a helper method to do work on the chunk, per the comments on variable scope.
using (var stream = File.Open(FILE_NAME, FileMode.Open, FileAccess.ReadWrite))
{
var chunk = new byte[numOfBytesInChunk];
var offsetOfChunkInFile = stream.Position; // It sounds like you've already calculated this.
stream.Read(chunk, 0, numOfBytesInChunk);
DoWorkOnChunk(ref chunk);
stream.Seek(offsetOfChunkInFile, SeekOrigin.Begin);
stream.Write(chunk, 0, numOfBytesInChunk);
}
private void DoWorkOnChunk(ref byte[] chunk)
{
//TODO: Any mutation done here to the data in 'chunk' will be written out to the stream.
}
I have an instance of MemoryStream that is closed (don't ask, can't change that and it is not that poor design as it seems at first glance :).
Anyway I found that I can take the byte[] using something like:
MemoryStream ms = SomeClass.GetMemoryStream();
byte[] myData = ms.GetBuffer();
Everything seems just great so far, the only problem I have is that the byte array returned from GetBuffer() is resized and contains unallocated space (byte)0;
Like so:
12,32,43,43,2,3,0,0,0,0,0,0,0
My question is is it safe to assume that I can read the array until I encounter the first 0?
Do you know any case in wich there will be (byte)0 in the middle of the data?
The data is a MIME Email Message.
Try ms.ToArray() : http://msdn.microsoft.com/en-us/library/system.io.memorystream.toarray.aspx
It works on a closed stream and returns a copy of the data, without the unused part of the buffer.
The msdn article on MemoryStream.GetBuffer Method says
Note that the buffer contains allocated bytes which might be unused.
For example, if the string "test" is written into the MemoryStream
object, the length of the buffer returned from GetBuffer is 256, not
4, with 252 bytes unused. To obtain only the data in the buffer, use
the ToArray method; however, ToArray creates a copy of the data in
memory.
Despite the copy it looks like what you are supposed to do...
what's the best way to write the binary representation of an int array (Int32[]) to a Stream?
Stream.Write only accepts byte[] as source and I would like to avoid converting/copying the array to an byte[] (array but instead streaming directly from the 'original location').
In a more system-oriented language (a.k.a. C++) I would simply cast the int array to a byte* but as far as I understood this isn't possible with C# (and moreover, casting byte* to byte[] wouldn't work out either way)
Thanks
Martin
PS: Actually, I would also like to stream single int values. Does using BinaryConverter.GetBytes() create a new byte array? In this case I extend my question to how to efficiently stream single int values ...
The simplest option would be to use BinaryWriter wrapping your output stream, and call Write(int) for each of your int values. If that doesn't use the right endianness for you, you could use EndianBinaryWriter from my MiscUtil library.
I don't know of anything built-in to do this more efficiently... I'd hope that the buffering within the stream would take care of it for the most part.
System.Array and System.Int32 both have the SerializableAttribute and so both support default serialization in a retrievable format.
http://msdn.microsoft.com/en-us/library/system.serializableattribute.aspx
There is sample code for Binary output and readback here:
http://msdn.microsoft.com/en-us/library/aa904194(VS.71).aspx
I have a byte[] array, the contents of which represent a TIFF file (as in, if I write out these bytes directly to a file using the BinaryWriter object, it forms a perfectly valid TIFF file) and I'm trying to turn it into a System.Drawing.Image object so that I can use it for later manipulation (feeding into a multipage TIFF object)
The problem I'm having is that the commonly accepted code for this task:
public Image byteArrayToImage(byte[] byteArrayIn)
{
MemoryStream ms = new MemoryStream(byteArrayIn);
Image returnImage = Image.FromStream(ms, true);
return returnImage;
}
doesn't work for me. The second line of the above method where it calls the Image.FromStream method dies at runtime, saying
Parameter Not Valid
I believe that the method is choking on the fact that this is a TIFF file but I cannot figure out how to make the FromStream method accept this fact.
How do I turn a byte array of a TIFF image into an Image object?
Also, like I said the end goal of this is to have a byte array representing a multipage TIFF file, which contains the TIFF files for which I have byte array objects of right now. If there's a much better way to go about doing this, I'm all for it.
OK, I found the issue, and it was from a part of the code unrelated to the part of the code I was asking about. The data was being passed as a string, I was converting it to a byte array (this was a test rig so I was trying to simulate the byte array that I get in the main app), then converting that to a MemoryStream, then making an Image from that.
What I failed to realize was that the string was Base64 encoded. Calling Convert.FromBase64String() caused it to turn into a byte array which wouldn't kill the Image.FromStream() method.
So basically it boiled down to a stupid mistake on my part. But hey, the code above is still useful and this page will probably serve as a Google result as to how to avoid this mistake to someone else.
Also, I found an easy way to construct a Multi-Page TIFF from my byte arrays here.
Edit: The assumption below is not correct, I had a chance to fire up my IDE later and tested with and without Write and both populated the MemoryStream correctly.
I think you need to write to your MemeoryStream first.
As if my memory (no pun intended) serves me correctly this:
MemoryStream ms = new MemoryStream(byteArrayIn);
Creates a memory stream of that size.
You then need to write your byte array contents to the memory stream:
ms.Write(byteArrayIn, 0, byteArrayIn.Length);
See if that fixes it.
All these were clues that helped me figure out my problem which was the same problem as the question asks. So i want to post my solution which i arrived at because of these helpful clues. Thanks for all the clues posted so far!
As Time Saunders posted in his answer, that Write method to actually write the bytes to the memory stream is essential. That was my first mistake.
Then my data was bad TIFF data too, but in my case, i had an extra character 13 at the beginning of my image data. Once i removed that, it all worked fine for me.
When i read about some basic TIFF file format specs, i found that TIFF files must begin with II or MM (two bytes with values of either 73 or 77). II means little-endian byte order ('Intel byte ordering') is used. MM means big-ending ('Motorola byte ordering') is used. The next two bytes are a two byte integer value ( = Int16 in .NET) of 42, binary 101010.
Thus a correct TIFF stream of bytes begins with the decimal byte values of: 73, 73, 42, 0 or 77, 77, 0, 42. I encourage anyone with the same problem that we experienced to inspect your TIFF data byte stream and make sure your data is valid TIFF data!
Thanks Schnapple and Tim Saunders!!