I have a byte[] array and want to write it to stdout: Console.Out.Write(arr2str(arr)). How to convert byte[] to string, so that app.exe > arr.txt does the expected thing? I just want to save the array to a file using a pipe, but encodings mess things up.
I'd later want to read that byte array from stdin: app.exe < arr.txt and get the same thing.
How can I do these two things: write and read byte arrays to/from stdin/stdout?
EDIT:
I'm reading with string s = Console.In.ReadToEnd(), and then System.Text.Encoding.Default.GetBytes(s). I'm converting from array to string with System.Text.Encoding.Default.GetString(bytes), but this doesn't work when used with <,>. By "doesn't work" I mean that writing and reading over a pipe does not return the same thing.
To work with binary files you want Console.OpenStandardInput() to retrieve a Stream that you can read from. This has been covered in other threads here at SO, this one for example: Read binary data from Console.In
If you are writing to Console.WriteLine you need to encode the text in to a printable format. If you want to output to a file as a binary you can't use Console.WriteLine
If you still need to output to the console you either need to open the raw stream with Console.OpenStandardOutput() or call Convert.ToBase64String to turn the byte array to a string. There is also Convert.FromBase64String to come back from base64 to a byte array.
Related
I have recently started learning C# Networking and I was wondering how would you tell if the received Byte array is a file or a string?
A byte array is just a byte array. It's just got data in.
How you interpret that data is up to you. What's the difference between a text file and a string, for example?
Fundamentally, if your application needs to know how to interpret the data, you've got to put that into the protocol.
A byte array is just a byte array. However, you could make the original byte array include a byte that describes what type it is (assuming you are the originator of it). Then you find this descriptor byte and use it to make decisions.
Strings are encoded byte arrays; files can contain strings and/or binary data.
ASCII strings use byte values between 0-127 to represent characters and control codes. For UTF8 people have written validation routines (https://stackoverflow.com/a/892443/884862).
You'd have to check the array for all of the string encoding characteristics before you could assume it's a binary file.
edit Here's an SO question about classifying a file type Using .NET, how can you find the mime type of a file based on the file signature not the extension using a signature (first X bytes) of the file to determine it's mimetype.
No you can't. Data is data, you must layer on top of your network communication form of protocol, it will need to say something like: "If the first byte I see is a 1 the next four bytes represent a int, if I see a 2 read the next byte and that is the length of the text string that follows that..."
A much easier solution than inventing your own protocol is use a prebuilt one that gives you a higher level abstraction like WCF so you don't need to deal with byte arrays.
Not quite a "file", an array contains data. You should loop through that array and write the data,
Try this:
foreach(string data in array)
{
Console.WriteLine(data);
}
Now, if it doesn't contain strings, but data, you can simply use a
foreach(var data in array)
{
Console.WriteLine(data.ToString());
}
I'm trying to create a class to manage the opening of a certain file. I would one of the properties to be a byte array of the file, but I don't know how big the file is going to be. I tried declaring the byte array as :
public byte[] file;
...but it won't allow me to set it the ways I've tried. br is my BinaryReader:
file = br.ReadBytes(br.BaseStream.Length);
br.Read(file,0,br.BaseStream.Length);
Neither way works. I assume it's because I have not initialized my byte array, but I don't want to give it a size if I don't know the size. Any ideas?
edit: Alright, I think it's because the Binary Reader's BaseStream length is a long, but its readers take int32 counts. If I cast the 64s into 32s, is it possible I will lose bytes in larger files?
I had no problems reading a file stream:
byte[] file;
var br = new BinaryReader(new FileStream("c:\\Intel\\index.html", FileMode.Open));
file = br.ReadBytes((int)br.BaseStream.Length);
Your code doesn't compile because the Length property of BaseStream is of type long but you are trying to use it as an int. Implicit casting which might lead to data loss is not allowed so you have to cast it to int explicitly.
Update
Just bear in mind that the code above aims to highlight your original problem and should not be used as it is. Ideally, you would use a buffer to read the stream in chunks. Have a look at this question and the solution suggested by Jon Skeet
You can't create unknown sized array.
byte []file=new byte[br.BaseStream.Length];
PS: You should have to repeatedly read chunks of bytes for larger files.
BinaryReader.ReadBytes returns a byte[]. There is no need to initialize a byte array because that method already does so internally and returns the complete array to you.
If you're looking to read all the bytes from a file, there's a handy method in the File class:
http://msdn.microsoft.com/en-us/library/system.io.file.readallbytes.aspx
If I have a single MemoryStream of which I know I sent multiple files (example 5 files) to this MemoryStream. Is it possible to read from this MemoryStream and be able to break apart file by file?
My gut is telling me no since when we Read, we are reading byte by byte... Any help and a possible snippet would be great. I haven't been able to find anything on google or here :(
You can't directly, not if you don't delimit the files in some way or know the exact size of each file as it was put into the buffer.
You can use a compressed file such as a zip file to transfer multiple files instead.
A stream is just a line of bytes. If you put the files next to each other in the stream, you need to know how to separate them. That means you must know the length of the files, or you should have used some separator. Some (most) file types have a kind of header, but looking for this in an entire stream may not be waterproof either, since the header of a file could just as well be data in another file.
So, if you need to write files to such a stream, it is wise to add some extra information. For instance, start with a version number, then, write the size of the first file, write the file itself and then write the size of the next file, etc....
By starting with a version number, you can make alterations to this format. In the future you may decide you need to store the file name as well. In that case, you can increase version number, make up a new format, and still be able to read streams that you created earlier.
This is of course especially useful if you store these streams too.
Since you're sending them, you'll have to send them into the stream in such a way that you'll know how to pull them out. The most common way of doing this is to use a length specification. For example, to write the files to the stream:
write an integer to the stream to indicate the number of files
Then for each file,
write an integer (or a long if the files are large) to indicate the number of bytes in the file
write the file
To read the files back,
read an integer (n) to determine the number of files in the stream
Then, iterating n times,
read an integer (or long if that's what you chose) to determine the number of bytes in the file
read the file
You could use an IEnumerable<Stream> instead.
You need to implement this yourself, what you would want to do is write in some sort of 'delimited' into the stream. As you're reading, look for that delimited, and you'll know when you have hit a new file.
Here's a quick and dirty example:
byte[] delimiter = System.Encoding.Default.GetBytes("++MyDelimited++");
ms.Write(myFirstFile);
ms.Write(delimiter);
ms.Write(mySecondFile);
....
int len;
do {
len = ms.ReadByte(buffer, lastOffest, delimiter.Length);
if(buffer == delimiter)
{
// Close and open a new file stream
}
// Write buffer to output stream
} while(len > 0);
Related to this question encrypt binary with 7z without filenames?
In C# how can i put binary in STDin? i was hoping the below would work but it doesnt. And it makes sense. So how do i push a byte[] array?
new BinaryWriter(p.StandardInput.FormatProvider);
Write directly to the base stream:
new BinaryWriter(p.StandardInput.BaseStream)
stdin is just another byte stream, one your program can read from
Stream st=Console.OpenStandardInput ();
StreamReader sr=new StreamReader(st);
etc. In the q. which you refer to, the material coming in from stdin is being piped from the output of another program. To do that part of the process, you use Console.OpenStandardOuput() to get a stream and push the binary out through that.
I need to be able to read a file format that mixes binary and non-binary data. Assuming I know the input is good, what's the best way to do this? As an example, let's take a file that has a double as the first line, a newline (0x0D 0x0A) and then ten bytes of binary data afterward. I could, of course, calculate the position of the newline, then make a BinaryReader and seek to that position, but I keep thinking that there has to be a better way.
You can use System.IO.BinaryReader. The problem with this though is you must know what type of data you are going to be reading before you call any of the Read methods.
Read(byte[], int, int)
Read(char[], int, int)
Read()
Read7BitEncodedInt()
ReadBoolean()
ReadByte()
ReadBytes(int)
ReadChar()
ReadChars()
ReadDecimal()
ReadDouble()
ReadInt16()
ReadInt32()
ReadInt64()
ReadSByte()
ReadSingle()
ReadString()
ReadUInt16()
ReadUInt32()
ReadUInt64()
And of course the same methods exist for writing in System.IO.BinaryWriter.
Is this file format already fixed? If it's not, it's a really good idea to change to use a length-prefixed format for the strings. Then you can read just the right amount and convert it to a string.
Otherwise, you'll need to read chunks from the file, scan for the newline, and decode the right amount of data or (if you don't find the newline) either buffer it somewhere else (e.g. a MemoryStream) or just remember the starting point and rewind the stream appropriately. It will be ugly, but that's just because of the deficiency of the file format.
I would suggest you don't "over-decode" (i.e. decode the arbitrary binary data after the string) - while it may well not do any harm, in some encodings you could be reading an impossible sequence of binary data, which then starts getting into the realms of DecoderFallbacks and the like.
I've had to deal with that when reading HTTP requests coming in over the wire on Compact Framework. My solution was to roll my own non-buffering ASCII-only StreamReader, so that it was safe to interleave calls to both the StreamReader and the underlying Stream.