I am trying to get a byte[] from a FileInfo.
Here, The FileInfo(fi) is a file I drop into my silverlight app.
So, as found on msnd, I am doing this :
byte[] b = new byte[fi.Length];
UTF8Encoding temp = new UTF8Encoding(true);
//Open the stream and read it back.
using (FileStream fs = fi.OpenRead())
{
while (fs.Read(b, 0, b.Length) > 0)
{
Console.WriteLine(temp.GetString(b));
}
}
But, do to it's protection level, I cannot use this.
So, I have done this :
byte[] b = new byte[fi.Length];
UTF8Encoding temp = new UTF8Encoding(true);
//Open the stream and read it back.
using (FileStream fs = fi.OpenRead())
{
while (fs.Read(b, 0, b.Length) > 0)
{
fs.Write(b, 0, b.Length);
}
}
But I got the message that I cannot Write from the FileStream.
Why I cannot write my File I drop into my app into a byte?
When The File is drop, it become a FileInfo.
Why I use OpenRead()? Because on the msdn, it seems it is writing the file : here
OpenWrite() rise an access error also.
Is there another way to do get yhe FileInfo document, into a byte?
To read a file into a byte[] the easies way would be:
byte[] myByteArray = File.ReadAllBytes(myFileInfo.FullName);
As #Dmitry Bychenko allready stated you try to write to a FileStream opened as readonly.
The other thing is that you want to write to the same FileStream you read from.
To solve the problem by correcting the attempt you did you can do:
byte[] b = new byte[fi.Length];
UTF8Encoding temp = new UTF8Encoding(true);
//Open the stream and read it back.
using (FileStream fs = fi.OpenRead())
{
using (MemoryStream ms = new MemoryStream(b))
{
while (fs.Read(b, 0, b.Length) > 0)
{
ms.Write(b, 0, b.Length);
}
}
}
In your case i would vote for the first example as its simple to read and hides the stream stuff perfectly.
Related
I need help converting a VERY LARGE binary file (ZIP file) to a Base64String and back again. The files are too large to be loaded into memory all at once (they throw OutOfMemoryExceptions) otherwise this would be a simple task. I do not want to process the contents of the ZIP file individually, I want to process the entire ZIP file.
The problem:
I can convert the entire ZIP file (test sizes vary from 1 MB to 800 MB at present) to Base64String, but when I convert it back, it is corrupted. The new ZIP file is the correct size, it is recognized as a ZIP file by Windows and WinRAR/7-Zip, etc., and I can even look inside the ZIP file and see the contents with the correct sizes/properties, but when I attempt to extract from the ZIP file, I get: "Error: 0x80004005" which is a general error code.
I am not sure where or why the corruption is happening. I have done some investigating, and I have noticed the following:
If you have a large text file, you can convert it to Base64String incrementally without issue. If calling Convert.ToBase64String on the entire file yielded: "abcdefghijklmnopqrstuvwx", then calling it on the file in two pieces would yield: "abcdefghijkl" and "mnopqrstuvwx".
Unfortunately, if the file is a binary then the result is different. While the entire file might yield: "abcdefghijklmnopqrstuvwx", trying to process this in two pieces would yield something like: "oiweh87yakgb" and "kyckshfguywp".
Is there a way to incrementally base 64 encode a binary file while avoiding this corruption?
My code:
private void ConvertLargeFile()
{
FileStream inputStream = new FileStream("C:\\Users\\test\\Desktop\\my.zip", FileMode.Open, FileAccess.Read);
byte[] buffer = new byte[MultipleOfThree];
int bytesRead = inputStream.Read(buffer, 0, buffer.Length);
while(bytesRead > 0)
{
byte[] secondaryBuffer = new byte[buffer.Length];
int secondaryBufferBytesRead = bytesRead;
Array.Copy(buffer, secondaryBuffer, buffer.Length);
bool isFinalChunk = false;
Array.Clear(buffer, 0, buffer.Length);
bytesRead = inputStream.Read(buffer, 0, buffer.Length);
if(bytesRead == 0)
{
isFinalChunk = true;
buffer = new byte[secondaryBufferBytesRead];
Array.Copy(secondaryBuffer, buffer, buffer.length);
}
String base64String = Convert.ToBase64String(isFinalChunk ? buffer : secondaryBuffer);
File.AppendAllText("C:\\Users\\test\\Desktop\\Base64Zip", base64String);
}
inputStream.Dispose();
}
The decoding is more of the same. I use the size of the base64String variable above (which varies depending on the original buffer size that I test with), as the buffer size for decoding. Then, instead of Convert.ToBase64String(), I call Convert.FromBase64String() and write to a different file name/path.
EDIT:
In my haste to reduce the code (I refactored it into a new project, separate from other processing to eliminate code that isn't central to the issue) I introduced a bug. The base 64 conversion should be performed on the secondaryBuffer for all iterations save the last (Identified by isFinalChunk), when buffer should be used. I have corrected the code above.
EDIT #2:
Thank you all for your comments/feedback. After correcting the bug (see the above edit), I re-tested my code, and it is actually working now. I intend to test and implement #rene's solution as it appears to be the best, but I thought that I should let everyone know of my discovery as well.
Based on the code shown in the blog from Wiktor Zychla the following code works. This same solution is indicated in the remarks section of Convert.ToBase64String as pointed out by Ivan Stoev
// using System.Security.Cryptography
private void ConvertLargeFile()
{
//encode
var filein= #"C:\Users\test\Desktop\my.zip";
var fileout = #"C:\Users\test\Desktop\Base64Zip";
using (FileStream fs = File.Open(fileout, FileMode.Create))
using (var cs=new CryptoStream(fs, new ToBase64Transform(),
CryptoStreamMode.Write))
using(var fi =File.Open(filein, FileMode.Open))
{
fi.CopyTo(cs);
}
// the zip file is now stored in base64zip
// and decode
using (FileStream f64 = File.Open(fileout, FileMode.Open) )
using (var cs=new CryptoStream(f64, new FromBase64Transform(),
CryptoStreamMode.Read ) )
using(var fo =File.Open(filein +".orig", FileMode.Create))
{
cs.CopyTo(fo);
}
// the original file is in my.zip.orig
// use the commandlinetool
// fc my.zip my.zip.orig
// to verify that the start file and the encoded and decoded file
// are the same
}
The code uses standard classes found in System.Security.Cryptography namespace and uses a CryptoStream and the FromBase64Transform and its counterpart ToBase64Transform
You can avoid using a secondary buffer by passing offset and length to Convert.ToBase64String, like this:
private void ConvertLargeFile()
{
using (var inputStream = new FileStream("C:\\Users\\test\\Desktop\\my.zip", FileMode.Open, FileAccess.Read))
{
byte[] buffer = new byte[MultipleOfThree];
int bytesRead = inputStream.Read(buffer, 0, buffer.Length);
while(bytesRead > 0)
{
String base64String = Convert.ToBase64String(buffer, 0, bytesRead);
File.AppendAllText("C:\\Users\\test\\Desktop\\Base64Zip", base64String);
bytesRead = inputStream.Read(buffer, 0, buffer.Length);
}
}
}
The above should work, but I think Rene's answer is actually the better solution.
Use this code:
public void ConvertLargeFile(string source , string destination)
{
using (FileStream inputStream = new FileStream(source, FileMode.Open, FileAccess.Read))
{
int buffer_size = 30000; //or any multiple of 3
byte[] buffer = new byte[buffer_size];
int bytesRead = inputStream.Read(buffer, 0, buffer.Length);
while (bytesRead > 0)
{
byte[] buffer2 = buffer;
if(bytesRead < buffer_size)
{
buffer2 = new byte[bytesRead];
Buffer.BlockCopy(buffer, 0, buffer2, 0, bytesRead);
}
string base64String = System.Convert.ToBase64String(buffer2);
File.AppendAllText(destination, base64String);
bytesRead = inputStream.Read(buffer, 0, buffer.Length);
}
}
}
i'm using the following function to compress(thanks to http://www.dotnetperls.com/):
public static void CompressStringToFile(string fileName, string value)
{
// A.
// Write string to temporary file.
string temp = Path.GetTempFileName();
File.WriteAllText(temp, value);
// B.
// Read file into byte array buffer.
byte[] b;
using (FileStream f = new FileStream(temp, FileMode.Open))
{
b = new byte[f.Length];
f.Read(b, 0, (int)f.Length);
}
// C.
// Use GZipStream to write compressed bytes to target file.
using (FileStream f2 = new FileStream(fileName, FileMode.Create))
using (GZipStream gz = new GZipStream(f2, CompressionMode.Compress, false))
{
gz.Write(b, 0, b.Length);
}
}
and for decompress:
static byte[] Decompress(byte[] gzip)
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (GZipStream stream = new GZipStream(new MemoryStream(gzip), CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
so my goal is actually compress log files and than to decompress them in memory and compare the uncompressed file to the original file in order to check that the compression succeeded and i'm able to open the compressed file successfuly.
the problem is that the uncompressed file is most of the time bigger than the original file and my compare check is failing altough the compression probably succeeded.
any idea why ?
btw here how i compare the uncompressed file to the original file:
static bool FileEquals(byte[] file1, byte[] file2)
{
if (file1.Length == file2.Length)
{
for (int i = 0; i < file1.Length; i++)
{
if (file1[i] != file2[i])
{
return false;
}
}
return true;
}
return false;
}
Try this method to compress a file:
public static byte[] Compress(byte[] raw)
{
using (MemoryStream memory = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(memory,
CompressionMode.Compress, true))
{
gzip.Write(raw, 0, raw.Length);
}
return memory.ToArray();
}
}
}
And this to decompress :
static byte[] Decompress(byte[] gzip)
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (GZipStream stream = new GZipStream(new MemoryStream(gzip), CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
}
Tell me if it worked.
Goodluck.
Think you'd be better off with the simplest API call, try Stream.CopyTo(). I can't find the error in your code. If I was working on it, I'd probably make sure everything is getting flushed properly.. can't recall if GZipStream is going to flush its output to FileStream when the using block closes.. but then you are also saying that the final file is larger, not smaller.
Anyhow, best policy in my experience.. don't rewrite gotcha prone code when you don't need to. At least you tested it ;)
I'm using the C# System.IO class (framework 4.0) to compress an image pulled off the file system with FileDialog and then inserted into a SQL Server database as a varbinary(max) data type. The problem I'm having is when I pull the data out of the database and attempt to decompress I get the subject error with an additional message -- make sure you are passing in a gzip stream.
The code to get the file:
OpenFileDialog dlgOpen = new OpenFileDialog();
if (dlgOpen.ShowDialog() == DialogResult.OK)
{
FileStream fs = File.OpenRead(dlgOpen.FileName);
byte[] picbyte1 = new byte[fs.Length];
byte[] picbyte = Compress(picbyte1);
fs.Read(picbyte, 0, System.Convert.ToInt32(picbyte.Length));
String ImageName = dlgOpen.FileName;
//String bs64OfBytes = Convert.ToBase64String(picbyte);
fs.Close();
//additional code inserts into database
....
}
The compress method:
private static byte[] Compress(byte[] data)
{
var output = new MemoryStream();
using (var gzip = new GZipStream(output, CompressionMode.Compress, true))
{
gzip.Write(data, 0, data.Length);
gzip.Close();
}
return output.ToArray();
}
The decompress method:
private static byte[] Decompress(byte[] data)
{
var output = new MemoryStream();
var input = new MemoryStream();
input.Write(data, 0, data.Length);
input.Position = 0;
using (var gzip = new GZipStream(input, CompressionMode.Decompress, true))
{
var buff = new byte[64];//also used 32
var read = gzip.Read(buff, 0, buff.Length);//error occurs here
while (read > 0)
{
output.Write(buff, 0, read);
read = gzip.Read(buff, 0, buff.Length);
}
gzip.Close();
}
return output.ToArray();
}
You need to insert a line and remove an other:
FileStream fs = File.OpenRead(dlgOpen.FileName);
byte[] picbyte1 = new byte[fs.Length];
fs.Read(picbyte1, 0, (int)fs.Length); // <-- Add this one
byte[] picbyte = Compress(picbyte1);
// fs.Read(picbyte, 0, System.Convert.ToInt32(picbyte.Length)); // <-- And remove this one
// ...
You are reading the image in your code, but something is in the wrong order:
// Original but incorrect sequence
FileStream fs = File.OpenRead(dlgOpen.FileName); // Open the file
byte[] picbyte1 = new byte[fs.Length]; // Assign the array
byte[] picbyte = Compress(picbyte1); // Compress the assigned array, but there is no contents...
fs.Read(picbyte, 0, System.Convert.ToInt32(picbyte.Length)); // You save the file to the already compressed bytes...
So you have saved the first part of the original file, not the compressed one (but the number of saved bytes corresponds with the number of compressed bytes). If you send this to the DB and read it back, the decompressor does not find it's Magic Number.
As an improvement, you can change these lines:
FileStream fs = File.OpenRead(dlgOpen.FileName);
byte[] picbyte1 = new byte[fs.Length];
fs.Read(picbyte1, 0, (int)fs.Length); // line that I suggested to add
probably change to:
byte[] picbyte1 = File.ReadAllBytes(dlgOpen.FileName);
I've got a rare case where a file cannot be read from a UNC path immediately after it was written. Here's the workflow:
plupload sends a large file in chunks to a WebAPI method
Method writes the chunks to a UNC path (a storage server). This loops until the file is completely uploaded.
After a few other operations, the same method tries to read the file again and sometimes it cannot find the file
It only seems to happen after our servers have been idle for a while. If I repeat the upload a few times, it starts to work.
I thought it might be a network configuration issue, or something to do with the file not completely closing before being read again.
Here's part of the code that writes the file (is the filestream OK in this case?)
SaveStream(stream, new FileStream(fileName, FileMode.Append, FileAccess.Write));
Here's SaveStream definition:
private static void SaveStream(Stream stream, FileStream fileStream)
{
using (var fs = fileStream)
{
var buffer = new byte[1024];
var l = stream.Read(buffer, 0, 1024);
while (l > 0)
{
fs.Write(buffer, 0, l);
l = stream.Read(buffer, 0, 1024);
}
fs.Flush();
fs.Close();
}
}
Here's the code that reads the file:
var fileInfo = new FileInfo(fileName);
var exists = fileInfo.Exists;
It's the fileInfo.Exists that is returning false.
Thank you
These kind of errors are mostly due to files not closed yet.
Try passing the fileName to SaveStream and then use it as follows:
private static void SaveStream(Stream stream, string fileName)
{
using (var fs = new FileStream(fileName, FileMode.Append, FileAccess.Write))
{
var buffer = new byte[1024];
var l = stream.Read(buffer, 0, 1024);
while (l > 0)
{
fs.Write(buffer, 0, l);
l = stream.Read(buffer, 0, 1024);
}
fs.Flush();
} // end of using will close and dispose fs properly
}
I'm trying code from http://www.paraesthesia.com/archive/2009/12/16/posting-multipartform-data-using-.net-webrequest.aspx to do a POST through httpwebrequest.
If I try this same code with a text file, it's fine. However if I do it with a zip file, then when re-download that file it's saying it's not a valid zip. I assume the zip portion is likely getting uploaded as text rather than binary. However, that page does say " It's OK to include binary content here. Don't base-64 encode it or anything, just stream it on in." But this doesn't seem to be working with the given code. I'm assuming I have to change the portion that reads the file to the stream:
using (FileStream fileStream = new FileStream(file.FullName, FileMode.Open, FileAccess.Read))
{
byte[] buffer = new byte[1024];
int bytesRead = 0;
while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
{
stream.Write(buffer, 0, bytesRead);
}
fileStream.Close();
}
Maybe to use BinaryReader? I'm a bit confused on how to use that in this context though, or if it's even what I need to do. A nudge in the right direction would be awesome. Thanks!
BinaryReader should work indeed:
FileInfo fInfo = new FileInfo(file.FullName);
//
long numBytes = fInfo.Length;
FileStream fStream = new FileStream(file.FullName, FileMode.Open, FileAccess.Read);
BinaryReader br = new BinaryReader(fStream);
byte[] bdata = br.ReadBytes((int)numBytes);
br.Close();
fStream.Close();
// Write bdata to the HttpStream
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("url-here");
// Additional webRequest parameters settings.
HttpStream stream = (Stream)webRequest.GetRequestStream();
stream .Write(bdata, 0, bdata.Length);
stream.Close();
HttpWebResponse response = (HttpWebRewponse)webRequest.GetResponse();