We Keep Coding

Create zip file in memory from bytes (text with arbitrary encoding)

The application i'm developing needs to compress xml files into zip files and send them through http requests to a web service. As I dont need to keep the zip files, i'm just performing the compression in memory. The web service is denying my requests because the zip files are apparently malformed.
I know there is a solution in this question which works perfectly, but it uses a StreamWriter. My problem with that solution is that StreamWriter requires an encoding or assumes UTF-8, and I do not need to know the enconding of the xml files. I just need to read the bytes from those files, and store them inside a zip file, whatever encoding they use.
So, to be clear, this question has nothing to do with encodings, as I don't need to transform the bytes into text or the oposite. I just need to compress a byte[].
I'm using the next code to test how my zip file is malformed:
static void Main(string[] args)
{
Encoding encoding = Encoding.GetEncoding("ISO-8859-1");
string xmlDeclaration = "<?xml version=\"1.0\" encoding=\"" + encoding.WebName.ToUpperInvariant() + "\"?>";
string xmlBody = "<Test>ª!\"·$%/()=?¿\\|##~€¬'¡º</Test>";
string xmlContent = xmlDeclaration + xmlBody;
byte[] bytes = encoding.GetBytes(xmlContent);
string fileName = "test.xml";
string zipPath = #"C:\Users\dgarcia\test.zip";
Test(bytes, fileName, zipPath);
}
static void Test(byte[] bytes, string fileName, string zipPath)
{
byte[] zipBytes;
using (var memoryStream = new MemoryStream())
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, leaveOpen: false))
{
var zipEntry = zipArchive.CreateEntry(fileName);
using (Stream entryStream = zipEntry.Open())
{
entryStream.Write(bytes, 0, bytes.Length);
}
//Edit: as the accepted answer states, the problem is here, because i'm reading from the memoryStream before disposing the zipArchive.
zipBytes = memoryStream.ToArray();
}
using (var fileStream = new FileStream(zipPath, FileMode.OpenOrCreate))
{
fileStream.Write(zipBytes, 0, zipBytes.Length);
}
}
If I try to open that file, I get an "Unexpected end of file" error. So apparently, the web service is correctly reporting a malformed zip file. What I have tried so far:
Flushing the entryStream.
Closing the entryStream.
Both flushing and closing the entryStream.
Note that if I open the zipArchive directly from the fileStream the zip file is formed with no errors. However, the fileStream is just there as a test, and I need to create my zip file in memory.

You are trying to get bytes from MemoryStream too early, ZipArchive did not write them all yet. Instead, do like this:
using (var memoryStream = new MemoryStream()) {
// note "leaveOpen" true, to not dispose memoryStream too early
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, leaveOpen: true)) {
var zipEntry = zipArchive.CreateEntry(fileName);
using (Stream entryStream = zipEntry.Open()) {
entryStream.Write(bytes, 0, bytes.Length);
}
}
// now, after zipArchive is disposed - all is written to memory stream
zipBytes = memoryStream.ToArray();
}

If you use a memory stream to load your text you can control the encoding type and it works across a WCF service. This is the implementation i am using currently and it works on my WCF services
private byte[] Zip(string text)
{
var bytes = Encoding.UTF8.GetBytes(text);
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
{
using (var gs = new GZipStream(mso, CompressionMode.Compress))
{
CopyTo(msi, gs);
}
return mso.ToArray();
}
}
private string Unzip(byte[] bytes)
{
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
{
using (var gs = new GZipStream(msi, CompressionMode.Decompress))
{
CopyTo(gs, mso);
}
return Encoding.UTF8.GetString(mso.ToArray());
}
}

getting the same directory structure after decompressing a zipped file

I downloaded the zip file from the amazon server(using AWS SDK for unity).The zip file has a folder which has one more folder inside it which contains png files.When I got the response object from the amazon server,I read it as a byte array and stored as a .zip file. When I double click on zip file I get directory and subdirectory inside it containing png files.Now I need to programatically unzip the file. I am trying to use GZipStream to decompress it,which returns the uncompressed byte array.Now how can I save this byte array so that I retain my folder structure?Also I don't want to use the third party library to decompress the zipped file.
void Start()
{
UnityInitializer.AttachToGameObject (this.gameObject);
client = new AmazonS3Client (mAccKey, mSecretKey, mRegion);
Debug.Log ("Getting the presigned url\n");
GetPreSignedUrlRequest request = new GetPreSignedUrlRequest ();
request.BucketName = mBucketName;
request.Key = mFileName;
request.Expires = DateTime.Now.AddMinutes (5);
request.Protocol = Protocol.HTTP;
GetObjectRequest requestObject = new GetObjectRequest ();
requestObject.BucketName = mBucketName;
requestObject.Key = mFileName;
Debug.Log ("Requesting for the " + mFileName + " contents" + "from the bucket\n" + mBucketName);
client.GetObjectAsync(mBucketName, mFileName, (responseObj) =>
{
var response = responseObj.Response;
if (response.ResponseStream != null)
{
Debug.Log("Recieving response\n");
using (BinaryReader bReader=new BinaryReader(response.ResponseStream))
{
byte[] buffer = bReader.ReadBytes((int)response.ResponseStream.Length);
var zippedPath=Application.persistentDataPath+"/"+zippedFile;
File.WriteAllBytes(zippedPath,buffer);
var unZippedPath=Application.persistentDataPath+"/"+unZipToFolder;
DirectoryInfo directory=Directory.CreateDirectory(unZippedPath);
byte[]compressedData=compress(buffer);
byte[] unCompressedData=decompress(compressedData);
//Debug.Log(unCompressedData.Length);
File.WriteAllBytes(unZippedPath+directory,unCompressedData);
}
Debug.Log("Response complete");
}
});
}
#region GZipStream
public static byte[] compress(byte[] data)
{
using (MemoryStream outStream = new MemoryStream())
{
using (GZipStream gzipStream = new GZipStream(outStream, CompressionMode.Compress))
using (MemoryStream srcStream = new MemoryStream(data))
CopyTo(srcStream, gzipStream);
return outStream.ToArray();
}
}
public static byte[] decompress(byte[] compressed)
{
using (MemoryStream inStream = new MemoryStream(compressed))
using (GZipStream gzipStream = new GZipStream(inStream, CompressionMode.Decompress))
using (MemoryStream outStream = new MemoryStream())
{
CopyTo(gzipStream, outStream);
return outStream.ToArray();
}
}
public static void CopyTo(Stream input, Stream output)
{
byte[] buffer = new byte[16 * 1024];
int bytesRead;
while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, bytesRead);
}
}
#endregion
}
folder structure inside the zip file images->sample images->10 png files

GZipStream can only (de)compress streams. In other words, you cannot restore folder structure using it. Use ZipFile or, if you cannot use framework 4.5, SharpZipLib.

How to compress a file using GZipStream in C# in .NET 2.0

I want to compress a file that has binary data and save the compressed data in another file:
FileStream fileStream = new FileStream("compressed_file.bin", FileMode.Create, FileAccess.Write);
GZipStream compressionStream = new GZipStream(fileStream, CompressionMode.Compress);
StreamWriter writer = new StreamWriter(compressionStream);
writer.Write(File.ReadAllBytes("file_to_be_compressed.bin"), 0, File.ReadAllBytes("file_to_be_compressed.bin").Length);
writer.Close();
I get following error:
cannot convert from 'byte[]' to 'char[]'
in line:
writer.Write(File.ReadAllBytes("file_to_be_compressed.bin"), 0, File.ReadAllBytes("file_to_be_compressed.bin").Length)
And is it fine to convert the binary data of file to byte array, or is it better to pass binary data of file as stream?
Note: CopyTo is not available in .NET 2.0

Try this, according to http://www.dotnetperls.com/gzipstream
using System.IO;
using System.IO.Compression;
using System.Text;
class Program
{
static void Main()
{
try
{
// 1.
// Starting file is 26,747 bytes.
string anyString = File.ReadAllText("TextFile1.txt");
// 2.
// Output file is 7,388 bytes.
CompressStringToFile("new.gz", anyString);
}
catch
{
// Could not compress.
}
}
public static void CompressStringToFile(string fileName, string value)
{
// A.
// Write string to temporary file.
string temp = Path.GetTempFileName();
File.WriteAllText(temp, value);
// B.
// Read file into byte array buffer.
byte[] b;
using (FileStream f = new FileStream(temp, FileMode.Open))
{
b = new byte[f.Length];
f.Read(b, 0, (int)f.Length);
}
// C.
// Use GZipStream to write compressed bytes to target file.
using (FileStream f2 = new FileStream(fileName, FileMode.Create))
using (GZipStream gz = new GZipStream(f2, CompressionMode.Compress, false))
{
gz.Write(b, 0, b.Length);
}
}
}

DeflateStream / GZipStream to CryptoStream and vice versa

I want to to compress and encrypt a file in one go by using this simple code:
public void compress(FileInfo fi, Byte[] pKey, Byte[] pIV)
{
// Get the stream of the source file.
using (FileStream inFile = fi.OpenRead())
{
// Create the compressed encrypted file.
using (FileStream outFile = File.Create(fi.FullName + ".pebf"))
{
using (CryptoStream encrypt = new CryptoStream(outFile, Rijndael.Create().CreateEncryptor(pKey, pIV), CryptoStreamMode.Write))
{
using (DeflateStream cmprss = new DeflateStream(encrypt, CompressionLevel.Optimal))
{
// Copy the source file into the compression stream.
inFile.CopyTo(cmprss);
Console.WriteLine("Compressed {0} from {1} to {2} bytes.", fi.Name, fi.Length.ToString(), outFile.Length.ToString());
}
}
}
}
}
The following lines will restore the encrypted and compressed file back to the original:
public void decompress(FileInfo fi, Byte[] pKey, Byte[] pIV)
{
// Get the stream of the source file.
using (FileStream inFile = fi.OpenRead())
{
// Get original file extension, for example "doc" from report.doc.gz.
String curFile = fi.FullName;
String origName = curFile.Remove(curFile.Length - fi.Extension.Length);
// Create the decompressed file.
using (FileStream outFile = File.Create(origName))
{
using (CryptoStream decrypt = new CryptoStream(inFile, Rijndael.Create().CreateDecryptor(pKey, pIV), CryptoStreamMode.Read))
{
using (DeflateStream dcmprss = new DeflateStream(decrypt, CompressionMode.Decompress))
{
// Copy the uncompressed file into the output stream.
dcmprss.CopyTo(outFile);
Console.WriteLine("Decompressed: {0}", fi.Name);
}
}
}
}
}
This works also with GZipStream.

A decompressing stream is expected to be read from, not written to. (unlike a CryptoStream, which supports all four combinations of read/write and encrypt/decrypt)
You should create the DeflateStream around a CryptoStreamMode.Read stream around the input file, then copy from that directly to the output stream.

How to read byte array into FileStream

I have an byte array and I want to read the byte array into a FileStream. Below is my sample of code:
string fileName = "test.txt";
byte[] file = File.ReadAllBytes(Server.MapPath("~/Files/" + fileName));
FileStream fs = new FileStream();
fs.ReadByte(file);
object obj = LoadFile<object>(fs);
public static T LoadFile<T>(FileStream fs)
{
using (GZipStream gzip = new GZipStream(fs, CompressionMode.Decompress))
{
BinaryFormatter bf = new BinaryFormatter();
return (T)bf.Deserialize(gzip);
}
}
In the method above, I have use FileStream to read byte array, but unlucky fs.ReadByte cannot read byte array. Any help please focus on how to Read byte array into a FileStream for using as a parameter in method "LoadFile". Please do not read directly the file into FileStream because the file here is loaded from somewhere else like from database or other source.

string fileName = "test.txt";
byte[] file = File.ReadAllBytes(Server.MapPath("~/Files/" + fileName));
MemoryStream memStream = new MemoryStream();
BinaryFormatter binForm = new BinaryFormatter();
memStream.Write(file, 0, file.Length);
memStream.Seek(0, SeekOrigin.Begin);
Object obj = (Object)binForm.Deserialize(memStream);

I'm not sure where the misunderstanding is. FileStream represents a file on disk. You cannot "read bytes into it" without writing them to disk and you cannot read from it without reading from disk.
Maybe what you want is a MemoryStream which can contain arbitrary contents.
Both derive from Stream.

Yeah! Now I got a good solution after doing some more research. As the topic I have posted "How to read byte array into FileStream". We cannot read byte array into FileStream, it just use to read a file on driver to byte array. So I have change a little bit on my code, and now I have a file to read it using FileStream. How I made a file?
In this context I have an object. The object is anything as you want!
I use a collection as a samble object.
Collection<object> list = new Collection<object>();
//Now I will write this list to a file. fileName is what you want and be sure that folder Files is exist on server or at the root folder of your project
WriteFile(list, Server.MapPath("~/Files/" + fileName));
//The method to write object to file is here
public static void WriteFile<T>(T obj, string path)
{
FileStream serializeStream = new FileStream(path, FileMode.Create);
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(serializeStream, obj);
serializeStream.Flush();
serializeStream.Close();
}
After I have wrote my object to a file, I need a method to read it back to object. So I do write this method:
public static Collection<object> ReatFile(string fileName){
//I have to read the file which I have wrote to an byte array
byte[] file;
using (var stream = new FileStream(Server.MapPath("~/Files/" + fileName), FileMode.Open, FileAccess.Read))
{
using (var reader = new BinaryReader(stream))
{
file = reader.ReadBytes((int)stream.Length);
}
}
//And now is what I have to do with the byte array of file is to convert it back to object which I have wrote it into a file
//I am using MemoryStream to convert byte array back to the original object.
MemoryStream memStream = new MemoryStream();
BinaryFormatter binForm = new BinaryFormatter();
memStream.Write(file, 0, file.Length);
memStream.Seek(0, SeekOrigin.Begin);
Object obj = (Object)binForm.Deserialize(memStream);
Collection<object> list = (Collection<object>)obj;
return list;
}
After doing some steps above, I am now can write any type object to file and then read it back to original object. Thank too much for any help I have got there.

Why do you run File.ReadAllBytes prior to the usage of your FileStream?
string fileName = "test.txt";
using(FileStream fs = new FileStream(Server.MapPath("~/Files/" + fileName), FileMode.Open, FileAccess.Read))
{
object obj = LoadFile<object>(fs);
fs.Close();
}