My application will be storing large amounts of cached data to local storage for performance and disconnected purposes. I have tried to use SharpZipLib to compress the cache files that are created, but I'm having some difficulty.
I can get the file created, but it is invalid. Windows' built-in zip system and 7-zip both indicate that the file is invalid. When I attempt to open the file programmatically through SharpZipLib, I get the exception "Wrong Central Directory signature." I think part of the problem is that I'm creating the zip file directly from a MemoryStream, so there is no "root" directory. Not sure how to create one programmatically with SharpZipLib.
The EntityManager below is the IdeaBlade DevForce-generated "datacontext." It can save its contents to a stream for purposes of serializing to disk for caching.
Here's my code:
private void SaveCacheFile(string FileName, EntityManager em)
{
using (IsolatedStorageFile isf = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream isfs = new IsolatedStorageFileStream(FileName, System.IO.FileMode.CreateNew, isf))
{
MemoryStream inStream = new MemoryStream();
MemoryStream outStream = new MemoryStream();
Crc32 crc = new Crc32();
em.CacheStateManager.SaveCacheState(inStream, false, true);
inStream.Position = 0;
ZipOutputStream zipStream = new ZipOutputStream(outStream);
zipStream.IsStreamOwner = false;
zipStream.SetLevel(3);
ZipEntry newEntry = new ZipEntry(FileName);
byte[] buffer = new byte[inStream.Length];
inStream.Read(buffer, 0, buffer.Length);
newEntry.DateTime = DateTime.Now;
newEntry.Size = inStream.Length;
crc.Reset();
crc.Update(buffer);
newEntry.Crc = crc.Value;
zipStream.PutNextEntry(newEntry);
buffer = null;
outStream.Position = 0;
inStream.Position = 0;
StreamUtils.Copy(inStream, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.Finish();
zipStream.Close();
outStream.Position = 0;
StreamUtils.Copy(outStream, isfs, new byte[4096]);
outStream.Close();
}
}
}
Creating a zip file straight from memory is not your problem. SharpZipLib uses the parameter in the ZipEntry constructor to determine the path, and does not care if that path has subdirectories or not.
using (ZipOutputStream zipStreamOut = new ZipOutputStream(outputstream))
{
zipStreamOut.PutNextEntry(new ZipEntry("arbitrary.ext"));
zipstreamOut.Write(mybytearraydata, 0, mybytearraydata.Length);
zipStreamOut.Finish();
//Line below needed if outputstream is a MemoryStream and you are
//passing it to a function expecting a stream.
outputstream.Position = 0;
//DoStuff. Optional; Not necessary if e.g., outputstream is a FileStream.
}
Remove outStream.Position = 0; and it works.
Related
I am trying to use SevenZipSharp to compress and decompress a memory stream. Compression is working fine but decompression is not. I think SevenZipSharp is not able to figure the archive type from the stream.
SevenZipCompressor compress = new SevenZip.SevenZipCompressor();
compress.CompressionLevel = CompressionLevel.Normal;
compress.CompressionMethod = CompressionMethod.Lzma
using (MemoryStream memStream = new MemoryStream())
{
compress.CompressFiles(memStream, #"d:\Temp1\MyFile.bmp");
using (FileStream file = new FileStream(#"d:\arch.7z", FileMode.Create, System.IO.FileAccess.Write))
{
memStream.CopyTo(file);
}
}
//works till here, file is created
Console.Read();
using (FileStream file = new FileStream(#"d:\arch.7z", FileMode.Open, System.IO.FileAccess.Read))
{
using (MemoryStream memStream = new MemoryStream())
{
file.CopyTo(memStream);
//throws exception here on this line
using (var extractor = new SevenZipExtractor(memStream))
{
extractor.ExtractFiles(#"d:\x", 0);
}
}
}
Try to see if your output file can be loaded using the 7Zip client. I'm guessing that it will fail.
The problem lies in the writing to the memorystream. Say, you write 100 bytes to the stream, it will be on position 100. When you use CopyTo, the stream will be copied from the current position, not the start of the stream.
So you'll have to reset the position to 0 after reading/writing to allow the next reader to read all the data. For instance when creating the 7Zip file:
using (MemoryStream memStream = new MemoryStream())
{
// Position starts at 0
compress.CompressFiles(memStream, #"d:\Temp1\MyFile.bmp");
// Position is now N
memStream.Position = 0; // <-- Reset the position to 0.
using (FileStream file = new FileStream(#"d:\arch.7z", FileMode.Create, System.IO.FileAccess.Write))
{
// Will copy all data in the stream from current position till the end of the stream.
memStream.CopyTo(file);
}
}
I am currently working on integrating Amazon Prime on our system and being stuck at getting the label back as ZPL format.
Basically, Amazon returns a base64 string, we will need to convert that string to a byte array, then save that array as a *.gzip file. From that gzip file, we can extract the content and get the zpl label content.
My question is, how we can do all of above without storing any temp files to system. I have researched some solutions but none is working for me.
My current code as below:
var str = "base64string";
var label = Convert.FromBase64String(str);
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("label.zip");
var entryStream = demoFile.Open();
using (var bw = new BinaryWriter(entryStream))
{
bw.Write(label);
}
var data = new MemoryStream();
using (var zip = ZipFile.Read(entryStream))
{
zip["label"].Extract(data);
}
data.Seek(0, SeekOrigin.Begin);
entryStream.Close();
}
using (var fileStream = new FileStream(#"D:\test.zip", FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
If I save the file as test.zip, I can successfully get the label back. But if I try to extract it directly to another stream, I get an error
A stream from ZipArchiveEntry has been disposed
I've done something similar, taking PNG label data from a zipped web response. This is how I went about that
using (WebClient webClient = new WebClient())
{
// Download. Expect this to be a zip file
byte[] data = webClient.DownloadData(urlString);
MemoryStream memoryStream = new MemoryStream(data);
ZipArchive zipArchive = new ZipArchive(memoryStream);
foreach (var zipEntry in zipArchive.Entries)
{
// Can check file name here and ignore anything in zip we're not expecting
if (!zipEntry.Name.EndsWith(".png")) continue;
// Open zip entry as stream
Stream extractedFile = zipEntry.Open();
// Convert stream to memory stream
MemoryStream extractedMemoryStream = new MemoryStream();
extractedFile.CopyTo(extractedMemoryStream);
// At this point the extractedMemoryStream is a sequence of bytes containing image data.
// In this test project I'm pushing that into a bitmap image, just to see something on screen, but could as easily be written to a file or passed for storage to sql or whatever.
BitmapDecoder decoder = PngBitmapDecoder.Create(extractedMemoryStream, BitmapCreateOptions.None, BitmapCacheOption.OnLoad);
BitmapFrame frame = decoder.Frames.First();
frame.Freeze();
this.LabelImage.Source = frame;
}
}
I was overthinking it. I finally found a simple way to do it. We just need to convert that base64 string to bytes array and use GzipStream to directly decompress it. I leave the solution here in case someone needs it. Thanks!
var label = Convert.FromBase64String(str);
using (var compressedStream = new MemoryStream(label))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
I am usig Ionic.Zip.ZipFile.I want to create 10 mb zip file.I need get size of ZipFile before save to disk.Is it possible ?
private static string tempPath = "#Temp Folder";
List<string> fileNames = new List<string>();
using (Ionic.Zip.ZipFile zf = new Ionic.Zip.ZipFile())
{
for (int i = 0; i < fileNames.Count; i++)
{
zf.AddFile(tempPath + fileNames[i], string.Empty);
//How can I get size of zf before save here ?
if(zf size==10mb)
{
zf.Save(tempPath + string.Format("{0}-{1}-{2}.zip","XXX", "XXX",
DateTime.Now.ToString("yyyyMMdd")));
}
}
}
You can save your zip into a MemoryStream:
var ms = new MemoryStream();
zip.Save(ms);
then read the MemoryStream.Length Property and get the size.
If you then still want to save it to disk just use the memory stream you already have:
FileStream file = new FileStream("file.zip", FileMode.Create, FileAccess.Write);
// I believe you'll need to rewind the stream before saving
ms.Seek(0, SeekOrigin.Begin);
ms.WriteTo(file);
file.Close();
ms.Close();
I am trying to implement SharpZipLib
So I copied the below snippet from their samples:
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName)
{
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}
I copy pasted that function and called it this way:
using (MemoryStream ms = new MemoryStream())
using (FileStream file = new FileStream(#"c:\file.jpg", FileMode.Open, FileAccess.Read))
{
byte[] bytes = new byte[file.Length];
file.Read(bytes, 0, (int)file.Length);
ms.Write(bytes, 0, (int)file.Length);
var result = SharpZip.CreateToMemoryStream(ms, "file.jpg");
result.WriteTo(new FileStream(#"c:\myzip.zip", FileMode.Create, System.IO.FileAccess.Write));
}
The myzip.zip is sucessfully created, but the file.jpg inside has zero bytes.
Any ideas?
Thanks a lot
It is necessary to "rewind" the input MemoryStream after writing the input file data:
using (MemoryStream ms = new MemoryStream())
using (FileStream file = File.OpenRead(#"input file path"))
{
byte[] bytes = new byte[file.Length];
file.Read(bytes, 0, (int)file.Length);
ms.Write(bytes, 0, (int)file.Length);
ms.Position = 0; // "Rewind" the stream to the beginning.
var result = SharpZip.CreateToMemoryStream(ms, "file.jpg");
using (var outputStream = File.Create(#"output file path"))
{
result.WriteTo(outputStream);
}
}
Alternative (slightly simplified) version of the implementation:
var bytes = File.ReadAllBytes(#"input file path");
using (MemoryStream ms = new MemoryStream(bytes))
{
var result = SharpZip.CreateToMemoryStream(ms, "file.jpg");
using (var outputStream = File.Create(#"output file path"))
{
result.WriteTo(outputStream);
}
}
I have an application in ASP.NET where user can upload ZIP file. I'm trying to extract file using ICSharpZipLib (I also tried DotNetZip, but had same issue).
This zip file contains single xml document (9KB before compress).
When I open this file with other applications on my desktop (7zip, windows explorer) it seems to be ok.
My unzip method throws System.OutOfMemoryException and I have no idea why is that. When I debugged my unziping method I noticed that zipInputStreams' Length property throws Exception and is not available:
Stream UnZipSingleFile(Stream memoryStream)
{
var zipInputStream = new ZipInputStream(memoryStream);
memoryStream.Position = 0;
zipInputStream.GetNextEntry();
MemoryStream unzippedStream = new MemoryStream();
int len;
byte[] buf = new byte[4096];
while ((len = zipInputStream.Read(buf, 0, buf.Length)) > 0)
{
unzippedStream.Write(buf, 0, len);
}
unzippedStream.Position = 0;
memoryStream.Position = 0;
return unzippedStream;
}
and here's how I get string of unzippedStream:
string GetString()
{
var reader = new StreamReader(unzippedStream);
var result = reader.ReadToEnd();
unzippedStream.Position = 0;
return result;
}
From their wiki:
"Sharpzip supports Zip files using both stored and deflate compression methods and also supports old (PKZIP 2.0) style and AES encryption"
Are you sure the format of the uploaded zip file is acceptable for SharpZipLib?
While this post is quite old, I think it could be beneficial to illustrate how I did this for compression and decompression using ICSharpZipLib (C# package version 1.1.0). I put this together by looking into the examples shown here (see ie. these compression and decompression examples).
Assumption: The input to the compression and decompression below should be in bytes. If you have ie. an xml file you could load it to an XDocument, and convert it into an XmlDocument with .ToXmlDocument(). From there, you could access the string contents by calling .OuterXml, and converting the string to a byte array.
// Compression (inputBytes = ie. string-to-compress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipOutputStream(outputStream))
{
zipStream.SetLevel(3);
var newEntry = new ZipEntry("someFilename.someExtension");
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(dataStream, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false;
}
outputStream.Position = 0;
var outputBytes = outputStream.ToArray();
// Decompression (inputBytes = ie. string-to-decompress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipInputStream(dataStream))
{
while (zipStream.GetNextEntry() is ZipEntry zipEntry)
{
var buffer = new byte[4096];
StreamUtils.Copy(zipStream, outputStream, buffer);
}
}
var outputBytes = outputStream.ToArray();