I have an application in ASP.NET where user can upload ZIP file. I'm trying to extract file using ICSharpZipLib (I also tried DotNetZip, but had same issue).
This zip file contains single xml document (9KB before compress).
When I open this file with other applications on my desktop (7zip, windows explorer) it seems to be ok.
My unzip method throws System.OutOfMemoryException and I have no idea why is that. When I debugged my unziping method I noticed that zipInputStreams' Length property throws Exception and is not available:
Stream UnZipSingleFile(Stream memoryStream)
{
var zipInputStream = new ZipInputStream(memoryStream);
memoryStream.Position = 0;
zipInputStream.GetNextEntry();
MemoryStream unzippedStream = new MemoryStream();
int len;
byte[] buf = new byte[4096];
while ((len = zipInputStream.Read(buf, 0, buf.Length)) > 0)
{
unzippedStream.Write(buf, 0, len);
}
unzippedStream.Position = 0;
memoryStream.Position = 0;
return unzippedStream;
}
and here's how I get string of unzippedStream:
string GetString()
{
var reader = new StreamReader(unzippedStream);
var result = reader.ReadToEnd();
unzippedStream.Position = 0;
return result;
}
From their wiki:
"Sharpzip supports Zip files using both stored and deflate compression methods and also supports old (PKZIP 2.0) style and AES encryption"
Are you sure the format of the uploaded zip file is acceptable for SharpZipLib?
While this post is quite old, I think it could be beneficial to illustrate how I did this for compression and decompression using ICSharpZipLib (C# package version 1.1.0). I put this together by looking into the examples shown here (see ie. these compression and decompression examples).
Assumption: The input to the compression and decompression below should be in bytes. If you have ie. an xml file you could load it to an XDocument, and convert it into an XmlDocument with .ToXmlDocument(). From there, you could access the string contents by calling .OuterXml, and converting the string to a byte array.
// Compression (inputBytes = ie. string-to-compress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipOutputStream(outputStream))
{
zipStream.SetLevel(3);
var newEntry = new ZipEntry("someFilename.someExtension");
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(dataStream, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false;
}
outputStream.Position = 0;
var outputBytes = outputStream.ToArray();
// Decompression (inputBytes = ie. string-to-decompress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipInputStream(dataStream))
{
while (zipStream.GetNextEntry() is ZipEntry zipEntry)
{
var buffer = new byte[4096];
StreamUtils.Copy(zipStream, outputStream, buffer);
}
}
var outputBytes = outputStream.ToArray();
Related
I am currently working on integrating Amazon Prime on our system and being stuck at getting the label back as ZPL format.
Basically, Amazon returns a base64 string, we will need to convert that string to a byte array, then save that array as a *.gzip file. From that gzip file, we can extract the content and get the zpl label content.
My question is, how we can do all of above without storing any temp files to system. I have researched some solutions but none is working for me.
My current code as below:
var str = "base64string";
var label = Convert.FromBase64String(str);
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("label.zip");
var entryStream = demoFile.Open();
using (var bw = new BinaryWriter(entryStream))
{
bw.Write(label);
}
var data = new MemoryStream();
using (var zip = ZipFile.Read(entryStream))
{
zip["label"].Extract(data);
}
data.Seek(0, SeekOrigin.Begin);
entryStream.Close();
}
using (var fileStream = new FileStream(#"D:\test.zip", FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
If I save the file as test.zip, I can successfully get the label back. But if I try to extract it directly to another stream, I get an error
A stream from ZipArchiveEntry has been disposed
I've done something similar, taking PNG label data from a zipped web response. This is how I went about that
using (WebClient webClient = new WebClient())
{
// Download. Expect this to be a zip file
byte[] data = webClient.DownloadData(urlString);
MemoryStream memoryStream = new MemoryStream(data);
ZipArchive zipArchive = new ZipArchive(memoryStream);
foreach (var zipEntry in zipArchive.Entries)
{
// Can check file name here and ignore anything in zip we're not expecting
if (!zipEntry.Name.EndsWith(".png")) continue;
// Open zip entry as stream
Stream extractedFile = zipEntry.Open();
// Convert stream to memory stream
MemoryStream extractedMemoryStream = new MemoryStream();
extractedFile.CopyTo(extractedMemoryStream);
// At this point the extractedMemoryStream is a sequence of bytes containing image data.
// In this test project I'm pushing that into a bitmap image, just to see something on screen, but could as easily be written to a file or passed for storage to sql or whatever.
BitmapDecoder decoder = PngBitmapDecoder.Create(extractedMemoryStream, BitmapCreateOptions.None, BitmapCacheOption.OnLoad);
BitmapFrame frame = decoder.Frames.First();
frame.Freeze();
this.LabelImage.Source = frame;
}
}
I was overthinking it. I finally found a simple way to do it. We just need to convert that base64 string to bytes array and use GzipStream to directly decompress it. I leave the solution here in case someone needs it. Thanks!
var label = Convert.FromBase64String(str);
using (var compressedStream = new MemoryStream(label))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
I am trying to implement SharpZipLib
So I copied the below snippet from their samples:
// Compresses the supplied memory stream, naming it as zipEntryName, into a zip,
// which is returned as a memory stream or a byte array.
//
public MemoryStream CreateToMemoryStream(MemoryStream memStreamIn, string zipEntryName)
{
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipStream = new ZipOutputStream(outputMemStream);
zipStream.SetLevel(3); //0-9, 9 being the highest level of compression
ZipEntry newEntry = new ZipEntry(zipEntryName);
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(memStreamIn, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false; // False stops the Close also Closing the underlying stream.
zipStream.Close(); // Must finish the ZipOutputStream before using outputMemStream.
outputMemStream.Position = 0;
return outputMemStream;
// Alternative outputs:
// ToArray is the cleaner and easiest to use correctly with the penalty of duplicating allocated memory.
byte[] byteArrayOut = outputMemStream.ToArray();
// GetBuffer returns a raw buffer raw and so you need to account for the true length yourself.
byte[] byteArrayOut = outputMemStream.GetBuffer();
long len = outputMemStream.Length;
}
I copy pasted that function and called it this way:
using (MemoryStream ms = new MemoryStream())
using (FileStream file = new FileStream(#"c:\file.jpg", FileMode.Open, FileAccess.Read))
{
byte[] bytes = new byte[file.Length];
file.Read(bytes, 0, (int)file.Length);
ms.Write(bytes, 0, (int)file.Length);
var result = SharpZip.CreateToMemoryStream(ms, "file.jpg");
result.WriteTo(new FileStream(#"c:\myzip.zip", FileMode.Create, System.IO.FileAccess.Write));
}
The myzip.zip is sucessfully created, but the file.jpg inside has zero bytes.
Any ideas?
Thanks a lot
It is necessary to "rewind" the input MemoryStream after writing the input file data:
using (MemoryStream ms = new MemoryStream())
using (FileStream file = File.OpenRead(#"input file path"))
{
byte[] bytes = new byte[file.Length];
file.Read(bytes, 0, (int)file.Length);
ms.Write(bytes, 0, (int)file.Length);
ms.Position = 0; // "Rewind" the stream to the beginning.
var result = SharpZip.CreateToMemoryStream(ms, "file.jpg");
using (var outputStream = File.Create(#"output file path"))
{
result.WriteTo(outputStream);
}
}
Alternative (slightly simplified) version of the implementation:
var bytes = File.ReadAllBytes(#"input file path");
using (MemoryStream ms = new MemoryStream(bytes))
{
var result = SharpZip.CreateToMemoryStream(ms, "file.jpg");
using (var outputStream = File.Create(#"output file path"))
{
result.WriteTo(outputStream);
}
}
I want to iterate through the contents of a zipped archive and, where the contents are readable, display them. I can do this for text based files, but can't seem to work out how to pull out binary data from things like images. Here's what I have:
var zipArchive = new System.IO.Compression.ZipArchive(stream);
foreach (var entry in zipArchive.Entries)
{
using (var entryStream = entry.Open())
{
if (IsFileBinary(entry.Name))
{
using (BinaryReader br = new BinaryReader(entryStream))
{
//var fileSize = await reader.LoadAsync((uint)entryStream.Length);
var fileSize = br.BaseStream.Length;
byte[] read = br.ReadBytes((int)fileSize);
binaryContent = read;
I can see inside the zip file, but calls to Length result in an OperationNotSupported error. Also, given that I'm getting a long and then having to cast to an integer, it feels like I'm missing something quite fundamental about how this should work.
I think the stream will decompress the data as it is read, which means that the stream cannot know the decompressed length. Calling entry.Length should return the correct size value that you can use. You can also call entry.CompressedLength to get the compressed size.
Just copy the stream into a file or another stream:
using (var fs = await file.OpenStreamForWriteAsync())
{
using (var src = entry.Open())
{
var buffLen = 1024;
var buff = new byte[buffLen];
int read;
while ((read = await src.ReadAsync(buff, 0, buffLen)) > 0)
{
await fs.WriteAsync(buff, 0, read);
await fs.FlushAsync();
}
}
}
My application will be storing large amounts of cached data to local storage for performance and disconnected purposes. I have tried to use SharpZipLib to compress the cache files that are created, but I'm having some difficulty.
I can get the file created, but it is invalid. Windows' built-in zip system and 7-zip both indicate that the file is invalid. When I attempt to open the file programmatically through SharpZipLib, I get the exception "Wrong Central Directory signature." I think part of the problem is that I'm creating the zip file directly from a MemoryStream, so there is no "root" directory. Not sure how to create one programmatically with SharpZipLib.
The EntityManager below is the IdeaBlade DevForce-generated "datacontext." It can save its contents to a stream for purposes of serializing to disk for caching.
Here's my code:
private void SaveCacheFile(string FileName, EntityManager em)
{
using (IsolatedStorageFile isf = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream isfs = new IsolatedStorageFileStream(FileName, System.IO.FileMode.CreateNew, isf))
{
MemoryStream inStream = new MemoryStream();
MemoryStream outStream = new MemoryStream();
Crc32 crc = new Crc32();
em.CacheStateManager.SaveCacheState(inStream, false, true);
inStream.Position = 0;
ZipOutputStream zipStream = new ZipOutputStream(outStream);
zipStream.IsStreamOwner = false;
zipStream.SetLevel(3);
ZipEntry newEntry = new ZipEntry(FileName);
byte[] buffer = new byte[inStream.Length];
inStream.Read(buffer, 0, buffer.Length);
newEntry.DateTime = DateTime.Now;
newEntry.Size = inStream.Length;
crc.Reset();
crc.Update(buffer);
newEntry.Crc = crc.Value;
zipStream.PutNextEntry(newEntry);
buffer = null;
outStream.Position = 0;
inStream.Position = 0;
StreamUtils.Copy(inStream, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.Finish();
zipStream.Close();
outStream.Position = 0;
StreamUtils.Copy(outStream, isfs, new byte[4096]);
outStream.Close();
}
}
}
Creating a zip file straight from memory is not your problem. SharpZipLib uses the parameter in the ZipEntry constructor to determine the path, and does not care if that path has subdirectories or not.
using (ZipOutputStream zipStreamOut = new ZipOutputStream(outputstream))
{
zipStreamOut.PutNextEntry(new ZipEntry("arbitrary.ext"));
zipstreamOut.Write(mybytearraydata, 0, mybytearraydata.Length);
zipStreamOut.Finish();
//Line below needed if outputstream is a MemoryStream and you are
//passing it to a function expecting a stream.
outputstream.Position = 0;
//DoStuff. Optional; Not necessary if e.g., outputstream is a FileStream.
}
Remove outStream.Position = 0; and it works.
I'm interested in pulling a file from online a .txt file.
The txt file stores:
filename
md5 hash
filename
md5 hash
I am interested in getting the data from online then comparing the data to local files.
byte[] buffer = new byte[512];
WebRequest test = WebRequest.Create("http://www.domain.com/file.txt");
Stream something = test.GetRequestStream();
something.Read(buffer,0,20);
I don't quite understand streams and how to go about reading just one line from the file. I do not want to download the file first then retrieve the data. I'm interested in just pulling it from online. How different are "streams" vs normal IO, with StreamWriter and StreamReader?
EDIT--
WebRequest myWebRequest = WebRequest.Create("http://www.domain.com/file.txt");
WebResponse myReponse = myWebRequest.GetResponse();
Stream recStream = myReponse.GetResponseStream();
StreamReader reader = new StreamReader(recStream);
txt_status.Text = reader.ReadLine();
GetRequestStream provides a stream for writing to. If you want the returned data to walk through make use of GetResponseStream
...
Stream ReceiveStream = myWebResponse.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
// Pipe the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader( ReceiveStream, encode );
Console.WriteLine("\nResponse stream received");
Char[] read = new Char[256];
// Read 256 charcters at a time.
int count = readStream.Read( read, 0, 256 );
Console.WriteLine("HTML...\r\n");
while (count > 0)
{
// Dump the 256 characters on a string and display the string onto the console.
String str = new String(read, 0, count);
Console.Write(str);
count = readStream.Read(read, 0, 256);
}
...
If you're reading text, try using a TextReader
WebRequest test = WebRequest.Create("http://www.domain.com/file.txt");
Stream something = test.GetRequestStream();
TextReader reader = (TextReader)new StreamReader(something);
string textfile = reader.ReadToEnd();
All a stream is, is a sequence of bytes. MemoryStreams, FileStream, etc. all inherit from System.IO.Stream
If you are simply attempting to compare the MD5 has against a local file, you could do something such as the following (not tested):
// Download File
WebClient wc = new WebClient();
byte[] bytes = wc.DownloadData();
MD5 md5 = System.Security.Cryptography.MD5.Create();
byte[] hash = md5.ComputeHash(bytes);
StringBuilder onlineFile = new StringBuilder();
for (int i = 0; i < hash.Length; i++)
{
onlineFile.Append(hash[i].ToString("X2"));
}
// Load Local File
FileStream fs = new FileStream(#"c:\yourfile.txt",FileMode.Open);
byte[] fileBytes = new byte[fs.Length];
fs.Read(fileBytes, 0, fileBytes.Length);
byte[] hash = md5.ComputeHash(fileBytes);
StringBuilder localFile = new StringBuilder();
for (int i = 0; i < hash.Length; i++)
{
onlineFile.Append(hash[i].ToString("X2"));
}
if(localFile.ToString() == onlineFile.ToString())
{
// Match
}