I want to iterate through the contents of a zipped archive and, where the contents are readable, display them. I can do this for text based files, but can't seem to work out how to pull out binary data from things like images. Here's what I have:
var zipArchive = new System.IO.Compression.ZipArchive(stream);
foreach (var entry in zipArchive.Entries)
{
using (var entryStream = entry.Open())
{
if (IsFileBinary(entry.Name))
{
using (BinaryReader br = new BinaryReader(entryStream))
{
//var fileSize = await reader.LoadAsync((uint)entryStream.Length);
var fileSize = br.BaseStream.Length;
byte[] read = br.ReadBytes((int)fileSize);
binaryContent = read;
I can see inside the zip file, but calls to Length result in an OperationNotSupported error. Also, given that I'm getting a long and then having to cast to an integer, it feels like I'm missing something quite fundamental about how this should work.
I think the stream will decompress the data as it is read, which means that the stream cannot know the decompressed length. Calling entry.Length should return the correct size value that you can use. You can also call entry.CompressedLength to get the compressed size.
Just copy the stream into a file or another stream:
using (var fs = await file.OpenStreamForWriteAsync())
{
using (var src = entry.Open())
{
var buffLen = 1024;
var buff = new byte[buffLen];
int read;
while ((read = await src.ReadAsync(buff, 0, buffLen)) > 0)
{
await fs.WriteAsync(buff, 0, read);
await fs.FlushAsync();
}
}
}
Related
I've got a couple of questions regarding the use of ShareZipLib.
I'm using 2 different methods to create a zip file and updating one which seems like overkill. Is there a better way?
Create Zip File:
using (var zipFile = File.Open(zipFilename, FileMode.CreateNew))
using (var outputStream = new ZipOutputStream(zipFile))
{
outputStream.Password = password;
outputStream.SetLevel(compressionLevel); // 0 - store only to 9 - means best compression
var buffer = new byte[4096];
foreach (var file in filenames)
{
var entry = new ZipEntry(Path.GetFileName(file));
entry.DateTime = DateTime.Now;
outputStream.PutNextEntry(entry);
using (var fs = File.OpenRead(file))
{
var sourceBytes = 0;
do
{
sourceBytes = fs.Read(buffer, 0, buffer.Length);
outputStream.Write(buffer, 0, sourceBytes);
} while (sourceBytes > 0);
}
}
outputStream.Finish();
outputStream.Close();
}
Update ZipFile:
using (var zipFile = File.Open(zipFilename, FileMode.Append))
using (var outputStream = new ZipFile(zipFile))
{
outputStream.Password = password;
outputStream.BeginUpdate();
foreach (var filename in filenames)
{
var entry = new ZipEntry(Path.GetFileName(filename));
entry.DateTime = DateTime.Now;
outputStream.Add(entry);
}
outputStream.CommitUpdate();
}
I've tried using the File.Open(zipFilename, FileMode.Append) on the create method hoping I'd be able to append files to the zip file but it just overrides it. The only other way I found is the second method but as I said, it seems like an overkill. Hopefully, this can be simplified and this brings my second question/issue.
Why is the compression level set at the stream level, which makes sense to me by the way, when creating a new zip file but when you are updating a zip file, you have to set it at the file level.
Also when setting it at the stream level, it requires a number from 0 to 9.
outputStream.SetLevel(compressionLevel);
But when setting it at the file level on the update, it uses an enum:
Surely, you don't want to have different level of compression when adding and updating but 0 to 9 are not the values provided in the enum.
Thanks.
I have a big file, and I want to send it to Web API which will send it to Amazon. Since file is big I want to send file to Amazon in chunk wise.
So If I have 1 GB file, I want my API to receive file in let's say 20 MB chunk so that I can send it to Amazon and then again receive 20 MB chunk. How is this doable. Below is my attempt.
public async Task<bool> Upload()
{
var fileuploadPath = ConfigurationManager.AppSettings["FileUploadLocation"];
var provider = new MultipartFormDataStreamProvider(fileuploadPath);
var content = new StreamContent(HttpContext.Current.Request.GetBufferlessInputStream(true));
// Now code below writes to a folder, but I want to make sure I read it as soon as I receive some chunk
await content.ReadAsMultipartAsync(provider);
return true;
}
Pseudo Code:
While (await content.ReadAsMultipartAsync(provider) == 20 MB chunk)
{
//Do something
// Then again do something with rest of chunk and so on.
}
File is as large as 1 GB.
As of now entire file is getting sent by this line of code:
await content.ReadAsMultipartAsync(provider);
I am lost here please help me. All I want is receive file in small chunk and process it.
P.S: I am sending file as MultiPart/Form-Data from Postman to test.
Attempt No 2:
var filesReadToProvider = await Request.Content.ReadAsMultipartAsync();
foreach (var content in filesReadToProvider.Contents)
{
var stream = await content.ReadAsStreamAsync();
using (StreamReader sr = new StreamReader(stream))
{
string line = "";
while ((line = sr.ReadLine()) != null)
{
using (MemoryStream outputStream = new MemoryStream())
using (StreamWriter sw = new StreamWriter(outputStream))
{
sw.WriteLine(line);
sw.Flush();
// Do Something
}
}
}
}
No time to test this, but the ReadBlock method seems to be what you want to use.
Should look something like what I have below, but it assumes all your other code is good and you just needed some help with the buffering. This is a "blocking" read operation, but there is also a ReadBlockAsync method which returns a Task.
const int bufferSize= 1024;
var filesReadToProvider = await Request.Content.ReadAsMultipartAsync();
foreach (var content in filesReadToProvider.Contents)
{
var stream = await content.ReadAsStreamAsync();
using (StreamReader sr = new StreamReader(stream))
{
int bytesRead;
char[] buffer = new char[bufferSize];
while ((bytesRead = sr.ReadBlock(buffer, 0, bufferSize)) > 0)
{
// Do something with the first <bytesRead> of buffer and
// not with <bufferSize> as <bytesRead> will contain the
// number of bytes actually read by the call to ReadBlock
}
}
}
I am currently working on integrating Amazon Prime on our system and being stuck at getting the label back as ZPL format.
Basically, Amazon returns a base64 string, we will need to convert that string to a byte array, then save that array as a *.gzip file. From that gzip file, we can extract the content and get the zpl label content.
My question is, how we can do all of above without storing any temp files to system. I have researched some solutions but none is working for me.
My current code as below:
var str = "base64string";
var label = Convert.FromBase64String(str);
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("label.zip");
var entryStream = demoFile.Open();
using (var bw = new BinaryWriter(entryStream))
{
bw.Write(label);
}
var data = new MemoryStream();
using (var zip = ZipFile.Read(entryStream))
{
zip["label"].Extract(data);
}
data.Seek(0, SeekOrigin.Begin);
entryStream.Close();
}
using (var fileStream = new FileStream(#"D:\test.zip", FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
If I save the file as test.zip, I can successfully get the label back. But if I try to extract it directly to another stream, I get an error
A stream from ZipArchiveEntry has been disposed
I've done something similar, taking PNG label data from a zipped web response. This is how I went about that
using (WebClient webClient = new WebClient())
{
// Download. Expect this to be a zip file
byte[] data = webClient.DownloadData(urlString);
MemoryStream memoryStream = new MemoryStream(data);
ZipArchive zipArchive = new ZipArchive(memoryStream);
foreach (var zipEntry in zipArchive.Entries)
{
// Can check file name here and ignore anything in zip we're not expecting
if (!zipEntry.Name.EndsWith(".png")) continue;
// Open zip entry as stream
Stream extractedFile = zipEntry.Open();
// Convert stream to memory stream
MemoryStream extractedMemoryStream = new MemoryStream();
extractedFile.CopyTo(extractedMemoryStream);
// At this point the extractedMemoryStream is a sequence of bytes containing image data.
// In this test project I'm pushing that into a bitmap image, just to see something on screen, but could as easily be written to a file or passed for storage to sql or whatever.
BitmapDecoder decoder = PngBitmapDecoder.Create(extractedMemoryStream, BitmapCreateOptions.None, BitmapCacheOption.OnLoad);
BitmapFrame frame = decoder.Frames.First();
frame.Freeze();
this.LabelImage.Source = frame;
}
}
I was overthinking it. I finally found a simple way to do it. We just need to convert that base64 string to bytes array and use GzipStream to directly decompress it. I leave the solution here in case someone needs it. Thanks!
var label = Convert.FromBase64String(str);
using (var compressedStream = new MemoryStream(label))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
I am try to download a zip file via a url to extract files from. I would rather not have to save it a temp file (which works fine) and rather keep it in memory - it is not very big. For example, if I try to download this file:
http://phs.googlecode.com/files/Download%20File%20Test.zip
using this code:
using Ionic.Zip;
...
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(URL);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.ContentLength > 0)
{
using (MemoryStream zipms = new MemoryStream())
{
int bytesRead;
byte[] buffer = new byte[32768];
using (Stream stream = response.GetResponseStream())
{
while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
zipms.Write(buffer, 0, bytesRead);
ZipFile zip = ZipFile.Read(stream); // <--ERROR: "This stream does not support seek operations. "
}
using (ZipFile zip = ZipFile.Read(zipms)) // <--ERROR: "Could not read block - no data! (position 0x00000000) "
using (MemoryStream txtms = new MemoryStream())
{
ZipEntry csentry= zip["Download File Test.cs"];
csentry.Extract(txtms);
txtms.Position = 0;
using (StreamReader reader = new StreamReader(txtms))
{
string csentry = reader.ReadToEnd();
}
}
}
}
...
Note where i flagged the errors I am receiving. With the first one, it does not like the System.Net.ConnectStream. If I comment that line out and allow it to hit the line where I note the second error, it does not like the MemoryStream. I did see this posting: https://stackoverflow.com/a/6377099/1324284 but I am having the same issues that others mention about not having more then 4 overloads of the Read method so I cannot try the WebClient.
However, if I do everything via a FileStream and save it to a temp location first, then point ZipFile.Read at that temp location, everything works including extracting any contained files into a MemoryStream.
Thanks for any help.
You need to Flush() your MemoryStream and set the Position to 0 before you read from it, otherwise you are trying to read from the current position (where there is nothing).
For your code:
ZipFile zip;
using (Stream stream = response.GetResponseStream())
{
while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
zipms.Write(buffer, 0, bytesRead);
zipms.Flush();
zipms.Position = 0;
zip = ZipFile.Read(zipms);
}
I have an application in ASP.NET where user can upload ZIP file. I'm trying to extract file using ICSharpZipLib (I also tried DotNetZip, but had same issue).
This zip file contains single xml document (9KB before compress).
When I open this file with other applications on my desktop (7zip, windows explorer) it seems to be ok.
My unzip method throws System.OutOfMemoryException and I have no idea why is that. When I debugged my unziping method I noticed that zipInputStreams' Length property throws Exception and is not available:
Stream UnZipSingleFile(Stream memoryStream)
{
var zipInputStream = new ZipInputStream(memoryStream);
memoryStream.Position = 0;
zipInputStream.GetNextEntry();
MemoryStream unzippedStream = new MemoryStream();
int len;
byte[] buf = new byte[4096];
while ((len = zipInputStream.Read(buf, 0, buf.Length)) > 0)
{
unzippedStream.Write(buf, 0, len);
}
unzippedStream.Position = 0;
memoryStream.Position = 0;
return unzippedStream;
}
and here's how I get string of unzippedStream:
string GetString()
{
var reader = new StreamReader(unzippedStream);
var result = reader.ReadToEnd();
unzippedStream.Position = 0;
return result;
}
From their wiki:
"Sharpzip supports Zip files using both stored and deflate compression methods and also supports old (PKZIP 2.0) style and AES encryption"
Are you sure the format of the uploaded zip file is acceptable for SharpZipLib?
While this post is quite old, I think it could be beneficial to illustrate how I did this for compression and decompression using ICSharpZipLib (C# package version 1.1.0). I put this together by looking into the examples shown here (see ie. these compression and decompression examples).
Assumption: The input to the compression and decompression below should be in bytes. If you have ie. an xml file you could load it to an XDocument, and convert it into an XmlDocument with .ToXmlDocument(). From there, you could access the string contents by calling .OuterXml, and converting the string to a byte array.
// Compression (inputBytes = ie. string-to-compress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipOutputStream(outputStream))
{
zipStream.SetLevel(3);
var newEntry = new ZipEntry("someFilename.someExtension");
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(dataStream, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false;
}
outputStream.Position = 0;
var outputBytes = outputStream.ToArray();
// Decompression (inputBytes = ie. string-to-decompress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipInputStream(dataStream))
{
while (zipStream.GetNextEntry() is ZipEntry zipEntry)
{
var buffer = new byte[4096];
StreamUtils.Copy(zipStream, outputStream, buffer);
}
}
var outputBytes = outputStream.ToArray();