We Keep Coding

System.IO.Compression.ZipArchive corrupt when downloading [duplicate]

I'm trying to create a zip stream on the fly with some byte array data and make it download via my MVC action.
But the downloaded file always gives the following corrupted error when opened in windows.
And this error when I try to xtract from 7z
But note that the files extracted from the 7z is not corrupted.
I'm using ZipArchive and the below is my code.
private byte[] GetZippedPods(IEnumerable<POD> pods, long consignmentID)
{
using (var zipStream = new MemoryStream())
{
//Create an archive and store the stream in memory.
using (var zipArchive = new ZipArchive(zipStream, ZipArchiveMode.Create, true))
{
int index = 1;
foreach (var pod in pods)
{
var zipEntry = zipArchive.CreateEntry($"POD{consignmentID}{index++}.png", CompressionLevel.NoCompression);
using (var originalFileStream = new MemoryStream(pod.ByteData))
{
using (var zipEntryStream = zipEntry.Open())
{
originalFileStream.CopyTo(zipEntryStream);
}
}
}
return zipStream.ToArray();
}
}
}
public ActionResult DownloadPOD(long consignmentID)
{
var pods = _consignmentService.GetPODs(consignmentID);
var fileBytes = GetZippedPods(pods, consignmentID);
return File(fileBytes, MediaTypeNames.Application.Octet, $"PODS{consignmentID}.zip");
}
What am I doing wrong here.
Any help would be highly appreciated as I'm struggling with this for a whole day.
Thanks in advance

Move zipStream.ToArray() outside of the zipArchive using.
The reason for your problem is that the stream is buffered. There's a few ways to deal wtih it:
You can set the stream's AutoFlush property to true.
You can manually call .Flush() on the stream.
Or, since it's MemoryStream and you're using .ToArray(), you can simply allow the stream to be Closed/Disposed first (which we've done by moving it outside the using).

I Dispose ZipArchive And error solved
public static byte[] GetZipFile(Dictionary<string, List<FileInformation>> allFileInformations)
{
MemoryStream compressedFileStream = new MemoryStream();
//Create an archive and store the stream in memory.
using (var zipArchive = new ZipArchive(compressedFileStream, ZipArchiveMode.Create, true))
{
foreach (var fInformation in allFileInformations)
{
var files = allFileInformations.Where(x => x.Key == fInformation.Key).SelectMany(x => x.Value).ToList();
for (var i = 0; i < files.Count; i++)
{
ZipArchiveEntry zipEntry = zipArchive.CreateEntry(fInformation.Key + "/" + files[i].FileName);
var caseAttachmentModel = Encoding.UTF8.GetBytes(files[i].Content);
//Get the stream of the attachment
using (var originalFileStream = new MemoryStream(caseAttachmentModel))
using (var zipEntryStream = zipEntry.Open())
{
//Copy the attachment stream to the zip entry stream
originalFileStream.CopyTo(zipEntryStream);
}
}
}
//i added this line
zipArchive.Dispose();
return compressedFileStream.ToArray();
}
}
public void SaveZipFile(){
var zipFileArray = Global.GetZipFile(allFileInformations);
var zipFile = new MemoryStream(zipFileArray);
FileStream fs = new FileStream(path + "\\111.zip",
FileMode.Create,FileAccess.Write);
zipFile.CopyTo(fs);
zipFile.Flush();
fs.Close();
zipFile.Close();
}

I was also having problems with this and I found my issue was not the generation of the archive itself but rather how I was handing my GET request in AngularJS.
This post helped me: how to download a zip file using angular
The key was adding responseType: 'arraybuffer' to my $http call.
factory.serverConfigExportZIP = function () {
return $http({
url: dataServiceBase + 'serverConfigExport',
method: "GET",
responseType: 'arraybuffer'
})
};

you can remove "using" and use Dispose and Close methods
it's work for me
...
zip.Dispose();
zipStream.Close();
return zipStream.ToArray();

I know this is a C# question but for managed C++, delete the ZipArchive^ after you're done with it to fix the error.
ZipArchive^ zar = ZipFile::Open(starget, ZipArchiveMode::Create);
ZipFileExtensions::CreateEntryFromFile(zar, sfile1, "file.txt");
ZipFileExtensions::CreateEntryFromFile(zar, sfile2, "file2.txt");
delete zar;

when i wanted to create zip file directly from MemoryStream which i used for ZipArchive i was getting error ( "unexpected end of data" or zero length file )
there are three points to get ride of this error
set the last parameter of ZipArchive constructor to true ( it leaves to leave stream open after ZipArchive disposed )
call dispose() on ZipArchive and dispose it manually.
create another MemoryStream based on which you set in ZipArchive constructor, by calling ToArray() method.
here is sample code :
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create,))
{
foreach (var s3Object in objectList.S3Objects)
{
var entry = archive.CreateEntry(s3Object.Key, CompressionLevel.NoCompression);
using (var entryStream = entry.Open())
{
var request = new GetObjectRequest { BucketName = command.BucketName, Key = s3Object.Key };
using (var getObjectResponse = await client.GetObjectAsync(request))
{
await getObjectResponse.ResponseStream.CopyToAsync(entryStream);
}
}
}
archive.Dispose();
using (var fileStream = new FileStream(outputFileName, FileMode.Create, FileAccess.Write))
{
var zipFileMemoryStream = new MemoryStream(memoryStream.ToArray());
zipFileMemoryStream.CopyTo(fileStream);
zipFileMemoryStream.Flush();
fileStream.Close();
zipFileMemoryStream.Close();
}
}
}

I had the same problem... In this case I just needed to move the ToArray() (byte[]) from MemoryStream outside the using (var zipArchive = new ZipArchive...
I think it is necessary for using related to ZipArchive to completely close and dispose of the file before converting it into a byte array

How to unzip a file in memory to an Azure CloudBlob [duplicate]

I'm probably doing something obviously stupid here. Please point it out!
I have some C# code that is pulling down a bunch of .gz files from SFTP (using the SSH.NET Nuget package - works great!). Each gz contains only a single .CSV file inside of them. I want to keep these files in memory without hitting disk (yes, I know, server memory management concerns exist - that's fine as these files are fairly small), decompress them in memory to extract the CSV file inside, and then return a collection of CSV files in a custom DTO (FtpFile).
My problem is that while my MemoryStream from the SFTP connection has data in it, either it doesn't ever seem to be populated in my GZipStream or the copy from the GZipStream to my output MemoryStream is failing. I have tried with the more traditional looping over Read with my own buffer but it had the same results as this code.
Aside from connection details (it connects successfully, so no worries there), here's all of my code:
Logic:
public static List<FtpFile> Foo()
{
var connectionInfo = new ConnectionInfo("example.com",
"username",
new PasswordAuthenticationMethod("username", "password"));
using (var client = new SftpClient(connectionInfo))
{
client.Connect();
var searchResults = client.ListDirectory("/testdir")
.Where(obj => obj.IsRegularFile
&& obj.Name.ToLowerInvariant().StartsWith("test_")
&& obj.Name.ToLowerInvariant().EndsWith(".gz"))
.Take(2)
.ToList();
var fileResults = new List<FtpFile>();
foreach (var file in searchResults)
{
var ftpFile = new FtpFile { FileName = file.Name, FileSize = file.Length };
using (var fileStream = new MemoryStream())
{
client.DownloadFile(file.FullName, fileStream); // Success! All is good here, so far. :)
using (var gzStream = new GZipStream(fileStream, CompressionMode.Decompress))
{
using (var outputStream = new MemoryStream())
{
gzStream.CopyTo(outputStream);
byte[] outputBytes = outputStream.ToArray(); // No data. Sad panda. :'(
ftpFile.FileContents = Encoding.ASCII.GetString(outputBytes);
fileResults.Add(ftpFile);
}
}
}
}
return fileResults;
}
}
FtpFile (just a simple DTO I'm populating):
public class FtpFile
{
public string FileName { get; set; }
public long FileSize { get; set; }
public string FileContents { get; set; }
}
PSA If anybody comes and copies this code, be aware that this is NOT good code in that you could have some serious memory management problems with this code! It's best practice to instead stream it to disk, which is not being done in this code! My needs are very specific in that I have to have these files simultaneously in memory for what I'm building with them.

If you are inserting data into the stream, make sure to seek back to its origin before un-gzipping it.
The following should fix your troubles:
using (var fileStream = new MemoryStream())
{
client.DownloadFile(file.FullName, fileStream); // Success! All is good here, so far. :)
fileStream.Seek(0, SeekOrigin.Begin);
using (var gzStream = new GZipStream(fileStream, CompressionMode.Decompress))
{
using (var outputStream = new MemoryStream())
{
gzStream.CopyTo(outputStream);
byte[] outputBytes = outputStream.ToArray(); // No data. Sad panda. :'(
ftpFile.FileContents = Encoding.ASCII.GetString(outputBytes);
fileResults.Add(ftpFile);
}
}
}

PDF File Zip in Memory Stream Showing not supported file or damaged My Code attached

I have pdf file in directory that can be open successfully. I am actually changing into byte[] and zip it in memory stream and return it.
I got all files as zipped but when i am opening files getting error
"File can not be open either it has not supported type or file is damage"
I am changing pdf file to byte[],
ArrayList PDFbyteList = new ArrayList();
byte[] bytes = System.IO.File.ReadAllBytes(#"C:\Windows\SysWOW64\config\systemprofile\Desktop\" + PdfFile);
PDFbyteList.Add(bytes);
Then I change it to zip using memory stream,
public FileResult DownloadMultiplePDFFiles(List<FileResult> list)
{
int count = list.Count();
byte[] file1;
using (MemoryStream ms = new MemoryStream())
{
using (var archive = new ZipArchive(ms, ZipArchiveMode.Create, true))
{
for (var i = 0; i < count; i++)
{
file1 = ObjectToByteArray(PDFbyteList[i]);
string name = list[i].FileDownloadName.Split('.')[0];
name = name + ".pdf";
var zipArchiveEntry = archive.CreateEntry(name, CompressionLevel.Fastest);
using (var zipStream = zipArchiveEntry.Open()) zipStream.Write(file1, 0, file1.Length);
file1 = null;
}
}
//return File(ms.ToArray(), "application/zip", "Reports.zip");
return File(ms.ToArray(), "application/octet-stream", "Reports.zip");
}
}
Parameter list having content-type , file name and bytes. I am taking only file name from here and bytes from array list. Well content type in parameter list is "application/octet-stream".
I tried with "application/octet-stream" as well as "application/zip" having same error.
Hopes for your help

Azure Data Lake : Facing issue while moving data from Blob to ADLS

I am creating an Azure Function in C# which does following things:
extracts the zipped file from a blob,
unzips it and copies it to Azure Data Lake Store.
I was able to unzip the file and upload it into another blob using the UploadFromStreamAsync(stream) function.
However, I am facing issue while doing the same For ADLS
I referred to the below link Upload to ADLS from file stream and tried to first create the file using adlsFileSystemClient.FileSystem.Create and then append the stream using adlsFileSystemClient.FileSystem.Append in data lake but it did not work.
- The create method creates a zero byte file but the append does nothing and the azure function still completes successfully without any error. Also, tried with adlsFileSystemClient.FileSystem.AppendAsync and still the same problem.
Code:
// Save blob(zip file) contents to a Memory Stream.
using (var zipBlobFileStream = new MemoryStream())
{
await blockBlob.DownloadToStreamAsync(zipBlobFileStream);
await zipBlobFileStream.FlushAsync();
zipBlobFileStream.Position = 0;
//use ZipArchive from System.IO.Compression to extract all the files from zip file
using (var zip = new ZipArchive(zipBlobFileStream))
{
//Each entry here represents an individual file or a folder
foreach (var entry in zip.Entries)
{
string destfilename = $"{destcontanierPath2}/"+entry.FullName;
log.Info($"DestFilename: {destfilename}");
//creating an empty file (blobkBlob) for the actual file with the same name of file
var blob = extractcontainer.GetBlockBlobReference($"{destfilename}");
using (var stream = entry.Open())
{
//check for file or folder and update the above blob reference with actual content from stream
if (entry.Length > 0)
{
await blob.UploadFromStreamAsync(stream);
//Creating a file and then append
adlsFileSystemClient.FileSystem.Create(_adlsAccountName, "/raw/Hello.txt",overwrite:true);
// Appending the stream to Azure Data Lake
using(var ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0; // rewind
log.Info($"**********MemoryStream: {ms}");
// do something with ms
await adlsFileSystemClient.FileSystem.AppendAsync(_adlsAccountName, "/raw/Hello.txt",ms,0);
}
}
}
}
}
}
New Interim Solution:
using (var zipBlobFileStream = new MemoryStream())
{
await blockBlob.DownloadToStreamAsync(zipBlobFileStream);
using (var zip = new ZipArchive(zipBlobFileStream))
{
//Each entry here represents an individual file or a folder
foreach (var entry in zip.Entries)
{
entry.ExtractToFile(directoryPath + entry.FullName, true);
//Upload the File to ADLS
var parameters = new UploadParameters(directoryPath + entry.FullName, "/raw/" + md5, _adlsAccountName, isOverwrite: true, maxSegmentLength: 268435456 * 2);
var frontend = new Microsoft.Azure.Management.DataLake.StoreUploader.DataLakeStoreFrontEndAdapter(_adlsAccountName, adlsFileSystemClient);
var uploader = new DataLakeStoreUploader(parameters, frontend);
uploader.Execute();
File.Delete(directoryPath + entry.FullName);
}
}
}

In your case, You could change your code as following, then it should work. You should remove the create file code out of the foreach clause.
//Creating a file and then append
adlsFileSystemClient.FileSystem.Create(_adlsAccountName, "/raw/Hello.txt",overwrite:true);
using (var zipBlobFileStream = new MemoryStream())
{
await blockBlob.DownloadToStreamAsync(zipBlobFileStream);
await zipBlobFileStream.FlushAsync();
zipBlobFileStream.Position = 0;
//use ZipArchive from System.IO.Compression to extract all the files from zip file
using (var zip = new ZipArchive(zipBlobFileStream))
{
//Each entry here represents an individual file or a folder
foreach (var entry in zip.Entries)
{
string destfilename = $"{destcontanierPath2}/"+entry.FullName;
log.Info($"DestFilename: {destfilename}");
//creating an empty file (blobkBlob) for the actual file with the same name of file
var blob = extractcontainer.GetBlockBlobReference($"{destfilename}");
using (var stream = entry.Open())
{
//check for file or folder and update the above blob reference with actual content from stream
if (entry.Length > 0)
{
using (MemoryStream ms = new MemoryStream())
{
stream.CopyTo(ms);
ms.Position = 0;
blob.UploadFromStream(ms);
ms.Position = 0;
adlsFileSystemClient.FileSystem.Append(adlsAccountName, "/raw/Hello.txt", ms);
}
}
}
}
}
}
}

'System.OutOfMemoryException' was thrown in ZipArchive for large file from URL

I have an URL which contains a zip file. The files need to be unzipped from the URL. The URL is Opened and Read using webclient and then added to a Stream. It is then used in the ZipArchive object which will unzip the files and store them in the D:\ drive. When a file is around 400Mb I get the 'System.OutOfMemoryException'.
Stream has to be used since the webClient.OpenRead(Uri Address) returns a Stream. As well as the use ZipArchive(Stream stream).
How can I stop from getting this message?
string zipFileUrl = "https://www.dropbox.com/s/clersbjdcshpdy6/oversize_zip_test_0.zip?dl=0"
string output_path = #"D:\";
using (WebClient webClient = new WebClient())
{
using (Stream streamFile = webClient.OpenRead(zipFileUrl))
{
using (ZipArchive archive = new ZipArchive(streamFile))//ERROR HERE
{
var entries = archive.Entries;
//Loops thru each file in Zip and adds it to directory
foreach (var entry in entries)
{
if (entry.FullName != "/" && entry.Name != "")
{
string completeFileName = Path.Combine(output_path, entry.FullName);
string directory = Path.GetDirectoryName(completeFileName);
//If directory does not exist then we create it.
if (!Directory.Exists(directory))
{
Directory.CreateDirectory(directory);
}
//Extracts zip from URL to extract path, and overwrites if file exists.
entry.ExtractToFile(completeFileName, true);
}
}
}
}

I think here might be your problem, from the ZipArchive.Init method
private void Init(Stream stream, ZipArchiveMode mode, Boolean leaveOpen)
{
Stream extraTempStream = null;
try
{
_backingStream = null;
//check stream against mode
switch (mode)
{
case ZipArchiveMode.Create:
// (SNIP)
case ZipArchiveMode.Read:
if (!stream.CanRead)
throw new ArgumentException(SR.ReadModeCapabilities);
if (!stream.CanSeek)
{
_backingStream = stream;
extraTempStream = stream = new MemoryStream();
_backingStream.CopyTo(stream);
stream.Seek(0, SeekOrigin.Begin);
}
break;
case ZipArchiveMode.Update:
// (SNIP)
default:
// (SNIP)
}
// (SNIP)
}
if streamFile.CanSeek is false (which from a WebClient it will be) it copies the entire file in to memory then works on the file. This is what is using up all the memory.
Try to find a 3rd party library that handles Zip files and does not need a stream that supports seeking. If you can't, copy the file to disk first to the temp folder with a FileStream with the FileOptions.DeleteOnClose option passed in, then use that stream in your zip before you close the stream.
string zipFileUrl = "https://www.dropbox.com/s/clersbjdcshpdy6/oversize_zip_test_0.zip?dl=0";
string output_path = #"D:\";
using (var tempFileStream = new FileStream(Path.GetTempFileName(), FileMode.Create,
FileAccess.ReadWrite, FileShare.None,
4096, FileOptions.DeleteOnClose))
{
using (WebClient webClient = new WebClient())
{
using (Stream streamFile = webClient.OpenRead(zipFileUrl))
{
streamFile.CopyTo(tempFileStream);
}
}
tempFileStream.Position = 0;
using (ZipArchive archive = new ZipArchive(tempFileStream))
{
var entries = archive.Entries;
//Loops thru each file in Zip and adds it to directory
foreach (var entry in entries)
{
if (entry.FullName != "/" && entry.Name != "")
{
string completeFileName = Path.Combine(output_path, entry.FullName);
string directory = Path.GetDirectoryName(completeFileName);
//If directory does not exist then we create it.
if (!Directory.Exists(directory))
{
Directory.CreateDirectory(directory);
}
//Extracts zip from URL to extract path, and overwrites if file exists.
entry.ExtractToFile(completeFileName, true);
}
}
}
}