Getting extra space in last line after file seek and zipping it - c#

Using below code I am using File seek and convert to result byte to compressed stream and generating the zip file,
public static async Task Get(string filename)
{
byte[] result;
byte[] compressedBytes;
using (FileStream SourceStream = File.Open(filename, FileMode.Open))
{
SourceStream.Seek(20, SeekOrigin.Begin);
result = new byte[SourceStream.Length];
await SourceStream.ReadAsync(result, 0, (int)SourceStream.Length);
}
string fileName = "Export_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
using (var outStream = File.Create(fileName))
{
using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
var fileInArchive = archive.CreateEntry("test.txt", CompressionLevel.Optimal);
using (var entryStream = fileInArchive.Open())
using (var fileToCompressStream = new MemoryStream(result))
{
fileToCompressStream.CopyTo(entryStream);
}
}
}
}
Now when I unzip the resultant file has extra space. What's the reason for it and how to resolve it?

You're seeking 20 bytes into the stream, but the length of your array is the complete length of the stream. Therefore the final 20 bytes in your array are being ignored.
The simple fix for this is just to allocate less space, and then only ask to read the reduced number of bytes:
result = new byte[SourceStream.Length - 20];
await SourceStream.ReadAsync(result, 0, result.Length);
Note that you're also assuming that a single call to ReadAsync will read all the data. That may be the case in many situations, but it's generally not a good idea to assume that about streams.
It would be simpler just to copy straight from the file stream to the compressed stream though, instead of reading the whole file into memory first:
public static async Task Get(string filename)
{
string outputFile = "Export_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
using (var outStream = File.Create(outputFile))
{
using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
var fileInArchive = archive.CreateEntry("test.txt", CompressionLevel.Optimal);
using (var entryStream = fileInArchive.Open())
using (var fileToCompressStream = File.Open(filename, FileMode.Open))
{
// Skip the first 20 bytes
fileToCompressStream.Position = 20;
fileToCompressStream.CopyTo(entryStream);
}
}
}
}

Related

Invalid zip file reading from a Stream in C#

I have the following code:
private static byte[] ConverterStringToByte(Stream body)
{
string fileName = "data_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
// Take out the bytes from the memory stream and safely close the stream
using (var ms = new MemoryStream())
{
body.CopyTo(ms);
using (var zipArchive = new ZipArchive(ms, ZipArchiveMode.Create, false))
{
var zipEntry = zipArchive.CreateEntry(fileName, CompressionLevel.Optimal);
using (BinaryWriter writer = new BinaryWriter(zipEntry.Open()))
{
ms.Position = 0;
writer.Write(ms.ToArray());
}
}
return ms.ToArray();
}
}
I am downloading the file successfully, however I'm getting
invalid file
when trying to open
I think it should be something like this. Not sure about fileName though, because it's the name of the file being put into archive, so I don't think it should have *.zip extension. Unless you are creating a zip of zips.
static byte[] ConverterStringToByte(Stream body)
{
string fileName = #"data_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
using (var ms = new MemoryStream())
{
using (var zipArchive = new ZipArchive(ms, ZipArchiveMode.Create, false))
{
var zipEntry = zipArchive.CreateEntry(fileName, CompressionLevel.Optimal);
using (var destStream = zipEntry.Open())
{
body.CopyTo(destStream);
}
}
return ms.ToArray();
}
}

System.IO.Compression.ZipArchive corrupt when downloading [duplicate]

I'm trying to create a zip stream on the fly with some byte array data and make it download via my MVC action.
But the downloaded file always gives the following corrupted error when opened in windows.
And this error when I try to xtract from 7z
But note that the files extracted from the 7z is not corrupted.
I'm using ZipArchive and the below is my code.
private byte[] GetZippedPods(IEnumerable<POD> pods, long consignmentID)
{
using (var zipStream = new MemoryStream())
{
//Create an archive and store the stream in memory.
using (var zipArchive = new ZipArchive(zipStream, ZipArchiveMode.Create, true))
{
int index = 1;
foreach (var pod in pods)
{
var zipEntry = zipArchive.CreateEntry($"POD{consignmentID}{index++}.png", CompressionLevel.NoCompression);
using (var originalFileStream = new MemoryStream(pod.ByteData))
{
using (var zipEntryStream = zipEntry.Open())
{
originalFileStream.CopyTo(zipEntryStream);
}
}
}
return zipStream.ToArray();
}
}
}
public ActionResult DownloadPOD(long consignmentID)
{
var pods = _consignmentService.GetPODs(consignmentID);
var fileBytes = GetZippedPods(pods, consignmentID);
return File(fileBytes, MediaTypeNames.Application.Octet, $"PODS{consignmentID}.zip");
}
What am I doing wrong here.
Any help would be highly appreciated as I'm struggling with this for a whole day.
Thanks in advance
Move zipStream.ToArray() outside of the zipArchive using.
The reason for your problem is that the stream is buffered. There's a few ways to deal wtih it:
You can set the stream's AutoFlush property to true.
You can manually call .Flush() on the stream.
Or, since it's MemoryStream and you're using .ToArray(), you can simply allow the stream to be Closed/Disposed first (which we've done by moving it outside the using).
I Dispose ZipArchive And error solved
public static byte[] GetZipFile(Dictionary<string, List<FileInformation>> allFileInformations)
{
MemoryStream compressedFileStream = new MemoryStream();
//Create an archive and store the stream in memory.
using (var zipArchive = new ZipArchive(compressedFileStream, ZipArchiveMode.Create, true))
{
foreach (var fInformation in allFileInformations)
{
var files = allFileInformations.Where(x => x.Key == fInformation.Key).SelectMany(x => x.Value).ToList();
for (var i = 0; i < files.Count; i++)
{
ZipArchiveEntry zipEntry = zipArchive.CreateEntry(fInformation.Key + "/" + files[i].FileName);
var caseAttachmentModel = Encoding.UTF8.GetBytes(files[i].Content);
//Get the stream of the attachment
using (var originalFileStream = new MemoryStream(caseAttachmentModel))
using (var zipEntryStream = zipEntry.Open())
{
//Copy the attachment stream to the zip entry stream
originalFileStream.CopyTo(zipEntryStream);
}
}
}
//i added this line
zipArchive.Dispose();
return compressedFileStream.ToArray();
}
}
public void SaveZipFile(){
var zipFileArray = Global.GetZipFile(allFileInformations);
var zipFile = new MemoryStream(zipFileArray);
FileStream fs = new FileStream(path + "\\111.zip",
FileMode.Create,FileAccess.Write);
zipFile.CopyTo(fs);
zipFile.Flush();
fs.Close();
zipFile.Close();
}
I was also having problems with this and I found my issue was not the generation of the archive itself but rather how I was handing my GET request in AngularJS.
This post helped me: how to download a zip file using angular
The key was adding responseType: 'arraybuffer' to my $http call.
factory.serverConfigExportZIP = function () {
return $http({
url: dataServiceBase + 'serverConfigExport',
method: "GET",
responseType: 'arraybuffer'
})
};
you can remove "using" and use Dispose and Close methods
it's work for me
...
zip.Dispose();
zipStream.Close();
return zipStream.ToArray();
I know this is a C# question but for managed C++, delete the ZipArchive^ after you're done with it to fix the error.
ZipArchive^ zar = ZipFile::Open(starget, ZipArchiveMode::Create);
ZipFileExtensions::CreateEntryFromFile(zar, sfile1, "file.txt");
ZipFileExtensions::CreateEntryFromFile(zar, sfile2, "file2.txt");
delete zar;
when i wanted to create zip file directly from MemoryStream which i used for ZipArchive i was getting error ( "unexpected end of data" or zero length file )
there are three points to get ride of this error
set the last parameter of ZipArchive constructor to true ( it leaves to leave stream open after ZipArchive disposed )
call dispose() on ZipArchive and dispose it manually.
create another MemoryStream based on which you set in ZipArchive constructor, by calling ToArray() method.
here is sample code :
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create,))
{
foreach (var s3Object in objectList.S3Objects)
{
var entry = archive.CreateEntry(s3Object.Key, CompressionLevel.NoCompression);
using (var entryStream = entry.Open())
{
var request = new GetObjectRequest { BucketName = command.BucketName, Key = s3Object.Key };
using (var getObjectResponse = await client.GetObjectAsync(request))
{
await getObjectResponse.ResponseStream.CopyToAsync(entryStream);
}
}
}
archive.Dispose();
using (var fileStream = new FileStream(outputFileName, FileMode.Create, FileAccess.Write))
{
var zipFileMemoryStream = new MemoryStream(memoryStream.ToArray());
zipFileMemoryStream.CopyTo(fileStream);
zipFileMemoryStream.Flush();
fileStream.Close();
zipFileMemoryStream.Close();
}
}
}
I had the same problem... In this case I just needed to move the ToArray() (byte[]) from MemoryStream outside the using (var zipArchive = new ZipArchive...
I think it is necessary for using related to ZipArchive to completely close and dispose of the file before converting it into a byte array

Generated Zip file is invalid after downloading from S3

I am creating a zip file that, appears, valid but is always invalid after I have put it to a Amazon S3 bucket. I am using System.IO.Compression for the task and AmazonS3Client for uploading:
private byte[] GenerateZipFile(string tenant)
{
byte[] zipData;
var results = QueryAggregateTable(tenant);
using (var memoryStream = new MemoryStream())
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
foreach (var item in results)
{
var archiveEntry = archive.CreateEntry($"{item.RowKey:D3}.json", CompressionLevel.Fastest);
using (var entryStream = archiveEntry.Open())
{
var entryBytes = Encoding.UTF8.GetBytes(item.Data);
entryStream.Write(entryBytes, 0, item.Data.Length);
}
}
zipData = memoryStream.ToArray();
}
return zipData;
}
and
private async Task UploadToAmazon(byte[] zipData, string tenant)
{
var bucketName = _config["egestionBucketName"];
var configCreds = _config["egestionAwsCredentials"].Split(":");
var awsCreds = new BasicAWSCredentials(configCreds[0], configCreds[1]);
var awsRegion = Amazon.RegionEndpoint.GetBySystemName(_config["egestionRegionEndpointSystemName"]);
var s3Client = new AmazonS3Client(awsCreds, awsRegion);
using (var stream = new MemoryStream(zipData))
{
var putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = $"{tenant}-{DateTime.UtcNow.ToString("s")}.zip",
InputStream = stream,
CannedACL = S3CannedACL.BucketOwnerFullControl
};
await s3Client.PutObjectAsync(putRequest);
}
}
The byte array looks good after returning from generation and the upload method does, in fact, load a file with the correct name to the bucket. When I attempt to download the file to check it I cannot open it with a message stating it is invalid.
I have had some problems with async/await and suspect it may be something related but there is no non async option for PutObject that I can find. Any help appreciated.
This is not an async-await issue.
The bytes from the memory stream are being collected before the archive has had a chance to write all the data to the stream. The uploaded archive is incomplete and therefore invalid when downloaded.
Move
zipData = memoryStream.ToArray();
to outside of the archive using block so that any buffered data is flushed to the backing stream when the archive is disposed.
//...
using (var memoryStream = new MemoryStream()) {
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true)) {
foreach (var item in results) {
var archiveEntry = archive.CreateEntry($"{item.RowKey:D3}.json", CompressionLevel.Fastest);
using (var entryStream = archiveEntry.Open()) {
var entryBytes = Encoding.UTF8.GetBytes(item.Data);
entryStream.Write(entryBytes, 0, entryBytes.Length);
}
}
}//Archive disposed and pushed any remaining buffered data to the stream.
zipData = memoryStream.ToArray();
}
//...

Create zip file in memory from bytes (text with arbitrary encoding)

The application i'm developing needs to compress xml files into zip files and send them through http requests to a web service. As I dont need to keep the zip files, i'm just performing the compression in memory. The web service is denying my requests because the zip files are apparently malformed.
I know there is a solution in this question which works perfectly, but it uses a StreamWriter. My problem with that solution is that StreamWriter requires an encoding or assumes UTF-8, and I do not need to know the enconding of the xml files. I just need to read the bytes from those files, and store them inside a zip file, whatever encoding they use.
So, to be clear, this question has nothing to do with encodings, as I don't need to transform the bytes into text or the oposite. I just need to compress a byte[].
I'm using the next code to test how my zip file is malformed:
static void Main(string[] args)
{
Encoding encoding = Encoding.GetEncoding("ISO-8859-1");
string xmlDeclaration = "<?xml version=\"1.0\" encoding=\"" + encoding.WebName.ToUpperInvariant() + "\"?>";
string xmlBody = "<Test>ª!\"·$%/()=?¿\\|##~€¬'¡º</Test>";
string xmlContent = xmlDeclaration + xmlBody;
byte[] bytes = encoding.GetBytes(xmlContent);
string fileName = "test.xml";
string zipPath = #"C:\Users\dgarcia\test.zip";
Test(bytes, fileName, zipPath);
}
static void Test(byte[] bytes, string fileName, string zipPath)
{
byte[] zipBytes;
using (var memoryStream = new MemoryStream())
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, leaveOpen: false))
{
var zipEntry = zipArchive.CreateEntry(fileName);
using (Stream entryStream = zipEntry.Open())
{
entryStream.Write(bytes, 0, bytes.Length);
}
//Edit: as the accepted answer states, the problem is here, because i'm reading from the memoryStream before disposing the zipArchive.
zipBytes = memoryStream.ToArray();
}
using (var fileStream = new FileStream(zipPath, FileMode.OpenOrCreate))
{
fileStream.Write(zipBytes, 0, zipBytes.Length);
}
}
If I try to open that file, I get an "Unexpected end of file" error. So apparently, the web service is correctly reporting a malformed zip file. What I have tried so far:
Flushing the entryStream.
Closing the entryStream.
Both flushing and closing the entryStream.
Note that if I open the zipArchive directly from the fileStream the zip file is formed with no errors. However, the fileStream is just there as a test, and I need to create my zip file in memory.
You are trying to get bytes from MemoryStream too early, ZipArchive did not write them all yet. Instead, do like this:
using (var memoryStream = new MemoryStream()) {
// note "leaveOpen" true, to not dispose memoryStream too early
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, leaveOpen: true)) {
var zipEntry = zipArchive.CreateEntry(fileName);
using (Stream entryStream = zipEntry.Open()) {
entryStream.Write(bytes, 0, bytes.Length);
}
}
// now, after zipArchive is disposed - all is written to memory stream
zipBytes = memoryStream.ToArray();
}
If you use a memory stream to load your text you can control the encoding type and it works across a WCF service. This is the implementation i am using currently and it works on my WCF services
private byte[] Zip(string text)
{
var bytes = Encoding.UTF8.GetBytes(text);
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
{
using (var gs = new GZipStream(mso, CompressionMode.Compress))
{
CopyTo(msi, gs);
}
return mso.ToArray();
}
}
private string Unzip(byte[] bytes)
{
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
{
using (var gs = new GZipStream(msi, CompressionMode.Decompress))
{
CopyTo(gs, mso);
}
return Encoding.UTF8.GetString(mso.ToArray());
}
}

Byte array getting corrupted when passed to another method

I have a bunch of Jpg images in byte array form. I want to add these to a zip file, turn the zip file into a byte array, and pass it to somewhere else. In a method, I have this code:
var response = //some response object that will hold a byte array
using (var ms = new MemoryStream())
{
using (var zipArchive = new ZipArchive(ms, ZipArchiveMode.Create, true))
{
var i = 1;
foreach (var image in images) // some collection that holds byte arrays.
{
var entry = zipArchive.CreateEntry(i + ".jpg");
using (var entryStream = entry.Open())
using (var compressStream = new MemoryStream(photo.ImageOriginal))
{
compressStream.CopyTo(entryStream);
}
i++;
}
response.ZipFile = ms.ToArray();
}
using (var fs = new FileStream(#"C:\Users\MyName\Desktop\image.zip", FileMode.Create))
{
ms.Position = 0;
ms.CopyTo(fs);
}
}
return response;
Now, I've added a filestream near the bottom to write it to a zipfile right away for testing purposes. This works, I get a zip file with 1 or more images in it on my desktop. However: response.ZipFile can not be made into a valid zipfile in the same way. I have tried this:
using (var ms2 = new MemoryStream(response.ZipFile))
using (var fs = new FileStream(#"C:\Users\Bara\Desktop\image.zip", FileMode.Create))
{
ms2.Position = 0;
ms2.CopyTo(fs);
}
But that creates a zipfile that can not be opened.
What I'm trying to do: turn response.ZipFileinto an array that can be turned into a working zipfile again. What am I doing wrong in this code?
How do you know that ZipArchive's Dispose doesn't write more to the underlying stream?
You should move this line to be after disposing the ZipArchive:
response.ZipFile = ms.ToArray();
Full code:
var response = //some response object that will hold a byte array
using (var ms = new MemoryStream())
{
using (var zipArchive = new ZipArchive(ms, ZipArchiveMode.Create, true))
{
var i = 1;
foreach (var image in images) // some collection that holds byte arrays.
{
var entry = zipArchive.CreateEntry(i + ".jpg");
using (var entryStream = entry.Open())
using (var compressStream = new MemoryStream(photo.ImageOriginal))
{
compressStream.CopyTo(entryStream);
}
i++;
}
}
response.ZipFile = ms.ToArray();
}
return response;

Categories