The application i'm developing needs to compress xml files into zip files and send them through http requests to a web service. As I dont need to keep the zip files, i'm just performing the compression in memory. The web service is denying my requests because the zip files are apparently malformed.
I know there is a solution in this question which works perfectly, but it uses a StreamWriter. My problem with that solution is that StreamWriter requires an encoding or assumes UTF-8, and I do not need to know the enconding of the xml files. I just need to read the bytes from those files, and store them inside a zip file, whatever encoding they use.
So, to be clear, this question has nothing to do with encodings, as I don't need to transform the bytes into text or the oposite. I just need to compress a byte[].
I'm using the next code to test how my zip file is malformed:
static void Main(string[] args)
{
Encoding encoding = Encoding.GetEncoding("ISO-8859-1");
string xmlDeclaration = "<?xml version=\"1.0\" encoding=\"" + encoding.WebName.ToUpperInvariant() + "\"?>";
string xmlBody = "<Test>ª!\"·$%/()=?¿\\|##~€¬'¡º</Test>";
string xmlContent = xmlDeclaration + xmlBody;
byte[] bytes = encoding.GetBytes(xmlContent);
string fileName = "test.xml";
string zipPath = #"C:\Users\dgarcia\test.zip";
Test(bytes, fileName, zipPath);
}
static void Test(byte[] bytes, string fileName, string zipPath)
{
byte[] zipBytes;
using (var memoryStream = new MemoryStream())
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, leaveOpen: false))
{
var zipEntry = zipArchive.CreateEntry(fileName);
using (Stream entryStream = zipEntry.Open())
{
entryStream.Write(bytes, 0, bytes.Length);
}
//Edit: as the accepted answer states, the problem is here, because i'm reading from the memoryStream before disposing the zipArchive.
zipBytes = memoryStream.ToArray();
}
using (var fileStream = new FileStream(zipPath, FileMode.OpenOrCreate))
{
fileStream.Write(zipBytes, 0, zipBytes.Length);
}
}
If I try to open that file, I get an "Unexpected end of file" error. So apparently, the web service is correctly reporting a malformed zip file. What I have tried so far:
Flushing the entryStream.
Closing the entryStream.
Both flushing and closing the entryStream.
Note that if I open the zipArchive directly from the fileStream the zip file is formed with no errors. However, the fileStream is just there as a test, and I need to create my zip file in memory.
You are trying to get bytes from MemoryStream too early, ZipArchive did not write them all yet. Instead, do like this:
using (var memoryStream = new MemoryStream()) {
// note "leaveOpen" true, to not dispose memoryStream too early
using (var zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, leaveOpen: true)) {
var zipEntry = zipArchive.CreateEntry(fileName);
using (Stream entryStream = zipEntry.Open()) {
entryStream.Write(bytes, 0, bytes.Length);
}
}
// now, after zipArchive is disposed - all is written to memory stream
zipBytes = memoryStream.ToArray();
}
If you use a memory stream to load your text you can control the encoding type and it works across a WCF service. This is the implementation i am using currently and it works on my WCF services
private byte[] Zip(string text)
{
var bytes = Encoding.UTF8.GetBytes(text);
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
{
using (var gs = new GZipStream(mso, CompressionMode.Compress))
{
CopyTo(msi, gs);
}
return mso.ToArray();
}
}
private string Unzip(byte[] bytes)
{
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream())
{
using (var gs = new GZipStream(msi, CompressionMode.Decompress))
{
CopyTo(gs, mso);
}
return Encoding.UTF8.GetString(mso.ToArray());
}
}
Related
Using below code I am using File seek and convert to result byte to compressed stream and generating the zip file,
public static async Task Get(string filename)
{
byte[] result;
byte[] compressedBytes;
using (FileStream SourceStream = File.Open(filename, FileMode.Open))
{
SourceStream.Seek(20, SeekOrigin.Begin);
result = new byte[SourceStream.Length];
await SourceStream.ReadAsync(result, 0, (int)SourceStream.Length);
}
string fileName = "Export_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
using (var outStream = File.Create(fileName))
{
using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
var fileInArchive = archive.CreateEntry("test.txt", CompressionLevel.Optimal);
using (var entryStream = fileInArchive.Open())
using (var fileToCompressStream = new MemoryStream(result))
{
fileToCompressStream.CopyTo(entryStream);
}
}
}
}
Now when I unzip the resultant file has extra space. What's the reason for it and how to resolve it?
You're seeking 20 bytes into the stream, but the length of your array is the complete length of the stream. Therefore the final 20 bytes in your array are being ignored.
The simple fix for this is just to allocate less space, and then only ask to read the reduced number of bytes:
result = new byte[SourceStream.Length - 20];
await SourceStream.ReadAsync(result, 0, result.Length);
Note that you're also assuming that a single call to ReadAsync will read all the data. That may be the case in many situations, but it's generally not a good idea to assume that about streams.
It would be simpler just to copy straight from the file stream to the compressed stream though, instead of reading the whole file into memory first:
public static async Task Get(string filename)
{
string outputFile = "Export_" + DateTime.Now.ToString("yyyyMMddhhmmss") + ".zip";
using (var outStream = File.Create(outputFile))
{
using (var archive = new ZipArchive(outStream, ZipArchiveMode.Create, true))
{
var fileInArchive = archive.CreateEntry("test.txt", CompressionLevel.Optimal);
using (var entryStream = fileInArchive.Open())
using (var fileToCompressStream = File.Open(filename, FileMode.Open))
{
// Skip the first 20 bytes
fileToCompressStream.Position = 20;
fileToCompressStream.CopyTo(entryStream);
}
}
}
}
I'm trying to download a zip file and modify it before returning it. I expect the stream to be modified after adding additional files with ZipArchive in Update mode. However, it stays the same. What am I doing wrong?
using (WebClient webClient = new WebClient())
{
string url = "http://www.dynaexamples.com/examples-manual/ls-dyna_example.zip/at_download/file";
byte[] downloadedData = webClient.DownloadData(url);
using (MemoryStream stream = new MemoryStream())
{
stream.Write(downloadedData, 0, downloadedData.Length);
Console.WriteLine(stream.Length.ToString()); //911616
ZipArchive archive = new ZipArchive(stream, ZipArchiveMode.Update);
ZipArchiveEntry testFile = archive.CreateEntry("test.txt");
using (StreamWriter writer = new StreamWriter(testFile.Open()))
{
writer.WriteLine("test");
writer.Flush();
}
Console.WriteLine(stream.Length.ToString()); //911616
}
}
I am currently working on integrating Amazon Prime on our system and being stuck at getting the label back as ZPL format.
Basically, Amazon returns a base64 string, we will need to convert that string to a byte array, then save that array as a *.gzip file. From that gzip file, we can extract the content and get the zpl label content.
My question is, how we can do all of above without storing any temp files to system. I have researched some solutions but none is working for me.
My current code as below:
var str = "base64string";
var label = Convert.FromBase64String(str);
using (var memoryStream = new MemoryStream())
{
using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
{
var demoFile = archive.CreateEntry("label.zip");
var entryStream = demoFile.Open();
using (var bw = new BinaryWriter(entryStream))
{
bw.Write(label);
}
var data = new MemoryStream();
using (var zip = ZipFile.Read(entryStream))
{
zip["label"].Extract(data);
}
data.Seek(0, SeekOrigin.Begin);
entryStream.Close();
}
using (var fileStream = new FileStream(#"D:\test.zip", FileMode.Create))
{
memoryStream.Seek(0, SeekOrigin.Begin);
memoryStream.CopyTo(fileStream);
}
}
If I save the file as test.zip, I can successfully get the label back. But if I try to extract it directly to another stream, I get an error
A stream from ZipArchiveEntry has been disposed
I've done something similar, taking PNG label data from a zipped web response. This is how I went about that
using (WebClient webClient = new WebClient())
{
// Download. Expect this to be a zip file
byte[] data = webClient.DownloadData(urlString);
MemoryStream memoryStream = new MemoryStream(data);
ZipArchive zipArchive = new ZipArchive(memoryStream);
foreach (var zipEntry in zipArchive.Entries)
{
// Can check file name here and ignore anything in zip we're not expecting
if (!zipEntry.Name.EndsWith(".png")) continue;
// Open zip entry as stream
Stream extractedFile = zipEntry.Open();
// Convert stream to memory stream
MemoryStream extractedMemoryStream = new MemoryStream();
extractedFile.CopyTo(extractedMemoryStream);
// At this point the extractedMemoryStream is a sequence of bytes containing image data.
// In this test project I'm pushing that into a bitmap image, just to see something on screen, but could as easily be written to a file or passed for storage to sql or whatever.
BitmapDecoder decoder = PngBitmapDecoder.Create(extractedMemoryStream, BitmapCreateOptions.None, BitmapCacheOption.OnLoad);
BitmapFrame frame = decoder.Frames.First();
frame.Freeze();
this.LabelImage.Source = frame;
}
}
I was overthinking it. I finally found a simple way to do it. We just need to convert that base64 string to bytes array and use GzipStream to directly decompress it. I leave the solution here in case someone needs it. Thanks!
var label = Convert.FromBase64String(str);
using (var compressedStream = new MemoryStream(label))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
I have a JSON file created, and I am going to zip it using DotNetZip.
Using with StreamWriter to zip it is working, if I try to use MemoryStream it will not working.
StreamWriter :
sw = new StreamWriter(assetsFolder + #"manifest.json");
sw.Write(strManifest);
sw.Close();
zip.AddFile(Path.Combine(assetsFolder, "manifest.json"), "/");
zip.AddFile(Path.Combine(assetsFolder, "XXXXXXX"), "/");
zip.Save(outputStream);
MemoryStream :
var manifestStream = GenerateStreamFromString(strManifest);
public static Stream GenerateStreamFromString(string s)
{
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(s);
writer.Flush();
stream.Position = 0;
return stream;
}
zip.AddEntry("manifest.json", manifestStream);
zip.AddFile(Path.Combine(assetsFolder, "XXXXXXX"), "/");
zip.Save(outputStream);
I must using the .JSON file type to zip it, Can any one told me where have a mistake?
To create a Gzipped Json you need to use GZipStream. Try method below.
https://www.dotnetperls.com/gzipstream
GZipStream compresses data. It saves data efficiently—such as in
compressed log files. We develop a utility method in the C# language
that uses the System.IO.Compression namespace. It creates GZIP files.
It writes them to the disk.
public static void CompressStringToFile(string fileName, string value)
{
// A.
// Write string to temporary file.
string temp = Path.GetTempFileName();
File.WriteAllText(temp, value);
// B.
// Read file into byte array buffer.
byte[] b;
using (FileStream f = new FileStream(temp, FileMode.Open))
{
b = new byte[f.Length];
f.Read(b, 0, (int)f.Length);
}
// C.
// Use GZipStream to write compressed bytes to target file.
using (FileStream f2 = new FileStream(fileName, FileMode.Create))
using (GZipStream gz = new GZipStream(f2, CompressionMode.Compress, false))
{
gz.Write(b, 0, b.Length);
}
}
Is DeflateStream supposed to create archived stream that can be stored as standard .zip archive?
I'm trying to create in-memory zip (to be sent remotely) from a local file.
I used a DeflateStream to get a compressed byte array from the file on local disk:
public static byte[] ZipFile(string csvFullPath)
{
using (FileStream csvStream = File.Open(csvFullPath, FileMode.Open, FileAccess.Read))
{
using (MemoryStream compressStream = new MemoryStream())
{
using (DeflateStream deflateStream = new DeflateStream(compressStream, CompressionLevel.Optimal))
{
csvStream.CopyTo(deflateStream);
deflateStream.Close();
return compressStream.ToArray();
}
}
}
}
This works great.
However when I dump the resulting bytes to a zip file:
byte[] zippedBytes = ZipFile(FileName);
File.WriteAllBytes("Sample.zip", zippedBytes);
I cannot open the resulting .zip archive with windows build-in .zip functionality (or with any other 3rd party archive tool).
An alternative I'm planning now is using ZipArchive - however that would require creating temporary files on disk (first copy the file into separate directory, then zip it, then read it into byte array and then delete it)
You can use this nice library https://dotnetzip.codeplex.com/
or you can use ZipArchive and it works with MemoryStream pretty good:
public static byte[] ZipFile(string csvFullPath)
{
using (FileStream csvStream = File.Open(csvFullPath, FileMode.Open, FileAccess.Read))
{
using (MemoryStream zipToCreate = new MemoryStream())
{
using (ZipArchive archive = new ZipArchive(zipToCreate, ZipArchiveMode.Create, true))
{
ZipArchiveEntry fileEntry = archive.CreateEntry(Path.GetFileName(csvFullPath));
using (var entryStream = fileEntry.Open())
{
csvStream.CopyTo(entryStream);
}
}
return zipToCreate.ToArray();
}
}
}