We Keep Coding

Unable to create zip file using Ionic.Zip

I'm not sure where and what am I doing wrong, but the zip that I'm creating using DotNetZip library, is creating a zip file whose contents are blank. Or the size of file in zip is showing as 0Kb and unable to open it.
Code:
public static async Task DotNetZipFileAsync(MemoryStream stream, string bucket, List<List<string>> pdfFileSet, IAmazonS3 s3Client)
{
using Ionic.Zip.ZipFile zip = new ZipFile();
foreach (var pdfFile in pdfFileSet)
{
foreach (var file in pdfFile)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucket,
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
ZipEntry zipEntry = zip.AddEntry(file.Split('/')[^1], responseStream);
await responseStream.CopyToAsync(stream);
}
}
zip.Save(stream);
stream.Seek(0,SeekOrigin.Begin);
await stream.CopyToAsync(new FileStream(#"C:\LocalRepo\Temp.zip", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite));
}
}
}

Your code has at least two problems:
The read stream is completely consumed by the await responseStream.CopyToAsync(stream). You could rewind the responseStream to cope with this, but saving the data into the memory stream is completely useless.
The response stream is disposed before zip.Save is called.
What you could do: keep the streams open until Save is called and dispose them afterwards. As Alexey Rumyantsev discovered (see comments), also the GetObjectResponse objects need to be kept until the ZIP file is saved.
using Ionic.Zip.ZipFile zip = new ZipFile();
var disposables = List<IDisposable>();
try
{
foreach (var pdfFile in pdfFileSet)
{
foreach (var file in pdfFile)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucket,
Key = file
};
var response = await s3Client.GetObjectAsync(request);
disposables.Add(response);
var responseStream = response.ResponseStream;
disposables.Add(responseStream);
ZipEntry zipEntry = zip.AddEntry(file.Split('/')[^1], responseStream);
}
}
using var fileStream = new FileStream(#"C:\LocalRepo\Temp.zip", FileMode.Create, FileAccess.Write);
zip.Save(fileStream);
}
finally
{
foreach (var disposable in disposables)
{
disposable.Dispose();
}
}
The documentation has some hints ony how this could be made smarter.

public static async Task DotNetZipFileAsync(string bucket, List<List<string>> pdfFileSet, IAmazonS3 s3Client)
{
int read;
using Ionic.Zip.ZipFile zip = new ZipFile();
byte[] buffer = new byte[16 * 1024];
foreach (var pdfFile in pdfFileSet)
{
foreach (var file in pdfFile)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucket,
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
using (MemoryStream ms = new MemoryStream())
{
while ((read = responseStream.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
zip.AddEntry(file.Split('/')[^1], ms.ToArray());
}
}
}
using var fileStream = new FileStream(#"C:\LocalRepo\Temp.zip", FileMode.Create, FileAccess.Write);
zip.Save(fileStream);
}

sending binary file byte array to web api method

I have a requirement to post binary file of size 100MB data in the format of either JSON or byte array to Web API 1.1.
My client application is C# winforms application with x32 bit architecture. Where as I want to perform reading binary file from this client application and send this binary file byte array to Web API.
Current implementation in my winforms application is as below
var sFile = #"C"\binary.zip";
var mybytearray = File.ReadAllBytes(sFile);
var webRequest =
(HttpWebRequest)WebRequest.Create("http://localhost/filewriter");
webRequest.ContentType = "text/plain";
webRequest.Method = WebRequestMethods.Http.Post;
webRequest.AllowWriteStreamBuffering = true;
webRequest.Timeout = 100000;
webRequest.Headers.Add("fileName", Path.GetFileName(sFile));
webRequest.ContentLength = mybytearray.Length;
using (var dataStream = new StreamWriter(webRequest.GetRequestStream()))
dataStream.Write(mybytearray);
using (var response = webRequest.GetResponse())
{
if(response.StatusCode = HttpStatusCode.Ok;
return true;
}
below is written at my Web api method
[HttpPost]
public HttpResponseMessage filewriter(byte[] binaryData)
{
using (FileStream binaryFileStream = new FileStream("C:\\myNewFile.zip", FileMode.Create, FileAccess.ReadWrite))
{
binaryFileStream.Write(binaryData, 0, binaryData.Length);
}
}
As you can see, in above code I was not able to send byte array to web api method filewriter. Am I missing something that should work in this case.
Other way as I said I was tried same but instead of byte array with Json one as below
var sFile = #"C"\binary.zip";
var mybytearray = File.ReadAllBytes(sFile);
var mymodel = new model
{
fileName = sFile,
binaryData = mybytearray
};
var jsonResendObjects = JsonConvert.SerializeObject(mymodel);
var webRequest = (HttpWebRequest)WebRequest.Create("http://localhost/filewriter");
webRequest.ContentType = "application/json";
webRequest.Method = WebRequestMethods.Http.Post;
webRequest.AllowWriteStreamBuffering = true;
webRequest.Timeout = 100000;
webRequest.Headers.Add("fileName", Path.GetFileName(sFile));
webRequest.ContentLength = jsonResendObjects.Length;
byte[] responseData = null;
webRequest.AllowWriteStreamBuffering = true;
using (var dataStream = new StreamWriter(webRequest.GetRequestStream()))
dataStream.Write(jsonResendObjects);
On web api side
[HttpPost]
public HttpResponseMessage filewriter([FromBody]model mymodel)
{
using (FileStream binaryFileStream = new FileStream("C:\\myNewFile.zip", FileMode.Create, FileAccess.ReadWrite))
{
binaryFileStream.Write(mymodel.binarydata, 0, binaryDatabinarydat.Length);
}
}

According to me, it would be easy to use base64 encoding for
communication.
If you want to do so
First, convert your file to byte[] and then to base64 string
Like this:
byte[] bytes = File.ReadAllBytes("path");
string file = Convert.ToBase64String(bytes);
// You have base64 Data in "file" variable
On your WebAPI Endpoint accept string
[HttpPost]
public HttpResponseMessage filewriter(string fileData)
{
}
Then convert your base64 string back to byte[] and write it to file or whatever you want to do with that.
Like This:
// put your base64 string in b64str
Byte[] bytes = Convert.FromBase64String(b64Str);
File.WriteAllBytes(path, bytes);
And you can Compress your string Using GZIP Like this
public static void CopyTo(Stream src, Stream dest) {
byte[] bytes = new byte[4096];
int cnt;
while ((cnt = src.Read(bytes, 0, bytes.Length)) != 0) {
dest.Write(bytes, 0, cnt);
}
}
public static byte[] Zip(string str) {
var bytes = Encoding.UTF8.GetBytes(str);
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream()) {
using (var gs = new GZipStream(mso, CompressionMode.Compress)) {
//msi.CopyTo(gs);
CopyTo(msi, gs);
}
return mso.ToArray();
}
}
public static string Unzip(byte[] bytes) {
using (var msi = new MemoryStream(bytes))
using (var mso = new MemoryStream()) {
using (var gs = new GZipStream(msi, CompressionMode.Decompress)) {
//gs.CopyTo(mso);
CopyTo(gs, mso);
}
return Encoding.UTF8.GetString(mso.ToArray());
}
}
Reference:-
Convert file to base64 and back
GZip Compression

How to convert GZipStream to HttpContent?

I have written a Web API code to return a zip file. But I am not able to convert the GZipStream content to HttpContent. I get the following error:
cannot implicitly convert type 'system.io.compression.GZipStream' to
'System.Net.Http.HttpContent'
Where did I go wrong?
My WebApi Code:
var content =
new GZipStream(memStream, CompressionMode.Compress);
HttpResponseMessage httpResponseMessage = new HttpResponseMessage();
httpResponseMessage.Content = content;
httpResponseMessage.Content.Headers.Add("x-filename", document.FileName);
httpResponseMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("application/zip");
httpResponseMessage.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
httpResponseMessage.Content.Headers.ContentDisposition.FileName = "xyz.zip";
httpResponseMessage.StatusCode = HttpStatusCode.OK;

I got a solution using Zip archive and I have created a static class to push the memory stream into a zip archive as below,
var pushStreamContent = ZipStreamContent.Create("MultipleDocument.zip", memStList);
ZipStreamContent class,
public static class ZipStreamContent
{
public static PushStreamContent Create(string fileName, List<MemoryStream> msList)
{
var content = new PushStreamContent((outputStream, httpContent, transportContext) =>
{
using (var zip = new ZipArchive(outputStream, ZipArchiveMode.Create, leaveOpen: false))
{
msList[0].Position = 0;
var createenter = zip.CreateEntry("xyz.jpg", CompressionLevel.Optimal);
using (var s = createenter.Open())
{
msList[0].CopyTo(s);
}
}
});
content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
content.Headers.ContentDisposition.FileName = fileName;
return content;
}
}

I have simplified the above given GipStreamContent static class like following, It's working well, so I hope It will help all others.
CloudBlockBlob blob = null;
//azure storage connection
var container = GetBlobClient(tenantInfo);
//directory reference
var directory = container.GetDirectoryReference(
string.Format(DirectoryNameConfigValue, tenantInfo.TenantId.ToString(), documentList[0].ProjectId));
var pushStreamContent = new PushStreamContent(async (outputStream, httpContent, transportContext) =>
{
//zip the multiple files
using (var zipEntry = new ZipArchive(outputStream, ZipArchiveMode.Create, leaveOpen: false))
{
for (int docId = 0; docId < documentList.Count; docId++)
{
blob = directory.GetBlockBlobReference(DocumentNameConfigValue + documentList[docId].DocumentId);
if (!blob.Exists()) continue;
MemoryStream memStream = new MemoryStream();
await blob.DownloadToStreamAsync(memStream);
memStream.Position = 0;
var createEntry = zipEntry.CreateEntry(documentList[docId].FileName, CompressionLevel.Fastest);
using (var stream = createEntry.Open())
{
memStream.CopyTo(stream);
}
}
}
});

Convert wav streamed over HTTP to mp3, in real-time

Background: I am consuming a service which returns data with a MIME type of audio/wav. I need to provide a playback mechanism for this audio (currently built as an MVC application). As an example, my endpoint looks something like https://audio.fooservice.com/GetAudio?audioId=123
The audio is 8kHz, 1-channel u-law.
Due to varying format support across browsers when using the HTML5 <audio> tag, I am unable to use the original u-law wav because Internet Explorer will not play it.
My proposed solution is to do a real-time conversion from the source format to mp3.
I've cobbled together a partially working solution from various other questions here and in the NAudio forums, but it throws an exception as noted in the comments below:
private void NAudioTest(string url)
{
Stream outStream = new MemoryStream();
var format = WaveFormat.CreateMuLawFormat(8000, 1);
using (Stream ms = new MemoryStream())
{
var request = (HttpWebRequest)WebRequest.Create(url);
request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
using (Stream stream = request.GetResponse().GetResponseStream())
{
using (var reader = new RawSourceWaveStream(stream, format))
{
// reader is not seekable; we need to convert to a byte array to seek
var bytes = reader.ToByteArray();
// create a new stream from the byte aray
var seekableStream = new MemoryStream(bytes);
// instantiating a WaveFileReader as follows will throw an exception:
// "System.FormatException: Not a WAVE file - no RIFF header"
using (var waveReader = new WaveFileReader(seekableStream))
{
using (var pcmStream = WaveFormatConversionStream.CreatePcmStream(waveReader))
{
var pcmBytes = pcmStream.ToByteArray();
var mp3 = pcmBytes.ToMp3();
}
}
}
}
}
}
public static class StreamExtensions
{
public static byte[] ToByteArray(this Stream stream)
{
var ms = new MemoryStream();
var buffer = new byte[1024];
int bytes = 0;
while ((bytes = stream.Read(buffer, 0, buffer.Length)) > 0)
ms.Write(buffer, 0, bytes);
return ms.ToArray();
}
}
public static class ByteExtensions
{
public static byte[] ToMp3(this byte[] bytes)
{
using (var outStream = new MemoryStream())
{
using (var ms = new MemoryStream(bytes))
{
using (var reader = new WaveFileReader(ms))
{
using (var writer = new LameMP3FileWriter(outStream, reader.WaveFormat, 64))
{
reader.CopyTo(writer);
return outStream.ToArray();
}
}
}
}
}
}
I've been poking around at this for most of the day and I feel like I'm introducing unnecessary complexity into something that seems like it should be fairly straightforward.
Any help would be much appreciated.
Note: I cannot change the source format and supporting IE is a requirement.
EDIT: I resolved the RIFF exception and am able to produce a stream of the MP3, but it's nothing but white noise. Hopefully I can resolve that as well. My new code is as follows:
[HttpGet]
public ActionResult GetMp3(string url)
{
if (String.IsNullOrWhiteSpace(url))
return null;
var muLawFormat = WaveFormat.CreateMuLawFormat(8000, 1);
var compressedStream = new MemoryStream();
using (var ms = new MemoryStream())
{
var request = (HttpWebRequest)WebRequest.Create(url);
request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
using (Stream webStream = request.GetResponse().GetResponseStream())
{
var buffer = new byte[4096];
int read;
while (webStream != null && (read = webStream.Read(buffer, 0, buffer.Length)) > 0)
ms.Write(buffer, 0, read);
}
ms.Position = 0;
using (WaveStream wav = WaveFormatConversionStream.CreatePcmStream(new RawSourceWaveStream(ms, muLawFormat)))
using (var mp3 = new LameMP3FileWriter(compressedStream, new WaveFormat(), LAMEPreset.MEDIUM_FAST))
wav.CopyTo(mp3);
}
compressedStream.Seek(0, 0);
return new FileStreamResult(compressedStream, "audio/mpeg");
}

This works for me (and I needed to do exactly what you wanted to do). Hope this helps someone else as well. I used NAudio with LAME.
You have to make sure that you copy the libmp3lamexx.dll files to your webserver's BIN location or to some folder in the %PATH% variable, else it won't work.
string sq = /* URL of WAV file (http://foo.com/blah.wav) */
Response.ContentType = "audio/mpeg";
using (WebClient wc = new WebClient())
{
if (!sq.ToLower().EndsWith(".wav"))
{
byte[] rawFile = wc.DownloadData(sq.Trim());
Response.OutputStream.Write(rawFile, 0, rawFile.Length);
}
else
{
using (var wavReader = new WaveFileReader(new MemoryStream(wc.DownloadData(sq.Trim()))))
{
try
{
using (var wavWriter = new LameMP3FileWriter(Response.OutputStream, wavReader.WaveFormat, LAMEPreset.ABR_128))
{
wavReader.CopyTo(wavWriter);
}
}
catch (ArgumentException)
{
var newFormat = new WaveFormat(wavReader.WaveFormat.SampleRate, 16, 2);
using (var pcmStream = new WaveFormatConversionStream(newFormat, wavReader))
{
using (var wavWriter = new LameMP3FileWriter(Response.OutputStream, pcmStream.WaveFormat, LAMEPreset.ABR_128))
{
pcmStream.CopyTo(wavWriter);
}
}
}
}
}
Response.Flush();
Response.End();
}

Can I upload excel file into Amazon S3 using AWSSDK.dll

Can I upload excel file into the AWS s3 account. What I have fount is that PutObject method provided in the Library can be used to upload the file from a location or using the Stream object.
PutObjectRequest request = new PutObjectRequest()
{
ContentBody = "this is a test",
BucketName = bucketName,
Key = keyName,
InputStream = stream
};
PutObjectResponse response = client.PutObject(request);
Key can be the absolute path on the machine or we give the stream of the file. But my doubt is how we can upload the excel file using the above method
P.S
This is the way I am using to convert stream to byte[] but input.ReadByte() is always equal to zero. So my doubt is, is it not reading the excel file?
FileStream str = new FileStream(#"C:\case1.xlsx", FileMode.Open);
byte[] arr = ReadFully(str);
public static byte[] ReadFully(FileStream input)
{
long size = 0;
while (input.ReadByte() > 0)
{
size++;
}
byte[] buffer = new byte[size];
//byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}

You should be able to upload any file via the file path or stream. It doesn't matter that it's an Excel file. When you run PutObject, it uploads the actual file data represented by that path or stream.
You can see the MIME types for MS Office formats at Filext. Doing it by file path would probably be easier:
PutObjectRequest request = new PutObjectRequest()
{
ContentBody = "this is a test",
BucketName = bucketName,
Key = keyName,
ContentType =
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", // xlsx
FilePath = #"\path\to\myfile.xlsx"
};
PutObjectResponse response = client.PutObject(request);
Or reading from a file stream:
PutObjectRequest request = new PutObjectRequest()
{
ContentBody = "this is a test",
BucketName = bucketName,
Key = keyName,
ContentType =
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" // xlsx
};
using (var stream = new FileStream(#"\path\to\myfile.xlsx", FileMode.Open))
{
request.InputStream = stream;
PutObjectResponse response = client.PutObject(request);
}