We Keep Coding

How can I give a *.csv extension to something written in a MemoryStream?

I’m developing an Azure Function to create a CSV file from a list of custom objects, gzip it and upload it to an Azure Storage container with this code:
var blobServiceClient = new BlobServiceClient("My connection string");
var containerClient = blobServiceClient.GetBlobContainerClient("My container name");
var config = new CsvConfiguration(CultureInfo.CurrentCulture) { Delimiter = ";", Encoding = Encoding.UTF8 };
var list = new List<FakeModel>
{
new FakeModel { Field1 = "A", Field2 = "B" },
new FakeModel { Field1 = "C", Field2 = "D" }
};
await using var memoryStream1 = new MemoryStream();
await using var streamWriter = new StreamWriter(memoryStream1);
await using var csvWriter = new CsvWriter(streamWriter, config);
await csvWriter.WriteRecordsAsync(list);
await csvWriter.FlushAsync();
memoryStream1.Position = 0;
await using var memoryStream2 = new MemoryStream();
await using var zip = new GZipStream(memoryStream2, CompressionMode.Compress, true);
await memoryStream1.CopyToAsync(zip);
memoryStream2.Position = 0;
var blockBlob = containerClient.GetBlockBlobClient("test.csv.gz");
await blockBlob.UploadAsync(memoryStream2);
It works. When I download the gzip from the cloud to check it, obviously I get that the file has the correct name, so it’s shown as a GZ file whose name is test.csv.gz, but when I download it and open it with an extractor, I get that the CSV file inside of it is something strange, like test.csv-3, that my computer can’t open. Of course, I need it to be a valid *.csv file. The problem here is that using memory streams I can only give a name to the blob, not for the inner CSV file. How can I do it? Keep in mind that I’d like so use memory streams to keep things simple with Azure Functions’s local storage. Can you help me?

Regarding the issue, please refer to the following code
var blobServiceClient = new BlobServiceClient("My connection string");
var containerClient = blobServiceClient.GetBlobContainerClient("My container name");
var config = new CsvConfiguration(CultureInfo.CurrentCulture) { Delimiter = ";", Encoding = Encoding.UTF8 };
var list = new List<FakeModel>
{
new FakeModel { Field1 = "A", Field2 = "B" },
new FakeModel { Field1 = "C", Field2 = "D" }
};
await using var memoryStream1 = new MemoryStream();
await using var streamWriter = new StreamWriter(memoryStream1);
await using var csvWriter = new CsvWriter(streamWriter, config);
await csvWriter.WriteRecordsAsync(list);
await csvWriter.FlushAsync();
memoryStream1.Position = 0;
var options = new BlockBlobOpenWriteOptions
{
HttpHeaders = new BlobHttpHeaders
{
ContentType = "application/gzip",
},
};
await using var outStream= await containerClient.GetBlockBlobClient("test.csv.gz").OpenWriteAsync(true,options);
await using var zip = new GZipStream(outStream, CompressionMode.Compress, true);
await memoryStream1.CopyToAsync(zip);
await using var input = await containerClient.GetBlockBlobClient("test.csv.gz").OpenReadAsync();
await using var file = File.Create("<file path>");
await using var zip1 = new GZipStream(input, CompressionMode.Decompress, true);
await zip1.CopyToAsync(file);

zip multiple pdfs from url link, how to

I have a project that requires pdf files to be zipped up from an URL link and then downloaded and clickable by the end users browser. So far, I was able to zip one pdf file which isn't nearly what I'm looking to for.
I'm not sure how to proceed from here. Below is the code. Any help would be very much appreciated.
ASP.NET Core
[HttpGet("zipFiles")]
public IActionResult ZipPDFFiles()
{
var fileNames = _repo.GetFileNames();
foreach (var filesName in fileNames)
{
var urlLink = "https://example.com/folder/" + $"{filesName.PdfFileName}";
var net = new System.Net.WebClient();
var data = net.DownloadData(urlLink);
var file = $"{filesName.PdfFileName}";
var contentType = "application/zip";
string zippedFolderName = "Archive.zip";
using (MemoryStream ms = new MemoryStream())
{
using (var archive = new ZipArchive(ms, ZipArchiveMode.Create, true))
{
var zipArchiveEntry = archive.CreateEntry($"{file}", System.IO.Compression.CompressionLevel.Fastest);
using (var zipStream = zipArchiveEntry.Open()) zipStream.Write(data, 0, data.Length);
}
return File(ms.ToArray(), contentType, $"{zippedFolderName}");
}
}
return NotFound();
}

Some improvements:
used HttpClient
files uploaded simultaneously
var fileUrls = new[]
{
new Uri("https://learn.microsoft.com/en-us/dotnet/standard/microservices-architecture/implement-resilient-applications/media/image3.5.png"),
new Uri("https://learn.microsoft.com/en-us/dotnet/standard/microservices-architecture/implement-resilient-applications/media/image4.png"),
new Uri("https://learn.microsoft.com/en-us/dotnet/standard/microservices-architecture/implement-resilient-applications/media/image6.png")
};
var downloadResults = fileUrls
.Select(uri => (uri: uri, response: HttpClientFactory.Create().SendAsync(new HttpRequestMessage(HttpMethod.Get, uri))))
.ToArray();
await Task.WhenAll(downloadResults.Select(v => v.response));
using (var ms = new MemoryStream())
{
using (var archive = new ZipArchive(ms, ZipArchiveMode.Create, true))
{
foreach (var download in downloadResults)
{
var entry = archive.CreateEntry(download.uri.Segments.Last(), CompressionLevel.Fastest);
using (var zipStream = entry.Open())
{
var data = await download.response.Result.Content.ReadAsByteArrayAsync();
zipStream.Write(data, 0, data.Length);
}
}
}
return File(ms.ToArray(), contentType, $"{zippedFolderName}");
}

How to convert GZipStream to HttpContent?

I have written a Web API code to return a zip file. But I am not able to convert the GZipStream content to HttpContent. I get the following error:
cannot implicitly convert type 'system.io.compression.GZipStream' to
'System.Net.Http.HttpContent'
Where did I go wrong?
My WebApi Code:
var content =
new GZipStream(memStream, CompressionMode.Compress);
HttpResponseMessage httpResponseMessage = new HttpResponseMessage();
httpResponseMessage.Content = content;
httpResponseMessage.Content.Headers.Add("x-filename", document.FileName);
httpResponseMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("application/zip");
httpResponseMessage.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
httpResponseMessage.Content.Headers.ContentDisposition.FileName = "xyz.zip";
httpResponseMessage.StatusCode = HttpStatusCode.OK;

I got a solution using Zip archive and I have created a static class to push the memory stream into a zip archive as below,
var pushStreamContent = ZipStreamContent.Create("MultipleDocument.zip", memStList);
ZipStreamContent class,
public static class ZipStreamContent
{
public static PushStreamContent Create(string fileName, List<MemoryStream> msList)
{
var content = new PushStreamContent((outputStream, httpContent, transportContext) =>
{
using (var zip = new ZipArchive(outputStream, ZipArchiveMode.Create, leaveOpen: false))
{
msList[0].Position = 0;
var createenter = zip.CreateEntry("xyz.jpg", CompressionLevel.Optimal);
using (var s = createenter.Open())
{
msList[0].CopyTo(s);
}
}
});
content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
content.Headers.ContentDisposition.FileName = fileName;
return content;
}
}

I have simplified the above given GipStreamContent static class like following, It's working well, so I hope It will help all others.
CloudBlockBlob blob = null;
//azure storage connection
var container = GetBlobClient(tenantInfo);
//directory reference
var directory = container.GetDirectoryReference(
string.Format(DirectoryNameConfigValue, tenantInfo.TenantId.ToString(), documentList[0].ProjectId));
var pushStreamContent = new PushStreamContent(async (outputStream, httpContent, transportContext) =>
{
//zip the multiple files
using (var zipEntry = new ZipArchive(outputStream, ZipArchiveMode.Create, leaveOpen: false))
{
for (int docId = 0; docId < documentList.Count; docId++)
{
blob = directory.GetBlockBlobReference(DocumentNameConfigValue + documentList[docId].DocumentId);
if (!blob.Exists()) continue;
MemoryStream memStream = new MemoryStream();
await blob.DownloadToStreamAsync(memStream);
memStream.Position = 0;
var createEntry = zipEntry.CreateEntry(documentList[docId].FileName, CompressionLevel.Fastest);
using (var stream = createEntry.Open())
{
memStream.CopyTo(stream);
}
}
}
});

How to generate a zip file within the HttpResponseMessage from within an Api Controller [duplicate]

I have a web service that I can call and save the returned csv file. Everything seems to be working OK. What I am now interested in doing is returning multiple CSV files for the user to download. What is the proper way to handle this? I'm guessing I need a way to package them up (zip? perhaps)?
[HttpPost]
[Route("OutputTemplate")]
public HttpResponseMessage OutputTemplate()
{
HttpResponseMessage msg = new HttpResponseMessage();
string body = this.Request.Content.ReadAsStringAsync().Result;
try
{
string contents = DoStuff(body) // get contents based on body
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(contents);
writer.Flush();
stream.Position = 0;
msg.StatusCode = HttpStatusCode.OK;
msg.Content = new StreamContent(stream);
msg.Content.Headers.ContentType = new MediaTypeHeaderValue("text/csv");
msg.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = "fileexport"
};
return msg;
}
...
}

Using the following model to abstract file name and content
public class FileModel {
public string FileName { get; set; }
public byte[] FileContent { get; set; }
}
The following extension was derived to compress the file content
public static class ZipArchiveExtensions {
public static Stream Compress(this IEnumerable<FileModel> files) {
if (files.Any()) {
var ms = new MemoryStream();
using(var archive = new ZipArchive(
stream: ms,
mode: ZipArchiveMode.Create,
leaveOpen: true
)){
foreach (var file in files) {
var entry = archive.add(file);
}
}
ms.Position = 0;
return ms;
}
return null;
}
private static ZipArchiveEntry add(this ZipArchive archive, FileModel file) {
var entry = archive.CreateEntry(file.FileName, CompressionLevel.Fastest);
using (var stream = entry.Open()) {
stream.Write(file.FileContent, 0, file.FileContent.Length);
}
return entry;
}
}
With that in place, the example API controller action could look something like this.
public class ExampleApiController : ApiController {
public async Task<IHttpActionResult> OutputTemplate() {
IHttpActionResult result = BadRequest();
var body = await Request.Content.ReadAsStreamAsync();
List<FileModel> files = DoSomething(body);
if (files.Count > 1) {
//compress the files.
var archiveStream = files.Compress();
var content = new StreamContent(archiveStream);
var response = Request.CreateResponse(System.Net.HttpStatusCode.OK);
response.Content = content;
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/zip");
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") {
FileName = "fileexport.zip"
};
result = ResponseMessage(response);
} else if (files.Count == 1) {
//return the single file
var fileName = files[0].FileName; //"fileexport.csv"
var content = new ByteArrayContent(files[0].FileContent);
var response = Request.CreateResponse(System.Net.HttpStatusCode.OK);
response.Content = content;
response.Content.Headers.ContentType = new MediaTypeHeaderValue("text/csv");
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") {
FileName = fileName
};
result = ResponseMessage(response);
}
return result;
}
private List<FileModel> DoSomething(System.IO.Stream body) {
//...TODO: implement file models
throw new NotImplementedException();
}
}

How can I get the bytes of a GetObjectResponse from S3?

I'm retrieving a file from Amazon S3. I want to convert the file to bytes so that I can download it as follows:
var download = new FileContentResult(bytes, "application/pdf");
download.FileDownloadName = filename;
return download;
I have the file here:
var client = Amazon.AWSClientFactory.CreateAmazonS3Client(
accessKey,
secretKey,
config
);
GetObjectRequest request = new GetObjectRequest();
GetObjectResponse response = client.GetObject(request);
I know about response.WriteResponseStreamToFile() but I want to download the file to the regular downloads folder. If I convert the GetObjectResponse to bytes, I can return the file. How can I do this?

Here's the solution I found for anyone else who needs it:
GetObjectResponse response = client.GetObject(request);
using (Stream responseStream = response.ResponseStream)
{
var bytes = ReadStream(responseStream);
var download = new FileContentResult(bytes, "application/pdf");
download.FileDownloadName = filename;
return download;
}
public static byte[] ReadStream(Stream responseStream)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = responseStream.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}

Just another option:
Stream rs;
using (IAmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client())
{
GetObjectRequest getObjectRequest = new GetObjectRequest();
getObjectRequest.BucketName = "mybucketname";
getObjectRequest.Key = "mykey";
using (var getObjectResponse = client.GetObject(getObjectRequest))
{
getObjectResponse.ResponseStream.CopyTo(rs);
}
}

I struggled to get the cleaner method offered by Alex to work (not sure what I'm missing), but I wanted to do it w/o the extra ReadStream method offered by Erica (although it worked)... here is what I wound up doing:
var s3Client = new AmazonS3Client(AccessKeyId, SecretKey, Amazon.RegionEndpoint.USEast1);
using (s3Client)
{
MemoryStream ms = new MemoryStream();
GetObjectRequest getObjectRequest = new GetObjectRequest();
getObjectRequest.BucketName = BucketName;
getObjectRequest.Key = awsFileKey;
using (var getObjectResponse = s3Client.GetObject(getObjectRequest))
{
getObjectResponse.ResponseStream.CopyTo(ms);
}
var download = new FileContentResult(ms.ToArray(), "image/png"); //"application/pdf"
download.FileDownloadName = ToFilePath;
return download;
}

Stream now has asynchronous methods. In C# 8, you can do this:
public async Task<byte[]> GetAttachmentAsync(string objectPointer)
{
var objReq = new GetObjectRequest
{
BucketName = "bucket-name",
Key = objectPointer, // the file name
};
using var objResp = await _s3Client.GetObjectAsync(objReq);
using var ms = new MemoryStream();
await objResp.ResponseStream.CopyToAsync(ms, _ct); // _ct is a CancellationToken
return ms.ToArray();
}
This won't block any threads while the IO occurs.