We Keep Coding

how to make file content as base64 encoded before making zip file

I have a directory where I have CSV files which I need to first encode the file content as base64 string and then make it as zip file.
I am able to make file as zip with below code, but in between on the fly how to make file content as base64 encoded? Thanks!
var csvFiles = Directory.GetFiles(#"C:\Temp", "*.csv")
.Select(f => new FileInfo(f));
foreach (var file in csvFiles)
{
using (var newFile = ZipFile.Open($#"C:\tmp\{Path.GetFileNameWithoutExtension(file.Name)}.zip",
ZipArchiveMode.Create))
{
newFile.CreateEntryFromFile($#"C:\Temp\{file.Name}",
file.Name);
}
}

Disregarding your motives or other problems (conceptual or otherwise)
Here is a fully streamed solution with minimal allocations (let's be nice to your Large Object Heap). The CryptoStream with ToBase64Transform, is just a way to stream base64 encoding
var csvFiles = Directory.GetFiles(#"D:\Temp");
using var outputStream = new FileStream(#"D:\Test.zip", FileMode.Create);
using var archive = new ZipArchive(outputStream, ZipArchiveMode.Create, true);
foreach (var file in csvFiles)
{
using var inputFile = new FileStream(file, FileMode.Open, FileAccess.Read);
using var base64Stream = new CryptoStream(inputFile, new ToBase64Transform(), CryptoStreamMode.Read);
var entry = archive.CreateEntry(Path.GetFileName(file));
using var zipStream = entry.Open();
base64Stream.CopyTo(zipStream);
}

You need to create the base64 string, convert it to a byte array, and then create the archive entry from the byte array (by creating a stream).
Something like this should do the job:
var dirInfo = new DirectoryInfo(#"C:\Temp");
var csvFiles = dirInfo.GetFiles("*.csv"); // This already returns a `FileInfo[]`.
foreach (var file in csvFiles)
{
var fileBytes = File.ReadAllBytes(file.FullName);
var base64String = Convert.ToBase64String(fileBytes);
var base64Bytes = Encoding.UTF8.GetBytes(base64String);
string newFilePath = $#"C:\tmp\{Path.GetFileNameWithoutExtension(file.Name)}.zip";
using (var newFile = ZipFile.Open(newFilePath, ZipArchiveMode.Create))
{
// You might want to change the extension
// since the file is no longer in CSV format.
var zipEntry = newFile.CreateEntry(file.Name);
using (var base64Stream = new MemoryStream(base64Bytes))
using (var zipEntryStream = zipEntry.Open())
{
base64Stream.CopyTo(zipEntryStream);
}
}
}
Alternatively, you could save the base64 string to a temporary file, create the entry from that file, and then delete it; but I don't prefer writing dummy data to the disk when the job can be done in memory.

Asp .NET Read a file from a tar.gz archive

I have some files inside in one .tar.gz archive. These files are on a linux server.How can I read from a specific file inside this archive if I know it's name?
For reading direct from the txt file, I used the following code:
Uri urlFile = new Uri("ftp://" + ServerName + "/%2f" + FilePath + "/" + fileName);
WebClient req = new WebClient() { Credentials=new NetworkCredential("user","psw")};
string result = req.DownloadString(urlFile);
It's possible to read this file without copying the archive on the local machine, something like the code above?

I found a solution. Maybe this can help you guys.
// archivePath="ftp://myLinuxServer.com/%2f/move/files/archive/20170225.tar.gz";
public static string ExtractFileFromArchive(string archivePath, string fileName)
{
string stringFromFile="File not found";
WebClient wc = new WebClient() { Credentials = cred, Proxy= webProxy }; //Create webClient with all necessary settings
using (Stream source = new GZipInputStream(wc.OpenRead(archivePath))) //wc.OpenRead() create one stream with archive tar.gz from our server
{
using (TarInputStream tarStr =new TarInputStream(source)) //TarInputStream is a stream from ICSharpCode.SharpZipLib.Tar library(need install SharpZipLib in nutgets)
{
TarEntry te;
while ((te = tarStr.GetNextEntry())!=null) // Go through all files from archive
{
if (te.Name == fileName)
{
using (Stream fs = new MemoryStream()) //Create a empty stream that we will be fill with file contents.
{
tarStr.CopyEntryContents(fs);
fs.Position = 0; //Move stream position to 0, in order to read from beginning
stringFromFile = new StreamReader(fs).ReadToEnd(); //Convert stream to string
}
break;
}
}
}
}
return stringFromFile;
}

'System.OutOfMemoryException' was thrown in ZipArchive for large file from URL

I have an URL which contains a zip file. The files need to be unzipped from the URL. The URL is Opened and Read using webclient and then added to a Stream. It is then used in the ZipArchive object which will unzip the files and store them in the D:\ drive. When a file is around 400Mb I get the 'System.OutOfMemoryException'.
Stream has to be used since the webClient.OpenRead(Uri Address) returns a Stream. As well as the use ZipArchive(Stream stream).
How can I stop from getting this message?
string zipFileUrl = "https://www.dropbox.com/s/clersbjdcshpdy6/oversize_zip_test_0.zip?dl=0"
string output_path = #"D:\";
using (WebClient webClient = new WebClient())
{
using (Stream streamFile = webClient.OpenRead(zipFileUrl))
{
using (ZipArchive archive = new ZipArchive(streamFile))//ERROR HERE
{
var entries = archive.Entries;
//Loops thru each file in Zip and adds it to directory
foreach (var entry in entries)
{
if (entry.FullName != "/" && entry.Name != "")
{
string completeFileName = Path.Combine(output_path, entry.FullName);
string directory = Path.GetDirectoryName(completeFileName);
//If directory does not exist then we create it.
if (!Directory.Exists(directory))
{
Directory.CreateDirectory(directory);
}
//Extracts zip from URL to extract path, and overwrites if file exists.
entry.ExtractToFile(completeFileName, true);
}
}
}
}

I think here might be your problem, from the ZipArchive.Init method
private void Init(Stream stream, ZipArchiveMode mode, Boolean leaveOpen)
{
Stream extraTempStream = null;
try
{
_backingStream = null;
//check stream against mode
switch (mode)
{
case ZipArchiveMode.Create:
// (SNIP)
case ZipArchiveMode.Read:
if (!stream.CanRead)
throw new ArgumentException(SR.ReadModeCapabilities);
if (!stream.CanSeek)
{
_backingStream = stream;
extraTempStream = stream = new MemoryStream();
_backingStream.CopyTo(stream);
stream.Seek(0, SeekOrigin.Begin);
}
break;
case ZipArchiveMode.Update:
// (SNIP)
default:
// (SNIP)
}
// (SNIP)
}
if streamFile.CanSeek is false (which from a WebClient it will be) it copies the entire file in to memory then works on the file. This is what is using up all the memory.
Try to find a 3rd party library that handles Zip files and does not need a stream that supports seeking. If you can't, copy the file to disk first to the temp folder with a FileStream with the FileOptions.DeleteOnClose option passed in, then use that stream in your zip before you close the stream.
string zipFileUrl = "https://www.dropbox.com/s/clersbjdcshpdy6/oversize_zip_test_0.zip?dl=0";
string output_path = #"D:\";
using (var tempFileStream = new FileStream(Path.GetTempFileName(), FileMode.Create,
FileAccess.ReadWrite, FileShare.None,
4096, FileOptions.DeleteOnClose))
{
using (WebClient webClient = new WebClient())
{
using (Stream streamFile = webClient.OpenRead(zipFileUrl))
{
streamFile.CopyTo(tempFileStream);
}
}
tempFileStream.Position = 0;
using (ZipArchive archive = new ZipArchive(tempFileStream))
{
var entries = archive.Entries;
//Loops thru each file in Zip and adds it to directory
foreach (var entry in entries)
{
if (entry.FullName != "/" && entry.Name != "")
{
string completeFileName = Path.Combine(output_path, entry.FullName);
string directory = Path.GetDirectoryName(completeFileName);
//If directory does not exist then we create it.
if (!Directory.Exists(directory))
{
Directory.CreateDirectory(directory);
}
//Extracts zip from URL to extract path, and overwrites if file exists.
entry.ExtractToFile(completeFileName, true);
}
}
}
}

Base64 to StreamContent to upload image without pulling from file system

I'd like to be able to change this code so that I don't have to pull from a file on a file system, but rather use a base64 value that is saved in a database. Does anyone here know enough about StreamContent to know what I need to do to accomplish this?
The file is a jpeg file.
private static StreamContent FileMultiPartBody(string fullFilePath)
{
var fileInfo = new FileInfo(fullFilePath);
var fileContent = new StreamContent(fileInfo.OpenRead());
// Manually wrap the string values in escaped quotes.
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
{
FileName = string.Format("\"{0}\"", fileInfo.Name),
Name = "\"signature\"",
};
fileContent.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
return fileContent;
}

StreamContent is just a wrapper of another stream (the stream returned from fileInfo.OpenRead() in your example). All you need to do is replace that stream with a stream from your database and return that. You can also replace the fileInfo.Name with a Path.GetFileName(fullFilePath) call.
private Stream GetStreamFromDatabase(string fullFilePath)
{
//TODO
}
private static StreamContent FileMultiPartBody(string fullFilePath)
{
var fileContent = new StreamContent(GetStreamFromDatabase(fullFilePath))
// Manually wrap the string values in escaped quotes.
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
{
FileName = string.Format("\"{0}\"", Path.GetFileName(fullFilePath)),
Name = "\"signature\"",
};
fileContent.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
return fileContent;
}
If you need help converting a base64 value from a database to a stream I would suggest asking a separate question about that.

Extract a zipped file only (without the folder)

With the SharpZip lib I can easily extract a file from a zip archive:
FastZip fz = new FastZip();
string path = "C:/bla.zip";
fz.ExtractZip(bla,"C:/Unzips/",".*");
However this puts the uncompressed folder in the output directory.
Suppose there is a foo.txt file within bla.zip which I want. Is there an easy way to just extract that and place it in the output directory (without the folder)?

The FastZip does not seem to provide a way to change folders, but the "manual" way of doing supports this.
If you take a look at their example:
public void ExtractZipFile(string archiveFilenameIn, string outFolder) {
ZipFile zf = null;
try {
FileStream fs = File.OpenRead(archiveFilenameIn);
zf = new ZipFile(fs);
foreach (ZipEntry zipEntry in zf) {
if (!zipEntry.IsFile) continue; // Ignore directories
String entryFileName = zipEntry.Name;
// to remove the folder from the entry:
// entryFileName = Path.GetFileName(entryFileName);
byte[] buffer = new byte[4096]; // 4K is optimum
Stream zipStream = zf.GetInputStream(zipEntry);
// Manipulate the output filename here as desired.
String fullZipToPath = Path.Combine(outFolder, entryFileName);
string directoryName = Path.GetDirectoryName(fullZipToPath);
if (directoryName.Length > 0)
Directory.CreateDirectory(directoryName);
using (FileStream streamWriter = File.Create(fullZipToPath)) {
StreamUtils.Copy(zipStream, streamWriter, buffer);
}
}
} finally {
if (zf != null) {
zf.IsStreamOwner = true;stream
zf.Close();
}
}
}
As they note, instead of writing:
String entryFileName = zipEntry.Name;
you can write:
String entryFileName = Path.GetFileName(entryFileName)
to remove the folders.

Assuming you know this is the only file (not folder) in the zip:
using(ZipFile zip = new ZipFile(zipStm))
{
foreach(ZipEntry ze in zip)
if(ze.IsFile)//must be our foo.txt
{
using(var fs = new FileStream(#"C:/Unzips/foo.txt", FileMode.OpenOrCreate, FileAccess.Write))
zip.GetInputStream(ze).CopyTo(fs);
break;
}
}
If you need to handle other possibilities, or e.g. getting the name of the zip-entry, the complexity rises accordingly.