Scenario
I have a object that I convert to a flat CSV and then compress and upload to a filestore.
I could easily do this by following the below steps.
Convert object to CSV file.
Compress file
Upload file.
However
I do not want the penalty that comes with touching physical storage so would like to do all this in memory.
Current Incorrect Implementation
Convert object to CSV byte array
Compress byte array
Upload byte array to file store
Problem
What i'm essentially doing is compressing a byte array and uploading that. which is obviously wrong. (Because when the compressed Gzip file is uncompressed, it contains a byte array of the csv and not the actual csv itself.)
Is it possible to create a file like "file.csv" in memory and then compress that in memory, instead of compressing a byte array?
The problem I'm having is it would seem I can only name the file and specify its extension when saving to a physical location.
Code Example of Current Implementation
public byte[] Example(IEnumerable<object> data)
{
// Convert Object to CSV and write to byte array.
byte[] bytes = null;
using (var ms = new MemoryStream())
{
TextWriter writer = new StreamWriter(ms);
var csv = new CsvWriter(writer);
csv.WriteRecords(data);
writer.Flush();
ms.Position = 0;
bytes = ms.ToArray();
}
//Compress byte array
using (var compressedStream = new MemoryStream(bytes))
using (var resultStream = new MemoryStream())
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
{
zipStream.CopyTo(resultStream);
zipStream.Close();
var gzipByteArray = resultStream.ToArray();
//Upload to AzureStorage
new AzureHelper().UploadFromByteArray(gzipByteArray, 0, gzipByteArray.Length);
}
}
Wrap the Stream you use for the upload in a GZipStream, write your CSV to that, and the then you'll have uploaded the compressed CSV.
Related
I am building a XmlDocument in memory (I am not writing it to disk). I need to be able to create a zip archive that would contain the Xml file and then get the zip archive as byte array (all of this without actually writing/creating anything on the hard-disk). Is this possible?
I should mention that I am trying to do this in C#.
var buffer = new MemoryStream();
using (buffer)
using (var zip = new ZipArchive(buffer, ZipArchiveMode.Create) )
{
var entry = zip.CreateEntry("content.xml", CompressionLevel.Optimal);
using (var stream = entry.Open())
{
xmlDoc.Save(stream);
}
}
var bytes = buffer.ToArray();
I need to convert a byte array into an excel file using C# to upload it in Sharepoint.
The following code read an input file from client as a byte array:
public object UploadFile(HttpPostedFile file)
{
byte[] fileData = null;
using (var binaryReader = new BinaryReader(file.InputStream))
{
fileData = binaryReader.ReadBytes(imageFile.ContentLength);
// convert fileData to excel
}
}
How can I do it?
It sounds like you're just after File.WriteAllBytes(path, contents). However, if the input file could be large, you may be better off using the Stream API:
using(var destination = File.Create(path)) {
file.InputStream.CopyTo(destination);
}
Edit: it looks like HttpPostedFile has a SaveAs method, so just:
file.SaveAs(path);
I am facing issue while converting byte array to zip file.Even though zip file is created using the below code but when I am extracting the zip file I am getting error "Cannot open file. It does not appear to be a valid archive".
private static void ShowZipFile(string fileName, byte[] data)
{
byte[] compress = Compress(data);
File.WriteAllBytes(fileName, compress);
}
private static byte[] Compress(byte[] data)
{
using (MemoryStream memory = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(memory,
CompressionMode.Compress, true))
{
gzip.Write(data, 0, data.Length);
}
return memory.ToArray();
}
}
A GZipStream isn't a zip file, basically - it's a gzip file. That's just compressed data, without any notions of multiple files, file names etc. If you save the file as foo.gz you may find that the zip tool you use knows how to decompress that, but you definitely need to understand that it's not the same as a foo.zip with file entries etc.
If you want to create an actual zip file, you might want to look at SharpZipLib, System.IO.Compression.ZipFile or similar libraries.
I want to convert mp3 files to WAV. In DirectX.DirectSound the secondary buffer is supporting only WAV. I am converting the files using naudio and this is working fine.
using (Mp3FileReader reader = new Mp3FileReader(mp3File))
{
WaveFileWriter.CreateWaveFile(outputFile, reader);
}
The problem is that I can't save the files on disk so I have to use them with stream.
Mp3FileReader mp3reader = new Mp3FileReader(filenamePathMp3);
var stream=WaveFormatConversionStream.CreatePcmStream(mp3reader);
which throws an exception Value does not fall within the expected range. How Can I make a stream with the audio converted to WAV or Raw audio for the secondaryBuffer without writing on disk?
Thank you in advance,
WaveFileWriter can write to a Stream instead of a file so you can just pass in a MemoryStream. Also, Mp3FileReader.Read already returns PCM so you could read out into a byte[] and then insert that into your memory stream if you prefer.
I've achieved to convert MP3 to WAV without writing it to the disk by using MemoryStream and WaveFileWriter.WriteWavFileToStream
You can also set up sampleRate bitrate and channel using RawSourceWaveStream
public static byte[] ConvertMp3ToWav(byte[] mp3File)
{
using (var retMs = new MemoryStream())
using (var ms = new MemoryStream(mp3File))
using (Mp3FileReader reader = new Mp3FileReader(ms))
{
var rs = new RawSourceWaveStream(reader, new WaveFormat(16000, 1));
using (WaveStream pcmStream = WaveFormatConversionStream.CreatePcmStream(rs))
{
WaveFileWriter.WriteWavFileToStream(retMs,pcmStream);
return retMs.ToArray();
}
}
}
I ave writing an XML File of size more than 1GB but at the time of writing I want to compress that file so that the size of an xml file is reduces so that at tile of xmlDoc.Load(fileName) load the file in minimum time duration.
my code for Writing an XML File is
using (FileStream fileStream = new FileStream(_logFilePath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite))
{
xmlDoc.Load(fileStream);
int byteLenght = fileStream.ReadByte();
byte[] intBytes = BitConverter.GetBytes(byteLenght);
intBytes = Compress(intBytes);
xmlDoc.DocumentElement.AppendChild(newelement);
fileStream.SetLength(0);
xmlDoc.Save(fileStream);
}
also for compression
private static byte[] Compress(byte[] data)
{
byte[] retVal;
using (MemoryStream compressedMemoryStream = new MemoryStream())
{
DeflateStream compressStream = new DeflateStream(compressedMemoryStream, CompressionMode.Compress, true);
compressStream.Write(data, 0, data.Length);
compressStream.Close();
retVal = new byte[compressedMemoryStream.Length];
compressedMemoryStream.Position = 0L;
compressedMemoryStream.Read(retVal, 0, retVal.Length);
compressedMemoryStream.Close();
compressStream.Close();
}
return retVal;
}
but its not work for compression the file.
Compressing the file on disk won't do much to improve the time spent loading the document, because the larger part of the time is in building up the object graph for the XmlDocument. It is so slow that it can take as long (or longer) as reading the uncompressed XML from disk. Although compression can save time here, it's only a minor gain if a fast media like an internal hdd is used.
If you want to improve performance working with large XML files, you'll need to use something like an XmlReader that streams the file instead of loading it all at once.