I need compress and uncompress strings only in memory, without saving it inside file.
It's easy to do that with one string but the problem appears when I need do this with many files. I don't want do compress every string creating array of bytes array. Is it possible to compress many strings into one object (zip with many files in memory)and uncompress this dynamically ? (yield)
Also I have to deal with big strings(xml files) and I copy this data from procedure into MemotySream. I'd like to just compress data from this streams
Related
I have a application that I want to copy directories within a internal ZIP to a path.
Did some searching and found this: Decompress byte array to string via BinaryReader yields empty string. However, the result is simply bytes. I haven't a clue about how to translate this back into folders that can then be moved to a path. (Working with just bytes is confusing to me)
Doing some more searching on here pointed me to the .NET 4.5 feature:
https://learn.microsoft.com/en-us/dotnet/standard/io/how-to-compress-and-extract-files
There's one complication, I don't have a zip path, rather a array of bytes from the zip kept internally inside my application. Keeping this in mind, how would I go about using this ZipFile feature but instead with a array of bytes as a input?
Some other things I've looked at:
Compress a single file using C#
https://msdn.microsoft.com/en-us/library/system.io.compression.zipfile%28v=vs.110%29.aspx
How to extract zip file contents into a folder in .NET 4.5
Note, for this particular application, I'd like to refrain from using external DLL's. A portable CLI executable is what I'm aiming for.
In order to satisfy both the need that I have only bytes and unzip the bytes (without using MemoryBuffer as that still makes no sense to me), I ended up creating a temporary folder, creating a empty file in that folder, filling it with the bytes of the zipped file then using ZipFile.ExtractToDirectory() to extract it to the final destination.
It may not be the most efficient, but it works quite well.
I want to replace some data in a file, however I do not know exactly where this 200MB file would contain it. Is it possible to find (and replace them with something else) these values without loading a 200mb+ file into the memory?
Searching the file is not a problem. What you need is to work with the FileStream which is available via File.Open method. You can read through the file up to the bytes you need to replace.
Problem arises when you need to insert something. The FileStream allows you to overwrite some or all of the file contents from a particular byte forth and to append new content to its end but it does not allow you to insert data in the middle of the file. In order to overcome this problem you are going to need a temporary file. If you agree to that you could do the following:
Open the FileStream on the original file.
Create a temporary file that will hold the draft version.
Search through the original file and copy all "good" data into temporary file up to the point where modifications are to be made.
Insert modified and new data into the temporary file.
Finish up the temporary file with the remaining "good" content from the original file.
Replace the original file with the temporary one.
Delete the temporary file.
You could use the Path.GetTempFileName method for convenient way of utilizing a temporary file.
P.S. If you modify an exe then you probably make replacements on text constants and you neither need to insert new bytes nor to remove any. In such a case you do not need to bother with the temporary file and the FileStream is all you need.
P.P.S. Working with the FileStream you decide on size of a buffer you read from file and write back. Keep in mind that this size is the tradeoff between memory consumption, I/O performance and complexity of your code. Choose wisely. I would make it per-byte for the first time and try to optimize increasing the buffer to say 64k when it works. You can count on the FileStream to buffer data; it is not performing disk I/O each time you request another byte from it. If you dive into buffering yourself then try not to fragment the Large Object Heap. The threshold for .NET 4.5 is 85000 bytes.
Just a thought, how about reading your file line by line or may be in chunk of bytes and see in each chunk if u have the data that needs to be replaced. Also while reading make sure get the file pointer till where you have read the file so that when u find the match then u can go back to that location and over write those exact bytes which u have targetted.
I am currently having to stream multiple files to database and retrieve. The challenge is the number of files to stream is unknown, the file names are unknown and there is only one field in database to store all the files! The good thing is the location of the files to stream is fixed.
Eg.
Location to stream from : c:\Temp\FilesToStreamFolder
Current files (the number of files is unknown and names can differ too)
test.dat
background.jpeg
banner.gif
otherdata.dat
I would like to stream all these files to database (one data field) and retrieve the files back as an when required, with appropriate names?
Hope I have explained well. Any thoughts, ideas etc from you guys is greatly appreciated. Thank you.
I will suggest to convert a file to byte arrays first. Then record its Length and Name in datatable. Get byte array of another file and append it to the previous array and record its length and name in datatable. Continue the same procedure for all the files. Update the database with your byte array to store all files. Store your datatable as an XML file in system.
Whenever you need to get the files back, read the database field. Read your XML file. Split the array based on the File length from XML file. Use the corresponding name. You can optionally save the XML file data in database as well, if its possible. I hope you are looking for same thing.
Hope it helps.
Using the FileUpload control, please explain the difference between the following two methods of uploading file:
1. Using the FileUpload.SaveAs() method:
fileUploadControl.SaveAs(path)
2. Writing a byte array to disk from FileUpload.FileBytes using File.WriteAllBytes():
File.WriteAllBytes(path, fileUploadControl.FileBytes);
How would these compare when uploading large files?
These both have different purposes.
SaveAs lets you save as a file directly while WriteAllBytes gives you a byte array of contents.
Your file upload control will receive the bytes only after the file has been uploaded by the client, so there will be no difference in upload speeds.
A byte array is a value-type, so if you are passing copies of that around, note that it will create copies in memory whenever you pass it to functions.
I would use FileUpload.FileBytes when I want to access the bytes directly in memory and fileUploadControl.SaveAs whenever all I want to do is write the file to disk.
I'm writing an an application in C# that will record audio files (*.wav) and automatically tag and name them. Wave files are RIFF files (like AVI) which can contain meta data chunks in addition to the waveform data chunks. So now I'm trying to figure out how to read and write the RIFF meta data to and from recorded wave files.
I'm using NAudio for recording the files, and asked on their forums as well on SO for way to read and write RIFF tags. While I received a number of good answers, none of the solutions allowed for reading and writing RIFF chunks as easily as I would like.
But more importantly I have very little experience dealing with files at a byte level, and think this could be a good opportunity to learn. So now I want to try writing my own class(es) that can read in a RIFF file and allow meta data to be read, and written from the file.
I've used streams in C#, but always with the entire stream at once. So now I'm little lost that I have to consider a file byte by byte. Specifically how would I go about removing or inserting bytes to and from the middle of a file? I've tried reading a file through a FileStream into a byte array (byte[]) as shown in the code below.
System.IO.FileStream waveFileStream = System.IO.File.OpenRead(#"C:\sound.wav");
byte[] waveBytes = new byte[waveFileStream.Length];
waveFileStream.Read(waveBytes, 0, waveBytes.Length);
And I could see through the Visual Studio debugger that the first four byte are the RIFF header of the file.
But arrays are a pain to deal with when performing actions that change their size like inserting or removing values. So I was thinking I could then to the byte[] into a List like this.
List<byte> list = waveBytes.ToList<byte>();
Which would make any manipulation of the file byte by byte a whole lot easier, but I'm worried I might be missing something like a class in the System.IO name-space that would make all this even easier. Am I on the right track, or is there a better way to do this? I should also mention that I'm not hugely concerned with performance, and would prefer not to deal with pointers or unsafe code blocks like this guy.
If it helps at all here is a good article on the RIFF/WAV file format.
I did not write in C#, but can point on some places which are bad from my point of view:
1) Do not read whole WAV files in memory unless the files are your own files and knowingly have small size.
2) There is no need to insert a data in memory. You can simply for example do about the following: Analyze source file, store offsets of chunks, and read metadata in memory; present the metadata for editing in a dialog; while saving write RIFF-WAV header, fmt chunk, transfer audio data from source file (by reading and writing blocks), add metadata; update RIFF-WAV header.
3) Try save metadata in the tail of file. This will results in alternating only tag will not require re-writing of whole file.
It seems some sources regarding working with RIFF files in C# are present here.