We Keep Coding

Download a file from System.Io.Stream class C# asp.net [duplicate]

I have a StreamReader object that I initialized with a stream, now I want to save this stream to disk (the stream may be a .gif or .jpg or .pdf).
Existing Code:
StreamReader sr = new StreamReader(myOtherObject.InputStream);
I need to save this to disk (I have the filename).
In the future I may want to store this to SQL Server.
I have the encoding type also, which I will need if I store it to SQL Server, correct?

As highlighted by Tilendor in Jon Skeet's answer, streams have a CopyTo method since .NET 4.
var fileStream = File.Create("C:\\Path\\To\\File");
myOtherObject.InputStream.Seek(0, SeekOrigin.Begin);
myOtherObject.InputStream.CopyTo(fileStream);
fileStream.Close();
Or with the using syntax:
using (var fileStream = File.Create("C:\\Path\\To\\File"))
{
myOtherObject.InputStream.Seek(0, SeekOrigin.Begin);
myOtherObject.InputStream.CopyTo(fileStream);
}
You have to call Seek if you're not already at the beginning or you won't copy the entire stream.

You must not use StreamReader for binary files (like gifs or jpgs). StreamReader is for text data. You will almost certainly lose data if you use it for arbitrary binary data. (If you use Encoding.GetEncoding(28591) you will probably be okay, but what's the point?)
Why do you need to use a StreamReader at all? Why not just keep the binary data as binary data and write it back to disk (or SQL) as binary data?
EDIT: As this seems to be something people want to see... if you do just want to copy one stream to another (e.g. to a file) use something like this:
/// <summary>
/// Copies the contents of input to output. Doesn't close either stream.
/// </summary>
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[8 * 1024];
int len;
while ( (len = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, len);
}
}
To use it to dump a stream to a file, for example:
using (Stream file = File.Create(filename))
{
CopyStream(input, file);
}
Note that Stream.CopyTo was introduced in .NET 4, serving basically the same purpose.

public void CopyStream(Stream stream, string destPath)
{
using (var fileStream = new FileStream(destPath, FileMode.Create, FileAccess.Write))
{
stream.CopyTo(fileStream);
}
}

private void SaveFileStream(String path, Stream stream)
{
var fileStream = new FileStream(path, FileMode.Create, FileAccess.Write);
stream.CopyTo(fileStream);
fileStream.Dispose();
}

I don't get all of the answers using CopyTo, where maybe the systems using the app might not have been upgraded to .NET 4.0+. I know some would like to force people to upgrade, but compatibility is also nice, too.
Another thing, I don't get using a stream to copy from another stream in the first place. Why not just do:
byte[] bytes = myOtherObject.InputStream.ToArray();
Once you have the bytes, you can easily write them to a file:
public static void WriteFile(string fileName, byte[] bytes)
{
string path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
if (!path.EndsWith(#"\")) path += #"\";
if (File.Exists(Path.Combine(path, fileName)))
File.Delete(Path.Combine(path, fileName));
using (FileStream fs = new FileStream(Path.Combine(path, fileName), FileMode.CreateNew, FileAccess.Write))
{
fs.Write(bytes, 0, (int)bytes.Length);
//fs.Close();
}
}
This code works as I've tested it with a .jpg file, though I admit I have only used it with small files (less than 1 MB). One stream, no copying between streams, no encoding needed, just write the bytes! No need to over-complicate things with StreamReader if you already have a stream you can convert to bytes directly with .ToArray()!
Only potential downsides I can see in doing it this way is if there's a large file you have, having it as a stream and using .CopyTo() or equivalent allows FileStream to stream it instead of using a byte array and reading the bytes one by one. It might be slower doing it this way, as a result. But it shouldn't choke since the .Write() method of the FileStream handles writing the bytes, and it's only doing it one byte at a time, so it won't clog memory, except that you will have to have enough memory to hold the stream as a byte[] object. In my situation where I used this, getting an OracleBlob, I had to go to a byte[], it was small enough, and besides, there was no streaming available to me, anyway, so I just sent my bytes to my function, above.
Another option, using a stream, would be to use it with Jon Skeet's CopyStream function that was in another post - this just uses FileStream to take the input stream and create the file from it directly. It does not use File.Create, like he did (which initially seemed to be problematic for me, but later found it was likely just a VS bug...).
/// <summary>
/// Copies the contents of input to output. Doesn't close either stream.
/// </summary>
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[8 * 1024];
int len;
while ( (len = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, len);
}
}
public static void WriteFile(string fileName, Stream inputStream)
{
string path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
if (!path.EndsWith(#"\")) path += #"\";
if (File.Exists(Path.Combine(path, fileName)))
File.Delete(Path.Combine(path, fileName));
using (FileStream fs = new FileStream(Path.Combine(path, fileName), FileMode.CreateNew, FileAccess.Write)
{
CopyStream(inputStream, fs);
}
inputStream.Close();
inputStream.Flush();
}

Here's an example that uses proper usings and implementation of idisposable:
static void WriteToFile(string sourceFile, string destinationfile, bool append = true, int bufferSize = 4096)
{
using (var sourceFileStream = new FileStream(sourceFile, FileMode.OpenOrCreate))
{
using (var destinationFileStream = new FileStream(destinationfile, FileMode.OpenOrCreate))
{
while (sourceFileStream.Position < sourceFileStream.Length)
{
destinationFileStream.WriteByte((byte)sourceFileStream.ReadByte());
}
}
}
}
...and there's also this
public static void WriteToFile(Stream stream, string destinationFile, int bufferSize = 4096, FileMode mode = FileMode.OpenOrCreate, FileAccess access = FileAccess.ReadWrite, FileShare share = FileShare.ReadWrite)
{
using (var destinationFileStream = new FileStream(destinationFile, mode, access, share))
{
while (stream.Position < stream.Length)
{
destinationFileStream.WriteByte((byte)stream.ReadByte());
}
}
}
The key is understanding the proper use of using (which should be implemented on the instantiation of the object that implements idisposable as shown above), and having a good idea as to how the properties work for streams. Position is literally the index within the stream (which starts at 0) that is followed as each byte is read using the readbyte method. In this case I am essentially using it in place of a for loop variable and simply letting it follow through all the way up to the length which is LITERALLY the end of the entire stream (in bytes). Ignore in bytes because it is practically the same and you will have something simple and elegant like this that resolves everything cleanly.
Keep in mind, too, that the ReadByte method simply casts the byte to an int in the process and can simply be converted back.
I'm gonna add another implementation I recently wrote to create a dynamic buffer of sorts to ensure sequential data writes to prevent massive overload
private void StreamBuffer(Stream stream, int buffer)
{
using (var memoryStream = new MemoryStream())
{
stream.CopyTo(memoryStream);
var memoryBuffer = memoryStream.GetBuffer();
for (int i = 0; i < memoryBuffer.Length;)
{
var networkBuffer = new byte[buffer];
for (int j = 0; j < networkBuffer.Length && i < memoryBuffer.Length; j++)
{
networkBuffer[j] = memoryBuffer[i];
i++;
}
//Assuming destination file
destinationFileStream.Write(networkBuffer, 0, networkBuffer.Length);
}
}
}
The explanation is fairly simple: we know that we need to keep in mind the entire set of data we wish to write and also that we only want to write certain amounts, so we want the first loop with the last parameter empty (same as while). Next, we initialize a byte array buffer that is set to the size of what's passed, and with the second loop we compare j to the size of the buffer and the size of the original one, and if it's greater than the size of the original byte array, end the run.

Why not use a FileStream object?
public void SaveStreamToFile(string fileFullPath, Stream stream)
{
if (stream.Length == 0) return;
// Create a FileStream object to write a stream to a file
using (FileStream fileStream = System.IO.File.Create(fileFullPath, (int)stream.Length))
{
// Fill the bytes[] array with the stream data
byte[] bytesInStream = new byte[stream.Length];
stream.Read(bytesInStream, 0, (int)bytesInStream.Length);
// Use FileStream object to write to the specified file
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
}
}

//If you don't have .Net 4.0 :)
public void SaveStreamToFile(Stream stream, string filename)
{
using(Stream destination = File.Create(filename))
Write(stream, destination);
}
//Typically I implement this Write method as a Stream extension method.
//The framework handles buffering.
public void Write(Stream from, Stream to)
{
for(int a = from.ReadByte(); a != -1; a = from.ReadByte())
to.WriteByte( (byte) a );
}
/*
Note, StreamReader is an IEnumerable<Char> while Stream is an IEnumbable<byte>.
The distinction is significant such as in multiple byte character encodings
like Unicode used in .Net where Char is one or more bytes (byte[n]). Also, the
resulting translation from IEnumerable<byte> to IEnumerable<Char> can loose bytes
or insert them (for example, "\n" vs. "\r\n") depending on the StreamReader instance
CurrentEncoding.
*/

Another option is to get the stream to a byte[] and use File.WriteAllBytes. This should do:
using (var stream = new MemoryStream())
{
input.CopyTo(stream);
File.WriteAllBytes(file, stream.ToArray());
}
Wrapping it in an extension method gives it better naming:
public void WriteTo(this Stream input, string file)
{
//your fav write method:
using (var stream = File.Create(file))
{
input.CopyTo(stream);
}
//or
using (var stream = new MemoryStream())
{
input.CopyTo(stream);
File.WriteAllBytes(file, stream.ToArray());
}
//whatever that fits.
}

public void testdownload(stream input)
{
byte[] buffer = new byte[16345];
using (FileStream fs = new FileStream(this.FullLocalFilePath,
FileMode.Create, FileAccess.Write, FileShare.None))
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
fs.Write(buffer, 0, read);
}
}
}

uncompressed file is bigger than original file in GZIP

i'm using the following function to compress(thanks to http://www.dotnetperls.com/):
public static void CompressStringToFile(string fileName, string value)
{
// A.
// Write string to temporary file.
string temp = Path.GetTempFileName();
File.WriteAllText(temp, value);
// B.
// Read file into byte array buffer.
byte[] b;
using (FileStream f = new FileStream(temp, FileMode.Open))
{
b = new byte[f.Length];
f.Read(b, 0, (int)f.Length);
}
// C.
// Use GZipStream to write compressed bytes to target file.
using (FileStream f2 = new FileStream(fileName, FileMode.Create))
using (GZipStream gz = new GZipStream(f2, CompressionMode.Compress, false))
{
gz.Write(b, 0, b.Length);
}
}
and for decompress:
static byte[] Decompress(byte[] gzip)
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (GZipStream stream = new GZipStream(new MemoryStream(gzip), CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
so my goal is actually compress log files and than to decompress them in memory and compare the uncompressed file to the original file in order to check that the compression succeeded and i'm able to open the compressed file successfuly.
the problem is that the uncompressed file is most of the time bigger than the original file and my compare check is failing altough the compression probably succeeded.
any idea why ?
btw here how i compare the uncompressed file to the original file:
static bool FileEquals(byte[] file1, byte[] file2)
{
if (file1.Length == file2.Length)
{
for (int i = 0; i < file1.Length; i++)
{
if (file1[i] != file2[i])
{
return false;
}
}
return true;
}
return false;
}

Try this method to compress a file:
public static byte[] Compress(byte[] raw)
{
using (MemoryStream memory = new MemoryStream())
{
using (GZipStream gzip = new GZipStream(memory,
CompressionMode.Compress, true))
{
gzip.Write(raw, 0, raw.Length);
}
return memory.ToArray();
}
}
}
And this to decompress :
static byte[] Decompress(byte[] gzip)
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (GZipStream stream = new GZipStream(new MemoryStream(gzip), CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
}
Tell me if it worked.
Goodluck.

Think you'd be better off with the simplest API call, try Stream.CopyTo(). I can't find the error in your code. If I was working on it, I'd probably make sure everything is getting flushed properly.. can't recall if GZipStream is going to flush its output to FileStream when the using block closes.. but then you are also saying that the final file is larger, not smaller.
Anyhow, best policy in my experience.. don't rewrite gotcha prone code when you don't need to. At least you tested it ;)

how to buffer an input stream until it is complete

i'm implementing a wcf service that accepts image streams. however i'm currently getting an exception when i run it. as its trying to get the length of the stream before the stream is complete. so what i'd like to do is buffer the stream until its complete. however i cant find any examples of how to do this...
can anyone help?
my code so far:
public String uploadUserImage(Stream stream)
{
Stream fs = stream;
BinaryReader br = new BinaryReader(fs);
Byte[] bytes = br.ReadBytes((Int32)fs.Length);// this causes exception
File.WriteAllBytes(filepath, bytes);
}

Rather than try to fetch the length, you should read from the stream until it returns that it's "done". In .NET 4, this is really easy:
// Assuming we *really* want to read it into memory first...
MemoryStream memoryStream = new MemoryStream();
stream.CopyTo(memoryStream);
memoryStream.Position = 0;
File.WriteAllBytes(filepath, memoryStream);
In .NET 3.5 there's no CopyTo method, but you can write something similar yourself:
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, bytesRead);
}
}
However, now we've got something to copy a stream, why bother reading it all into memory first? Let's just write it straight to a file:
using (FileStream output = File.OpenWrite(filepath))
{
CopyStream(stream, output); // Or stream.CopyTo(output);
}

I'm not sure what you are returning (or not returning), but something like this might work for you:
public String uploadUserImage(Stream stream) {
const int KB = 1024;
Byte[] bytes = new Byte[KB];
StringBuilder sb = new StringBuilder();
using (BinaryReader br = new BinaryReader(stream)) {
int len;
do {
len = br.Read(bytes, 0, KB);
string readData = Encoding.UTF8.GetString(bytes);
sb.Append(readData);
} while (len == KB);
}
//File.WriteAllBytes(filepath, bytes);
return sb.ToString();
}
A string can hold up to 2 GB, I believe.

Try this :
using (StreamWriter sw = File.CreateText(filepath))
{
stream.CopyTo(sw);
sw.Close();
}

Jon Skeets answer for .Net 3.5 and below using a Buffer Read is actually done incorrectly.
The buffer isn't cleared between reads which can result in issues on any read that returns less than 8192, for example if the 2nd read, read 192 bytes, the 8000 last bytes from the first read would STILL be in the buffer which would then be returned to the stream.
My code below you supply it a Stream and it will return a IEnumerable array.
Using this you can for-each it and Write to a MemoryStream and then use .GetBuffer() to end up with a compiled merged byte[].
private IEnumerable<byte[]> ReadFullStream(Stream stream) {
while(true) {
byte[] buffer = new byte[8192];//since this is created every loop, its buffer is cleared
int bytesRead = stream.Read(buffer, 0, buffer.Length);//read up to 8192 bytes into buffer
if (bytesRead == 0) {//if we read nothing, stream is finished
break;
}
if(bytesRead < buffer.Length) {//if we read LESS than 8192 bytes, resize the buffer to essentially remove everything after what was read, otherwise you will have nullbytes/0x00bytes at the end of your buffer
Array.Resize(ref buffer, bytesRead);
}
yield return buffer;//yield return the buffer data
}//loop here until we reach a read == 0 (end of stream)
}

C# Trying to replace a byte while using MemoryStream class

I get a text file from a mainframe and sometimes there are some 0x0D injected into the middle of the text lines.
The previos programmer created a method using the FileStream class. This method works fine but is taking around 30 minutes to go thru the entire file.
My thought was to pass the text lines that are needed (about 25 lines) to a method to decrease the processing time.
I've been working with the MemoryStream class but am having issue where it does not find the 0x0D control code.
Here is the current FileStream method:
private void ReplaceFileStream(string strInputFile)
{
FileStream fileStream = new FileStream(strInputFile, FileMode.Open, FileAccess.ReadWrite);
byte filebyte;
while (fileStream.Position < fileStream.Length)
{
filebyte = (byte)fileStream.ReadByte();
if (filebyte == 0x0D)
{
filebyte = 0x20;
fileStream.Position = fileStream.Position - 1;
fileStream.WriteByte(filebyte);
}
}
fileStream.Close();
}
and here is the MemoryStream method:
private void ReplaceMemoryStream(string strInputLine)
{
byte[] byteArray = Encoding.ASCII.GetBytes(strInputLine);
MemoryStream fileStream = new MemoryStream(byteArray);
byte filebyte;
while (fileStream.Position < fileStream.Length)
{
filebyte = (byte)fileStream.ReadByte();
if (filebyte == 0x0D)
{
filebyte = 0x20;
fileStream.Position = fileStream.Position - 1;
fileStream.WriteByte(filebyte);
}
}
fileStream.Close();
}
As I have not used the MemoryStream class before am not that familar with it. Any tips or ideas?

I don't know the size of your files, but if they are small enough that you can load the whole thing in memory at once, then you could do something like this:
private void ReplaceFileStream(string strInputFile)
{
byte[] fileBytes = File.ReadAllBytes(strInputFile);
bool modified = false;
for(int i=0; i < fileBytes.Length; ++i)
{
if (fileByte[i] == 0x0D)
{
fileBytes[i] = 0x20;
modified = true;
}
}
if (modified)
{
File.WriteAllBytes(strInputFile, fileBytes);
}
}
If you can't read the whole file in at once, then you should switch to a buffered reading type of setup, here is an example that reads from the file, writes to a temp file, then in the end copies the temp file over the original file. This should yield better performance then reading a file one byte at a time:
private void ReplaceFileStream(string strInputFile)
{
string tempFile = Path.GetTempFileName();
try
{
using(FileStream input = new FileStream(strInputFile,
FileMode.Open, FileAccess.Read))
using(FileStream output = new FileStream(tempFile,
FileMode.Create, FileAccess.Write))
{
byte[] buffer = new byte[4096];
bytesRead = input.Read(buffer, 0, 4096);
while(bytesRead > 0)
{
for(int i=0; i < bytesRead; ++i)
{
if (buffer[i] == 0x0D)
{
buffer[i] = 0x20;
}
}
output.Write(buffer, 0, bytesRead);
bytesRead = input.Read(buffer, 0, 4096);
}
output.Flush();
}
File.Copy(tempFile, strInputFile);
}
finally
{
if (File.Exists(tempFile))
{
File.Delete(tempFile);
}
}
}

if your replacement code does not find the 0x0D in the stream and the previous method with the FileStream does it, I think it could be because of the Encoding you are using to get the bytes of the file, you can try with some other encoding types.
otherwise your code seems to be fine, I would use a using around the MemoryStream to be sure it gets closed and disposed, something like this:
using(var fileStream = new MemoryStream(byteArray))
{
byte filebyte;
// your while loop...
}
looking at your code I am not 100% sure the changes you make to the memory stream will be persisted; Actually I think that if you do not save it after the changes, your changes will be lost. I can be wrong in this but you should test and see, if it does not save you should use StreamWriter to save it after the changes.

compressing and decompressing source data gives result different than source data

In my app I need to Decompress data written by DataContractSerializer to compression Deflate Stream in another app, edit the decompressed data and Compress it again.
Decompression works fine, but not for data compressed by me.
The problem is that when I do this:
byte[] result = Compressor.Compress(Compressor.Decompress(sourceData));
the length of the result byte array is different than sourceData array.
For example:
string source = "test value";
byte[] oryg = Encoding.Default.GetBytes(source);
byte[] comp = Compressor.Compress(oryg);
byte[] result1 = Compressor.Decompress(comp);
string result2 = Encoding.Default.GetString(res);
and here result1.Length is 0 and result2 is "" of course
Here is the code of my Compressor class.
public static class Compressor
{
public static byte[] Decompress(byte[] data)
{
byte[] result;
using (MemoryStream baseStream = new MemoryStream(data))
{
using (DeflateStream stream = new DeflateStream(baseStream, CompressionMode.Decompress))
{
result = ReadFully(stream, -1);
}
}
return result;
}
public static byte[] Compress(byte[] data)
{
byte[] result;
using (MemoryStream baseStream = new MemoryStream())
{
using (DeflateStream stream = new DeflateStream(baseStream, CompressionMode.Compress, true))
{
stream.Write(data, 0, data.Length);
result = baseStream.ToArray();
}
}
return result;
}
/// <summary>
/// Reads data from a stream until the end is reached. The
/// data is returned as a byte array. An IOException is
/// thrown if any of the underlying IO calls fail.
/// </summary>
/// <param name="stream">The stream to read data from</param>
/// <param name="initialLength">The initial buffer length</param>
private static byte[] ReadFully(Stream stream, int initialLength)
{
// If we've been passed an unhelpful initial length, just
// use 32K.
if (initialLength < 1)
{
initialLength = 65768 / 2;
}
byte[] buffer = new byte[initialLength];
int read = 0;
int chunk;
while ((chunk = stream.Read(buffer, read, buffer.Length - read)) > 0)
{
read += chunk;
// If we've reached the end of our buffer, check to see if there's
// any more information
if (read == buffer.Length)
{
int nextByte = stream.ReadByte();
// End of stream? If so, we're done
if (nextByte == -1)
{
return buffer;
}
// Nope. Resize the buffer, put in the byte we've just
// read, and continue
byte[] newBuffer = new byte[buffer.Length * 2];
Array.Copy(buffer, newBuffer, buffer.Length);
newBuffer[read] = (byte)nextByte;
buffer = newBuffer;
read++;
}
}
// Buffer is now too big. Shrink it.
byte[] ret = new byte[read];
Array.Copy(buffer, ret, read);
return ret;
}
}
Please help me with this case if You can.
Best regards,
Adam

(edited: switched from using flush, which still might not flush out all bytes, to now ensuring deflate is disposed first, as per Phil's answer here: zip and unzip string with Deflate)
Before attempting to read from backing store, you have to ensure the deflate stream has fully flushed itself when compressing, allowing deflate to finish compressing and write final bytes. Closing the deflate steam, or disposing of it, will achieve this.
public static byte[] Compress(byte[] data)
{
byte[] result;
using (MemoryStream baseStream = new MemoryStream())
{
using (DeflateStream stream = new DeflateStream(baseStream, CompressionMode.Compress, true))
{
stream.Write(data, 0, data.Length);
}
result = baseStream.ToArray(); // only safe to read after deflate closed
}
return result;
}
Also your ReadFully routine looks incredibly complicated and likely to have bugs.
One being:
while ((chunk = stream.Read(buffer, read, buffer.Length - read)) > 0)
When reading the 2nd chunk, read will be greater than the length of the buffer, meaning it'll always pass a negative value to stream.Read for the number of bytes to read. My guess is that it'll never read the 2nd chunk, returning zero, and fall out of the while loop.
I recommend Jon's version of ReadFully for this purpose: Creating a byte array from a stream