We Keep Coding

Download a file from System.Io.Stream class C# asp.net [duplicate]

I have a StreamReader object that I initialized with a stream, now I want to save this stream to disk (the stream may be a .gif or .jpg or .pdf).
Existing Code:
StreamReader sr = new StreamReader(myOtherObject.InputStream);
I need to save this to disk (I have the filename).
In the future I may want to store this to SQL Server.
I have the encoding type also, which I will need if I store it to SQL Server, correct?

As highlighted by Tilendor in Jon Skeet's answer, streams have a CopyTo method since .NET 4.
var fileStream = File.Create("C:\\Path\\To\\File");
myOtherObject.InputStream.Seek(0, SeekOrigin.Begin);
myOtherObject.InputStream.CopyTo(fileStream);
fileStream.Close();
Or with the using syntax:
using (var fileStream = File.Create("C:\\Path\\To\\File"))
{
myOtherObject.InputStream.Seek(0, SeekOrigin.Begin);
myOtherObject.InputStream.CopyTo(fileStream);
}
You have to call Seek if you're not already at the beginning or you won't copy the entire stream.

You must not use StreamReader for binary files (like gifs or jpgs). StreamReader is for text data. You will almost certainly lose data if you use it for arbitrary binary data. (If you use Encoding.GetEncoding(28591) you will probably be okay, but what's the point?)
Why do you need to use a StreamReader at all? Why not just keep the binary data as binary data and write it back to disk (or SQL) as binary data?
EDIT: As this seems to be something people want to see... if you do just want to copy one stream to another (e.g. to a file) use something like this:
/// <summary>
/// Copies the contents of input to output. Doesn't close either stream.
/// </summary>
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[8 * 1024];
int len;
while ( (len = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, len);
}
}
To use it to dump a stream to a file, for example:
using (Stream file = File.Create(filename))
{
CopyStream(input, file);
}
Note that Stream.CopyTo was introduced in .NET 4, serving basically the same purpose.

public void CopyStream(Stream stream, string destPath)
{
using (var fileStream = new FileStream(destPath, FileMode.Create, FileAccess.Write))
{
stream.CopyTo(fileStream);
}
}

private void SaveFileStream(String path, Stream stream)
{
var fileStream = new FileStream(path, FileMode.Create, FileAccess.Write);
stream.CopyTo(fileStream);
fileStream.Dispose();
}

I don't get all of the answers using CopyTo, where maybe the systems using the app might not have been upgraded to .NET 4.0+. I know some would like to force people to upgrade, but compatibility is also nice, too.
Another thing, I don't get using a stream to copy from another stream in the first place. Why not just do:
byte[] bytes = myOtherObject.InputStream.ToArray();
Once you have the bytes, you can easily write them to a file:
public static void WriteFile(string fileName, byte[] bytes)
{
string path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
if (!path.EndsWith(#"\")) path += #"\";
if (File.Exists(Path.Combine(path, fileName)))
File.Delete(Path.Combine(path, fileName));
using (FileStream fs = new FileStream(Path.Combine(path, fileName), FileMode.CreateNew, FileAccess.Write))
{
fs.Write(bytes, 0, (int)bytes.Length);
//fs.Close();
}
}
This code works as I've tested it with a .jpg file, though I admit I have only used it with small files (less than 1 MB). One stream, no copying between streams, no encoding needed, just write the bytes! No need to over-complicate things with StreamReader if you already have a stream you can convert to bytes directly with .ToArray()!
Only potential downsides I can see in doing it this way is if there's a large file you have, having it as a stream and using .CopyTo() or equivalent allows FileStream to stream it instead of using a byte array and reading the bytes one by one. It might be slower doing it this way, as a result. But it shouldn't choke since the .Write() method of the FileStream handles writing the bytes, and it's only doing it one byte at a time, so it won't clog memory, except that you will have to have enough memory to hold the stream as a byte[] object. In my situation where I used this, getting an OracleBlob, I had to go to a byte[], it was small enough, and besides, there was no streaming available to me, anyway, so I just sent my bytes to my function, above.
Another option, using a stream, would be to use it with Jon Skeet's CopyStream function that was in another post - this just uses FileStream to take the input stream and create the file from it directly. It does not use File.Create, like he did (which initially seemed to be problematic for me, but later found it was likely just a VS bug...).
/// <summary>
/// Copies the contents of input to output. Doesn't close either stream.
/// </summary>
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[8 * 1024];
int len;
while ( (len = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, len);
}
}
public static void WriteFile(string fileName, Stream inputStream)
{
string path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
if (!path.EndsWith(#"\")) path += #"\";
if (File.Exists(Path.Combine(path, fileName)))
File.Delete(Path.Combine(path, fileName));
using (FileStream fs = new FileStream(Path.Combine(path, fileName), FileMode.CreateNew, FileAccess.Write)
{
CopyStream(inputStream, fs);
}
inputStream.Close();
inputStream.Flush();
}

Here's an example that uses proper usings and implementation of idisposable:
static void WriteToFile(string sourceFile, string destinationfile, bool append = true, int bufferSize = 4096)
{
using (var sourceFileStream = new FileStream(sourceFile, FileMode.OpenOrCreate))
{
using (var destinationFileStream = new FileStream(destinationfile, FileMode.OpenOrCreate))
{
while (sourceFileStream.Position < sourceFileStream.Length)
{
destinationFileStream.WriteByte((byte)sourceFileStream.ReadByte());
}
}
}
}
...and there's also this
public static void WriteToFile(Stream stream, string destinationFile, int bufferSize = 4096, FileMode mode = FileMode.OpenOrCreate, FileAccess access = FileAccess.ReadWrite, FileShare share = FileShare.ReadWrite)
{
using (var destinationFileStream = new FileStream(destinationFile, mode, access, share))
{
while (stream.Position < stream.Length)
{
destinationFileStream.WriteByte((byte)stream.ReadByte());
}
}
}
The key is understanding the proper use of using (which should be implemented on the instantiation of the object that implements idisposable as shown above), and having a good idea as to how the properties work for streams. Position is literally the index within the stream (which starts at 0) that is followed as each byte is read using the readbyte method. In this case I am essentially using it in place of a for loop variable and simply letting it follow through all the way up to the length which is LITERALLY the end of the entire stream (in bytes). Ignore in bytes because it is practically the same and you will have something simple and elegant like this that resolves everything cleanly.
Keep in mind, too, that the ReadByte method simply casts the byte to an int in the process and can simply be converted back.
I'm gonna add another implementation I recently wrote to create a dynamic buffer of sorts to ensure sequential data writes to prevent massive overload
private void StreamBuffer(Stream stream, int buffer)
{
using (var memoryStream = new MemoryStream())
{
stream.CopyTo(memoryStream);
var memoryBuffer = memoryStream.GetBuffer();
for (int i = 0; i < memoryBuffer.Length;)
{
var networkBuffer = new byte[buffer];
for (int j = 0; j < networkBuffer.Length && i < memoryBuffer.Length; j++)
{
networkBuffer[j] = memoryBuffer[i];
i++;
}
//Assuming destination file
destinationFileStream.Write(networkBuffer, 0, networkBuffer.Length);
}
}
}
The explanation is fairly simple: we know that we need to keep in mind the entire set of data we wish to write and also that we only want to write certain amounts, so we want the first loop with the last parameter empty (same as while). Next, we initialize a byte array buffer that is set to the size of what's passed, and with the second loop we compare j to the size of the buffer and the size of the original one, and if it's greater than the size of the original byte array, end the run.

Why not use a FileStream object?
public void SaveStreamToFile(string fileFullPath, Stream stream)
{
if (stream.Length == 0) return;
// Create a FileStream object to write a stream to a file
using (FileStream fileStream = System.IO.File.Create(fileFullPath, (int)stream.Length))
{
// Fill the bytes[] array with the stream data
byte[] bytesInStream = new byte[stream.Length];
stream.Read(bytesInStream, 0, (int)bytesInStream.Length);
// Use FileStream object to write to the specified file
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
}
}

//If you don't have .Net 4.0 :)
public void SaveStreamToFile(Stream stream, string filename)
{
using(Stream destination = File.Create(filename))
Write(stream, destination);
}
//Typically I implement this Write method as a Stream extension method.
//The framework handles buffering.
public void Write(Stream from, Stream to)
{
for(int a = from.ReadByte(); a != -1; a = from.ReadByte())
to.WriteByte( (byte) a );
}
/*
Note, StreamReader is an IEnumerable<Char> while Stream is an IEnumbable<byte>.
The distinction is significant such as in multiple byte character encodings
like Unicode used in .Net where Char is one or more bytes (byte[n]). Also, the
resulting translation from IEnumerable<byte> to IEnumerable<Char> can loose bytes
or insert them (for example, "\n" vs. "\r\n") depending on the StreamReader instance
CurrentEncoding.
*/

Another option is to get the stream to a byte[] and use File.WriteAllBytes. This should do:
using (var stream = new MemoryStream())
{
input.CopyTo(stream);
File.WriteAllBytes(file, stream.ToArray());
}
Wrapping it in an extension method gives it better naming:
public void WriteTo(this Stream input, string file)
{
//your fav write method:
using (var stream = File.Create(file))
{
input.CopyTo(stream);
}
//or
using (var stream = new MemoryStream())
{
input.CopyTo(stream);
File.WriteAllBytes(file, stream.ToArray());
}
//whatever that fits.
}

public void testdownload(stream input)
{
byte[] buffer = new byte[16345];
using (FileStream fs = new FileStream(this.FullLocalFilePath,
FileMode.Create, FileAccess.Write, FileShare.None))
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
fs.Write(buffer, 0, read);
}
}
}

Converting byte array to pdf throws error when opening the dcoument

I am developing a WCF service that downloads a pdf file from a internet portal converts it into byte array and sends it to the client. On client side i am converting this byte array to pdf by using WriteAllBytes method. But while opening the pdf document it displays "There is error while opening the documnet. The file might be damaged or corrupted"
WCF Code
//
FileInformation fileInfo = File.OpenBinaryDirect(clientContext, fileRef.ToString());
byte[] Bytes = new byte[Convert.ToInt32(fileSize)];
fileInfo.Stream.Read(Bytes, 0, Bytes.Length);
return Bytes;
Client code
byte[] recievedBytes = <call to wcf method returing byte array>;
File.WriteAllBytes(path, recievedBytes);

I strongly suspect this is the problem:
byte[] Bytes = new byte[Convert.ToInt32(fileSize)];
fileInfo.Stream.Read(Bytes, 0, Bytes.Length);
You're assuming a single call to Read will read everything. Instead, you should loop round until you have read everything. For example:
byte[] bytes = new byte[(int) fileSize];
int index = 0;
while (index < bytes.Length)
{
int bytesRead = fileInfo.Stream.Read(bytes, index, bytes.Length - index);
if (bytesRead == 0)
{
throw new IOException("Unable to read whole file");
}
index += bytesRead;
}
Alternatively:
MemoryStream output = new MemoryStream((int) fileSize];
fileInfo.Stream.CopyTo(output);
return output.ToArray();

System.OutOfMemory exception when trying to read large files

public static byte[] ReadMemoryMappedFile(string fileName)
{
long length = new FileInfo(fileName).Length;
using (var stream = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.ReadWrite))
{
using (var mmf = MemoryMappedFile.CreateFromFile(stream, null, length, MemoryMappedFileAccess.Read, null, HandleInheritability.Inheritable, false))
{
using (var viewStream = mmf.CreateViewStream(0, length, MemoryMappedFileAccess.Read))
{
using (BinaryReader binReader = new BinaryReader(viewStream))
{
var result = binReader.ReadBytes((int)length);
return result;
}
}
}
}
}
OpenFileDialog openfile = new OpenFileDialog();
openfile.Filter = "All Files (*.*)|*.*";
openfile.ShowDialog();
byte[] buff = ReadMemoryMappedFile(openfile.FileName);
texteditor.Text = BitConverter.ToString(buff).Replace("-"," "); <----A first chance exception of type 'System.OutOfMemoryException' occurred in mscorlib.dll
I get a System.OutOfMemory exception when trying to read large files.
I've read a lot for 4 weeks in all the web... and tried a lot!!! But still, I can't seem to find a good solution to my problem.
Please help me..
Update
public byte[] FileToByteArray(string fileName)
{
byte[] buff = null;
FileStream fs = new FileStream(fileName,
FileMode.Open,
FileAccess.Read);
BinaryReader br = new BinaryReader(fs);
long numBytes = new FileInfo(fileName).Length;
buff = br.ReadBytes((int)numBytes);
//return buff;
return File.ReadAllBytes(fileName);
}
OR
public static byte[] FileToByteArray(FileStream stream, int initialLength)
{
// If we've been passed an unhelpful initial length, just
// use 32K.
if (initialLength < 1)
{
initialLength = 32768;
}
BinaryReader br = new BinaryReader(stream);
byte[] buffer = new byte[initialLength];
int read = 0;
int chunk;
while ((chunk = br.Read(buffer, read, buffer.Length - read)) > 0)
{
read += chunk;
// If we've reached the end of our buffer, check to see if there's
// any more information
if (read == buffer.Length)
{
int nextByte = br.ReadByte();
// End of stream? If so, we're done
if (nextByte == -1)
{
return buffer;
}
// Nope. Resize the buffer, put in the byte we've just
// read, and continue
byte[] newBuffer = new byte[buffer.Length * 2];
Array.Copy(buffer, newBuffer, buffer.Length);
newBuffer[read] = (byte)nextByte;
buffer = newBuffer;
read++;
}
}
// Buffer is now too big. Shrink it.
byte[] ret = new byte[read];
Array.Copy(buffer, ret, read);
return ret;
}
I still get a System.OutOfMemory exception when trying to read large files.

If your file is 4GB, then BitConverter will turn each byte into XX- string, each char in string is 2 bytes * 3 chars per byte * 4 294 967 295 bytes = 25 769 803 770. You need +25Gb of free memory to fit entire string, plus you already have your file in memory as byte array.
Besides, no single object in a .Net program may be over 2GB. Theoretical limit for a string length would be 1,073,741,823 chars, but you also need to have a 64-bit process.
So solution in your case - open FileStream. Read first 16384 bytes (or how much can fit on your screen), convert to hex and display, and remember file offset. When user wants to navigate to next or previous page - seek to that position in file on disk, read and display again, etc.

You need to read the file in chunks, keep track of where you are in the file, page the contents on screen and use seek and position to move up and down in the file stream.

You will not be able to display 4Gb file reading all of it in memory first by any approach.
The approach is to virtualize the data, reading only the visible lines when user scrolls. If you need to do a read-only text viewer then you can use WPF ItemsControl with virtulizing stack panel and bind to custom IList collection which will lazily fetch lines from the file calculating file offset by for the line index.

how to buffer an input stream until it is complete

i'm implementing a wcf service that accepts image streams. however i'm currently getting an exception when i run it. as its trying to get the length of the stream before the stream is complete. so what i'd like to do is buffer the stream until its complete. however i cant find any examples of how to do this...
can anyone help?
my code so far:
public String uploadUserImage(Stream stream)
{
Stream fs = stream;
BinaryReader br = new BinaryReader(fs);
Byte[] bytes = br.ReadBytes((Int32)fs.Length);// this causes exception
File.WriteAllBytes(filepath, bytes);
}

Rather than try to fetch the length, you should read from the stream until it returns that it's "done". In .NET 4, this is really easy:
// Assuming we *really* want to read it into memory first...
MemoryStream memoryStream = new MemoryStream();
stream.CopyTo(memoryStream);
memoryStream.Position = 0;
File.WriteAllBytes(filepath, memoryStream);
In .NET 3.5 there's no CopyTo method, but you can write something similar yourself:
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, bytesRead);
}
}
However, now we've got something to copy a stream, why bother reading it all into memory first? Let's just write it straight to a file:
using (FileStream output = File.OpenWrite(filepath))
{
CopyStream(stream, output); // Or stream.CopyTo(output);
}

I'm not sure what you are returning (or not returning), but something like this might work for you:
public String uploadUserImage(Stream stream) {
const int KB = 1024;
Byte[] bytes = new Byte[KB];
StringBuilder sb = new StringBuilder();
using (BinaryReader br = new BinaryReader(stream)) {
int len;
do {
len = br.Read(bytes, 0, KB);
string readData = Encoding.UTF8.GetString(bytes);
sb.Append(readData);
} while (len == KB);
}
//File.WriteAllBytes(filepath, bytes);
return sb.ToString();
}
A string can hold up to 2 GB, I believe.

Try this :
using (StreamWriter sw = File.CreateText(filepath))
{
stream.CopyTo(sw);
sw.Close();
}

Jon Skeets answer for .Net 3.5 and below using a Buffer Read is actually done incorrectly.
The buffer isn't cleared between reads which can result in issues on any read that returns less than 8192, for example if the 2nd read, read 192 bytes, the 8000 last bytes from the first read would STILL be in the buffer which would then be returned to the stream.
My code below you supply it a Stream and it will return a IEnumerable array.
Using this you can for-each it and Write to a MemoryStream and then use .GetBuffer() to end up with a compiled merged byte[].
private IEnumerable<byte[]> ReadFullStream(Stream stream) {
while(true) {
byte[] buffer = new byte[8192];//since this is created every loop, its buffer is cleared
int bytesRead = stream.Read(buffer, 0, buffer.Length);//read up to 8192 bytes into buffer
if (bytesRead == 0) {//if we read nothing, stream is finished
break;
}
if(bytesRead < buffer.Length) {//if we read LESS than 8192 bytes, resize the buffer to essentially remove everything after what was read, otherwise you will have nullbytes/0x00bytes at the end of your buffer
Array.Resize(ref buffer, bytesRead);
}
yield return buffer;//yield return the buffer data
}//loop here until we reach a read == 0 (end of stream)
}

Create a Stream without having a physical file to create from

I'm needing to create a zip file containing documents that exist on the server. I am using the .Net Package class to do so, and to create a new Package (which is the zip file) I have to have either a path to a physical file or a stream. I am trying to not create an actual file that would be the zip file, instead just create a stream that would exist in memory or something.
My question is how do you instantiate a new Stream (i.e. FileStream, MemoryStream, etc) without having a physical file to instantiate from.

MemoryStream has several constructor overloads, none of which require a file.

There is an example of how to do this on the MSDN page for MemoryStream:
using System;
using System.IO;
using System.Text;
class MemStream
{
static void Main()
{
int count;
byte[] byteArray;
char[] charArray;
UnicodeEncoding uniEncoding = new UnicodeEncoding();
// Create the data to write to the stream.
byte[] firstString = uniEncoding.GetBytes(
"Invalid file path characters are: ");
byte[] secondString = uniEncoding.GetBytes(
Path.GetInvalidPathChars());
using(MemoryStream memStream = new MemoryStream(100))
{
// Write the first string to the stream.
memStream.Write(firstString, 0 , firstString.Length);
// Write the second string to the stream, byte by byte.
count = 0;
while(count < secondString.Length)
{
memStream.WriteByte(secondString[count++]);
}
// Write the stream properties to the console.
Console.WriteLine(
"Capacity = {0}, Length = {1}, Position = {2}\n",
memStream.Capacity.ToString(),
memStream.Length.ToString(),
memStream.Position.ToString());
// Set the position to the beginning of the stream.
memStream.Seek(0, SeekOrigin.Begin);
// Read the first 20 bytes from the stream.
byteArray = new byte[memStream.Length];
count = memStream.Read(byteArray, 0, 20);
// Read the remaining bytes, byte by byte.
while(count < memStream.Length)
{
byteArray[count++] =
Convert.ToByte(memStream.ReadByte());
}
// Decode the byte array into a char array
// and write it to the console.
charArray = new char[uniEncoding.GetCharCount(
byteArray, 0, count)];
uniEncoding.GetDecoder().GetChars(
byteArray, 0, count, charArray, 0);
Console.WriteLine(charArray);
}
}
}
Is this what you are looking for?

You can create a new stream and write to it. You don't need a file to construct the object.
http://msdn.microsoft.com/en-us/library/system.io.memorystream.aspx
Write Method:
http://msdn.microsoft.com/en-us/library/system.io.memorystream.write.aspx
Constructors for Memory Stream:
http://msdn.microsoft.com/en-us/library/system.io.memorystream.memorystream.aspx