We Keep Coding

Can't create MemoryStream

Is there any reason why this code shouldn't produce a memory stream with the word Slappy in it?
private MemoryStream StringBuilderToMemoryStream(StringBuilder source)
{
MemoryStream memoryStream = new MemoryStream();
StreamWriter streamWriter = new StreamWriter(memoryStream);
streamWriter.Write("slappy");
return memoryStream;
}
Even if I say streamWriter.Write(source.toString()); it fails.
Funny thing is, that it works on one of the methods that calls this routine but not on any of the others.
And the order I call them in makes no difference either.
But regardless, even when I call the above, from the method that works, the output is still an empty MemoryStream.
Any thoughts?

You don't flush the stream writer so the word never gets written to the memory stream.
Add the following after the call to streamWriter.Write:
streamWriter.Flush();
Furthermore, if you want to read that word later from the memory stream, make sure to reset its position, because after the Write it is located after the word slappy:
memoryStream.Position = 0;

If you don't want to call streamWriter.Flush(); you can set the AutoFlush-Property of the StreamWriter, at the moment you create it.
MemoryStream memoryStream = new MemoryStream();
StreamWriter streamWriter = new StreamWriter(memoryStream)
{
AutoFlush = true
}

Why is copying a stream and then deserializing using a BinaryFormatter faster than just deserializing

This code takes about 8 seconds with a stream containing about 65K coming from a blob in a database
private string[] GetArray(Stream stream)
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
object result = binaryFormatter.Deserialize(stream);
return (string[])result;
}
This code takes a few milliseconds:
private string[] GetArray(Stream stream)
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
MemoryStream memoryStream = new MemoryStream();
Copy(stream, memoryStream);
memoryStream.Position = 0;
object result = binaryFormatter.Deserialize(memoryStream);
return (string[])result;
}
Why?

So you say the problem disappears when the database is taken out of the equation. Here is my theory:
BinaryFormatter reads from the stream in tiny increments. It has to read as little as possible so that it does not accidentally swallow a few bytes after the serialized object. That means it is issuing tons of read commands (I verified this with Reflector).
Probably, every read of the blob stream is causing a network roundtrip (or some other major overhead). That gives you millions of roundtrips if using BinaryFormatter right away.
Buffering first causes the network to be utilized more efficiently because the read buffer size is much bigger.

Capture Stream Output to String

I'm calling a library method that writes to a stream. But I want to write to a string. Is this possible? (I do not control the source code of the method I'm calling and so changing that is not an option.)
Experimenting, I tried something like this:
iCalendarSerializer serializer = new iCalendarSerializer();
MemoryStream stream = new MemoryStream();
serializer.Serialize(new iCalendar(), stream, System.Text.Encoding.UTF8);
byte[] buff = new byte[stream.Length];
stream.Read(buff, 0, (int)stream.Length);
But I get an error on the last line that's something about not being able to access a closed stream. Apparently, the Serialize() method closes the stream when it's done.
Are there other options?

How about byte[] buff = stream.ToArray()?

ToArray is one of 2 correct way of getting the data out of memory stream (the other one is GetBuffer and Length). It looks like you just want byte array sized to data of the stream and ToArray does exactly that.
Note that it is by design safe to call these 3 methods on disposed stream, so you can safely wrap using(stream) around the code that write some data to the stream.
In you case stream look to be disposed by serialization code (.Serialize).
iCalendarSerializer serializer = new iCalendarSerializer();
MemoryStream stream = new MemoryStream();
using(stream)
{
serializer.Serialize(new iCalendar(), stream, System.Text.Encoding.UTF8);
}
byte[] buff = stream.ToArray();

In your example you need to change the position of the stream before read takes place:
stream.Position = 0;
stream.Read(buff, 0, (int)stream.Length);
In order to write stream to string you can use StreamReader.ReadToEnd() method:
var reader = new StreamReader(stream);
var text = reader.ReadToEnd();

unable to save dynamically created MemoryStream with rebex sftp

I'm using StreamWriter to generate a dynamic file and holding it in a MemoryStream. Everything appears to be alright until I go to save the file using rebex sftp.
The example they give on their site works fine:
// upload a text using a MemoryStream
string message = "Hello from Rebex FTP for .NET!";
byte[] data = System.Text.Encoding.Default.GetBytes(message);
System.IO.MemoryStream ms = new System.IO.MemoryStream(data);
client.PutFile(ms, "message.txt");
However the code below does not:
using (var stream = new MemoryStream())
{
using (var writer = new StreamWriter(stream))
{
writer.AutoFlush = true;
writer.Write("test");
}
client.PutFile(stream, "test.txt");
}
The file "test.txt" is saved, however it is empty. Do I need to do more than just enable AutoFlush for this to work?

After writing to the MemoryStream, the stream is positioned at the end. The PutFile method reads from the current position to the end. That's exactly 0 bytes.
You need to position the stream at the beginning before passing it to PutFile:
...
}
stream.Seek(0, SeekOrigin.Begin);
client.PutFile(stream, "test.txt");
You may also need to prevent the StreamWriter from disposing the MemoryStream:
var writer = new StreamWriter(stream);
writer.Write("test");
writer.Flush();
stream.Seek(0, SeekOrigin.Begin);
client.PutFile(stream, "test.txt");

Writing to then reading from a MemoryStream

I'm using DataContractJsonSerializer, which likes to output to a Stream. I want to top-and-tail the outputs of the serializer so I was using a StreamWriter to alternately write in the extra bits I needed.
var ser = new DataContractJsonSerializer(typeof (TValue));
using (var stream = new MemoryStream())
{
using (var sw = new StreamWriter(stream))
{
sw.Write("{");
foreach (var kvp in keysAndValues)
{
sw.Write("'{0}':", kvp.Key);
ser.WriteObject(stream, kvp.Value);
}
sw.Write("}");
}
using (var streamReader = new StreamReader(stream))
{
return streamReader.ReadToEnd();
}
}
When I do this I get an ArgumentException "Stream was not readable".
I'm probably doing all sorts wrong here so all answers welcome. Thanks.

Three things:
Don't close the StreamWriter. That will close the MemoryStream. You do need to flush the writer though.
Reset the position of the stream before reading.
If you're going to write directly to the stream, you need to flush the writer first.
So:
using (var stream = new MemoryStream())
{
var sw = new StreamWriter(stream);
sw.Write("{");
foreach (var kvp in keysAndValues)
{
sw.Write("'{0}':", kvp.Key);
sw.Flush();
ser.WriteObject(stream, kvp.Value);
}
sw.Write("}");
sw.Flush();
stream.Position = 0;
using (var streamReader = new StreamReader(stream))
{
return streamReader.ReadToEnd();
}
}
There's another simpler alternative though. All you're doing with the stream when reading is converting it into a string. You can do that more simply:
return Encoding.UTF8.GetString(stream.GetBuffer(), 0, (int) stream.Length);
Unfortunately MemoryStream.Length will throw if the stream has been closed, so you'd probably want to call the StreamWriter constructor that doesn't close the underlying stream, or just don't close the StreamWriter.
I'm concerned by you writing directly to the the stream - what is ser? Is it an XML serializer, or a binary one? If it's binary, your model is somewhat flawed - you shouldn't mix binary and text data without being very careful about it. If it's XML, you may find that you end up with byte-order marks in the middle of your string, which could be problematic.

setting the memory streams position to the beginning might help.
stream.Position = 0;
But the core problem is that the StreamWriter is closing your memory stream when it is closed.
Simply flushing that stream where you end the using block for it and only disposing of it fter you have read the data out of the memory stream will solve this for you.
You may also want to consider using a StringWriter instead...
using (var writer = new StringWriter())
{
using (var sw = new StreamWriter(stream))
{
sw.Write("{");
foreach (var kvp in keysAndValues)
{
sw.Write("'{0}':", kvp.Key);
ser.WriteObject(writer, kvp.Value);
}
sw.Write("}");
}
return writer.ToString();
}
This would require your serialization WriteObject call can accept a TextWriter instead of a Stream.

To access the content of a MemoryStream after it has been closed use the ToArray() or GetBuffer() methods. The following code demonstrates how to get the content of the memory buffer as a UTF8 encoded string.
byte[] buff = stream.ToArray();
return Encoding.UTF8.GetString(buff,0,buff.Length);
Note: ToArray() is simpler to use than GetBuffer() because ToArray() returns the exact length of the stream, rather than the buffer size (which might be larger than the stream content). ToArray() makes a copy of the bytes.
Note: GetBuffer() is more performant than ToArray(), as it doesn't make a copy of the bytes. You do need to take care about possible undefined trailing bytes at the end of the buffer by considering the stream length rather than the buffer size. Using GetBuffer() is strongly advised if stream size is larger than 80000 bytes because the ToArray copy would be allocated on the Large Object Heap where it's lifetime can become problematic.
It is also possible to clone the original MemoryStream as follows, to facilitate accessing it via a StreamReader e.g.
using (MemoryStream readStream = new MemoryStream(stream.ToArray()))
{
...
}
The ideal solution is to access the original MemoryStream before it has been closed, if possible.

Just a wild guess: maybe you need to flush the streamwriter? Possibly the system sees that there are writes "pending". By flushing you know for sure that the stream contains all written characters and is readable.