We Keep Coding

Why do these two StreamWriter constructors give me different results?

Long story short I am trying to send a string via TcpClient using StreamWriter.
Without changing any other code except swapping out these samples. They produce different results.
In code sample 1 the StreamReader picks up that it has DataAvailable and the message is received.
In code sample 2 it does not have DataAvailable so no message is received. I need to keep my underlying stream open hence needing to use the constructor of StreamWrite in sample 2.
Sample 1 - Write Method
public void SendMessage(string message)
{
message = "TestMessage";
//WORKING - Sample 1
using (var sw = new StreamWriter(stream))
{
sw.Write(message);
sw.Flush();
}
}
Sample 2 - Write Method
public void SendMessage(string message)
{
message = "TestMessage";
//NOT WORKING - Sample 2
var encoding = new UTF8Encoding(false, true);
using (var sw = new StreamWriter(stream, encoding, 1024, true))
{
sw.Write(message);
sw.Flush();
}
}
Read Method
public string ReadMessage()
{
if (!stream.DataAvailable)
return null;
//I have also tried
//if(sr.Peek() == 0)
// return null;
string message = sr.ReadToEnd();
return message;
}
NOTE: If I put both samples together with the working one last I get the message received "TestMessageTestMessage" so it is definitely writing to stream however it is not setting DataAvailable to true?
Any Idea's why?

The problem is your ReadToEnd() command which blocks indefinitely on a NetworkStream which has no end until closed. I tested your code and I went past the DataAvailable query and blocked on the ReadToEnd() command.
Your method that uses a constructor that allows the BaseStream to stay open means that you never have an end to your stream. When the working method closes the stream the ReadMessage method returns with everything in the stream.
The solution: Do not attempt to read to the end. Read in blocks while the data is available or introduce a terminating character and read to that character.
From MSDN:
ReadToEnd assumes that the stream knows when it has reached an end. For interactive protocols in which the server sends data only when you ask for it and does not close the connection, ReadToEnd might block indefinitely because it does not reach an end, and should be avoided.

Reading String from Stream

I am encrypting data to a stream. If, for example, my data is of type Int32, I will use BitConverter.GetBytes(myInt) to get the bytes and then write those bytes to the stream.
To read the data back, I read sizeof(Int32) to determine the number of bytes to read, read those bytes, and then use BitConverter.ToInt32(byteArray, 0) to convert the bytes back to an Int32.
So how would I do this with a string? Writing the string is no problem. But the trick when reading the string is knowing how many bytes to read before I can then convert it back to a string.
I have found similar questions, but they seem to assume the string occupies the entire stream and just read to the end of the stream. But here, I can have any number of other items before and after the string.
Note that StringReader is not an option here since I want the option of handling file data that may be larger than I want to load into memory.

You would normally send a content length header, and then read the length determined by that information.
Here is some sample code:
public async Task ContinouslyReadFromStream(NetworkStream sourceStream, CancellationToken token)
{
while (!ct.IsCancellationRequested && sourceStream.CanRead)
{
while (sourceStream.CanRead && !sourceStream.DataAvailable)
{
// Avoid potential high CPU usage when doing stream.ReadAsync
// while waiting for data
Thread.Sleep(10);
}
var lengthOfMessage = BitConverter.ToInt32(await ReadExactBytesAsync(stream, 4, ct), 0);
var content = await ReadExactBytesAsync(stream, lengthOfMessage, ct);
// Assuming you use UTF8 encoding
var stringContent = Encoding.UTF8.GetString(content);
}
}
protected static async Task<byte[]> ReadExactBytesAsync(Stream stream, int count, CancellationToken ct)
{
var buffer = new byte[count];
var totalBytesRemaining = count;
var totalBytesRead = 0;
while (totalBytesRemaining != 0)
{
var bytesRead = await stream.ReadAsync(buffer, totalBytesRead, totalBytesRemaining, ct);
ct.ThrowIfCancellationRequested();
totalBytesRead += bytesRead;
totalBytesRemaining -= bytesRead;
}
return buffer;
}

The solutions that come to mind are to either provide a predetermined sentinel value to signal the end of the string (ASM uses a 0 byte for this, for example), or to provide a fixed-length block of metadata ahead of each new datatype. In that block of metadata would be the type and the length, plus whatever other information you found it useful to include.
For compactness I would use the sentinel value if it will work in your system.

Understanding the NetworkStream.EndRead()-example from MSDN

I tried to understand the MSDN example for NetworkStream.EndRead(). There are some parts that i do not understand.
So here is the example (copied from MSDN):
// Example of EndRead, DataAvailable and BeginRead.
public static void myReadCallBack(IAsyncResult ar ){
NetworkStream myNetworkStream = (NetworkStream)ar.AsyncState;
byte[] myReadBuffer = new byte[1024];
String myCompleteMessage = "";
int numberOfBytesRead;
numberOfBytesRead = myNetworkStream.EndRead(ar);
myCompleteMessage =
String.Concat(myCompleteMessage, Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));
// message received may be larger than buffer size so loop through until you have it all.
while(myNetworkStream.DataAvailable){
myNetworkStream.BeginRead(myReadBuffer, 0, myReadBuffer.Length,
new AsyncCallback(NetworkStream_ASync_Send_Receive.myReadCallBack),
myNetworkStream);
}
// Print out the received message to the console.
Console.WriteLine("You received the following message : " +
myCompleteMessage);
}
It uses BeginRead() and EndRead() to read asynchronously from the network stream.
The whole thing is invoked by calling
myNetworkStream.BeginRead(someBuffer, 0, someBuffer.Length, new AsyncCallback(NetworkStream_ASync_Send_Receive.myReadCallBack), myNetworkStream);
somewhere else (not displayed in the example).
What I think it should do is print the whole message received from the NetworkStream in a single WriteLine (the one at the end of the example). Notice that the string is called myCompleteMessage.
Now when I look at the implementation some problems arise for my understanding.
First of all: The example allocates a new method-local buffer myReadBuffer. Then EndStream() is called which writes the received message into the buffer that BeginRead() was supplied. This is NOT the myReadBuffer that was just allocated. How should the network stream know of it? So in the next line numberOfBytesRead-bytes from the empty buffer are appended to myCompleteMessage. Which has the current value "". In the last line this message consisting of a lot of '\0's is printed with Console.WriteLine.
This doesn't make any sense to me.
The second thing I do not understand is the while-loop.
BeginRead is an asynchronous call. So no data is immediately read. So as I understand it, the while loop should run quite a while until some asynchronous call is actually executed and reads from the stream so that there is no data available any more. The documentation doesn't say that BeginRead immediately marks some part of the available data as being read, so I do not expect it to do so.
This example does not improve my understanding of those methods. Is this example wrong or is my understanding wrong (I expect the latter)? How does this example work?

I think the while loop around the BeginRead shouldn't be there. You don't want to execute the BeginRead more than ones before the EndRead is done. Also the buffer needs to be specified outside the BeginRead, because you may use more than one reads per packet/buffer.
There are some things you need to think about, like how long are my messages/blocks (fixed size). Shall I prefix it with a length. (variable size) <datalength><data><datalength><data>
Don't forget it is a Streaming connection, so multiple/partial messages/packets can be read in one read.
Pseudo example:
int bytesNeeded;
int bytesRead;
public void Start()
{
bytesNeeded = 40; // u need to know how much bytes you're needing
bytesRead = 0;
BeginReading();
}
public void BeginReading()
{
myNetworkStream.BeginRead(
someBuffer, bytesRead, bytesNeeded - bytesRead,
new AsyncCallback(EndReading),
myNetworkStream);
}
public void EndReading(IAsyncResult ar)
{
numberOfBytesRead = myNetworkStream.EndRead(ar);
if(numberOfBytesRead == 0)
{
// disconnected
return;
}
bytesRead += numberOfBytesRead;
if(bytesRead == bytesNeeded)
{
// Handle buffer
Start();
}
else
BeginReading();
}

Is Stream.Copy piped?

Suppose I am writing a tcp proxy code.
I am reading from the incoming stream and writing to the output stream.
I know that Stream.Copy uses a buffer, but my question is:
Does the Stream.Copy method writes to the output stream while fetching the next chunk from the input stream or it a loop like "read chunk from input, write chunk to ouput, read chunk from input, etc" ?

Here's the implementation of CopyTo in .NET 4.5:
private void InternalCopyTo(Stream destination, int bufferSize)
{
int num;
byte[] buffer = new byte[bufferSize];
while ((num = this.Read(buffer, 0, buffer.Length)) != 0)
{
destination.Write(buffer, 0, num);
}
}
So as you can see, it reads from the source, then writes to the destination. This could probably be improved ;)
EDIT: here's a possible implementation of a piped version:
public static void CopyToPiped(this Stream source, Stream destination, int bufferSize = 0x14000)
{
byte[] readBuffer = new byte[bufferSize];
byte[] writeBuffer = new byte[bufferSize];
int bytesRead = source.Read(readBuffer, 0, bufferSize);
while (bytesRead > 0)
{
Swap(ref readBuffer, ref writeBuffer);
var iar = destination.BeginWrite(writeBuffer, 0, bytesRead, null, null);
bytesRead = source.Read(readBuffer, 0, bufferSize);
destination.EndWrite(iar);
}
}
static void Swap<T>(ref T x, ref T y)
{
T tmp = x;
x = y;
y = tmp;
}
Basically, it reads a chunk synchronously, starts to copy it to the destination asynchronously, then read the next chunk and waits for the write to complete.
I ran a few performance tests:
using MemoryStreams, I didn't expect a significant improvement, since it doesn't use IO completion ports (AFAIK); and indeed, the performance is almost identical
using files on different drives, I expected the piped version to perform better, but it doesn't... it's actually slightly slower (by 5 to 10%)
So it apparently doesn't bring any benefit, which is probably the reason why it isn't implemented this way...

According to Reflector it does not. Such behavior better be documented because it would introduce concurrency. This is never safe to do in general. So the API design to not "pipe" is sound.
So this is not just a question of Stream.Copy being more or less smart. Copying in a concurrent way is not an implementation detail.

Stream.Copy is synchronous operation. I don't think it is reasonable to expect it to use asynchronous read/write to make simultaneous read and write.
I would expect asynchrounous version (like RandomAccessStream.CopyAsync) to use simultaneous read and write.
Note: using multiple threads during copy would be unwelcome behavior, but using asynchronous read and write to run them at the same time is ok.

Writing to the output stream is impossible (when using one buffer) while fetching next chunk because fetching the next chunk can overwrite the buffer while its being used for output.
You can say use double buffering but its pretty much the same as using a double sized buffer.

networkstream always empty!

hey I'm writing on an Server-Client program
but when my client sends something, it never reaches my server!
I'm sending like this:
public void Send(string s)
{
char[] chars = s.ToCharArray();
byte[] bytes = chars.CharToByte();
nstream.Write(bytes, 0, bytes.Length);
nstream.Flush();
}
and Receiving in a background thread like this
void CheckIncoming(object dd)
{
RecievedDelegate d = (RecievedDelegate)dd;
try
{
while (true)
{
List<byte> bytelist = new List<byte>();
System.Threading.Thread.Sleep(1000);
int ssss;
ssss = nstream.ReadByte();
if (ssss > 1)
{
System.Diagnostics.Debugger.Break();
}
if (bytelist.Count != 0)
{
d.Invoke(bytelist.ToArray());
}
}
}
catch (Exception exp)
{
MSGBOX("ERROR:\n" + exp.Message);
}
}
the ssss int is never > 1
whats happening here???

NetworkStream.Flush() actually has no effect:
The Flush method implements the Stream..::.Flush method; however, because NetworkStream is not buffered, it has no affect [sic] on network streams. Calling the Flush method does not throw an exception
How much data is being sent?
If you don't send enough data it may remain buffered at the network level, until you close the stream or write more data.
See the TcpClient.NoDelay property for a way to disable this, if you are only going to be sending small chunks of data and require low latency.

You should change the check of the return value to if (ssss >= 0).
ReadByte returns a value greater or equal 0 if it succeeds to read a byte (source).
To elaborate on Marc's comment: How is nstream created? Maybe there is an underlying class that does not flush.

well, Im creating a TcpClient, and use GetStream(); to get the NetworkStream