Is it possible to use a BinaryReader and BinaryWriter at the same time with the same underlying NetworkStream
using a single thread sequentially interlacing reads and writes?
using 1 thread for reading and 1 thread for writing?
(My goal is to simultaneously send and receive data via a TcpClient connection)
So far I've come across two related posts:
One references the NetworkStream docs:
Read and write operations can be performed simultaneously on an instance of the NetworkStream class without the need for synchronization. As long as there is one unique thread for the write operations and one unique thread for the read operations, there will be no cross-interference between read and write threads and no synchronization is required.
The second references the BinaryReader docs:
Using the underlying stream while reading or while using the BinaryReader can cause data loss and corruption. For example, the same bytes might be read more than once, bytes might be skipped, or character reading might become unpredictable.
I'm not 100% sure how to interpret these quotes and I'm not sure which of my 2 cases above are possible if any.
"yes" is the short answer; a NetworkStream essentially acts as a duplex stream, with the read operations completely separate to the write operations. A BinaryReader will only be using the read operations, and BinaryWriter the write operations. Concepts like Length, Position and Seek do not make sense on NetworkStream and are not supported. So: there should be no conflict here using BinaryReader / BinaryWriter. However, at the same time I would probably advise against using them - simply because they usually don't actually add much vs using raw Stream operations, and aren't compatible with arbitrary network protocols. If you're implementing something custom that just uses your code: you'll probably be fine, of course.
The warning about touching the Stream while using BinaryReader/BinaryWriter still very much applies to some other streams - FileStream for example: you could trivially corrupt the data by repositioning the stream while a reader/writer thinks it is in charge - and since there is a single position: this means that reads will impact writes, etc. But: this simply doesn't apply to NetworkStream. In this regard: NetworkStream is the exception, not the rule: in terms of what you usually expect from a Stream, NetworkStream is very unusual.
Related
i want to transfer data over sockets and currently i am creating a memory stream.
i can also use a network stream.
Can anyone please help me understand the difference between c# network stream and memory stream?
A NetworkStream is directly related to a socket; it does not know it's own length, you cannot seek, and the read/write functions are directly bound to the receive/send APIs (and therefore, read and write are entirely unrelated to eachother). It can timeout, and a read can take a considerable time if waiting for more data.
A MemoryStream is basically a wrapper over a local byte[]. It has a known length (which can change), you can seek, and read/write are directly related: both increment the same position cursor, and you can write something, rewind, and then read it. All operations are very timely.
It might be easier to ask "what are the similarities", which would be simply: both have a read/write API, by virtue of being subclasses of Stream.
both streams are derive of Stream, this classes are warper for different purpose
According to my understanding, Network Stream reads from the network interface, where if you use a Memory Stream (I mean, in the same scenario), all the data will be loaded to memory first (I assume it reads to the end of the actual stream), then the read operations will read from memory.
The first read operation to occur on the Memory Stream, all the data needs to be loaded in to memory.
Where network stream, you can read the data as they arrive.
I have a class that is wrapping a stream, with the intention of that stream likely being a NetworkStream. However in unit testing it is much easier to use a MemoryStream to verify functionality. However I have noticed that MemoryStream and NetworkStream don't actually act the same on write.
When I write to a MemoryStream the seek pointer of the stream is set to the end of what I wrote. In order to read out what I wrote I have to adjust the Seek pointer back to the beginning of what I wrote.
When I write to a NetworkStream the other end of the stream can read that data without adjusting the seek pointer.
I assume that a NetworkStream is handling the concept of the seek pointer internally and making sure that even though it is filling data into the stream on the other side it is not adjusting the pointer.
My question then is, what is the best way to mock that same behavior in the MemoryStream? I thought I might just seek the pointer back the same number of bytes I write but that seems cludgy. Also If I stack several of these streams together or other wrapped streams, how will they know whether the pointer needs reset or not?
MemoryStream isn't designed for how you are using it. A NetworkStream is designed to allow you to fill up a block of memory with your output. It is not designed to have a second object reading from what the original object is writing.
I would suggest that you create a FakeNetworkStream class. You would have two objects, writing to one would add data to the other. But the advantage of a FakeNetworkStream is that you can implement pathological socket behavior to detect as many bugs as possible.
Don't send any data until the user flushes the stream. Subtle bugs can arise when a socket is not flushed because where or not the data is actually sent can vary. By always waiting until the flush you can have your tests fail when a flush actually need to happen.
Read isn't guaranteed to produce all of the data you request. Simulate this by only ever producing a single byte at a time.
This might help.
https://github.com/billpg/POP3Listener/commit/a73a325f21a955edd9f1023a94c586aba29cfadc#diff-80d1609ed97fdaa9538dea7547301577cc19b6f4466a1a0a3a0ac08ad3eea23e
This function returns two NetworkStream objects, a client and a server, by briefly opening a server, connecting to that server, and returning both ends.
Question:
What is different between FileStream and StreamWriter in .Net?
What context are you supposed to use it? What is their advantage and disadvantage?
Is it possible to combine these two into one?
What is different between FileStream and StreamWriter in dotnet?
A FileStream is a Stream. Like all Streams it only deals with byte[] data.
A StreamWriter : TextWriter is a Stream-decorator. A TextWriter encodes the primitive type like string, int and char to byte[] and then writes hat to the linked Stream.
What context are you supposed to use it? What is their advantage and disadvantage?
You use a bare FileStream when you have byte[] data. You add a StreamWriter when you want to write text. Use a Formatter or a Serializer to write more complex data.
Is it possible to combine these two into one?
Yes. You always need a Stream to create a StreamWriter. The helper method System.IO.File.CreateText("path") will create them in combination and then you only have to Dispose() the outer writer.
FileStream writes bytes, StreamWriter writes text. That's all.
A FileStream is explicitly intended for working files.
A StreamWriter can be used to stream to any type of Stream - network sockets, files, etc.
ScottGu explains the different Stream objects quite nicely here: http://www.codeguru.com/Csharp/Csharp/cs_data/streaming/article.php/c4223
They are two different levels used in outputting information to known data sources.
A FileStream is a type of Stream, which is conceptually a mechanism that points to some location and can handle incoming and/or outgoing data to and from that location. Streams exist for reading/writing to files, network connections, memory, pipes, the console, debug and trace listeners, and a few other types of data sources. Specifically, a FileStream exists to perform reads and writes to the file system. Most streams are pretty low-level in their usage, and deal with data as bytes.
A StreamWriter is a wrapper for a Stream that simplifies using that stream to output plain text. It exposes methods that take strings instead of bytes, and performs the necessary conversions to and from byte arrays. There are other Writers; the other main one you'd use is the XmlTextWriter, which facilitates writing data in XML format. There are also Reader counterparts to the Writers that similarly wrap a Stream and facilitate getting the data back out.
Well, from the MSDN for FileStream:
Exposes a Stream around a file, supporting both synchronous and asynchronous read and write operations.
and the MSDN for StreamWriter:
Implements a TextWriter for writing characters to a stream in a particular encoding.
The most obvious difference is that FileStream allows read/write operations, while StreamWriter is write only.
The StreamWriter page goes on to add:
StreamWriter is designed for character output in a particular encoding, whereas classes derived from Stream are designed for byte input and output.
So a second difference is that FileStream is for bytes, while StreamWriter is for text.
One key difference (in addition to the above comments), could be that FileStream supports random disk access read and writes to any specified FileStream.Position. For large file modifications, that can be invaluable.
Why do we need Reader/Writer Streams while Using TCPListner in C# networking?
example :
TcpClient client = server.AcceptTcpClient();
client.ReceiveTimeout = 75000;
Stream s = client.GetStream();
StreamReader reader = new StreamReader(s);
StreamWriter writer = new StreamWriter(s);
writer.AutoFlush = true;
You only need StreamReader/StreamWriter if you want to deal with text instead of binary data.
Sockets deal with binary data naturally, which is why TcpClient exposes a Stream. Stream is a binary abstraction, hence its Read/Write methods deal in byte arrays. TextReader/TextWriter (from which StreamReader/StreamWriter are derived) deal with text - their methods deal with strings and char arrays.
StreamReader/StreamWriter basically wrap a stream and do the conversion (via an Encoding) between text and binary data for you.
The TcpListener is designed to use streams for a couple of reasons.
First, it makes the API consistent with other, mid-high level (compared to using raw sockets) APIs in the BCL. For example, the standard way of handling File IO is to use Streams.
Second, streams provide a huge advantage for reading and writing to TCP channels. True, when you are working on very simple cases, they make life slightly, but just slightly, more difficult. However, when you start looking at more complex scenarios, they make everything much simpler.
For example, if the Tcp API didn't use streams, it would be much more difficult to do things like compress traffic across the connection (just pass the writer+reader through a GZipStream or similar), or much more difficult to encrypt all of your communication across the wire. Streams provide huge amounts of opportunity to easily extend your algorithm to handle situations that may arise in the future.
You don't need them. You can use raw sockets if you don't like the Tcp wrapper classes. Take a look at System.Net.Sockets.
I receive the follow exception:
System.NotSupportedException : This stream does not support seek operations.
at System.Net.Sockets.NetworkStream.Seek(Int64 offset, SeekOrigin origin)
at System.IO.BufferedStream.FlushRead()
at System.IO.BufferedStream.WriteByte(Byte value)
The follow link show that this is a known problem for microsoft.
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=273186
This stacktrace show 2 things:
The System.IO.BufferedStream do some absurd pointer move operation. A BufferedStream should buffer the underlying stream and not more. The quality of buffer will be bad if there are such seek operation.
It will never work stable with a stream that not support Seek.
Are there any alternatives?
Does I need a buffer together with a NetworkStream in C# or is this already buffered.
Edit: I want simply reduce the number of read/write calls to the underlying socket stream.
The NetworkStream is already buffered. All data that is received is kept in a buffer waiting for you to read it. Calls to read will either be very fast, or will block waiting for data to be received from the other peer on the network, a BufferedStream will not help in either case.
If you are concerned about the blocking then you can look at switching the underlying socket to non-blocking mode.
The solution is to use two independent BufferedStreams, one for receiving and one for sending. And don't forget to flush the sending BufferedStream appropriately.
Since even in 2018 it seems hard to get a satisfying answer to this question, for the sake of humanity, here are my two cents:
The NetworkStream is buffered on the OS side. However, that does not mean there are no reasons to buffer on the .net side. TCP behaves well on Write-Read (repeat), but stalls on Write-Write-Read due to delayed ack, etc, etc.
If you, like me, have a bunch of sub-par protocol code to take into the twentyfirst century, you want to buffer.
Alternatively, if you stick to the above, you could also buffer only reads/rcvs or only writes/sends, and use the NetworkStream directly for the other side, depending on how broken what code is. You just have to be consistent!
What BufferedStream docs fail to make abundantly clear is that you should only switch reading and writing if your stream is seekable. This is because it buffers reads and writes in the same buffer. BufferedStream simply does not work well for NetworkStream.
As Marc pointed out, the cause of this lameness is the conflation of two streams into one NetworkStream which is not one of .net's greatest design decisions.
A BufferedStream simply acts to reduce the number of read/write calls to the underlying stream (which may be IO/hardware bound). It cannot provide seek capability (and indeed, buffering and seeking are in many ways contrary to eachother).
Why do you need to seek? Perhaps copy the stream to something seekable first - a MemoryStream or a FileStream - then do your actual work from that second, seekable stream.
Do you have a specific purpose in mind? I may be able to suggest more appropriate options with more details...
In particular: note that NetworkStream is a curiosity - with most streams, read/write relate to the same physical stream; however, a NetworkStream actually represents two completely independent pipes; read and write are completely unrelated. Likewise, you can't seek in bytes that have already zipped past you... you can skip data, but that is better done by doing a few Read opdrations and discarding the data.