I receive the follow exception:
System.NotSupportedException : This stream does not support seek operations.
at System.Net.Sockets.NetworkStream.Seek(Int64 offset, SeekOrigin origin)
at System.IO.BufferedStream.FlushRead()
at System.IO.BufferedStream.WriteByte(Byte value)
The follow link show that this is a known problem for microsoft.
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=273186
This stacktrace show 2 things:
The System.IO.BufferedStream do some absurd pointer move operation. A BufferedStream should buffer the underlying stream and not more. The quality of buffer will be bad if there are such seek operation.
It will never work stable with a stream that not support Seek.
Are there any alternatives?
Does I need a buffer together with a NetworkStream in C# or is this already buffered.
Edit: I want simply reduce the number of read/write calls to the underlying socket stream.
The NetworkStream is already buffered. All data that is received is kept in a buffer waiting for you to read it. Calls to read will either be very fast, or will block waiting for data to be received from the other peer on the network, a BufferedStream will not help in either case.
If you are concerned about the blocking then you can look at switching the underlying socket to non-blocking mode.
The solution is to use two independent BufferedStreams, one for receiving and one for sending. And don't forget to flush the sending BufferedStream appropriately.
Since even in 2018 it seems hard to get a satisfying answer to this question, for the sake of humanity, here are my two cents:
The NetworkStream is buffered on the OS side. However, that does not mean there are no reasons to buffer on the .net side. TCP behaves well on Write-Read (repeat), but stalls on Write-Write-Read due to delayed ack, etc, etc.
If you, like me, have a bunch of sub-par protocol code to take into the twentyfirst century, you want to buffer.
Alternatively, if you stick to the above, you could also buffer only reads/rcvs or only writes/sends, and use the NetworkStream directly for the other side, depending on how broken what code is. You just have to be consistent!
What BufferedStream docs fail to make abundantly clear is that you should only switch reading and writing if your stream is seekable. This is because it buffers reads and writes in the same buffer. BufferedStream simply does not work well for NetworkStream.
As Marc pointed out, the cause of this lameness is the conflation of two streams into one NetworkStream which is not one of .net's greatest design decisions.
A BufferedStream simply acts to reduce the number of read/write calls to the underlying stream (which may be IO/hardware bound). It cannot provide seek capability (and indeed, buffering and seeking are in many ways contrary to eachother).
Why do you need to seek? Perhaps copy the stream to something seekable first - a MemoryStream or a FileStream - then do your actual work from that second, seekable stream.
Do you have a specific purpose in mind? I may be able to suggest more appropriate options with more details...
In particular: note that NetworkStream is a curiosity - with most streams, read/write relate to the same physical stream; however, a NetworkStream actually represents two completely independent pipes; read and write are completely unrelated. Likewise, you can't seek in bytes that have already zipped past you... you can skip data, but that is better done by doing a few Read opdrations and discarding the data.
Related
Is it possible to use a BinaryReader and BinaryWriter at the same time with the same underlying NetworkStream
using a single thread sequentially interlacing reads and writes?
using 1 thread for reading and 1 thread for writing?
(My goal is to simultaneously send and receive data via a TcpClient connection)
So far I've come across two related posts:
One references the NetworkStream docs:
Read and write operations can be performed simultaneously on an instance of the NetworkStream class without the need for synchronization. As long as there is one unique thread for the write operations and one unique thread for the read operations, there will be no cross-interference between read and write threads and no synchronization is required.
The second references the BinaryReader docs:
Using the underlying stream while reading or while using the BinaryReader can cause data loss and corruption. For example, the same bytes might be read more than once, bytes might be skipped, or character reading might become unpredictable.
I'm not 100% sure how to interpret these quotes and I'm not sure which of my 2 cases above are possible if any.
"yes" is the short answer; a NetworkStream essentially acts as a duplex stream, with the read operations completely separate to the write operations. A BinaryReader will only be using the read operations, and BinaryWriter the write operations. Concepts like Length, Position and Seek do not make sense on NetworkStream and are not supported. So: there should be no conflict here using BinaryReader / BinaryWriter. However, at the same time I would probably advise against using them - simply because they usually don't actually add much vs using raw Stream operations, and aren't compatible with arbitrary network protocols. If you're implementing something custom that just uses your code: you'll probably be fine, of course.
The warning about touching the Stream while using BinaryReader/BinaryWriter still very much applies to some other streams - FileStream for example: you could trivially corrupt the data by repositioning the stream while a reader/writer thinks it is in charge - and since there is a single position: this means that reads will impact writes, etc. But: this simply doesn't apply to NetworkStream. In this regard: NetworkStream is the exception, not the rule: in terms of what you usually expect from a Stream, NetworkStream is very unusual.
i want to transfer data over sockets and currently i am creating a memory stream.
i can also use a network stream.
Can anyone please help me understand the difference between c# network stream and memory stream?
A NetworkStream is directly related to a socket; it does not know it's own length, you cannot seek, and the read/write functions are directly bound to the receive/send APIs (and therefore, read and write are entirely unrelated to eachother). It can timeout, and a read can take a considerable time if waiting for more data.
A MemoryStream is basically a wrapper over a local byte[]. It has a known length (which can change), you can seek, and read/write are directly related: both increment the same position cursor, and you can write something, rewind, and then read it. All operations are very timely.
It might be easier to ask "what are the similarities", which would be simply: both have a read/write API, by virtue of being subclasses of Stream.
both streams are derive of Stream, this classes are warper for different purpose
According to my understanding, Network Stream reads from the network interface, where if you use a Memory Stream (I mean, in the same scenario), all the data will be loaded to memory first (I assume it reads to the end of the actual stream), then the read operations will read from memory.
The first read operation to occur on the Memory Stream, all the data needs to be loaded in to memory.
Where network stream, you can read the data as they arrive.
In dealing with server/client connections, I have seen both of these used effectively without any apparent advantages to either, yet I doubt they would both exist if there weren't any known advantages to one or the other. Does anyone know any clear differences between the two? Help would be much appreciated, thanks.
Socket.Send is a raw send of data directly through the WINSOCK layer... Stream buffers and processes data as you send it. It's generally used for situations where you need stream-like functionality. There's some extra overhead there, obviously. In comparison to Send, this overhead includes creating a new class to manage a "stream" and introducing a couple of layers of indirection between you and Socket.Send. NetworkStream.Write eventually just calls Socket.Send; but that's a couple of layers of overhead down.
There's also a bit more work to get the stream set up, through the use of the NetworkStream type.
If course, a stream can also be bidirectional, meaning you can both read and write to it. You can read from a socket perfectly fine with the Receive methods.
Use of Send or Receive directly couples you to socket. If you use a Stream-derivative, you're decoupled from Socket and free to use methods/classes that work with Stream.
I have a class that is wrapping a stream, with the intention of that stream likely being a NetworkStream. However in unit testing it is much easier to use a MemoryStream to verify functionality. However I have noticed that MemoryStream and NetworkStream don't actually act the same on write.
When I write to a MemoryStream the seek pointer of the stream is set to the end of what I wrote. In order to read out what I wrote I have to adjust the Seek pointer back to the beginning of what I wrote.
When I write to a NetworkStream the other end of the stream can read that data without adjusting the seek pointer.
I assume that a NetworkStream is handling the concept of the seek pointer internally and making sure that even though it is filling data into the stream on the other side it is not adjusting the pointer.
My question then is, what is the best way to mock that same behavior in the MemoryStream? I thought I might just seek the pointer back the same number of bytes I write but that seems cludgy. Also If I stack several of these streams together or other wrapped streams, how will they know whether the pointer needs reset or not?
MemoryStream isn't designed for how you are using it. A NetworkStream is designed to allow you to fill up a block of memory with your output. It is not designed to have a second object reading from what the original object is writing.
I would suggest that you create a FakeNetworkStream class. You would have two objects, writing to one would add data to the other. But the advantage of a FakeNetworkStream is that you can implement pathological socket behavior to detect as many bugs as possible.
Don't send any data until the user flushes the stream. Subtle bugs can arise when a socket is not flushed because where or not the data is actually sent can vary. By always waiting until the flush you can have your tests fail when a flush actually need to happen.
Read isn't guaranteed to produce all of the data you request. Simulate this by only ever producing a single byte at a time.
This might help.
https://github.com/billpg/POP3Listener/commit/a73a325f21a955edd9f1023a94c586aba29cfadc#diff-80d1609ed97fdaa9538dea7547301577cc19b6f4466a1a0a3a0ac08ad3eea23e
This function returns two NetworkStream objects, a client and a server, by briefly opening a server, connecting to that server, and returning both ends.
I'm using C#.Net and the Socket class from the System.Net.Sockets namespace. I'm using the asynchronous receive methods. I understand this can be more easily done with something like a web service; this question is borne out of my curiosity rather than a practical need.
My question is: assume the client is sending some binary-serialized object of an unknown length. On my server with the socket, how do I know the entire object has been received and that it is ready for deserialization? I've considered prepending the object with the length of the object in bytes, but this seems unnecessary in the .Net world. What happens if the object is larger than the buffer? How would I know, 'hey, gotta resize the buffer because the object is too big'?
You either need the protocol to be self-terminating (like XML is, effectively - you know when you've finished receiving an XML document when it closes the root element) or you need to length-prefix the data, or you need the other end to close the stream when it's done.
In the case of a self-terminated protocol, you need to have enough hooks in so that the reading code can tell when it's finished. With binary serialization you may well not have enough hooks. Length-prefix is by far the easiest solution here.
If you use pure sockets, you need to know the length. Otherwise, the size of the buffer is not relevant, because even if you have a buffer of the size of the whole data, it still may not read all into it - check Stream.Read method, it returns the nr of bites actually read, so you need to loop until all data is received.
Yeah, you won't deserialize until you've rxed all the bytes.