I have a NetworkStream that I use to get data from another program. The data arrives as a Byte[64], which I then Enqueue to a ConcurrentQueue so that another thread can dequeue for analysis later.
The queue is instantiated:
ConcurrentQueue<byte[]> fifo = new ConcurrentQueue<byte[]>();
then I Enqueue all the data being sent:
Byte[] bytesIn = new Byte[64];
int i;
while ((i = stream.Read(bytesIn, 0, bytesIn.Length)) != 0)
{
fifo.Enqueue(bytesIn);
}
If I then look at (during debug) the data in fifo it turns out that every byte[64] contained therin is identical to the latest bytesIn. How do I ensure that the arrays I'm adding to fifo are the values and not the pointers (if that's the correct terminology)?
Enqueue a copy of the array instead. You can use the ToArray extension for this.
while ((i = stream.Read(bytesIn, 0, bytesIn.Length)) != 0)
{
var received = bytesIn.Take(i).ToArray();
fifo.Enqueue(received);
}
I also used Take to trim the buffer, and copy only the bytes that were received.
Alternatively, as suggested by #hvd in the comments, using Array.Copy will be faster
while ((i = stream.Read(bytesIn, 0, bytesIn.Length)) != 0)
{
var received = new byte[i];
Array.Copy(bytesIn, 0, received, 0, i);
fifo.Enqueue(received);
}
I think the main issue here is your misunderstanding of adding a reference type to the queue which you have declared outside your while-loop.
If you take a close look at the code you provided, you can see you only declare bytesIn once. You enqueue the bytesIn, and then rewrite the value of the array. The array is, however, still the same object as before and can thus not be queued again, hence it changes the array to the new value.
So what's it what we actually want to do? We want to;
Read the stream
Put the output in a new array-object
Enqueue the new object
Which is exactly what #dcastro does, but I'll strip down the code for you;
while ((
i = stream.Read(bytesIn, 0, bytesIn.Length)) != 0 //read the contents of the
//stream and put it in
//bytesIn, if available
)
{
var received = new byte[i]; //Create a new, empty array, which we are
//going to put in the queue.
Array.Copy(bytesIn, 0, received, 0, i); //Copy the contents of bytesIn into our new
//array. This way we can reuse bytesIn while
//maintaining the received data.
fifo.Enqueue(received); //Enqueue the new array and thus saving it.
}
For more information, perhaps you should read about Reference types.
Related
I have a Socket code which is communicating through TCP/IP.The machine to which i am communicating has buffer data in its buffer.At present i am trying to get the buffer data using this code.
byte data = new byte[1024];
int recv = sock.Receive(data);
stringData = Encoding.ASCII.GetString(data, 0, recv);
But this code retrieves only 11 lines of data whereas more data is there in the machines buffer.Is this because i have used int recv = sock.Receive(data); and data is 1024 ?
If yes ,How to get the total buffer size and retrieve it into string.
If you think you are missing some data, then you need to check recv and almost certainly: loop. Fortunately, ASCII is always single byte - in most other encodings you would also have to worry about receiving partial characters.
A common approach is basically:
int recv;
while((recv = sock.Receive(data)) > 0)
{
// process recv-many bytes
// ... stringData = Encoding.ASCII.GetString(data, 0, recv);
}
Keep in mind that there is no guarantee that stringData will be any particular entire unit of work; what you send is not always what you receive, and that could be a single character, 14 lines, or the second half of one word and the first half of another. You generally need to maintain your own back-buffer of received data until you have a complete logical frame to process.
Note, however, Receive always tries to return something (at least one byte), unless the inbound stream has closed - and will block to do so. If this is a problem, you may need to check the available buffer (sock.Available) to decide whether to do synchronous versus asynchronous receive (i.e. read synchronously while data is available, otherwise request an asynchronous read).
Try something along these lines:
StringBuilder sbContent=new StringBuilder();
byte data = new byte[1024];
int numBytes;
while ((numBytes = sock.Receive(data))>0)
{
sbContent.Append(Encoding.UTF8.GetString(data));
}
// use sbContent.ToString()
Socket tcpSocket = new Socket(ipe.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
Console.WriteLine(" ReceiveBufferSize {0}", tcpSocket.ReceiveBufferSize);
For actual data you can put below condition:-
int receiveBytes;
while((receiveBytes = tcpSocket.Receive.Data(receiveBytes)) > 0)
{
}
I am writing a telnet server using the async Begin/End methods. The issue that I am having is determining what within my buffer is actual data and what is not. Network coding is a bit new to me, but I've tried to research this and have not been able to find a answer.
public bool Start(IGame game)
{
// Get our server address information.
IPHostEntry serverHost = Dns.GetHostEntry(Dns.GetHostName());
IPEndPoint serverEndPoint = new IPEndPoint(IPAddress.Any, this.Port);
// Instance the server socket, bind it to a port and begin listening for connections.
this._ServerSocket = new Socket(serverEndPoint.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
this._ServerSocket.Bind(serverEndPoint);
this._ServerSocket.Listen(this.MaxQueuedConnections);
this._ServerSocket.BeginAccept(new AsyncCallback(Connect), this._ServerSocket);
return true;
}
private void Connect(IAsyncResult result)
{
var player = new BasePlayer();
try
{
player.Game = this.Game;
player.Connection = this._ServerSocket.EndAccept(result);
lock (this.Connections)
{
this.Connections.Add(player);
}
// Pass all of the data handling for the player to itself.
player.Connection.BeginReceive(player.Buffer, 0, player.BufferSize, SocketFlags.None, new AsyncCallback(player.ReceiveData), player);
// Fetch the next incoming connection.
this._ServerSocket.BeginAccept(new AsyncCallback(Connect), this._ServerSocket);
}
catch (Exception)
{
}
}
and then the player.ReceiveData..
public void ReceiveData(IAsyncResult result)
{
int bytesRead = this.Connection.EndReceive(result);
if (bytesRead > 0)
{
// TODO: Parse data received by the user.
//Queue the next receive data.
this.Connection.BeginReceive(this.Buffer, 0, this.BufferSize, SocketFlags.None, new AsyncCallback(ReceiveData), this);
var str = System.Text.Encoding.Default.GetString(Buffer);
}
else
{
this.Disconnect(result);
}
}
So when I call BeginReceive, I need to provide a buffer of a predetermined size. In doing that, I end up with unused bytes in my buffer array. They all have the value of 0, so I am assuming that I can loop through the array and build a new one starting at index 0 and working until I hit a value of 0.
I imagine there is a better way to do this? Can someone please point me in the right direction as to how I should determine what the data is within my buffer or perhaps a way that I can do this without having to use a predetermined buffer size.
So when call BeginReceive, I need to provide a buffer of a predetermined size. In doing that, I end up with unused bytes in my buffer array. They all have the value of 0, so I am assuming that I can loop through the array and build a new one starting at index 0 and working until I hit a value of 0.
No, that's not what you should do. Instead, in your callback (ReceiveData) you're already calling EndReceive - and the result of that is the number of bytes you read. That's how much of the buffer you should use.
However, you should copy the data you've read out of the buffer before you call BeginReceive again, otherwise you may end up with the next bit of data overwriting the just-read data before you get to use it.
So something like:
string text = Encoding.ASCII.GetString(Buffer, 0, bytesRead);
Connection.BeginReceive(this.Buffer, 0, this.BufferSize, SocketFlags.None,
new AsyncCallback(ReceiveData), this);
I would not suggest that you use Encoding.Default to convert the bytes to text - instead, you should decide which encoding you're using, and stick to that. If you use an encoding which isn't always one-byte-per-character, you'll end up in a slightly trickier situation, as then you might end up receiving a buffer with part of a character. At that point you need to keep a Decoder which can maintain state about partial characters read.
I am currently using System.Collections.Concurrent.BlockingCollection, and it´s very good with what it´s doing.
However it seems that it only keeps the reference of an object.
So if i have one byte[] object, which is written to and added to the Queue 100 times.
And after it reaches 100, i want to read all those, i will only get 100 copies of the current data "byte[]" holds.
Hope that explains it, at least it seems it´s doing this from my tests.
So if it´s doing this, is there another one that can keep copies of the data and just add it and add it till i read it?
Like for example, i would have 100 byte[] files, write it to a MemoryStream in the correct order, then i can read them in that order.
Though a Memory Stream isn´t what i would prefer to use, but works as an example.
Here is my code:
try
{
Thread.Sleep(100);
for (int i = Queue.Count; i <= Queue.Count; i++)
if (Queue.TryTake(out AudioData, 300))
{
if (Record)
waveWriter.Write(AudioData, 0, AudioData.Length);
}
}
catch (Exception e)
{
if (e is ArgumentNullException)
return;
}
Here is the part which receives the data
using (ms = new MemoryStream(TcpSize))
using (var tt1 = tcplisten.AcceptTcpClient())
{
ReceiveData = new byte[TcpSize];
tt1.NoDelay = true;
using (var tcpstream = tt1.GetStream())
while (connect)
{
if (Record)
Queue.Add(ReceiveData);
tcpstream.Read(ReceiveData, 0, TcpSize);
waveProvider.AddSamples(ReceiveData, 0, TcpSize);
}
}
You may wonder why i use a for loop and all that for writing, but it´s just there for debug purposes. I wanted to test if the objects in the Queue was copies, cause if so, it shouldn´t matter when i write it, but it does which means it must be reference.
Thanks
If you want to queue copies of the data, just make a copy and then queue the copy.
Queue.Add((byte[])ReceiveData.Clone());
But I think you also need to sort out the fact that you're writing the data to the queue before filling the buffer...
Alternatively, create a new buffer on each iteration and queue that instead:
while (connect)
{
ReceiveData = new byte[TcpSize];
tcpstream.Read(ReceiveData, 0, TcpSize);
waveProvider.AddSamples(ReceiveData, 0, TcpSize);
if (Record)
Queue.Add(ReceiveData);
}
I tried to understand the MSDN example for NetworkStream.EndRead(). There are some parts that i do not understand.
So here is the example (copied from MSDN):
// Example of EndRead, DataAvailable and BeginRead.
public static void myReadCallBack(IAsyncResult ar ){
NetworkStream myNetworkStream = (NetworkStream)ar.AsyncState;
byte[] myReadBuffer = new byte[1024];
String myCompleteMessage = "";
int numberOfBytesRead;
numberOfBytesRead = myNetworkStream.EndRead(ar);
myCompleteMessage =
String.Concat(myCompleteMessage, Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));
// message received may be larger than buffer size so loop through until you have it all.
while(myNetworkStream.DataAvailable){
myNetworkStream.BeginRead(myReadBuffer, 0, myReadBuffer.Length,
new AsyncCallback(NetworkStream_ASync_Send_Receive.myReadCallBack),
myNetworkStream);
}
// Print out the received message to the console.
Console.WriteLine("You received the following message : " +
myCompleteMessage);
}
It uses BeginRead() and EndRead() to read asynchronously from the network stream.
The whole thing is invoked by calling
myNetworkStream.BeginRead(someBuffer, 0, someBuffer.Length, new AsyncCallback(NetworkStream_ASync_Send_Receive.myReadCallBack), myNetworkStream);
somewhere else (not displayed in the example).
What I think it should do is print the whole message received from the NetworkStream in a single WriteLine (the one at the end of the example). Notice that the string is called myCompleteMessage.
Now when I look at the implementation some problems arise for my understanding.
First of all: The example allocates a new method-local buffer myReadBuffer. Then EndStream() is called which writes the received message into the buffer that BeginRead() was supplied. This is NOT the myReadBuffer that was just allocated. How should the network stream know of it? So in the next line numberOfBytesRead-bytes from the empty buffer are appended to myCompleteMessage. Which has the current value "". In the last line this message consisting of a lot of '\0's is printed with Console.WriteLine.
This doesn't make any sense to me.
The second thing I do not understand is the while-loop.
BeginRead is an asynchronous call. So no data is immediately read. So as I understand it, the while loop should run quite a while until some asynchronous call is actually executed and reads from the stream so that there is no data available any more. The documentation doesn't say that BeginRead immediately marks some part of the available data as being read, so I do not expect it to do so.
This example does not improve my understanding of those methods. Is this example wrong or is my understanding wrong (I expect the latter)? How does this example work?
I think the while loop around the BeginRead shouldn't be there. You don't want to execute the BeginRead more than ones before the EndRead is done. Also the buffer needs to be specified outside the BeginRead, because you may use more than one reads per packet/buffer.
There are some things you need to think about, like how long are my messages/blocks (fixed size). Shall I prefix it with a length. (variable size) <datalength><data><datalength><data>
Don't forget it is a Streaming connection, so multiple/partial messages/packets can be read in one read.
Pseudo example:
int bytesNeeded;
int bytesRead;
public void Start()
{
bytesNeeded = 40; // u need to know how much bytes you're needing
bytesRead = 0;
BeginReading();
}
public void BeginReading()
{
myNetworkStream.BeginRead(
someBuffer, bytesRead, bytesNeeded - bytesRead,
new AsyncCallback(EndReading),
myNetworkStream);
}
public void EndReading(IAsyncResult ar)
{
numberOfBytesRead = myNetworkStream.EndRead(ar);
if(numberOfBytesRead == 0)
{
// disconnected
return;
}
bytesRead += numberOfBytesRead;
if(bytesRead == bytesNeeded)
{
// Handle buffer
Start();
}
else
BeginReading();
}
I can use SetBuffer with SocketAsyncEventArgs just fine.
If I try to use BufferList (after doing SetBuffer(null, 0, 0)) I always and immediately get SocketError InvalidArgument (10022) when I do SendAsync on the socket.
There are NO examples or documentation on how to use BufferList and what I am doing makes sense (to me anyway).
Can someone point out an example program or code snippet?
I'm tearing my hair out over this and don't have much left ...
Here is basically what I am doing (e is SocketAsyncEventArgs and lSocket is the same socket I use for SetBuffer which works)
// null the buffer since we will use a buffer list
e.SetBuffer(null, 0, 0);
// create a bufferlist
e.BufferList = new List<ArraySegment<byte>>();
// create the bufferlist with the network header and the response bytes
e.BufferList.Add(new ArraySegment<byte>(lTxBytes)); // add the 4 character total length
e.BufferList.Add(new ArraySegment<byte>(Encoding.ASCII.GetBytes(lTx.Identity))); // echo back the incoming sequence number
e.BufferList.Add(new ArraySegment<byte>(Encoding.ASCII.GetBytes(lResponse)));
// *** the SendAsync always completes IMMEDIATELY (returns false) gets SocketError InvalidArgument (10022)
if (lSocket.SendAsync(e) == false)
{
// data was already sent back to the client.
AppSupport.WriteLog(LogLevel.Flow, "ProcessReceive had SendAsync complete synchronously (bytes transferred {0}).", e.BytesTransferred);
ProcessSend(e);
}
The reason you are getting an exception is that under the hood the SocketAsyncEventArgs only uses the buffers present in the list at the time of setting the BufferList property.
Basically you are trying to send en empty buffer with the code :
e.BufferList = new List<ArraySegment<byte>>();
e.BufferList.Add(new ArraySegment<byte>(lTxBytes));
e.BufferList.Add(new ArraySegment<byte>(Encoding.ASCII.GetBytes(lTx.Identity)));
e.BufferList.Add(new ArraySegment<byte>(Encoding.ASCII.GetBytes(lResponse)));
Instead try to do :
var list = new List<ArraySegment<byte>>();
list.Add(new ArraySegment<byte>(lTxBytes));
list.Add(new ArraySegment<byte>(Encoding.ASCII.GetBytes(lTx.Identity)));
list.Add(new ArraySegment<byte>(Encoding.ASCII.GetBytes(lResponse)));
e.BufferList = list;
This behavior is not well documented at all and can only be understood by looking at the BufferList setter code in detail. Behind the scenes the SocketAsyncEventArgs has a WSABuffer array field(for interop with native code) where it copies and pins the byte arrays references when you set the BufferList. Since it is this WSABuffer[] that is sent to native code, that explains why your code throws an exception.