I am very new in Socket Programming.
I am using the following code to receive incoming data from a pathology machine.
byte[] buffer = new byte[2048];
IPAddress ipAddress = IPAddress.Parse(SERVER_IP);
IPEndPoint localEndpoint = new IPEndPoint(ipAddress, PORT_NO);
Socket sock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
try
{
sock.Connect(localEndpoint);
}
catch (Exception ex)
{
throw ex;
}
int recv = 0;
string Printed = string.Empty;
StringBuilder sb = new StringBuilder();
while ((recv = sock.Receive(buffer)) > 0)
{
if (sock.Receive(buffer).ToString().Length > 1) // I used this line because it's receiving some garbage value all the time.
{
sb.Append(Encoding.UTF8.GetString(buffer));
}
else
{
if (sb.Length > 50 && Printed == string.Empty)
{
Console.WriteLine(sb);
Printed = "Y";
}
}
}
Issues I am facing
My program is not receiving complete data. Maybe because of this line if (sock.Receive(buffer).ToString().Length > 1). But I used this line because it's always receiving something.
My program goes to endless loop. I am looking for the program which should stop for sometime after receiving the data and start listening again for new incoming data.
There's a few things here;
you need to store the read count, and use only that many bytes, i.e. var bytes = sock.Receive(buffer); (and use bytes for both the EOF test, and for how many bytes to process)
we can't use ToString().Length > 1 here, because it is an integer and every integer, as a string, has a non-zero length; instead, simply: if (bytes > 0) (minutiae: there is a scenario where an open socket can return zero without meaning EOF, but... it doesn't apply here)
even for a text protocol, you can't necessarily simply use Encoding.UTF8.GetString(buffer, 0, bytes), because UTF8 is a multi-byte encoding, meaning: you might have partial characters; additionally, you don't yet know whether that is one message, half a message, or 14 and a bit messages; you need to read about the protocol's "framing" - which might simply mean "buffer bytes until you see a newline ('\n') character, decode those buffered bytes via the encoding, process that message, and repeat"
Related
Here's the strangest problem I've ever seen. Consider this code I use to receive consecutive UDP datagrams (the sequence always ends with a datagram containing Base64-encoded message FEND):
var received = new List<string>(10);
while (received.LastOrDefault() != "FEND")
{
byte[] buffer = new byte[65535]; // 65535 is the maximum size of a UDP datagram
client.Receive(buffer);
string strBuffer = Convert.ToBase64String(buffer);
strBuffer = strBuffer.TrimEnd('A'); // The buffer ends with zeros that are translated to "A"
received.Add(strBuffer);
Console.WriteLine();
}
received = received.Take(received.Count - 1).ToList();
client is a Socket field initialized with this code:
client = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
client.Bind(new IPEndPoint(IPAddress.Any, port));
port is specified by the client code (I'm writing a class library).
Here's the issue itself: the received list always contains only one item. I have added a Console.WriteLine() call into the end of while cycle and put there a breakpoint. When it hits, strBuffer contains a string of type hoYfopfHUTDJf8erK... (it's the expected value, and it is different in different iterations), and received contains a List<string> which has its Count property equal to 1 in all iterations! Its only item is the strBuffer value from the iteration I'm watching.
It seems like the Add call replaces the item in received instead of adding a new one. Is it an underlying bug or a stupid error?
UPDATE: in an online C# compiler my code works fine.
Are you really getting BASE64 text in UDP datagrams? Maybe you need Encoding.GetString() instead?
I think you could try something like this:
var received = new List<string>(10);
byte[] buffer = new byte[65535]; // 65535 is the maximum size of a UDP datagram
while (true)
{
var bytesReceived = client.Receive(buffer);
if (bytesReceived == 0)
break;
string strBuffer = System.Text.Encoding.UTF8.GetString(buffer, 0, bytesReceived);
if (strBuffer == "FEND")
break;
received.Add(strBuffer);
Console.WriteLine();
}
I have a Socket code which is communicating through TCP/IP.The machine to which i am communicating has buffer data in its buffer.At present i am trying to get the buffer data using this code.
byte data = new byte[1024];
int recv = sock.Receive(data);
stringData = Encoding.ASCII.GetString(data, 0, recv);
But this code retrieves only 11 lines of data whereas more data is there in the machines buffer.Is this because i have used int recv = sock.Receive(data); and data is 1024 ?
If yes ,How to get the total buffer size and retrieve it into string.
If you think you are missing some data, then you need to check recv and almost certainly: loop. Fortunately, ASCII is always single byte - in most other encodings you would also have to worry about receiving partial characters.
A common approach is basically:
int recv;
while((recv = sock.Receive(data)) > 0)
{
// process recv-many bytes
// ... stringData = Encoding.ASCII.GetString(data, 0, recv);
}
Keep in mind that there is no guarantee that stringData will be any particular entire unit of work; what you send is not always what you receive, and that could be a single character, 14 lines, or the second half of one word and the first half of another. You generally need to maintain your own back-buffer of received data until you have a complete logical frame to process.
Note, however, Receive always tries to return something (at least one byte), unless the inbound stream has closed - and will block to do so. If this is a problem, you may need to check the available buffer (sock.Available) to decide whether to do synchronous versus asynchronous receive (i.e. read synchronously while data is available, otherwise request an asynchronous read).
Try something along these lines:
StringBuilder sbContent=new StringBuilder();
byte data = new byte[1024];
int numBytes;
while ((numBytes = sock.Receive(data))>0)
{
sbContent.Append(Encoding.UTF8.GetString(data));
}
// use sbContent.ToString()
Socket tcpSocket = new Socket(ipe.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
Console.WriteLine(" ReceiveBufferSize {0}", tcpSocket.ReceiveBufferSize);
For actual data you can put below condition:-
int receiveBytes;
while((receiveBytes = tcpSocket.Receive.Data(receiveBytes)) > 0)
{
}
I am writing a telnet server using the async Begin/End methods. The issue that I am having is determining what within my buffer is actual data and what is not. Network coding is a bit new to me, but I've tried to research this and have not been able to find a answer.
public bool Start(IGame game)
{
// Get our server address information.
IPHostEntry serverHost = Dns.GetHostEntry(Dns.GetHostName());
IPEndPoint serverEndPoint = new IPEndPoint(IPAddress.Any, this.Port);
// Instance the server socket, bind it to a port and begin listening for connections.
this._ServerSocket = new Socket(serverEndPoint.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
this._ServerSocket.Bind(serverEndPoint);
this._ServerSocket.Listen(this.MaxQueuedConnections);
this._ServerSocket.BeginAccept(new AsyncCallback(Connect), this._ServerSocket);
return true;
}
private void Connect(IAsyncResult result)
{
var player = new BasePlayer();
try
{
player.Game = this.Game;
player.Connection = this._ServerSocket.EndAccept(result);
lock (this.Connections)
{
this.Connections.Add(player);
}
// Pass all of the data handling for the player to itself.
player.Connection.BeginReceive(player.Buffer, 0, player.BufferSize, SocketFlags.None, new AsyncCallback(player.ReceiveData), player);
// Fetch the next incoming connection.
this._ServerSocket.BeginAccept(new AsyncCallback(Connect), this._ServerSocket);
}
catch (Exception)
{
}
}
and then the player.ReceiveData..
public void ReceiveData(IAsyncResult result)
{
int bytesRead = this.Connection.EndReceive(result);
if (bytesRead > 0)
{
// TODO: Parse data received by the user.
//Queue the next receive data.
this.Connection.BeginReceive(this.Buffer, 0, this.BufferSize, SocketFlags.None, new AsyncCallback(ReceiveData), this);
var str = System.Text.Encoding.Default.GetString(Buffer);
}
else
{
this.Disconnect(result);
}
}
So when I call BeginReceive, I need to provide a buffer of a predetermined size. In doing that, I end up with unused bytes in my buffer array. They all have the value of 0, so I am assuming that I can loop through the array and build a new one starting at index 0 and working until I hit a value of 0.
I imagine there is a better way to do this? Can someone please point me in the right direction as to how I should determine what the data is within my buffer or perhaps a way that I can do this without having to use a predetermined buffer size.
So when call BeginReceive, I need to provide a buffer of a predetermined size. In doing that, I end up with unused bytes in my buffer array. They all have the value of 0, so I am assuming that I can loop through the array and build a new one starting at index 0 and working until I hit a value of 0.
No, that's not what you should do. Instead, in your callback (ReceiveData) you're already calling EndReceive - and the result of that is the number of bytes you read. That's how much of the buffer you should use.
However, you should copy the data you've read out of the buffer before you call BeginReceive again, otherwise you may end up with the next bit of data overwriting the just-read data before you get to use it.
So something like:
string text = Encoding.ASCII.GetString(Buffer, 0, bytesRead);
Connection.BeginReceive(this.Buffer, 0, this.BufferSize, SocketFlags.None,
new AsyncCallback(ReceiveData), this);
I would not suggest that you use Encoding.Default to convert the bytes to text - instead, you should decide which encoding you're using, and stick to that. If you use an encoding which isn't always one-byte-per-character, you'll end up in a slightly trickier situation, as then you might end up receiving a buffer with part of a character. At that point you need to keep a Decoder which can maintain state about partial characters read.
I am writing an app that will require to make hundreds of socket connections over tcp to read/write data.
I have come across this code snippet here and I'm wondering how I can make this more robust.
This is currently how I am calling the code:
foreach (var ip in listofIps)
{
IPEndPoint remoteEP = new IPEndPoint(IPAddress.Parse(ip), 4001);
Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
client.Connect(remoteEP);
await ReadAsync(client);
}
Is there anything wrong with the above, and how can it be optimized such that it runs concurrently?
In the code snippet, the buffer size is set to 1000. Just as a simple illustration, if I were to attempt to print out only the bytes received, and not the remaining 0x00s, I have to do something like this:
while (true)
{
await s.ReceiveAsync(awaitable);
int bytesRead = args.BytesTransferred;
if (bytesRead <= 0) break;
var hex = new StringBuilder(bytesRead * 2);
var msg = new byte[bytesRead];
for (int i = 0; i < bytesRead; i++)
msg[i] = args.Buffer[i];
foreach (byte b in msg)
hex.AppendFormat("{0:x2} ", b);
AppendLog(string.Format("RX: {0}", hex));
}
Is there a more efficient way of doing this? Previously, I would iterate the whole buffer and print out the data, but that will give me a whole bunch of trailing 0x00s as my protocol is anywhere between 60 to 70 bytes long.
I am writing an app that will require to make hundreds of socket connections over tcp to read/write data.
You don't need "high-performance sockets" for that. The code is far simpler with regular-performance sockets.
For starters, don't use the custom awaitables from the link you posted. They are perfectly fine for some people (and completely "robust"), but you don't need them and your code will be simpler without them.
Is there anything wrong with the above, and can it be further optimized?
Yes. You shouldn't mix blocking (Connect) and asynchronous (ReadAsync) code. I would recommend something like this:
foreach (var ip in listofIps)
{
IPEndPoint remoteEP = new IPEndPoint(IPAddress.Parse(ip), 4001);
Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
await client.ConnectTaskAsync(remoteEP);
...
}
Where ConnectTaskAsync is a standard TAP-over-APM wrapper:
public static Task ConnectTaskAsync(this Socket socket, EndPoint endpoint)
{
return TaskFactory.FromAsync(socket.BeginConnect, socket.EndConnect, endpoint, null);
}
As Marc Gravell pointed out, this code (and your original code) is connecting the sockets one at a time. You could use Task.WhenAll to connect them all simultaneously.
2) Is there a more efficient way of doing this?
First, you should define a TAP-over-APM ReceiveTaskAsync wrapper similar to the above. When dealing with binary data, I also like to have an extension method on byte arrays for dumping:
public string DumpHex(this ArraySegment<byte> data)
{
return string.Join(" ", data.Select(b => b.ToString("X2")));
}
Then you can have code like this:
while (true)
{
int bytesRead = await socket.ReceiveTaskAsync(buffer);
if (bytesRead == 0) break;
var data = new ArraySegment<byte>(buffer, 0, bytesRead);
AppendLog("RX: " + data.HexDump());
...
}
If you do a lot of binary manipulation, you may find my ArraySegments library helpful.
3) Where and how should I include the logic to check if my whole data has arrived within a single read
Oh, it's more complex than that. :) Sockets are a stream abstraction, not a message abstraction. So if you want to define "messages" in your protocol, you need to include a length prefix or delimiter byte so you can detect the message boundaries. Then you need to write code that will parse out your messages, keeping in mind that blocks of data read from the socket may contain only a partial message (so you have to buffer it), a complete message, multiple complete messages, and may also end with a partial message (again, buffering). And you have to also consider your existing buffer when receiving the new block.
I have a TCP/IP .NET Sockets FAQ on my blog that addresses this specifically and has some example code using my personal default preference for message framing (4-byte little-endian length prefixing).
4) How should I include a writeasync method such that I can send data through the socket in the middle of reads.
That one's surprisingly tricky:
public static Task<int> SendTaskAsync(this Socket socket, byte[] buffer, int offset, int size, SocketFlags flags)
{
return Task<int>.Factory.FromAsync(socket.BeginSend, socket.EndSend, buffer, offset, size, flags, null);
}
public static Task WriteAsync(this Socket socket, byte[] buffer)
{
int bytesSent = 0;
while (bytesSent != buffer.Length)
{
bytesSent += await socket.SendTaskAsync(data, bytesSent, buffer.Length - bytesSent, SocketFlags.None);
}
}
There is a code of async server. Client sends Header - size of Data Block + Data Block.
Server reads asynchronously first Header and then Data Block.
I need, after I read Data Block run the BeginRead for Header reading part, to make threads async.
PROBLEM:
When I got DataCallBack, in line:
int bytesRead = ns.EndRead(result);
I get not all buffer i asked to read in
mc.Client.GetStream().BeginRead(mc.DataBuffer, 0, size, new AsyncCallback(DataCallBack), mc);
If client send 1MB of Data I can get different number of "bytesRead".
QUESTION:
How to force "BeginRead" to read all data from connection. It should cause the new loop of Header - Data.
MyClient - simply wrapper over TcpClient;
CODE:
public void DoAcceptTcpClientCallback(IAsyncResult ar)
{
TcpListener listener = (TcpListener)ar.AsyncState;
TcpClient client = listener.EndAcceptTcpClient(ar);
client.NoDelay = false;
// client.ReceiveBufferSize = 1024*1024;
listener.BeginAcceptTcpClient(new AsyncCallback(DoAcceptTcpClientCallback), listener);
MyClient mc = new MyClient(client);
ContinueRead(0,mc);
}
public void ContinueRead(int size, MyClient mc)
{
if (size != 0)
{
mc.DataBuffer = new byte[size];
mc.Client.GetStream().BeginRead(mc.DataBuffer, 0, size, new AsyncCallback(DataCallBack), mc);
}
mc.Client.GetStream().BeginRead(mc.HeaderBuffer, 0, 4, new AsyncCallback(HeaderCallBack), mc);
}
private void HeaderCallBack(IAsyncResult result)
{
MyClient mc = (MyClient)result.AsyncState;
NetworkStream ns = mc.Stream;
int bytesRead = ns.EndRead(result);
if (bytesRead == 0)
throw new Exception();
mc.TotalLengs = BitConverter.ToInt32(mc.HeaderBuffer, 0);
ContinueRead(mc.TotalLengs, mc);
}
private void DataCallBack(IAsyncResult result)
{
MyClient mc = (MyClient)result.AsyncState;
NetworkStream ns = mc.Stream;
int bytesRead = ns.EndRead(result);
if (bytesRead == 0)
throw new Exception();
BAD CODE - MAKES ASYNC READING - SYNC
while (bytesRead < mc.TotalLengs)
{
bytesRead += ns.Read(mc.DataBuffer, bytesRead, mc.TotalLengs - bytesRead);
}
END BAD CODE
ContinueRead(0, mc);
ProcessPacket(mc.DataBuffer, mc.IP);
}
"If client send 1MB of Data I can get different number of "bytesRead"."
Yes...this is simply how TCP works under the hood. You can't change this. TCP guarantees the order of packets, not how they are grouped. The hardware and traffic conditions along the route the packets travel determine how that data is grouped (or un-grouped).
"How to force "BeginRead" to read all data from connection."
TCP has no idea how much data is being sent. As far as it is concerned, the connection is simply an endless stream of bytes; therefore it cannot read "all data" since there is no end to the data (from its perspective). TCP also has no notion of what a "complete message" is with respect to your application. It is up to you, the programmer, to develop a protocol that allows your application to know when all data has been sent.
If you are expecting a certain number of bytes, then keep a running sum of the values returned by EndRead() and stop when that magic number is hit.