I have a server that is going to transfer multiple files to a client over a single connection. The packet from server is in the following format:
unique_packet_id | file_content
I have onDataReceived function which I need to work like this:
public class TRACK_ID {
public string id;
public string unknown_identifier;
}
List<TRACK_ID> TRACKER = new List<TRACK_ID>();
public void OnDataReceived(IAsyncResult asyn)
{
try
{
log("OnDataReceived");
SocketPacket theSockId = (SocketPacket)asyn.AsyncState;
int iRx = theSockId.thisSocket.EndReceive(asyn);
// .. read the data into data_chunk
// if seperator found, that means we got an first chunk with the id
if (data_chunk.Contains("|") == true)
{
// extract unique_packet_id from the data
// bind unique_packet_id to an some kind of identifier? how!!??
TRACK_ID new_track = new TRACK_ID();
new_track.id = unique_packet_id;
new_track.unknown_identifier = X;
TRACKER.add(new_track);
} else {
// no seperator found - we're getting the rest of the data
// determinate the unique_packet_id of the incoming data so we can distinguish data/files
string current_packet_id = "";
for(int i=0; i<TRACKER.count; i++){
if(TRACKER[i].unknown_identifier == X){
current_packet_id = TRACKER[i].id; // we found our packet id!
break;
}
}
// now we got our packet_id and know where to store the buffer
}
WaitForData..
}
}
I need a variable X that will allow me to track where to store each incoming buffer
If I closed connection for each file, I could bind unique_packet_id to socket_id (socket_id would be X), but since I'm using the same connection, socket_id always stays the same so I have to use something else for this X variable.
Only other solution I can think of is sending the unique_packet_id in each chunk of data. But that seems like not the best way to do it. Then I would have to split the file buffer into chunks and append the id to each chunk. Any other ways how to accomplish this? Thanks!
You didn't say if you're using a stream socket or a datagram socket.
If you're using a datagram socket (e.g. you're using UDP/IP) then you will always receive a whole packet all at once, so you can identify the data because it goes with the unique_packet_id that was found before the | at the beginning of the current packet.
If you're using a stream socket (e.g. you're using TCP/IP) then I think you have a problem. Your packet format isn't delimited or escaped, so how will you know where one packet ends and the next one begins?
If you are using a stream socket, you need to use, for example, a packet format like this:
unique packet ID (say, in ASCII, terminated with CRNL — or whatever you choose)
content length (same format)
packet payload
The receiver can find the end of the packet because it knows how many bytes will be part of the payload.
You will also need to be prepared for the case where you get one of your packets in small pieces. For example, your callback function might be called once with part of the unique packet ID, called again with the rest of the header and part of the payload, and again with the rest of the payload and the complete following packet tacked on to the end. Or you may get three whole packets and part of a fourth in a single call to your callback function.
The other possible solution you mention, that of sending the unique_packet_id in each chunk of data, is not possible, because the sender doesn't know how the data will be chunked up when it is delivered to the receiver.
Related
I want to create a TCP listener for my .NET Core project. I'm using Kestrel and configured a new ConnectionHandler for this via
kestrelServerOptions.ListenLocalhost(5000, builder =>
{
builder.UseConnectionHandler<MyTCPConnectionHandler>();
});
So what I have so far is
internal class MyTCPConnectionHandler : ConnectionHandler
{
public override async Task OnConnectedAsync(ConnectionContext connection)
{
IDuplexPipe pipe = connection.Transport;
PipeReader pipeReader = pipe.Input;
while (true)
{
ReadResult readResult = await pipeReader.ReadAsync();
ReadOnlySequence<byte> readResultBuffer = readResult.Buffer;
foreach (ReadOnlyMemory<byte> segment in readResultBuffer)
{
// read the current message
string messageSegment = Encoding.UTF8.GetString(segment.Span);
// send back an echo
await pipe.Output.WriteAsync(segment);
}
if (readResult.IsCompleted)
{
break;
}
pipeReader.AdvanceTo(readResultBuffer.Start, readResultBuffer.End);
}
}
}
When sending messages from a TCP client to the server application the code works fine. The line await pipe.Output.WriteAsync(segment); is acting like an echo for now.
Some questions come up
What response should I send back to the client so that it does not run into a timeout?
When should I send back the response? When readResult.IsCompleted returns true?
How should I change the code to fetch the whole message sent by the client? Should I store each messageSegment in a List<string> and join it to a single string when readResult.IsCompleted returns true?
that is entirely protocol dependent; in many cases, you're fine to do nothing; in others, there will be specific "ping"/"pong" frames to send if you just want to say "I'm still here"
the "when" is entirely protocol dependent; waiting for readResult.IsCompleted means that you're waiting for the inbound socket to be marked as closed, which means you won't send anything until the client closes their outbound socket; for single-shot protocols, that might be fine; but in most cases, you'll want to look for a single inbound frame, and reply to that frame (and repeat)
it sounds like you might indeed be writing a one-shot channel, i.e. the client only sends one thing to the server, and after that: the server only sends one thing to the client; in that case, you do something like:
while (true)
{
var readResult = await pipeReader.ReadAsync();
if (readResult.IsCompleted)
{
// TODO: not shown; process readResult.Buffer
// tell the pipe that we consumed everything, and exit
pipeReader.AdvanceTo(readResultBuffer.End, readResultBuffer.End);
break;
}
else
{
// wait for the client to close their outbound; tell
// the pipe that we couldn't consume anything
pipeReader.AdvanceTo(readResultBuffer.Start, readResultBuffer.End);
}
As for:
Should I store each messageSegment in a List<string> and join it to a single string when
The first thing to consider here is that it is not necessarily the case that each buffer segment contains an exact number of characters. Since you are using UTF-8, which is a multi-byte encoding, a segment might contain fractions of characters at the start and end, so: decoding it is a bit more involved than that.
Because of this, it is common to check IsSingleSegment on the buffer; if this is true, you can just use simple code:
if (buffer.IsSingleSegment)
{
string message = Encoding.UTF8.GetString(s.FirstSpan);
DoSomethingWith(message);
}
else
{
// ... more complex
}
The discontiguous buffer case is much harder; basically, you have two choices here:
linearize the segments into a contiguous buffer, probably leasing an oversized buffer from ArrayPool<byte>.Shared, and use UTF8.GetString on the correct portion of the leased buffer
use the GetDecoder() API on the encoding, and use that to populate a new string, which on older frameworks means overwriting a newly allocated string, or in newer frameworks means using the string.Create API
Frankly, "1" is much simpler. For example (untested):
public static string GetString(in this ReadOnlySequence<byte> payload,
Encoding encoding = null)
{
encoding ??= Encoding.UTF8;
return payload.IsSingleSegment ? encoding.GetString(payload.FirstSpan)
: GetStringSlow(payload, encoding);
static string GetStringSlow(in ReadOnlySequence<byte> payload, Encoding encoding)
{
// linearize
int length = checked((int)payload.Length);
var oversized = ArrayPool<byte>.Shared.Rent(length);
try
{
payload.CopyTo(oversized);
return encoding.GetString(oversized, 0, length);
}
finally
{
ArrayPool<byte>.Shared.Return(oversized);
}
}
}
While setting up a TCP server-client connection, I realized that the server receive function hangs if the client does not send an '\n', but the client does not block if the sever doesn't. I tried searching for an explanation without finding a proper answer, so I came here to ask for your help.
I am using the same function to exchange data for both server and client, but I don't know why it works for one and doesn't for the other...
Here is my function in C#:
public bool sendToClient(int i, string msg)
{
try
{
(clientSockets.ElementAt(i)).mSocket.Send(Encoding.ASCII.GetBytes(msg));
}
catch(Exception e)
{
Console.WriteLine(e.Data.ToString());
return false;
}
return true;
}
private string getMessageFromConnection(Socket s)
{
byte[] buff;
string msg = "";
int k;
do
{
buff = new byte[100];
k = s.Receive(buff, 100, SocketFlags.None);
msg += Encoding.ASCII.GetString(buff, 0, k);
} while (k >= 100);
return msg;
}
The sockets are simple SOCK_STREAM ones, and the clientSockets is a list containing Client objects containing each client info including their socket.
I understand that one solution would be to detect a particular character to end the message, but I would like to know the reason behind it because I also had this issue using C.
Thanks in advance.
Your while loop continues only as long as you're reading exactly 100 bytes, and it seems that you intend to use that to detect the end of a message.
This will fail if the message is exactly 100 or any multitude of 100 bytes (in which case it will append a subsequent message to it).
But even worse - there is no guarantee that the socket will return 100 bytes, even if there is data still on its way. Receive does not wait until the underlying buffer has reached 100 bytes, it will return whatever it has available at that point.
You're going to have to either include a header that indicates the message length, or have a terminator character that indicates the end of the message.
I need to transfer a string over TCP connection. For this I serializable my object(over 10000 line list) in one stroke, without Intended. But large string won't transfer(As I understood due to buffer size). So MSDN, on this page (https://learn.microsoft.com/ru-ru/windows/uwp/networking/sockets) say me to use IBuffer for transfer my divided stroke. Here is a code:
// More efficient way to send packets.
// This way enables the system to do batched sends
IList<IBuffer> packetsToSend = PreparePackets();
var outputStream = stream.OutputStream;
int i = 0;
Task[] pendingTasks = new Tast[packetsToSend.Count];
foreach (IBuffer packet in packetsToSend)
{
pendingTasks[i++] = outputStream.WriteAsync(packet).AsTask();
}
// Now, wait for all of the pending writes to complete
await Task.WaitAll(pendingTasks);
What is the method PraparePackets()? How to prepare packets from my stroke?
Edit: I've found solution with DataReader and DataWriter, which has written in Albahari.(End of 16 chapter).
I've found solution with DataReader and DataWriter, which has written in Albahari.(End of 16 chapter).
Is sure that all data are read from a NetworkStream when DataAvailable is false?
Or does the sender of the data have to send the length of the data first.
And I have to read until I have read the number of bytes specified by the sender?
Sampel:
private Byte[] ReadStream(NetworkStream ns)
{
var bl = new List<Byte>();
var receivedBytes = new Byte[128];
while (ns.DataAvailable)
{
var bytesRead = ns.Read(receivedBytes, 0, receivedBytes.Length);
if (bytesRead == receivedBytes.Length)
bl.AddRange(receivedBytes);
else
bl.AddRange(receivedBytes.Take(bytesRead));
}
return bl.ToArray();
}
DataAvailable just tells you what is buffered and available locally. It means exactly nothing in terms of what is likely to arrive. The most common use of DataAvailable is to decide between a sync read and an async read.
If you are expecting the inbound stream to close after the send, then you can just keep using Read until a non-positive result is achieved, which tells you it has reached the end. If they are sending multiple frames, or just aren't closing - then yes: you'll need some way of detecting the end of a frame (=logical message). That can be via a length-prefix and counting, but it can also be via sentinel values. For example, in text-based protocols, \n or \r are often interpreted as "end of message".
So: it depends entirely on your protocol.
The easiest way would be to have a start/end character, so the message would be:
string message = "Hello";
string messageToSend = (char)2 + message + (char)3;
I've a tcp based client-server application. I'm able to send and receive strings, but don't know how to send an array of bytes instead.
I'm using the following function to send a string from the client to the server:
static void Send(string msg)
{
try
{
StreamWriter writer = new StreamWriter(client.GetStream());
writer.WriteLine(msg);
writer.Flush();
}
catch
{
}
}
Communication example
Client sends a string:
Send("CONNECTED| 84.56.32.14")
Server receives a string:
void clientConnection_ReceivedEvent(Connection client, String Message)
{
string[] cut = Message.Split('|');
switch (cut[0])
{
case "CONNECTED":
Invoke(new _AddClient(AddClient), client, null);
break;
case "STATUS":
Invoke(new _Status(Status), client, cut[1]);
break;
}
}
I need some help to modify the functions above in order to send and receive an array of bytes in addition to strings. I want to make a call like this:
Send("CONNECTED | 15.21.21.32", myByteArray);
Just use Stream - no need for a writer here. Basic sending is simple:
stream.Write(data, 0, data.Length);
However you probably need to think about "framing", i.e. how it knows where each sun-message starts and ends. With strings this is often special characters (maybe new line) - but this is rarely possible in raw binary. A common appoach is to proceed the message with the number f bytes to follow, in a pre-defined way (maybe network-byte-order fixed 4 byte unsigned integer, for example).
Reading: again, use the Stream Read method, but understand that you always need t check the return value; just because you say "read at most 20 bytes" doesn't mean you get that many, even if more is coming - you could read 3,3,3,11 bytes for example (unlikely, but you see what I mean). For example, to read exactly 20 bytes:
var buffer = new byte[...];
int count = 20, read, offset = 0;
while(count > 0 && ((read = source.Read(buffer, offset, count)) > 0) {
offset += read;
count -= read;
}
if(count != 0) throw new EndOfStreamException();
Since you seem new to networking you might want to use WCF or another framework. I've just written an article about my own framework: http://blog.gauffin.org/2012/05/griffin-networking-a-somewhat-performant-networking-library-for-net
You need to use a header for your packets as Mark suggested, since TCP uses streams and not packets. i.e. there is not 1-1 relation between send and receive operations.
This is the same problems I'm having. I only code the client and the server accepts byte arrays as proper data. The messages start with an ASCII STX character preceded by a bunch of bytes of any values except the STX and ETX characters. The message ends with a ETX ASCII CHARACTER. In C I could do this in my sleep, but I'm learning C# on the job. I don't understand why you would send bunches of double byte unicodes when single byte ASCII codes work just as well. Wasting double the bandwidth.