So I have written the code so that I can communicate with server and client.
The first question is how does the server identify that its communicating with an actual client, not someone else who's using the port, I've heard that browsers verify with servers using SHA hashing.
Second question is about the best way to send and receive data in variables, and also identifying which is which, because the current method of splitting data doesn't seem very elegant.
Server side code to receive and send data:
NetworkStream NetStream1 = TCPSocket.GetStream();
NetStream.Read(Buffer, 0, Buffer.Length);
ReceivedData = System.Text.Encoding.ASCII.GetString(Buffer);
string[] splitter = ReceivedData.Split('-');
Variable1 = splitter[0];
Variable2 = splitter[1];
//send response
SendBuffer = Encoding.ASCII.GetBytes(ResultINT1+"-"+ResultINT2);
NetStream.Write(SendBuffer, 0, SendBuffer.Length);
NetStream.Flush();
Client code to send and receive
NetworkStream SendStream = ClientSocket.GetStream();
byte[] SendBuffer = System.Text.Encoding.ASCII.GetBytes(V1+"-"+V2);
SendStream.Write(SendBuffer, 0, SendBuffer.Length);
SendStream.Flush();
//response
SendStream.Read(RecieveBuffer, 0, RecieveBuffer.Length);
string ResultString = System.Text.Encoding.ASCII.GetString(RecieveBuffer);
string[] splitted = ResultString.Split('-');
int R1 = Convert.ToInt32(splitted[0]);
int R2 = Convert.ToInt16(splitted[1]);
Provide some authentication mechanism
Use some serializer.
Your first question concerns authentication which is a huge subject and has many possible implementations although I'm not sure exactly what you mean by "someone else who's using the port". Your server should always be on the same port - that is how the client identifies a service.
Regarding your second question there are again many possibilities but I would suggest that the simplest for a beginner would be using XmlSerializer and a simple message envelope.
Create an XmlSerializable class either just using simple public properties or perhaps decorating with XmlElementAttribute, XmlRootAttribute etc.
Serialize to a MemoryStream
Write the bytes from the memory stream wrapped in an envelope (see later)
Receive a complete envelope into a byte array.
construct a MemoryStream from the byte array
Use XmlSerializer to reconstruct a copy of your original object.
The envelope is critical. The simplest one is just the binary length of the serialized object. Most protocols will typically extend that with CRC to handle possible corruption but since Ethernet uses a strong CRC and TCP is a reliable transport (albeit with a weak CRC) that is usually overkill. The key point that beginners miss is that TCP is a streaming protocol not a message based protocol thus it is perfectly possible for a sender to make a single write of say 1000 bytes and yet the receiver receives this as a number of smaller chunks. This is why you need some way to detect the end of a message such as using a length and why the receiver needs to accumulate received chunks until a complete message (and possibly part of the next) is received and can be deserialized.
This may seem complicated but unfortunately, at the TCP level, it doesn't get any simpler than that :(
The first question is how does the server identify that its communicating with an actual client, not someone else who's using the port, I've heard that browsers verify with servers using SHA hashing.
The server can identify different client by their IP addresses. See StreamReader.ReadToEnd
Second question is about the best way to send and receive data in variables, and also identifying which is which, because the current method of splitting data doesn't seem very elegant.
It depends on your protocol architecture, but a portable way to exchange values on network is to keep them in text format (this way no problem of endianness, type size...).
Said that, be careful of your variable separator : a '-' might be difficult to use with negative numbers, ' ' or ';' are more common.
You might want to define a communication protocol of some kind - a text-based protocol would be most straightforward to begin with - you can then read and write the "commands" each on a separate line.
First, there would be a "handshake", where the client would send something like "HELLO my-awesome-protocol-v1\n" and the server would respond similarly. This way you will be sure that the other person is a client, who understands the protocol or you can close a connection, which does not implement the protocol.
Then there could be some way of sending the values of variables with commands like "VAR variableName 123.45\n". You can read https://en.wikipedia.org/wiki/Text-based_protocol and see http://www.ncftp.com/libncftp/doc/ftp_overview.html for inspiration.
Related
If i send 1000 bytes in TCP, does it guarantee that the receiver will get the entire 1000 bytes "togther"? or perhaps he will first only get 500 bytes, and later he'll receive the other bytes?
EDIT: the question comes from the application's point of view. If the 1000 bytes are reassembles into a single buffer before they reach the application .. then i don't care if it was fragmented in the way..
See Transmission Control Protocol:
TCP provides reliable, ordered delivery of a stream of bytes from a program on one computer to another program on another computer.
A "stream" means that there is no message boundary from the receiver's point of view. You could get one 1000 byte message or one thousand 1 byte messages depending on what's underneath and how often you call read/select.
Edit: Let me clarify from the application's point of view. No, TCP will not guarantee that the single read would give you all of the 1000 bytes (or 1MB or 1GB) packet the sender may have sent. Thus, a protocol above the TCP usually contains fixed length header with the total content length in it. For example you could always send 1 byte that indicates the total length of the content in bytes, which would support up to 255 bytes.
As other answers indicated, TCP is a stream protocol -- every byte sent will be received (once and in the same order), but there are no intrinsic "message boundaries" -- whether all bytes are sent in a single .send call, or multiple ones, they might still be received in one or multiple .receive calls.
So, if you need "message boundaries", you need to impose them on top of the TCP stream, IOW, essentially, at application level. For example, if you know the bytes you're sending will never contain a \0, null-terminated strings work fine; various methods of "escaping" let you send strings of bytes which obey no such limitations. (There are existing protocols for this but none is really widespread or widely accepted).
Basically as far as TCP goes it only guarantees that the data sent from one end to the other end will be sent in the same order.
Now usually what you'll have to do is have an internal buffer that keeps looping until it has received your 1000 byte "packet".
Because the recv command as mentioned returns how much has actually been received.
So usually you'll have to then implement a protocol on top of TCP to make sure you send data at an appropriate speed. Because if you send() all the data in one run through it will overload the under lying networking stack, and which will cause complications.
So usually in the protocol there is a tiny acknowledgement packet sent back to confirm that the packet of 1000 bytes are sent.
You decide, in your message that how many bytes your message shall contain. For instance in your case its 1000. Following is up and running C# code to achieve the same. The method returns with 1000 bytes. The abort code is 0 bytes; you can tailor that according to your needs.
Usage:
strMsg = ReadData(thisTcpClient.Client, 1000, out bDisconnected);
Following is the method:
string ReadData(Socket sckClient, int nBytesToRead, out bool bShouldDisconnect)
{
bShouldDisconnect = false;
byte[] byteBuffer = new byte[nBytesToRead];
Array.Clear(byteBuffer, 0, byteBuffer.Length);
int nDataRead = 0;
int nStartIndex = 0;
while (nDataRead < nBytesToRead)
{
int nBytesRead = sckClient.Receive(byteBuffer, nStartIndex, nBytesToRead - nStartIndex, SocketFlags.None);
if (0 == nBytesRead)
{
bShouldDisconnect = true;
//0 bytes received; assuming disconnect signal
break;
}
nDataRead += nBytesRead;
nStartIndex += nBytesRead;
}
return Encoding.Default.GetString(byteBuffer, 0, nDataRead);
}
Let us know this didn't help you (0: Good luck.
Yes, there is a chance for receiving packets part by part. Hope this msdn article and following example (taken from the article in msdn for quick review) would be helpful to you if you are using windows sockets.
void CChatSocket::OnReceive(int nErrorCode)
{
CSocket::OnReceive(nErrorCode);
DWORD dwReceived;
if (IOCtl(FIONREAD, &dwReceived))
{
if (dwReceived >= dwExpected) // Process only if you have enough data
m_pDoc->ProcessPendingRead();
}
else
{
// Error handling here
}
}
TCP guarantees that they will recieve all 1000 bytes, but not necessarily in order (though, it will appear so to the recieving application) and not necessarily all at once (unless you craft the packet yourself and make it so.).
That said, for a packet as small as 1000 bytes, there is a good chance it'll send in one packet as long as you do it in one call to send, though for larger transmissions it may not.
The only thing that the TCP layer guarantees is that the receiver will receive:
all the bytes transmitted by the sender
in the same order
There are no guarantees at all about how the bytes might be split up into "packets". All the stuff you might read about MTU, packet fragmentation, maximum segment size, or whatever else is all below the layer of TCP sockets, and is irrelevant. TCP provides a stream service only.
With reference to your question, this means that the receiver may receive the first 500 bytes, then the next 500 bytes later. Or, the receiver might receive the data one byte at a time, if that's what it asks for. This is the reason that the recv() function takes a parameter that tells it how much data to return, instead of it telling you how big a packet is.
The transmission control protocol guarantees successful delivery of all packets by requiring acknowledgment of the successful delivery of each packet to the sender by the receiver. By this definition the receiver will always receive the payload in chunks when the size of the payload exceeds the MTU (maximum transmission unit).
For more information please read Transmission Control Protocol.
The IP packets may get fragmented during retransmission.
So the destination machine may receive multiple packets - which will be reassembled back by TCP/IP stack. Depending on the network API you are using - the data will be given to you either reassembled or in RAW packets.
It depends of the stablished MTU (Maximum transfer unit). If your stablished connection (once handshaked) refers to a MTU of 512 bytes you will need two or more TCP packets to send 1000 bytes.
I have the Problem, that I want to send a large string through a SocketConnection, but I can't receive the string at once because the Network is limited to 1500 bytes, so how can I Receive all bytes over the Socket without using a TCPClient, or is that just impossible.
public string ReceiveString(Socket connection)
{
byte[] buffer = new byte[1048576];
int recBytes = connection.Receive(buffer);
return UnicodeEncoding.Unicode.GetString(buffer, 0, recBytes);
}
Thats, what i have so far.
If it isn't possible, to get it over the Socket, can I get the EndPoint out of the Socket to use TCPClient as "last Option"?
TCP is a byte stream, it has no concept of messages. The size of the individual packets on the wire is irrelevant, it is just an implementation detail of the networking hardware. TCP guarantees that what you send is what you receive (but there is no 1-to-1 relationship between the size of individual sends and the size of individual reads, like there is in UDP).
The solution requires the sender to frame the string data in such a way that allows the reader to know when to stop reading. Either:
send the string length before sending the string data. The reader can then read the length first and then read the specified number of following bytes.
a. terminate the string with a unique delimiter that cannot appear in the string itself. The reader can then keep reading until it encounters the delimiter.
b. the delimiter can be closure of the connection. The reader can keep reading until a disconnect is detected.
Which solution you need to use depends on the particular protocol you are implementing. #1 is best for binary protocols, and allows for efficient memory managent, whereas #2 is more suited to text-based protocols, or streaming protocols where the final length is not know ahead of time. Sometimes protocols have to resort to #2b (HTTP and FTP both utilize it as times) when #1 and #2a are not possible.
In my application I have a serial port object and a listbox. In the DataRecieved event, I send serialPort.ReadLine() to the listbox. If I write a "n" character to the serial port, nothing will get added to the listbox because what gets recieved doesn't end in "\r" or "\n".
What is the correct way to read information from a serial port? (Keep in mind that I need to keep the full string/char[] of the last thing recieved.)
The 'correct' way depends heavily on implementation.
The SerialPort.ReadLine() method expects a CR/LF as a means to define a payload unit. And, by thing, I imagine that you mean exactly that - a message, payload or package (as in one meaningful, functional unit of information.)
What SerialPort.ReadLine() does is to wrap the whole 'receive everything coming from the buffer and wait for a end-of-payload mark before continuing' mechanism for you.
If you'd rather have the raw incoming content as soon as it arrives, then you may consider changing your code to use SerialPort.Read() instead.
If your message consists of an exact amount of bytes (sometimes the case with sensor data protocols) you can define the bytes you expect - but you should set a timeout in this case.
SerialPort.ReadTimeout = timeOut;
SerialPort.Read(responseBytes, 0, bytesExpected)
Im new to sockets, and Im creating a tictactoe online, I know how to make the connections with the clients and the server, but I will make a chat too.
Then I doing this, when a user chat I send a message with a prefix "CHAT: HELLO WORLD"
and when a user make a move I send a message without the prefix... this is the best way?
THX!!!
In defining a wire protocol over a stream-based protocol like TCP, you have a few options for constructing messages:
Fixed-length
All messages are the same length; every sequence of x bytes represents a new message.
Length-prefixed (variable length)
The first byte(s) of the message represent the length of the payload to follow.
String-terminated (variable length)
Read bytes from the stream until you come to a specified byte-string that represents the end of a message, i.e. the newline character \n.
If you ever intend on changing the protocol (protip: you will, even if you don't think you will), it is crucial that you include an identifier for the protocol version in each message to prevent issues when dealing with clients using an older iteration of the protocol. Clearly, this is the first thing you must determine before deciphering the rest of the payload, so this should be the first byte(s) of the message (following any length-prefix) - how could we determine the version if we don't know where it is located in every message we receive?
Typically you would go with a format that includes a packet length, type and payload.
In your case you could go with a Byte (type), Int16 (length), Byte[] (payload).
The type can be represented in code as an enum. Length would just represent the length of the payload.
public enum Byte PacketType {
PlayerMove = 1,
PlayerChat = 2
}
You need to define a protocol. Remember to allow room for additional features :-).
Eg. using regular expressions over complete lines (end with selected line terminator):
Matching ^:[a-c][1-3]:: is a move (colon, position, colon user name).
Matching ^!.*?:: is a chat message (exclamation point, name, colon, text).
and anything else (in V1) is an error.
Remember:
Data is sent in packets, you might need multiple reads from the socket to get a complete message.
Avoid ambiguity: resolving it might be x or y is hard.
Specify a text encoding (eg. UTF-8).
I assume you're using TCP?
You need to make sure you 'frame' both messages so you can identify them and also avoid potential blocking issues (in case the client stops sending while you are still expecting to read CHAT: or whatever you define). With TCP your byte order is guaranteed but reading does not guarantee a complete 'packet' so you'll need to implement some way of building up a buffer and identifying when your 'message' is complete.
A reasonably simple way of doing this is to make sure each 'message' has a header with the type and size specified.
EG:
Enumerate your message types (move and chat currently), so say 'chat' is 0x01 and your message is 1020 bytes. You can prefix your 'message' with 0x0103FC so the server knows how many bytes to expect, and build up a buffer using async socket calls until the 1020 bytes are read (or you arbitrarily decide that the client is not sending anymore)
I have the following code in C#:
Console.WriteLine("Connecting to server...");
TcpClient client = new TcpClient("127.0.0.1", 25565);
client.Client.Send(BitConverter.GetBytes(0x02));
client.Client.Send(BitConverter.GetBytes(0x0005));
client.Client.Send(Encoding.UTF8.GetBytes("wedtm"));
Console.Write("{0:x2}", client.GetStream().ReadByte());
For the life of me, I can't figure out how to transpose this to ruby. Any help here?
This is what I have so far, but it's not working as expected:
require 'socket'
s = TCPSocket.open("127.0.0.1", 25565)
s.write(0x02)
s.write(0x0005)
s.write("wedtm".bytes)
response = s.recvfrom(2)
puts "Response Size #{response.size}: #{response.to_s}"
The response should be 0x02
EDIT:
I'm assuming I have to use String#unpack on this, however, I can't figure out how to get "wedtm" to output to the appropriate \x000\x000\x000\x000 format.
There are at least two things to consider here:
Network byte order is big-endian. This means that you should always think in single bytes or arrays of bytes, as bytes are not subject to being shuffled around while larger types are.
C#'s BitConverter.GetBytes(int16) returns 2 bytes in little-endian format and GetBytes(int32) returns 4 bytes in little-endian format
Without knowing any Ruby or its string format, I'd guess you need to do something like this for the first part:
s.write("\x02\x00".bytes)
s.write("\x05\x00\x00\x00".bytes)
The second part should be okay.
WireShark is an invaluable tool when debugging network code and/or reverse engineering networking protocols, record the traffic of the C# app and compare the difference with yours.