I'm creating a client/server socket app. And I'm not being able to solve this problem, probably due to lack of knowledge on the matter.
The client must send an answer in order to proceed communication:
while(comunicate)
{
if (chkCorrectAnswer.Checked)
answer = encoder.GetBytes('\x02' + "82SP|" + '\x03');
else
answer = encoder.GetBytes("bla");
ServerStream.Write(answer, 0, answer.Length);
//or ??
//tcp.Client.Send(answer);
}
And the server recieves it:
while(comunicate)
{
var validanswer= encoder.GetBytes("myvalidanswer");
answer = new byte[validanswer.Length];
stream.Read(answer, 0, validanswer.Length);
//or ??
//tcp.Client.Receive(answer);
if (answer.SequenceEqual(validanswer))
// continue communication
}
Each code is in different app, looped "comunication thread".
The answer seems being sent correctly but the server doesn't seem to be recieveing it porperly. Sometimes it recieves blablab or lablabl and variations with 7 chars. I thoung the recieving would fill the buffer only with the incoming data and somehow it is filling the buffer with repeated data.
Two questions here:
What should I use, stream.write/read or client.send/recieve?
How to ensure this answer verification?
0x02 and 0x03 is called start of text (STX) and end of text (ETX) and are separators used to identify where your messages starts and ends. There is really no need to use both, it was a common practice when doing serial communication.
You need to continue building a message until ETX is received.
something like (easiest solution but not very efficient if you have lots of clients)
string buffer = "";
var readBuffer = new byte[1];
int readBytes = 0;
while ((readBytes = stream.Read(readBuffer, 0, 1)) == 1 && readBuffer[0] != 3)
{
buffer += readBuffer[0];
}
you can of course read larger chunks. But then you need to check if more than one message was arrived and process the buffer accordingly. That's how I would do it though.
I thoung the recieving would fill the buffer only with the incoming data and somehow it is filling the buffer with repeated data.
Well, you are repeatedly sending the data in a loop, so this is to be expected.
If you want to read only a certain number of bytes off the stream you need to also send the size of the logical packet ahead of it so that the receiving end can first read the size (say as a fixed int value) and then the actual response.
When you do the read you'll get everything you wrote including amything you've written previously.
Implement a length header or some kind of seperator so you know what's what --
length + message
or
message + seperator
Then parse it when you do the read.
Related
I have a TCP request response model in C# where I am communicating with a server. Once the server has written data to the stream, I am reading that data. But stream.read is taking 2 seconds to read the data. I need to send an explicit acknowledgement to the server, within 2 seconds but am unable to do so because of the time taken to read the data.
Below is my code to read data:
byte[] resp = new byte[100000];
var memoryStream = new MemoryStream();
int bytes;
String timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("Before reading data: ");
Console.WriteLine(timeStamp);
do
{
bytes = stream.Read(resp, 0, resp.Length);
memoryStream.Write(resp, 0, bytes);
}
while (bytes > 0);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After reading data: ");
Console.WriteLine(timeStamp);
GenerateAcknowledgemnt(stream);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After sending ack: ");
Console.WriteLine(timeStamp);
Below are the timestamps read, in the format yyyyMMddHHmmssff:
Before reading data:
2022050615490817
After reading data:
2022050615491019
After sending ack:
2022050615491020
I have highlighted the seconds bold.
How do I reduce the time that stream.read is taking to read? I have tried to wrap the network stream in a BufferedStream as well, but it didn't help.
At the moment, you are performing a read loop that keeps going until Read returns a non-positive number; in TCP, this means you are waiting until the other end hangs up (or at least hangs up their outbound socket) until you get out of that loop. I suspect what is happening is that the other end is giving up on you, closing their connection, and only then do you get out of the loop.
Basically: you can't loop like that; instead, what you need to do is to carefully read until either EOF (bytes <= 0) or until you have at least one complete frame that you can respond to, and in the latter case: respond then. This usually means a loop more like (pseudo-code):
while (TryReadSomeMoreData()) // performs a read into the buffer, positive result
{
// note: may have more than one frame per successful 'read'
while (TryParseOneFrame(out frame))
{
ProcessFrame(frame); // includes sending responses
// (and discard anything that you've now processed from the back-buffer)
}
}
(parsing a frame here means: following whatever rules apply about isolating a single message from the stream - this may mean looking for a sentinel value such as CR/LF/NUL, or may mean checking if you have enough bytes to read a header that includes a length, and then checking that you have however-many bytes the header indicates as the payload)
This is a little awkward if you're using MemoryStream as the backlog, as the discard step is not convenient; the "pipelines" API is more specifically designed for this, but: either way can work.
Secondly: you may prefer async IO, although sync IO is probably fine for a simple client application with only one connection (but not for servers, which may have many many connections).
I am trying to receive multiple messages send by Python TCP client to a C# server.
I do receive the data, but all at once and I don't want that to happen.
I tried to set the server buffer size to the size of the byte[] which I am sending, but it didn't work (source).
Any ideas how can I fix this issue?
Client sending code:
import socket
def send_to_remote(sock, data):
data_bytes = bytearray()
data_bytes.extend(map(ord, data)) # appending the array with string converted to bytes
bytes_length = len(data_bytes) # length of the array that I am sending
sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, bytes_length) # setting the buffer size to the array length
print(bytes_length)
sock.send(data_bytes) # sending the data
Server receiving code:
public static string ReadData(TcpClient connection)
{
StringBuilder receivedDataStr = new StringBuilder();
NetworkStream connectionStream = connection.GetStream();
if (connectionStream.CanRead)
{
byte[] connectionBuffer = new byte[connection.ReceiveBufferSize];
Console.WriteLine(">> Reading from NODE");
int bytesRead = 0;
bytesRead = connectionStream.Read(connectionBuffer, 0, 1024);
receivedDataStr.Append(Encoding.ASCII.GetString(connectionBuffer, 0, bytesRead));
Console.WriteLine(">> " + connection.ReceiveBufferSize);
return receivedDataStr.ToString();
}
else
{
Console.WriteLine(">> Can't read from network stream");
return "none-read";
}
}
EDIT:
What I do is:
send_to_remote(socekt, "msg1")
send_to_remote(socekt, "msg1")
And then:
string msg1 = ReadData(client);
string msg2 = ReadData(client);
I receive buffer array of 106200.
The result is:
"msg1 (then new line) msg2"
"" (string msg2 is waiting for data)
Thank you in advance!!
TCP is a stream protocol, not a packet protocol. There is absolutely no guarantee that you're going to get data in the same exact chunks that it was sent - all that is guaranteed is that you'll get the bytes in the correct order (or a failure, or a timeout, etc). You can get each byte separately, or 14 messages at once. Or anything in between, including half of a message in one receive, and the other half in the next. In the case of unicode (etc), this also means that a single character can be split over multiple receives.
Any multi-message protocol based on TCP needs to include some kind of framing, i.e. some way for you to know where each separate message starts and ends. Since you're using a text-based protocol, a common option is to separate logical messages with a CR, LF, or CR+LF - and then it is your job to buffer data until you have an entire message - by looking for the line ending. For completeness: in the case of binary protocols, it is common to length-prefix payloads with some kind of header that indicates the amount of data in the payload, then you just need to parse the header, determine the length, and buffer until you have that much data.
In simple applications with a text protocol and no challenging scalability concerns, it may be possible to use StreamReader (on top of a NetworkStream) to do that part for you, then just use ReadLine or ReadLineAsync. Note, however, that this would be a bad idea in "real" servers (especially ReadLine) - you don't want a malicious or just buggy client locking a thread forever just because it sent a few characters without the line ending, and then didn't send anything else ... ever. Of course, in a serious server you wouldn't be using a thread per client anyway.
so long story short, i am trying to develop an application which requires having Tcp Connection to a server. having no previous experiance in this field i ended up with having none of my functions working. so i decided to make a simple console app just to check my commands and their responses. the part about setting up the connections went well, so does the part about wrting into network stream(i think) but i hit a wall when trying to display server respons:
each time my porgram reaches the line where it has to Encode server respons and WriteLine, my console application goes all black and all texts goes away. can you tell what's wrong with the code i am using?everything up until the part where console trys to display response goes well, i checked. i want to have server's respones as normal strings(if possible i guess...)
static void ServerTalk(TcpClient tcpClient,NetworkStream networkStream, StreamWriter streamWriter, StreamReader streamReader)
{
//Messaging methods:
Console.WriteLine("Client: ");
string message = Console.ReadLine();
streamWriter.WriteLine(message);
streamWriter.Flush();
//Response methode:
byte[] data = new byte[tcpClient.ReceiveBufferSize];
int ret = networkStream.Read(data, 0, data.Length);
string respond = Encoding.ASCII.GetString(data).TrimEnd();
Console.WriteLine("Server: ");
Console.WriteLine(respond);
ServerTalk(tcpClient, networkStream, streamWriter, streamReader);
}
you need to process only ret bytes, you are translating all bytes in the array.
copy to a new array that is 'ret' long and then decode.
var retarr = new byte[ret];
Array.Copy(data,retarr,ret);
string respond = Encoding.ASCII.GetString(retarr);
More importantly note that this method of tcp communication is not going to work. You are assuming that when you send x bytes in one block you will receive x bytes in one block, this is not so, you can send 1 100 byte message and receive 100 1 byte messages
This means that you task is much more complex. YOu need to devise a way of sending self describing data - so that you know its completely received. classically you send a length followed by the data. You then keep looping in the receive code till you have received that many bytes
This is my first question posted on this forum, and I'm a beginner in c# world , so this is kind of exciting for me, but I'm facing some issues with sending a large amount of data through sockets so this is more details about my problem:
I'm sending a binary image of 5 Mo through a TCP socket, at the receiving part I'm saving the result(data received ) and getting only 1.5 Mo ==> data has been lost (I compared the original and the resulting file and it showed me the missed parts)
this is the code I use:
private void senduimage_Click(object sender, EventArgs e)
{
if (!user.clientSocket_NewSocket.Connected)
{
Socket clientSocket_NewSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
user.clientSocket_NewSocket = clientSocket_NewSocket;
System.IAsyncResult _NewSocket = user.clientSocket_NewSocket.BeginConnect(ip_address, NewSocket.Transceiver_TCP_Port, null, null);
bool successNewSocket = _NewSocket.AsyncWaitHandle.WaitOne(2000, true);
}
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(Uimage_Data);
user.clientSocket_NewSocket.Send(outStream);
}
In forums they say to divide data into chunks, is this a solution, if so how can I do this, I've tried but it didn't work!
There are lots of different solutions but chunking is usually a good solution, you can either do this blindly where you keep filling your temporary buffer and then putting it into some stateful buffer until you hit some arbitrary token or the buffer is not completely full, or you can adhere to some sort of contract per tcp message (a message being the overall data to recieve).
If you were to look at doing some sort of contract then do something like the first N bytes of a message is the descriptor, which you could make as big or as small as you want, but your temp buffer will ONLY read this size up front from the stream.
A typical header could be something like:
public struct StreamHeader // 5 bytes
{
public byte MessageType {get;set;} // 1 byte
public int MessageSize {get;set;} // 4 bytes
}
So you would read that in then if its small enough allocate the full message size to the temp buffer and read it all in, or if you deem it too big chunk it into sections and keep reading until the total bytes you have received match the MessageSize portion of your header structure.
Probably you haven't read the documentation on socket usage in C#. (http://msdn.microsoft.com/en-us/library/ms145160.aspx)
The internal buffer can not store all the data you provided to send methode. A possible solution to your problem can be is like you said to divide your data into chunks.
int totalBytesToSend = outstream.length; int bytesSend = 0;
while(bytesSend < totalBytesToSend )
bytesSend+= user.clientSocket_NewSocket.Send(outStream, bytesSend, totalBytesToSend - bytesSend,...);
I suspect that one of your problems is that you are not calling EndConnect. From the MSDN documentation:
The asynchronous BeginConnect operation must be completed by calling the EndConnect method.
Also, the wait:-
bool successNewSocket = _NewSocket.AsyncWaitHandle.WaitOne(2000, true);
is probably always false as there is nothing setting the event to the signaled state. Usually, you would specify a callback function to the BeginConnect function and in the callback you'd call EndConnect and set the state of the event to signaled. See the example code on this MSDN page.
UPDATE
I think I see another problem:-
byte[] outStream = System.Text.Encoding.ASCII.GetBytes(Uimage_Data);
I don't know what type Uimage_Data but I really don't think you want to convert it to ASCII. A zero in the data may signal an end of data byte (or maybe a 26 or someother ASCII code). The point is, the encoding process is likely to be changing the data.
Can you provide the type for the Uimage_Data object?
Most likely the problem is that you are closing the client-side socket before all the data has been transmitted to the server, and it is therefore getting discarded.
By default when you close a socket, all untransmitted data (sitting in the operating system buffers) is discarded. There are a few solutions:
[1] Set SO_LINGER (see http://developerweb.net/viewtopic.php?id=2982)
[2] Get the server to send an acknowledgement to the client, and don't close the client-side socket until you receive it.
[3] Wait until the output buffer is empty on the client side before closing the socket (test using getsocketopt SO_SND_BUF - I'm not sure of the syntax on c#).
Also you really should be testing the return value of Send(). Although in theory it should block until it sends all the data, I would want to actually verify that and at least print an error message if there is a mismatch.
I am playing with RserveCLI project, which is a .net client communicating with the statistical environment R. The basic idea is sending data/commends between this .NET client and an R session via TCP protocol.
One bug that others and I found is that big data trunk, say over 10k bytes, cannot get transfer successfully. I found the but in the following code snippet:
// send the commend to R, then R will do some computation and get the data ready to send back
int toConsume = this.SubmitCommand(cmd, data);
var res = new List<object>();
while (toConsume > 0)
{
var dhbuf = new byte[4];
if (this.socket.Receive(dhbuf) != 4)
{
throw new WebException("Didn't receive a header.");
}
byte typ = dhbuf[0];
// ReSharper disable RedundantCast
int dlength = dhbuf[1] + (((int)dhbuf[2]) << 8) + (((int)dhbuf[3]) << 16);
// ReSharper restore RedundantCast
var dvbuf = new byte[dlength];
// BUG: I added this sleep line, without this line, bug occures
System.Threading.Thread.Sleep(500);
// this line cannot receive the whole data at once
var received = this.socket.Receive(dvbuf);
// so the exception throws
if (received != dvbuf.Length)
{
var tempR = this.socket.Receive(dvbuf);
throw new WebException("Expected " + dvbuf.Length + " bytes of data, but received " + received + ".");
}
The reason is that the .NET code runs too fast and the R side cannot send the data that fast. So the receive line after my inserted Thread.Sleep(500) does not get all the data. If I wait some time there, then it can get all the data. But I don't know how long.
I have some basic idea to deal with the problem, for example, continuously use this.socket.Receive() to get data, but if there is no data there .Receive will block there.
I have little experience in socket programming, so I am asking the best practice for this kind of problem. Thanks!
According to the docs:
If you are using a connection-oriented Socket, the Receive method will read as much data as is available, up to the size of the buffer.
So you are never guranteed to get all the data asked for in the receive call. You need to check how many bytes were actually returned by the Receive, then issue another receive call for the remaining bytes. Continue that loop until you get all the bytes you were looking for.
By definition, TCP is a streaming protocol, whereas UDP is message based. If the data you are trying to receive does not contain a byte count for the entire message, or some sort of end-of-message indicator, you will just have to loop on the socket.receive until some arbitrary timeout has expired. At that point, check the accumulated received data for completeness.