Attempting to setup a TCP server to grab data from a stream. Seems to be working, but only when the stream is small. Once I start sending large amounts of data, this fails, only returns a portion of the characters. Can anyone help me out here? Why am I only getting a portion of the data I am sending?
Flow of the server should be, receive ALL data, store into database (RouteInboundXml()) and begin listening for more incoming data.
private void ReceivePortMessages()
{
string debug = string.Empty;
try
{
Debug.Print(" >> Starting Server");
IPAddress ipAddress = Dns.GetHostEntry(Dns.GetHostName()).AddressList.FirstOrDefault(ip => ip.AddressFamily == AddressFamily.InterNetwork);
_TcpListener = new TcpListener(ipAddress, TcpPort); ;
Debug.Print(string.Format("{0}:{1}", ipAddress.ToString(), TcpPort.ToString()));
_TcpListener.Start();
Stopwatch sw = new Stopwatch();
do
{
try
{
_TcpClient = _TcpListener.AcceptTcpClient();
Debug.Print(" >> Accept connection from client");
NetworkStream networkStream = _TcpClient.GetStream();
int receivingBufferSize = (int)_TcpClient.ReceiveBufferSize;
byte[] bytesFrom = new byte[receivingBufferSize];
int Read = 0;
string dataFromClient = string.Empty;
if (!sw.IsRunning)
{
sw.Start();
}
Read = networkStream.Read(bytesFrom, 0, receivingBufferSize);
dataFromClient = System.Text.Encoding.ASCII.GetString(bytesFrom);
dataFromClient = dataFromClient.Substring(0, dataFromClient.IndexOf("\0"));
if (dataFromClient != string.Empty)
{
XmlDocument xm = new XmlDocument();
debug = dataFromClient;
xm.LoadXml(string.Format("<root>{0}</root>", dataFromClient));
XmlElement root = xm.DocumentElement;
string rootName = root.FirstChild.Name;
RouteInboundXML(rootName, dataFromClient, sw);
sw.Restart();
}
}
catch (Exception ex)
{
Debug.Print("ReceivePortMessages: " + ex.ToString());
_TcpClient.Close();
_TcpListener.Stop();
ErrorLog.Write("XmlProcessing", ex.ToString() + "\r\n" + "DataFromClient: " + debug, "ReceivePortMessages()");
return;
}
} while (true);
}
catch (Exception ex)
{
Debug.Print("ReceivePortMessages: " + ex.ToString());
ErrorLog.Write("XmlProcessing", ex.ToString(), "ReceivePortMessages()");
}
}
You seem to be expecting to receive an entire XML document - and exactly the XML document - each time you call Stream.Read. That's a really, really dangerous assumption.
The stream is a stream of data - while it'll be segmented into packets for transmission, you shouldn't expect to receive the data in the same number of Read calls as there were Write calls.
Details for a single document per connection
If there's only one document per stream, you can probably simplify your code a lot and make it work properly. Just use the fact that there's an overload of XmlDocument.Load which accepts a stream:
using (var tcpClient = tcpListener.AcceptTcpClient())
{
XmlDocument doc = new XmlDocument();
using (var stream = tcpClient.GetStream())
{
doc.Load(stream);
}
// Use doc here
}
(If you can, I'd personally start using LINQ to XML instead, but that's a different matter.)
Details for multiple documents
If you want multiple messages on a single TCP stream, you should implement some sort of "chunking" protocol. One good way of doing this is to split each message into a "header" and a "body" where the header may be as simple as "the number of bytes in the body", so you know how much to read. (Alternatively you could design the protocol for the header to contain other metadata.)
After reading the header, you read the body from the stream until either you've read as many bytes as were indicated in the header, or you reach the end of the stream (which would usually indicate an error). That may require multiple Read calls. Then you're in a suitable position to parse the data into an XML document - ideally without imposing your own binary/text decoding first, as not all XML is ASCII...
You could design your protocol in such a way that you just have a delimiter between messages instead - but that's generally much harder to implement, as it mixes "reading the data" and "understanding the data". If you can use the length-prefix scheme described above instead, that's much simpler.
Here is some code i have used in the past.
Works for me.
using (TcpClient client = new TcpClient(ip, port))
{
var stm = client.GetStream();
stm.Write(data, 0, data.Length); //Write some data to the stream
byte[] resp = new byte[1024];
var memStream = new MemoryStream();
var bytes = 0;
client.Client.ReceiveTimeout = 200;
do
{
try
{
bytes = stm.Read(resp, 0, resp.Length);
memStream.Write(resp, 0, bytes);
}
catch (IOException ex)
{
// if the ReceiveTimeout is reached an IOException will be raised...
// with an InnerException of type SocketException and ErrorCode 10060
var socketExept = ex.InnerException as SocketException;
if (socketExept == null || socketExept.ErrorCode != 10060)
// if it's not the "expected" exception, let's not hide the error
throw ex;
// if it is the receive timeout, then reading ended
bytes = 0;
}
} while (bytes > 0);
return memStream.ToArray();
}
Related
To begin with, I'm a student, don't scold me too much for my code, I'm trying to get better.
I will accept any criticism about the code, it helps to get better)
The crux of my problem: when the server is disconnected, I send a command to it, process it, and send a response from the server stating that it has completed its work, in my second project (client) I receive a message and process it, and sometimes it is processed, and sometimes it happens the problem is that I just get an exception.
And it comes out of nothing, the server shutdown function works, I have to display a MessageBox, and it immediately switches me to another function and receives an exception for converting a byte to an object (the function works correctly, all other commands work without problems).
I have already climbed all the Internet resources, nothing helps.
Where does the exception occur
public Object ByteArrayToObject(byte[] arrBytes)
{
using (var memStream = new MemoryStream())
{
var binForm = new BinaryFormatter();
memStream.Write(arrBytes, 0, arrBytes.Length);
memStream.Seek(0, SeekOrigin.Begin);
object obj = binForm.Deserialize(memStream);***//ERROR HERE System.Runtime.Serialization.SerializationException: The input stream is not a valid binary format. The starting contents (in bytes) are:***
return obj;
}
}
The start of an exception occurs in this function after receiving a message from the server before or during the display of the MessageBox
at this moment it switches me to this function and catches an exception
private void DisconnectServer()
{
try
{
string theMessage = "Клиент выполнил дисконнект";
byte[] msg = Encoding.Default.GetBytes(theMessage);
int bytesSent = sock.Send(msg);
byte[] bytes = new byte[1024];
int bytesRec = sock.Receive(bytes);
string msgserv = Encoding.Default.GetString(bytes, 0, bytesRec);
//MessageBox.Show("Сервер (" + sock.RemoteEndPoint.ToString() + ") ответил: " + msgserv);
sock.Shutdown(SocketShutdown.Both);
sock.Close();
}
catch (Exception ex)
{
MessageBox.Show("Клиент: " + ex.Message);
}
}
The moment the server message is sent to the client about the disconnection and after which an error occurs
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine(client + ":" + data);
Console.ResetColor();
string theReply = "Сервер остановлен";
byte[] msg = Encoding.Default.GetBytes(theReply);
handler.Send(msg);
choice = 0;
handler.Shutdown(SocketShutdown.Both);
handler.Close();
break;
I have a little complication i encounter.
I may not be expert in TCP Connections but i hope someone here would help me.
This is my Client Code:
void Connect(String server, String message)
{
try
{
Int32 port = 8968;
TcpClient client = new TcpClient(server, port);
Byte[] data = File.ReadAllBytes(curSelectedFile);
NetworkStream stream = client.GetStream();
Byte[] fileData = File.ReadAllBytes(curSelectedFile);
Byte[] msgData = Encoding.ASCII.GetBytes("SendFile");
Byte[] sendData = new byte[fileData.Length + msgData.Length];
// Copy data to send package.
msgData.CopyTo(sendData, 0);
fileData.CopyTo(sendData, 4);
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close();
client.Close();
}
catch (ArgumentNullException e)
{
Console.WriteLine("ArgumentNullException: {0}", e);
}
catch (SocketException e)
{
Console.WriteLine("SocketException: {0}", e);
}
Console.WriteLine("\n Press Enter to continue...");
Console.Read();
}
This is my server one:
// Listen loop.
while(true)
{
using (TcpClient tcpClient = myListener.AcceptTcpClient())
{
Console.WriteLine("[Server] Acceptam client.");
using (NetworkStream networkStream = tcpClient.GetStream())
{
// Buffer for reading data
Byte[] bytes = new Byte[1024];
var data = new List<byte>();
int length;
while ((length = networkStream.Read(bytes, 0, bytes.Length)) != 0)
{
var copy = new byte[length];
Array.Copy(bytes, 0, copy, 0, length);
data.AddRange(copy);
}
// Incercam sa vedem ce doreste clientul.
string msg = Encoding.ASCII.GetString(data[0], 0, length);
if(msg.StartsWith("SendFile"))
{
using (Stream stream = new FileStream(#"C:\test.mp3", FileMode.Create, FileAccess.ReadWrite))
{
BinaryFormatter binaryFormatter = new BinaryFormatter();
networkStream.Position = 4;
binaryFormatter.Serialize(networkStream, data.ToArray());
}
}
}
}
}
What i'm trying to do here:
- I want the client to send a Message.. like "SaveFile" & after this string to be the filedata.
- The server should read the client message, and to process stuff according to the Client sentstring, before doing something with the file.
I believe that i don't know how to do it.
May i have an example on how to send/receive and read certain strings from the beggining of the file? How i can put them in the byte array and how to read it... It's quite overwhelming..
PS: The current Server Code is reading the data and CAN write as i coded it, without losing any packages. But also he's writing the aditional packets i sent before i converted the bytes of the file.
networkStream.Position = 4; isn't legal, because NetworkStream is not seekable.
I would discourage you from mixing text and binary data, just because of the complication it makes in the application protocol. But if you really want to do that, you should use BinaryWriter and BinaryReader, because it can write strings to a stream which can then be read later without consuming the bytes after the string.
Then you can do something like this...
In the client:
BinaryWriter writer = new BinaryWriter(networkStream);
writer.Write("SendFile");
writer.Write(fileData.Length);
writer.Write(fileData);
In the server:
BinaryReader reader = new BinaryReader(networkStream);
switch (reader.ReadString())
{
case "SendFile":
{
int length = reader.ReadInt32();
byte[] fileData = reader.ReadBytes(length);
// ... then do whatever with fileData, like write to a file
break;
}
}
The BinaryWriter/Reader implementation of the length-counted string is non-standard, so if you wanted to interact with any other non-.NET code using this technique, it would be more complicated because you have to replicate/reimplement the non-standard length-counting logic yourself.
IMHO a better approach is to encode commands as fixed-length data, e.g. an 8-, 16-, or 32-bit value, which is just some integer that specifies the command. Then you can list your commands in the code as an enum type, casting to/from the network stream for the I/O. This would be more portable, easier to implement on non-.NET platforms.
The following method is supposed to send a couple of commands to the server. It is also supposed to return full reply from the server as a string. The problem I have is with the reply section of the code; specifically, I am not getting the FULL reply back. If I add the following line Console.WriteLine(bytesRead); before the memoryStream.Write(buffer,0,bytesRead); I receive the full reply. Seems like this silly line of code helps "pause" something so that all data is returned. I really do not know what I am doing wrong and I need your help. Thanks
public string Send(List<string> commands)
{
try
{
// String that will contain full reply from server
string fullServerReply = string.Empty;
TcpClient tcpClient = new TcpClient(host, port);
NetworkStream networkStream = tcpClient.GetStream();
foreach (string command in commands)
{
// Check to see if this NetworkStream is writable
if (networkStream.CanWrite)
{
// Translate the passed message into UTF8 and store it as a byte array.
byte[] sendBytes = Encoding.UTF8.GetBytes(command);
// Send the message to the connected TcpServer.
networkStream.Write(sendBytes, 0, sendBytes.Length);
}
else
{
// Close everything.
networkStream.Close();
tcpClient.Close();
return "";
}
// Check to see if this NetworkStream is readable
if (networkStream.CanRead)
{
using (MemoryStream memoryStream = new MemoryStream())
{
// Buffer to store the response bytes.
byte[] buffer = new byte[1024];
do
{
int bytesRead = networkStream.Read(buffer, 0, buffer.Length);
if (bytesRead <= 0)
{
break;
}
//Console.WriteLine(bytesRead); <- BY ADDING THIS CODE I GET THE FULL REPLY
memoryStream.Write(buffer, 0, bytesRead);
} while (networkStream.DataAvailable);
memoryStream.Position = 0;
fullServerReply += Encoding.UTF8.GetString(memoryStream.ToArray()); // THIS STRING DOES NOT CONTAIN FULL REPLY
}
}
else
{
// Close everything.
networkStream.Close();
tcpClient.Close();
return "";
}
}
// Close everything.
networkStream.Close();
tcpClient.Close();
return fullServerReply.Trim();
}
catch (ArgumentNullException ex)
{
return "";
}
catch (SocketException ex)
{
return "";
}
return "";
}
Yup! DataAvailable is NOT indicator that a complete stream has been received. It only indicates if some data that hasn't yet been ready available in receive buffer.
When you add console.readline, you give network packets a chance to catch up and get more data in buffer.
Instead you should keep looping until either the network stream is closed, or since it is a tcp stream, whatever the protocol you're using that tells you how big your application packet is to keep reading for.
That's kind of how networking works - you cannot assume that you get everything back instantly, or in the same number of reads as there were writes. In your case, adding the write to the console was just enough to have the client spool everything.
Try this to flush all the writes to the stream after the loop:
memoryStream.Flush();
memoryStream.Position = 0;
fullServerReply += Encoding.UTF8.GetString(memoryStream.ToArray()); // THIS STRING DOES NOT CONTAIN FULL REPLY
In my client/server application my client wiil communicate with the server for 2 functions: the client will either request data from the server or it will send data so the server will save it. I'm using one socket for both methods, and the method to be used is defined by the first byte sent. If the first byte is "1" it is requesting data. If it is "2", it will send data (data bytes are sent after the "2" byte). It works perfectly for sending data. But when I'm requesting data it works, as long as I don't read the socket stream in the client. It's like if I make the client read data after sending data, the server will have no data to read, and it just crashes when trying to read the data.
Here is my server code:
private const int BufferSize = 1024;
NetworkStream netstream = null;
byte[] RecData = new byte[BufferSize];
int RecBytes;
try {
netstream = clientSocket.GetStream();
int totalrecbytes = 0;
using (MemoryStream ms = new MemoryStream()) {
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
byte[] bytes = ms.ToArray();
byte b = bytes[0];
switch (b) {
case 1:
//Here I gather data and put it in "stream" variable
byte[] SendingBuffer = null;
int NoOfPackets = Convert.ToInt32(Math.Ceiling(Convert.ToDouble(stream.Length) / Convert.ToDouble(BufferSize)));
int TotalLength = (int)stream.Length, CurrentPacketLength, counter = 0;
for (int i = 0; i < NoOfPackets; i++) {
if (TotalLength > BufferSize) {
CurrentPacketLength = BufferSize;
TotalLength = TotalLength - CurrentPacketLength;
}
else
CurrentPacketLength = TotalLength;
SendingBuffer = new byte[CurrentPacketLength];
stream.Read(SendingBuffer, 0, CurrentPacketLength);
netstream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
}
netstream.Flush();
}
catch (Exception e) {
Console.WriteLine("EXCEPTION:\n" + e.ToString());
}
break;
case 2:
//Code to read data
break;
}
}
netstream.Close()
clientSocket.Close();
And here is my client code:
using (System.Net.Sockets.TcpClient clientSocket = new System.Net.Sockets.TcpClient()) {
string returnData = "";
IAsyncResult ar = clientSocket.BeginConnect("127.0.0.1", 8080, null, null);
System.Threading.WaitHandle wh = ar.AsyncWaitHandle;
try {
if (!ar.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(5), false)) {
clientSocket.Close();
Console.WriteLine("Timeout");
return;
}
System.Net.Sockets.NetworkStream serverStream = clientSocket.GetStream();
byte b = 1;
byte[] outStream = { b };
serverStream.Write(outStream, 0, outStream.Length);
serverStream.Flush();
//If I comment following lines, the server can read sent data, but server can't otherwise
byte[] RecData = new byte[1024];
int RecBytes;
int totalrecbytes = 0;
MemoryStream MS = new MemoryStream();
while ((RecBytes = serverStream.Read(RecData, 0, RecData.Length)) > 0) {
MS.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
serverStream.Close();
clientSocket.Close();
clientSocket.EndConnect(ar);
}
catch (Exception ex) {
Console.WriteLine("Exceção: " + ex.ToString());
}
finally {
wh.Close();
}
}
So, how can I send data to server and read the response? (I tried even putting the thread to sleep after sending data, with no luck.)
Thanks in advance.
EDIT:
With some debug messages I discovered that the server do read the "1" byte that was sent, but somehow it gets stuck inside the while loop, like, the server just stops there, no more loops and it does not leave the while loop. I saw that after writing "loop" in console inside the while loop, and writing read bytes also in console. It wrote "loop" once, and the read byte.
This code worries me:
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
You are reading until the client closes the connection (or shuts down sending, which you don't do). But the client only closes when the server has replied. The server reply will never come. It is a deadlock.
Solution: Read a single byte to determine the requests command (b).
Unrelated to the question, your "packetised" sending (NoOfPackets, ...) does not seem to serve any purpose. Just use Stream.Copy to write. TCP does not have packets.
An even better solution would be to abandon your custom TCP protocol and use an HTTP library. All these concerns just go away. There are various smaller problems with your code that are very typical to see in TCP code.
I have implemented my Server application regarding this post here: http://www.codeguru.com/csharp/csharp/cs_network/sockets/article.php/c8781#Client1
Sum up: I am using async Sockets ala BeginAccept(..), BeginReceive(..).
My Server is capable of handling mutliple clients and everything works fine until a client performas two or more synchronous send operation without waiting some time. The client does not get any error and so is not notified, that the server does not get the second message! If the client waits approx. 100ms after the first send operation, everything works fine.
I thought that when i use TCP i can ensure that the server receives the message. (Except there is an exception thrown)!
Could you provide me a solution to fix this.
Here are the WaitForData(..) & OnDataReceive(..) Methods that i implemented in the server
public void WaitForData(MyClient client)
{
try
{
if (pfnCallBack == null)
{
pfnCallBack = new AsyncCallback(OnDataReceived);
}
iarResult = client.Socket.BeginReceive(client.DataBuffer,
0, client.DataBuffer.Length,
SocketFlags.None,
pfnCallBack,
client);
}
catch (SocketException se)
{
MessageBox.Show("SocketException#WaitForData" + se.Message);
}
}
public void OnDataReceived(IAsyncResult asyn)
{
try
{
MyClient user= (MyClient)asyn.AsyncState;
int iRx = user.Socket.EndReceive(asyn);
byte[] receivedData = user.DataBuffer;
MemoryStream memStream = new MemoryStream();
BinaryFormatter binForm = new BinaryFormatter();
memStream.Write(receivedData, 0, receivedData.Length);
memStream.Seek(0, SeekOrigin.Begin);
MyMessage msg = (MyMessage)binForm.Deserialize(memStream);
switch (msg.Command)
{
case (MyMessage.MyCommand.ConnId):
this.connId = (int) msg.MyObject;
tsslConnStatus.Text += " | ID: " + connId.ToString();
break;
case (MyMessage.MyCommand.Text):
MessageBox.Show(msg.MyObject.ToString());
break;
}
WaitForData(server);
}
catch (ObjectDisposedException ode)
{
MessageBox.Show("ObjectDisposedException#OnReceiveData" + ode.Message);
}
catch (SocketException se)
{
MessageBox.Show("SocketException#OnReceiveData" + se.Message);
}
}
The CLIENT calls a synchronous SEND METHOD TWICE or MORE! server INSTANCEOF MyClient
if (server.Socket.Connected)
{
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
bf.Serialize(ms, message);
MyMessage = new MyMessage(something);
server.Socket.Send(ms.ToArray());
}
so, i think this code snippets must be enough for you to get the idea i was trying to use!
If you need further details or code snippets, just tell me i will post it!
Thanx!
TCP is stream based and not message based. One Read can contain any of the following alternatives:
A teeny weeny part of message
A half message
Excactly one message
One and a half message
Two messages
Thus you need to use some kind of method to see if a complete message have arrived. The most common methods are:
Add a footer (for instance an empty line) which indicates end of message
Add a fixed length header containing the length of the message
Update
Simple example having just length as header.
Server side:
var buffer = binaryFormmater.Serialize(myobj);
var length = buffer.Length;
networkStream.Send(length);
networkStream.Send(buffer, 0, buffer.Length);
Client side:
var header = new buffer[4];
// TODO: You need to make sure that 4 bytes have been read.
networkStream.Read(header, 0, 4);
var length = BitConverter.ToInt32(buffer);
var readbuffer= new byte[65535];
var bytesLeft = length;
var messageStream = new MemoryStream();
while (bytesLeft > 0)
{
var read = networkStream.Read(readbuffer, 0, bytesLeft);
messageStream.Write(readbuffer, 0, read);
bytesLeft -= read,
}
messageStream.Seek(0, SeekOrigin.Begin);
MyMessage msg = (MyMessage)binForm.Deserialize(messageStream);