I am writing what is essentially an image backup server to store images. It is a one way service that will not return anything beyond a basic success or failure message to the client.
The issue that I am experienceing is that when I send a byte array through the network stream, it is being cut-off before the end of the stream at random locations. I do not have this issue when I run the server on my development machine and connect locally, but rather it only occurs when the server is deployed on a remote server.
When I send very small arrays ( < 512 bytes) the server recieves the entire stream successfully, but on streams larger than 2000 bytes I experience issues. The code for the client is as follows:
try
{
TcpClient Voice = new System.Net.Sockets.TcpClient();
//Obviously I use the remote IP when it is deployed - but have altered it for privacy.
IPEndPoint BackupServer = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 57000);
Voice.Connect(BackupServer);
NetworkStream DataStream = Voice.GetStream();
byte[] buffer = new ASCIIEncoding().GetBytes(ImageData.GetXml());
DataStream.Write(buffer, 0, buffer.Length);
DataStream.Flush();
}
catch
{
}
try
{
buffer = new byte[4096];
int read = DataStream.Read(buffer, 0, buffer.Length);
MessageBox.Show(new ASCIIEncoding().GetString(buffer) + " : " + read.ToString());
}
catch
{
}
The client code executes without any errors or problems regardless of the size of data I send.
And the code for the server side is as follows:
private void BackMeUp(object voice)
{
TcpClient Voice = (TcpClient)voice;
Voice.ReceiveTimeout = 30000;
NetworkStream DataStream = Voice.GetStream();
try
{
bool ShouldLoop = true;
//int loops = 0;
int loops = -1;
byte[] input = new byte[2048];
byte[] buffer = new byte[0];
//while (ShouldLoop)
while(loops != 0)
{
loops = DataStream.Read(input, 0, 2048);
for (int x = 0; x < loops; x++)
{
Array.Resize(ref buffer, buffer.Length + 1);
buffer[buffer.Length - 1] = input[x];
}
//if (loops < 2048)
//{
//ShouldLoop = false;
//break;
//}
}
while (true)
{
StringReader Reader = new StringReader(new ASCIIEncoding().GetString(buffer, 0, buffer.Length));
DataSet DS = new DataSet();
DS.ReadXml(Reader);
if (DS.Tables.Count > 0)
{
if (DS.Tables["Images"].Rows.Count > 0)
{
foreach (DataRow row in DS.Tables["Images"].Rows)
{
//
}
}
}
string response = "Got it!";
DataStream.Write(new ASCIIEncoding().GetBytes(response), 0, response.Length);
DataStream.Flush();
Voice.Close();
break;
}
}
catch (Exception Ex)
{
File.WriteAllText("Elog.txt", Ex.Message + " " + (Ex.InnerException != null ? Ex.InnerException.ToString() : " no Inner"));
Voice.Close();
}
}
The server recieves the data fine, and closes the stream when it reaches the end, however the data is cut-off and I get an error when I try to rebuild the dataset.
I have the impression this has to do with the time it takes to send the stream, and I have played around with the Close and Flush commands but I feel like I'm just shooting in the dark. Any help would be appreciated.
Concise version of question: What factors are involved with a TcpListener that could cause a) the truncation of the stream. or b) premature closing of the stream prior to all bytes being read. When the listener in question is on a remote host rather than a local server.
The Read method doesn't have to return the number of bytes that you requested, or the entire stream at once. Especially if the stream is slow, it will be returned in small chunks.
Call the Read method repeatedly, and handle the data for each block that you get. The Read method returns zero when the stream is read to the end:
buffer = new byte[4096];
do {
int read = DataStream.Read(buffer, 0, buffer.Length);
if (read != 0) {
// handle the first "read" bytes of the buffer (index 0 to read-1)
}
} while (read != 0);
If you know that your buffer is enough for any stream, you can fill up the buffer and handle it afterwards:
buffer = new byte[4096];
int offset = 0;
do {
int read = DataStream.Read(buffer, offset, buffer.Length - offset);
offset += read;
} while (read != 0);
// handle the first "offset" bytes of the buffer (index 0 to offset-1)
Related
I wrote a tcp listener to receive a sequence of images from single client , this is the code of server :
new Thread(() =>
{
while (true)
{
displayingFrame.Start(); // another thread to display images
Socket socket = listener.AcceptSocket();
TcpClient client = new TcpClient();
client.Client = socket;
Debug.WriteLine("Connection accepted.");
var childSocketThread = new Thread(() =>
{
NetworkStream ns = client.GetStream();
BinaryReader br = new BinaryReader(ns);
while (true)
{
using (MemoryStream ms = new MemoryStream())
{
int length = br.ReadInt32();
Debug.WriteLine("length : " + length);
byte[] buf = new byte[1024];
int totalReaded = 0;
int readed = 0;
while (totalReaded < length)
{
readed = br.Read(buf, 0, buf.Length);
ms.Write(buf, 0, readed);
totalReaded += readed;
}
byte[] frame = ms.ToArray();
this.frames.Enqueue(frame);
Debug.WriteLine("frame enqueued with length " + frame.Length);
}
}
});
childSocketThread.Start();
}
}).Start();
it receive frames very well but suddenly br.ReadInt32(); returns a very big length so br.Read(buf, 0, buf.Length); takes a very long time writing to memory stream and it writes a wrong data inside frame .
this is the client :
TcpClient client = new TcpClient();
client.Connect(new IPEndPoint(IPAddress.Loopback, 20000));
NetworkStream ns = client.GetStream();
BinaryWriter bw = new BinaryWriter(ns);
while ( true )
{
byte[] frame = Screenshot();
bw.Write(frame.Length);
Console.WriteLine("a frame length has flushed : " + frame.Length);
bw.Write(frame);
Console.WriteLine("a frame itself has flushed");
}
Console.ReadKey();
and here the debug info :
If you check the hex value you're getting - 1196314761 - you'll get 0x474E5089 and finally convert to ASCII you will get GNP\x89 which gives us the known magic value \x89PNG that is the marker of PNG file. You're actually reading the content of your screenshot as the length.
Make sure that you're code for reading the data does not read too much from the previous frame. I think you code for reading the data does not include the fact that you might get content of 2 frames in one .Read but then later you just don't care if you have too much data. You only check if it's not less than length.
I'm sending data to Lector device.
Normally I received data from device when I sending on hercules.
Hercules is returning "sRA eExtIn1 0 0 0".
The below code has waiting line stream.Read() function.
How can I getting data from device?
string responseData = null;
using (TcpClient client = new TcpClient("10.1.13.102", 2111))
{
using (NetworkStream stream = client.GetStream())
{
byte[] sentData = System.Text.Encoding.ASCII.GetBytes("<STX>sRN eExtIn1<ETX>");
stream.Write(sentData, 0, sentData.Length);
byte[] buffer = new byte[32];
int bytes;
if (client.Connected)
{
while ((bytes = stream.Read(buffer, 0, buffer.Length)) != 0)
{
for (int i = 0; i < bytes; i++)
{
responseData += (char)buffer[i];
}
}
}
}
}
The mistake you're making, and the other answer is also making, is assuming that stream.Read won't return until it has read 32 bytes. That is incorrect.
https://msdn.microsoft.com/en-us/library/system.net.sockets.networkstream.read(v=vs.110).aspx
This method reads data into the buffer parameter and returns the
number of bytes successfully read. If no data is available for
reading, the Read method returns 0. The Read operation reads as much
data as is available, up to the number of bytes specified by the size
parameter.
It will return when there is no data available to read or 32 bytes have been read, whichever comes first. So if, for example, the client is slow or the network very busy, the response may not have arrived yet when you call stream.Read. Consequently, there will be nothing to read so it will return 0 and you will exit, failing to read the data. In fact, you may have to call stream.Read any number of times to get the full 32 bytes if the network is very saturated and data is arriving a few bytes at a time (not likely with such a small packet, but you have to code it that way).
So your code needs to look like this (note the additional while loop):
using (TcpClient client = new TcpClient("10.1.13.102", 2111))
{
using (NetworkStream stream = client.GetStream())
{
byte[] sentData = System.Text.Encoding.ASCII.GetBytes("<STX>sRN eExtIn1<ETX>");
stream.Write(sentData, 0, sentData.Length);
byte[] buffer = new byte[32];
int bytes;
if (client.Connected)
{
int bytesRead = 0;
while (bytesRead < buffer.Length)
{
while ((bytes = stream.Read(buffer, 0, buffer.Length)) != 0)
{
for (int i = 0; i < bytes; i++)
{
responseData += (char)buffer[i];
}
bytesRead += bytes;
}
}
}
}
}
Thanks everybody.
I found solution of my question.
and tags should be describe as bytes. Like below.
byte[] byt = System.Text.Encoding.ASCII.GetBytes("sRN DItype");
stream.Write(STX, 0 , 1);
stream.Write(byt, 0, byt.Length);
stream.Write(ETX, 0, 1);
stream.Read(buffer, 0, buffer.Length);
In C/C++ read() on regular non-blocking network sockets will return immediately with the amount of data currently available in the buffer, up to the amount specified (so if we ask for 16 bytes and there are only 8 available at the moment, those 8 we'll get and it's up to us to call read() again and fetch all data).
In C# there's NetworkStream, which has built-in timeouts - does this mean that NetworkStream.Read() waits until either the timeout is reached or the amount of data requested is read, or will it give us any amount of data currently available in the buffer larger than 0 up to the amount requested (as the standard sockets do) even if there's time left?
It will read available data up to the number of bytes specified in the parameters, as described on MSDN, unless the stream is closed via timeout or other exception.
The Read operation reads as much data as is available, up to the number of bytes specified by the size parameter. If the remote host shuts down the connection, and all available data has been received, the Read method completes immediately and return zero bytes.
I solved it like this:
byte[] TotalData = new byte[0];
byte[] TempData = new byte[0];
using (TcpClient TCPClient = new TcpClient())
{
try
{
TCPClient.Connect(somehost, someport);
}
catch (Exception eee)
{
// Report the connection failed in some way if necessary
}
if (TCPClient.Connected)
{
using (NetworkStream clientStream = TCPClient.GetStream())
{
// You can reduce the size of the array if you know
// the data received is going to be small,
// don't forget to change it a little down too
byte[] TCPBuffer = new byte[524288];
int bytesRead = 0;
int loop = 0;
// Wait for data to begin coming in for up to 20 secs
while (!clientStream.DataAvailable && loop< 2000)
{
loop++;
Thread.Sleep(10);
}
// Keep reading until nothing comes for over 1 sec
while (clientStream.DataAvailable)
{
bytesRead = 0;
try
{
bytesRead = clientStream.Read(TCPBuffer, 0, 524288);
Array.Resize(ref TempData, bytesRead);
Array.Copy(TCPBuffer, TempData, bytesRead);
// Add data to TotalData
TotalData = JoinArrays(TotalData, TempData);
}
catch
{
break;
}
if (bytesRead == 0)
break;
Thread.Sleep(1000);
}
}
}
}
The JoinArrays() method:
byte[] JoinArrays(byte[] arrayA, byte[] arrayB)
{
byte[] outputBytes = new byte[arrayA.Length + arrayB.Length];
Buffer.BlockCopy(arrayA, 0, outputBytes, 0, arrayA.Length);
Buffer.BlockCopy(arrayB, 0, outputBytes, arrayA.Length, arrayB.Length);
return outputBytes;
}
The safe method is to use MemoryStream wich will make sure to wait and read all the stream to the memory , then u can use it as you like
public void SaveUserTemplate(Stream stream)
{
MemoryStream memoryStream = new MemoryStream();
stream.CopyTo(memoryStream);
memoryStream.Position = 0;
byte[] templatePathLength = new byte[4];
memoryStream.Read(templatePathLength, 0, templatePathLength.Length);
int nBytesTemplatePathLength = BitConverter.ToInt32(templatePathLength,0);
....
CopyTo function finally calls to this function:
github.com/microsoft/referencesource
private void InternalCopyTo(Stream destination, int bufferSize)
{
Contract.Requires(destination != null);
Contract.Requires(CanRead);
Contract.Requires(destination.CanWrite);
Contract.Requires(bufferSize > 0);
byte[] buffer = new byte[bufferSize];
int read;
while ((read = Read(buffer, 0, buffer.Length)) != 0)
destination.Write(buffer, 0, read);
}
In my client/server application my client wiil communicate with the server for 2 functions: the client will either request data from the server or it will send data so the server will save it. I'm using one socket for both methods, and the method to be used is defined by the first byte sent. If the first byte is "1" it is requesting data. If it is "2", it will send data (data bytes are sent after the "2" byte). It works perfectly for sending data. But when I'm requesting data it works, as long as I don't read the socket stream in the client. It's like if I make the client read data after sending data, the server will have no data to read, and it just crashes when trying to read the data.
Here is my server code:
private const int BufferSize = 1024;
NetworkStream netstream = null;
byte[] RecData = new byte[BufferSize];
int RecBytes;
try {
netstream = clientSocket.GetStream();
int totalrecbytes = 0;
using (MemoryStream ms = new MemoryStream()) {
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
byte[] bytes = ms.ToArray();
byte b = bytes[0];
switch (b) {
case 1:
//Here I gather data and put it in "stream" variable
byte[] SendingBuffer = null;
int NoOfPackets = Convert.ToInt32(Math.Ceiling(Convert.ToDouble(stream.Length) / Convert.ToDouble(BufferSize)));
int TotalLength = (int)stream.Length, CurrentPacketLength, counter = 0;
for (int i = 0; i < NoOfPackets; i++) {
if (TotalLength > BufferSize) {
CurrentPacketLength = BufferSize;
TotalLength = TotalLength - CurrentPacketLength;
}
else
CurrentPacketLength = TotalLength;
SendingBuffer = new byte[CurrentPacketLength];
stream.Read(SendingBuffer, 0, CurrentPacketLength);
netstream.Write(SendingBuffer, 0, (int)SendingBuffer.Length);
}
netstream.Flush();
}
catch (Exception e) {
Console.WriteLine("EXCEPTION:\n" + e.ToString());
}
break;
case 2:
//Code to read data
break;
}
}
netstream.Close()
clientSocket.Close();
And here is my client code:
using (System.Net.Sockets.TcpClient clientSocket = new System.Net.Sockets.TcpClient()) {
string returnData = "";
IAsyncResult ar = clientSocket.BeginConnect("127.0.0.1", 8080, null, null);
System.Threading.WaitHandle wh = ar.AsyncWaitHandle;
try {
if (!ar.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(5), false)) {
clientSocket.Close();
Console.WriteLine("Timeout");
return;
}
System.Net.Sockets.NetworkStream serverStream = clientSocket.GetStream();
byte b = 1;
byte[] outStream = { b };
serverStream.Write(outStream, 0, outStream.Length);
serverStream.Flush();
//If I comment following lines, the server can read sent data, but server can't otherwise
byte[] RecData = new byte[1024];
int RecBytes;
int totalrecbytes = 0;
MemoryStream MS = new MemoryStream();
while ((RecBytes = serverStream.Read(RecData, 0, RecData.Length)) > 0) {
MS.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
serverStream.Close();
clientSocket.Close();
clientSocket.EndConnect(ar);
}
catch (Exception ex) {
Console.WriteLine("Exceção: " + ex.ToString());
}
finally {
wh.Close();
}
}
So, how can I send data to server and read the response? (I tried even putting the thread to sleep after sending data, with no luck.)
Thanks in advance.
EDIT:
With some debug messages I discovered that the server do read the "1" byte that was sent, but somehow it gets stuck inside the while loop, like, the server just stops there, no more loops and it does not leave the while loop. I saw that after writing "loop" in console inside the while loop, and writing read bytes also in console. It wrote "loop" once, and the read byte.
This code worries me:
//When I get here, there is no data to read
while ((RecBytes = netstream.Read(RecData, 0, RecData.Length)) > 0) {
ms.Write(RecData, 0, RecBytes);
totalrecbytes += RecBytes;
}
You are reading until the client closes the connection (or shuts down sending, which you don't do). But the client only closes when the server has replied. The server reply will never come. It is a deadlock.
Solution: Read a single byte to determine the requests command (b).
Unrelated to the question, your "packetised" sending (NoOfPackets, ...) does not seem to serve any purpose. Just use Stream.Copy to write. TCP does not have packets.
An even better solution would be to abandon your custom TCP protocol and use an HTTP library. All these concerns just go away. There are various smaller problems with your code that are very typical to see in TCP code.
First i am n00b in socket programming. So i decided to write simple data over lan tcp server
My server code that handles incomming data is
private void HandleClientComm(object client)
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
clientStream.ReadTimeout = 10;
int size = 4096 * 1000;
byte[] message = new byte[size];
byte[] All = new byte[0];
int bytesRead;
string error = "";
lock (this)
{
while (true)
{
All = new byte[0];
while (true)
{
bytesRead = 0;
try
{
bytesRead = clientStream.Read(message, 0, size);
All = AddBArrays(All, message, bytesRead);
}
catch
{
break;
}
if (bytesRead == 0)
{
break;
}
}
if (All.Length > 0)
{
Message m = (Message)Tools.ByteArrayToObject(All);
OnRecived(new RecivedArgs("localhost", (Message)Tools.ByteArrayToObject(All)));
}
}
tcpClient.Close();
}
}
byte[] AddBArrays(byte[] ar1, byte[] ar2, int read)
{
byte[] concat = new byte[ar1.Length + read];
if (ar1.Length != 0)
System.Buffer.BlockCopy(ar1, 0, concat, 0, ar1.Length);
System.Buffer.BlockCopy(ar2, 0, concat, ar1.Length, read);
return concat;
}
it works but have some issues. It fales receiving files bigger then 100 mbs or smthng and also if i send data very often interval < 800 then data is lost. how should i improve my code? The large file issue is not so important the primary issue is the data loss in fast data sending.
tnx for help
Ok i now updated the code by the suggestions
private void HandleClientComm(object client)
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
clientStream.ReadTimeout = 10;
int size = 4096 * 1000;
List<byte> Test = new List<byte>();
byte[] message = new byte[size];
byte[] All = new byte[0];
int bytesRead;
while (true)
{
//All = new byte[0];
while (true)
{
bytesRead = 0;
try
{
bytesRead = clientStream.Read(message, 0, size);
for (int i = 0; i < bytesRead; i++)
{
Test.Add(message[i]);
}
}
catch
{
break;
}
if (bytesRead == 0)
{
break;
}
}
if (Test.Count > 0)
{
Message m = (Message)Tools.ByteArrayToObject(Test.ToArray());
OnRecived(new RecivedArgs("localhost", m));
Test = new List<byte>();
}
}
tcpClient.Close();
}
but the issues still there
Edit--> large file issue fixed it was just a 'System.OutOfMemoryException' but it didn't throw a error.
The All byte array you should change to a List<byte>. You are creating instances like crazy right now. The GC is probably working a lot more than it needs to. This might be slowing it down so much that it can't keep up.
Not really related to sockets:
Make size a const
NEVER lock this. Create a private field that you can lock. In fact, I don't even think you need a lock here.
remove the error string.
OK i solved the problem.
I simple send to much data to fast so data loss was unavoidable.
My optimized code is
private void HandleClientComm(object client)
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
int size = 1;
byte[] message = new byte[1];
int bytesRead;
while (true)
{
bytesRead = 0;
if (clientStream.DataAvailable)
bytesRead = clientStream.Read(message, 0, 1);
if (bytesRead > 0)
{
OnRecived(new RecivedArgs("tick", null));
}
Thread.Sleep(1);
}
}
i have tested intervals as low as 1 ms and no data loss :)
thanks for your help