We Keep Coding

C# TCP Socket Stream read wrong bytes

I Read Stream with:
private Stream _Stream;
private Socket _Socket;//TCP Socket
private void ReadStream()
{
if (Guest)
{
var sizeBuffer = ReadBytes(2);//the size buffer is always 2
int size = sizeBuffer[1];
size |= (sizeBuffer[0] << 8);
var data = ReadBytes(size);
string payload = System.Text.Encoding.UTF8.GetString(data, 0, data.Length);
var tokens = SplitPayload(payload);
if (tokens[0] == "nick")
{
SetNick(tokens);
}
else
{
throw new ApplicationException("Set your nick first");
}
}
else if(!Guest)
{
var sizeBuffer = ReadBytes(2);
int size = sizeBuffer[1];
size |= (sizeBuffer[0] << 8);
var data = ReadBytes(size);
string payload = System.Text.Encoding.UTF8.GetString(data, 0, data.Length);
var tokens = SplitPayload(payload);
CheckToken(tokens);
}
if (encrypted == null)
{
encrypted = false;
byte[] sizebuffer = new byte[2];
var size = _Socket.Receive(sizebuffer);
byte[] data = new byte[size];
var datasize = _Socket.Receive(data);
if (data[0] == 0x01)
{
encrypted = true;
_Stream = new NetworkStream(_Socket, true);
var sslStream = new SslStream(_Stream, false);
serverCertificate = new X509Certificate2(Path, "");
sslStream.AuthenticateAsServer(serverCertificate);
_Stream = sslStream;
}
else
{
encrypted = false;
_Stream = new NetworkStream(_Socket, true);
}
}
}
I write to the stream with:
private void SendServerMsg(string[] arguments)
{
var msg = string.Join("\0", arguments);
byte[] data = Encoding.UTF8.GetBytes(msg);
byte[] sizeinfo = new byte[2];
sizeinfo[1] = (byte)data.Length;
sizeinfo[0] = (byte)(data.Length >> 8);
_Stream.Write(sizeinfo, 0, sizeinfo.Length);
_Stream.Write(data, 0, data.Length);
}
my Problem is now:
1. When I make a Breakpoint at the line
_Stream.Write(sizeinfo, 0, sizeinfo.Length);
and step through manualy all work. The TCP _Socket gets the correct bytes,
but with no breakpoint the server stucks in the line
var data = ReadBytes(size);
and the size is more than 5000 but I send not more than 15 bytes (for example: "nick\0test")
hear is the readBytes method
private byte[] ReadBytes(int count)
{
byte[] buffer = new byte[count];
for (var i = 0; i < count; i++)
{
var oneByte = _Stream.ReadByte();
if (oneByte == -1)
{
break;
}
buffer[i] = (byte)oneByte;
}
return buffer;
}

Why I get a SerializationException with BinaryFormatter?

I am working on a .NET application where the server sends JPG compressed images from a webcam to the client via TCP/IP. For serialization/deserialization I am using the BinaryFormatter class. When testing between my Desktop computer (client/Windows 10) and my Laptop (server/Windows 10), everything works fine in the long run. When using a LattePanda (server/Windows 7), I get the following error after approximately 3-5 minutes runtime (I send/receive 30 frames per second):
An unhandled exception of type 'System.Runtime.Serialization.SerializationException' occurred in mscorlib.dll
Additional information: The input stream is not a valid binary format. The starting contents (in bytes) are: 00-00-00-01-00-00-00-FF-FF-FF-FF-01-00-00-00-00-00 ...
Here's a snippet from the server code:
private void distribute(Camera camera, Mat frame) {
if(clientSockets!=null) {
if(clientSockets.Count > 0) {
if(camera.Streaming) {
// compress and encapsulate raw image with jpg algorithm
CameraImage packet = new CameraImage(camera.Id, frame, camera.CodecInfo, camera.EncoderParams);
packet.SystemId = Program.Controller.Identity.Id;
packet.SequenceNumber = curSeqNum;
byte[] content;
using(MemoryStream ms = new MemoryStream()) {
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(ms, packet);
content = ms.ToArray();
}
byte[] payload = new byte[content.Length+4];
// prefix with packet length
Array.Copy(BitConverter.GetBytes(content.Length), 0, payload, 0, 4);
// append payload after length header
Array.Copy(content, 0, payload, 4, content.Length);
// distribute to connected clients
this.distribute(payload);
}
}
}
}
private void distribute(byte[] bytes) {
if(Program.Launched) {
lock(syncobj) {
// distribute to connected clients
for(int i=clientSockets.Count-1; i>=0; i--) {
try {
clientSockets[i].Send(bytes, bytes.Length, SocketFlags.None);
} catch(SocketException) {
clientSockets.RemoveAt(i);
}
}
}
}
}
Here's a snippet from the client code:
private void receive() {
try {
while(running) {
if((available = clientSocket.Receive(buffer, 4, SocketFlags.None)) > 0) {
// receive bytes from tcp stream
int offset = 0;
int bytesToRead = BitConverter.ToInt32(buffer, 0);
byte[] data = new byte[bytesToRead];
while(bytesToRead > 0) {
int bytesReceived = clientSocket.Receive(data, offset, bytesToRead, SocketFlags.None);
offset += bytesReceived;
bytesToRead -= bytesReceived;
}
// deserialize byte array to network packet
NetworkPacket packet = null;
using(MemoryStream ms = new MemoryStream(data)) {
BinaryFormatter bf = new BinaryFormatter();
packet = (NetworkPacket)bf.Deserialize(ms);
}
// deliver network packet to listeners
if(packet!=null) {
this.onNetworkPacketReceived?.Invoke(packet);
}
// update network statistics
NetworkStatistics.getInstance().TotalBytesRtpIn += data.Length;
}
}
} catch(SocketException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ObjectDisposedException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ThreadAbortException ex) {
// allows your thread to terminate gracefully
if(ex!=null) Thread.ResetAbort();
}
}
Any ideas why I get the SerializationException only on one machine and not on the other? Different mscorlib.dll libraries installed? How to check the version of the relevant (?) libraries?

Here is a tweaked version of your answer. Right now if clientSocket.Available < 4 and running == true you have a empty while(true) { } loop. This is going to take up 100% of one cpu core doing no useful work.
Instead of looping doing nothing till you have 4 bytes in the system I/O buffer use the same kind of loop you did for the payload for your message and load it to your byte array for the header. (I have also simplified the loop of reading the payload data to the loop I unusually use.)
private void receive() {
try {
while(running) {
int offset = 0;
byte[] header = new byte[4];
// receive header bytes from tcp stream
while (offset < header.Length) {
offset += clientSocket.Receive(header, offset, header.Length - offset, SocketFlags.None);
}
int bytesToRead = BitConverter.ToInt32(header, 0);
// receive body bytes from tcp stream
offset = 0;
byte[] data = new byte[bytesToRead];
while(offset < data.Length) {
offset += clientSocket.Receive(data, offset, data.Length - offset, SocketFlags.None);
}
// deserialize byte array to network packet
NetworkPacket packet = null;
using(MemoryStream ms = new MemoryStream(data)) {
BinaryFormatter bf = new BinaryFormatter();
packet = (NetworkPacket)bf.Deserialize(ms);
}
// deliver network packet to listeners
if(packet!=null) {
this.onNetworkPacketReceived?.Invoke(packet);
}
// update network statistics
NetworkStatistics.getInstance().TotalBytesRtpIn += data.Length;
}
}
} catch(SocketException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ObjectDisposedException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ThreadAbortException ex) {
// allows your thread to terminate gracefully
if(ex!=null) Thread.ResetAbort();
}
}
However, if you switched from the System.Net.Socket class to the System.Net.TcpClient class you could simplify your code a lot. First, if you don't need TotalBytesRtpIn to be updating you can stop sending the header, it is not needed for the deserialization as BinaryFormatter already has it's length stored as a internal field of the payload. Then all you need to do is get the NetworkStream from the TcpClient and process the packets as they come in.
private TcpClient _client; // Set this wherever you had your original Socket set up.
private void receive() {
try {
using(var stream = _client.GetStream()) {
BinaryFormatter bf = new BinaryFormatter();
while(running) {
#region This part is not needed if you are only going to deserialize the stream and not update TotalBytesRtpIn, make sure the server stops sending the header too!
int offset = 0;
byte[] header = new byte[4];
// receive header bytes from tcp stream
while (offset < header.Length) {
offset += stream.Read(header, offset, header.Length - offset);
}
int bytesToRead = BitConverter.ToInt32(header, 0);
#endregion
packet = (NetworkPacket)bf.Deserialize(stream);
// deliver network packet to listeners
if(packet!=null) {
this.onNetworkPacketReceived?.Invoke(packet);
}
// update network statistics
NetworkStatistics.getInstance().TotalBytesRtpIn += bytesToRead;
}
}
}
//These may need to get changed.
} catch(SocketException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ObjectDisposedException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ThreadAbortException ex) {
// allows your thread to terminate gracefully
if(ex!=null) Thread.ResetAbort();
}
}

As Scott suggested, I replaced the unsafe line that reads the header of the messages with a safer approach:
private void receive() {
try {
while(running) {
if(clientSocket.Available>=4) {
// receive header bytes from tcp stream
byte[] header = new byte[4];
clientSocket.Receive(header, 4, SocketFlags.None);
int bytesToRead = BitConverter.ToInt32(header, 0);
// receive body bytes from tcp stream
int offset = 0;
byte[] data = new byte[bytesToRead];
while(bytesToRead > 0) {
int bytesReceived = clientSocket.Receive(data, offset, bytesToRead, SocketFlags.None);
offset += bytesReceived;
bytesToRead -= bytesReceived;
}
// deserialize byte array to network packet
NetworkPacket packet = null;
using(MemoryStream ms = new MemoryStream(data)) {
BinaryFormatter bf = new BinaryFormatter();
packet = (NetworkPacket)bf.Deserialize(ms);
}
// deliver network packet to listeners
if(packet!=null) {
this.onNetworkPacketReceived?.Invoke(packet);
}
// update network statistics
NetworkStatistics.getInstance().TotalBytesRtpIn += data.Length;
}
}
} catch(SocketException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ObjectDisposedException ex) {
onNetworkClientDisconnected?.Invoke(agent.SystemId);
} catch(ThreadAbortException ex) {
// allows your thread to terminate gracefully
if(ex!=null) Thread.ResetAbort();
}
}
Now it's running flawlessly :) Thanks a lot for your help!

How can I make each flush() work, which make each buffer sent before next write() in C#

I am trying to sent very screenshot when i move the mouse on the screen. Therefore, I will sent the size of screenshot and image byte[]. In my client side, I will set the size byte array by the size received from server, and read the rest of byte[] as image. Ideally, I need each time I call the write(), the byte[] I want write into should be empty, so I use the flush(). In fact, after I received the getinputstream(), Sometime I will get the old data inside it. I don't know why flush() doesn't work. anyone has better ideas to solve this problem?
This is my code for server, I try use stream, and networkstream, it dosent works.
public void send(byte[] imageByte)
{
Stream stream = client.GetStream();
client.NoDelay = true;
client.Client.NoDelay = true;
Debug.WriteLine("noDelay:{0}, client.noDelay:{1}", client.NoDelay, client.Client.NoDelay);
byte[] imageSize;
string imgSize = Convert.ToString(imageByte.Length);
Debug.WriteLine("string=" + imgSize);
imageSize = Encoding.ASCII.GetBytes(imgSize);
Debug.WriteLine(" header size:" + imageSize.Length);
//ns.Write(imageSize, 0, imageSize.Length);
//ns.Flush();
//sleep 500 ms
stream.Write(imageSize, 0, imageSize.Length);
stream.Flush();
Thread.Sleep(150);
Debug.WriteLine("sent size");
//ns.Write(imageByte, 0, imageByte.Length);
//ns.Flush();
stream.Write(imageByte, 0, imageByte.Length);
stream.Flush();
Debug.WriteLine("First time sent image size:" + imageByte.Length);
client side:
public class connection extends AsyncTask {
private byte[] data;
public void setByteSize(int size) {
data = new byte[size];
}
public byte[] getByteSize() {
return data;
}
#Override
protected Object doInBackground(Object... arg0) {
try {
clientSocket = new Socket("134.129.125.126", 8080);
// clientSocket = new Socket("134.129.125.172", 8080);
System.out.println("client connect to server");
input = clientSocket.getInputStream();
System.out.println("getinputstream");
} catch (UnknownHostException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
while (true) {
int totalBytesRead = 0;
int chunkSize = 0;
int tempRead = 0;
String msg = null;
// byte[] data = null;
byte[] tempByte = new byte[1024 * 1024 * 4];
try {
// read from the inputstream
tempRead = input.read(tempByte);
// tempbyte has x+5 byte
System.out.println("Fist time read:" + tempRead);
// convert x+5 byte into String
String message = new String(tempByte, 0, tempRead);
msg = message.substring(0, 5);
int msgSize = Integer.parseInt(msg);
data = new byte[msgSize];
// for(int i =0;i<tempRead-5;i++)
// {
// data[i]=tempByte[i+5];
// }
for (int i = 0; i < msgSize; i++) {
data[i] = tempByte[i + 5];
}
// cut the header into imageMsg
// String imageMsg = new String(tempByte, 5, tempRead - 5);
// convert string into byte
System.out.println("message head:" + msg);
byteSize = Integer.parseInt(msg);
System.out.println("ByteSize:" + byteSize);
// convert String into byte[]
totalBytesRead = tempRead - 5;
System.out.println("total Byte Read=" + totalBytesRead);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
// get string for the size of image
try {
while (totalBytesRead != getByteSize().length) {
System.out.println("data length:"
+ getByteSize().length);
chunkSize = input.read(getByteSize(), totalBytesRead,
getByteSize().length - totalBytesRead);
System.out.println("chunkSize is " + chunkSize);
totalBytesRead += chunkSize;
// System.out.println("Total byte read "
// + totalBytesRead);
}
System.out.println("Complete reading - total read = "
+ totalBytesRead);
} catch (Exception e) {
}
bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);
System.out.println("deco");
runOnUiThread(new Runnable() {
public void run() {
image.setImageBitmap(bitmap);
System.out.println("setImage at less than 500");
}
});
}
}
}

Delay 3mins at the start of the program why? socket programming in C#

I will get 3mins of delay at the start of the below program.but where as in hyper terminal it shows every data for every 2 secs.
Her is my below code.please let me know where i went wrong .How i can rectify this delay. any suggestion?
private void StartReceiving()
{
// coding part of receiving changed.
try
{
string IPStr = textBox1_IPaddress.Text.Trim();
string portStr = textBox2_Port.Text.Trim();
int port = Convert.ToInt32(portStr);
int bytesRead = 0;
byte[] buffer = new byte[9];
//------------------------------------------------------------------
IPAddress ipAddress = System.Net.IPAddress.Parse(IPStr);
//create server's tcp listener for incoming connection
try
{
textBox3.Visible = true;
client = new TcpClient();
client.Connect(IPStr, port);
ns = client.GetStream();
while (true)
{
if (ns.DataAvailable)
{
byte[] data = ReadNumberBytes(ns, 9);
ASCIIEncoding encoder = new ASCIIEncoding();
msg = encoder.GetString(data);
textBox3.AppendText(msg);
textBox3.AppendText("\n");
GetData(msg);
}
else
{
Thread.Sleep(4000);
byte[] data = ReadNumberBytes(ns, 9);
ASCIIEncoding encoder = new ASCIIEncoding();
msg = encoder.GetString(data);
GetData(msg);
}
}
client.Close();
}
catch (SocketException se)
{
//message box;
}
}
}
public static byte[] ReadNumberBytes(NetworkStream stream, int n)
{
byte[] buffer = new byte[n];
int bytesRead = 0;
int chunk;
while (bytesRead < n)
{
chunk = stream.Read(buffer, (int)bytesRead, buffer.Length - int)bytesRead);
if (chunk == 0)
{
// error out.
throw new Exception("Unexpected disconnect");
}
bytesRead += chunk;
}
return buffer;
}

TCP server client issue

First i am n00b in socket programming. So i decided to write simple data over lan tcp server
My server code that handles incomming data is
private void HandleClientComm(object client)
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
clientStream.ReadTimeout = 10;
int size = 4096 * 1000;
byte[] message = new byte[size];
byte[] All = new byte[0];
int bytesRead;
string error = "";
lock (this)
{
while (true)
{
All = new byte[0];
while (true)
{
bytesRead = 0;
try
{
bytesRead = clientStream.Read(message, 0, size);
All = AddBArrays(All, message, bytesRead);
}
catch
{
break;
}
if (bytesRead == 0)
{
break;
}
}
if (All.Length > 0)
{
Message m = (Message)Tools.ByteArrayToObject(All);
OnRecived(new RecivedArgs("localhost", (Message)Tools.ByteArrayToObject(All)));
}
}
tcpClient.Close();
}
}
byte[] AddBArrays(byte[] ar1, byte[] ar2, int read)
{
byte[] concat = new byte[ar1.Length + read];
if (ar1.Length != 0)
System.Buffer.BlockCopy(ar1, 0, concat, 0, ar1.Length);
System.Buffer.BlockCopy(ar2, 0, concat, ar1.Length, read);
return concat;
}
it works but have some issues. It fales receiving files bigger then 100 mbs or smthng and also if i send data very often interval < 800 then data is lost. how should i improve my code? The large file issue is not so important the primary issue is the data loss in fast data sending.
tnx for help
Ok i now updated the code by the suggestions
private void HandleClientComm(object client)
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
clientStream.ReadTimeout = 10;
int size = 4096 * 1000;
List<byte> Test = new List<byte>();
byte[] message = new byte[size];
byte[] All = new byte[0];
int bytesRead;
while (true)
{
//All = new byte[0];
while (true)
{
bytesRead = 0;
try
{
bytesRead = clientStream.Read(message, 0, size);
for (int i = 0; i < bytesRead; i++)
{
Test.Add(message[i]);
}
}
catch
{
break;
}
if (bytesRead == 0)
{
break;
}
}
if (Test.Count > 0)
{
Message m = (Message)Tools.ByteArrayToObject(Test.ToArray());
OnRecived(new RecivedArgs("localhost", m));
Test = new List<byte>();
}
}
tcpClient.Close();
}
but the issues still there
Edit--> large file issue fixed it was just a 'System.OutOfMemoryException' but it didn't throw a error.

The All byte array you should change to a List<byte>. You are creating instances like crazy right now. The GC is probably working a lot more than it needs to. This might be slowing it down so much that it can't keep up.
Not really related to sockets:
Make size a const
NEVER lock this. Create a private field that you can lock. In fact, I don't even think you need a lock here.
remove the error string.

OK i solved the problem.
I simple send to much data to fast so data loss was unavoidable.
My optimized code is
private void HandleClientComm(object client)
{
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
int size = 1;
byte[] message = new byte[1];
int bytesRead;
while (true)
{
bytesRead = 0;
if (clientStream.DataAvailable)
bytesRead = clientStream.Read(message, 0, 1);
if (bytesRead > 0)
{
OnRecived(new RecivedArgs("tick", null));
}
Thread.Sleep(1);
}
}
i have tested intervals as low as 1 ms and no data loss :)
thanks for your help