We Keep Coding

async / await TCP listener saving images from cameras

Hi guys I have a big problem: I have createad a windows form that worked as a Server listening for TCP messages sended by some cameras connected to the same private network.
Each camera send a tcp message to the server when the camera detects something, the server has to work continuosly without freezing the gui interface and has to process every TCP message. The processing of this data includes the saving of an image shooted by the sender camera. The server saves these images into a folder and here there is my problem:
the server is not able to save correclty each image: it looks like that some bytes have been lost during the trasmission but I think that all the bytes have riched the server, something deeper has happend. Maybe it could be the way that I have programmed the async / await server?
I have a list of TcpListener because I have to use more server, one for each tab.
Here how the saved images look like
https://imgur.com/xtlgHPk
https://imgur.com/CcvWbDH
As you can see is not completely saved, except someone for some unkown reason
https://imgur.com/G25UPSS
public void TcpServer(int port)
{
IPAddress ipAddress = null;
string hostName = Dns.GetHostName();
IPHostEntry ipHostInfo = Dns.GetHostEntry(hostName);
for (int i = 0; i < ipHostInfo.AddressList.Length; ++i)
{
if (ipHostInfo.AddressList[i].AddressFamily ==
AddressFamily.InterNetwork)
{
ipAddress = ipHostInfo.AddressList[i];
_listener.Add( new TcpListener(ipAddress, port));
ReceiveDataAsync();
break;
}
}
if (ipAddress == null)
throw new Exception("No IPv4 address for server");
}
private async void ReceiveDataAsync()
{
try
{
_listener[tbServer.SelectedIndex].Start();
while (true)
{
var tcpClient = await _listener[tbServer.SelectedIndex].AcceptTcpClientAsync();
ReadDataFromClientAsync(tcpClient);
}
}
catch (Exception e)
{
MessageBox.Show("Errore: ", e.Message.ToString());
}
}
private async Task ReadDataFromClientAsync(TcpClient client)
{
try
{
using (NetworkStream stream=client.GetStream())
{
int selectedTabIndex, indexOfPort;
string endPoint;
while (client.Connected)
{
int count = 0;
var countBytes = new byte[4];
for (int i = 0; i < 6; i++)
{
count = await stream.ReadAsync(countBytes, 0, countBytes.Length);
}
//The data dimension of the TCP message is into his header, 24th byte.
if (count == 0)
{
break;
}
byte[] bytes = new byte[BitConverter.ToUInt32(countBytes, 0)];
await stream.ReadAsync(bytes, 0, bytes.Length);
indexOfPort = Settings.getIndexOfPort(client.Client.LocalEndPoint.ToString());
endPoint = client.Client.RemoteEndPoint.ToString();
selectedTabIndex = Settings.getIndexOfPort(client.Client.LocalEndPoint.ToString());
updateGui("entry", indexOfPort,endPoint);
BufferData bufferService = new BufferData();
CameraMessage cameraMessage = await bufferService.writeBufferData(bytes.ToList());
if (cameraMessage == null)
return;
//doing some stuff
msgToShow = await bufferService.SaveImageFromCamera(cameraMessage);
//doing other stuff
}
client.Close();
closeCommunication(selectedTabIndex,indexOfPort,endPoint);
}
}
}
catch (IOException e)
{
updateGui("stop");
}
}
The BufferData class:
public class BufferData
{
public async Task<CameraMessage> writeBufferData(List<byte> bytesList)
{
CameraMessage cameraMessage = new CameraMessage();
try
{
int num = 0;
int startSubData = 0;
for (int startData = 0; startData < bytesList.Count-8; startData = startSubData + num)
{
int codeDataMessage = BitConverter.ToInt32(bytesList.GetRange(startData, 4).ToArray(), 0);
int indexDataSize = startData + 4;
int SizeDataMessage = BitConverter.ToInt32(bytesList.GetRange(indexDataSize, 4).ToArray(), 0);
startSubData = indexDataSize + 4;
byte[] array = bytesList.GetRange(startSubData, SizeDataMessage).ToArray();
num = 0;
switch (codeDataMessage)
{
case 14020:
cameraMessage.image = await this.Base64ToImage(array);
num = this.OffSetStringType(SizeDataMessage);
break;
}
}
return cameraMessage;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message.ToString());
return null;
}
}
public async Task<string> SaveImageFromCamera( CameraMessage cameraMessage)
{
string path = Settings.pathFolder + cameraMessage.ld_I_SN + #"\Images\";
string res;
if (!Directory.Exists(Settings.pathFolder+cameraMessage.ld_I_SN))
{
Directory.CreateDirectory(Settings.pathFolder + cameraMessage.ld_I_SN + #"\Images");
}
try
{
await Task.Run(() => { cameraMessage.image.Save(path + cameraMessage.ld_I_FILENAME, ImageFormat.Jpeg); });
res = "#3 - OK - IMMAGINE SALVATA CON SUCCESSO";
cameraMessage.image.Dispose();
return res;
}
catch (Exception ex)
{
res = "#3 - ERR - IMMAGINE NON SALVATA";
return res;
}
}
public async static Task<Image> Base64ToImage(byte[] imageBytes)
{
MemoryStream ms = new MemoryStream(imageBytes, 0, imageBytes.Length);
ms.Write(imageBytes, 0, imageBytes.Length);
return System.Drawing.Image.FromStream(ms, true);
}
}

Your core problem is that you're assuming a socket read will read all the bytes you ask it to. That's not how socket reads work.
When you call await stream.ReadAsync(bytes, 0, bytes.Length), the stream will read any number of bytes from 1 to bytes.Length. It will return the number of bytes actually read. You need to take this into consideration and read again as long as your buffer hasn't been read completely.
Note that this has to be done both for the header bytes and the payload bytes.

Sending and receiving files keeps hanging

my program should be able to send and receive files but for some reason whenever i click on the send (button1) and receive (button2) buttons, it keeps hanging. Not sure if its something wrong my my codes? Also, I feel like my codes are pretty long as compared to other examples i found online but I'm not sure how to rectify.
Client codes
Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
IPEndPoint remoteEP = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9000);
private void Form1_Load(object sender, EventArgs e)
{
try
{
socket.Connect(remoteEP);
textBox2.Text = "Connected to Server";
}
catch (Exception ex)
{
textBox2.Text = "Unable to connect to Server";
textBox2.Text = ex.Message;
}
}
public const string SEND = "[SEND]";
public const string RECEIVE = "[RECEIVE]";
public const string QUIT = "[QUIT]";
private void button1_Click(object sender, EventArgs e)
{
textBox1.Clear();
textBox2.Clear();
NetworkStream stream = new NetworkStream(socket);
StreamReader reader = new StreamReader(stream);
StreamWriter writer = new StreamWriter(stream);
try
{
writer.WriteLine(RECEIVE);
writer.Flush();
writer.WriteLine(textBox1.Text);
writer.Flush();
Bitmap bmp = new Bitmap(#"C:\Users\Y400\Desktop\Lectures\Year 3\WAD\Week 11" + textBox1.Text);
MemoryStream ms = new MemoryStream();
bmp.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
byte[] bmpBytes = ms.GetBuffer();
bmp.Dispose();
ms.Close();
int sent;
sent = sendData(socket, bmpBytes);
textBox1.Text = "Transferring file complete\r\n";
textBox1.Text += bmpBytes.Length + " bytes sent to Server.";
}
catch (Exception ex)
{
textBox2.Text = ex.Message;
}
}
public static int sendData (Socket s, byte[] data)
{
int total = 0;
int size = data.Length;
int left = size;
int sent;
byte[] datasize = new byte[4];
datasize = BitConverter.GetBytes(size);
sent = s.Send(datasize);
while(total<size)
{
sent = s.Send(data, total, left, SocketFlags.None);
total += sent;
left -= sent;
}
return total;
}
private void button2_Click(object sender, EventArgs e)
{
textBox2.Clear();
textBox1.Clear();
byte[] data = new byte[1024];
string fileN = textBox2.Text.Trim();
NetworkStream ns = new NetworkStream(socket);
StreamReader reader = new StreamReader(ns);
StreamWriter writer = new StreamWriter(ns);
writer.WriteLine(SEND);
writer.Flush();
writer.WriteLine(fileN);
writer.Flush();
try
{
while (true)
{
data = receiveData(socket);
MemoryStream ms = new MemoryStream(data);
break;
}
textBox2.Text = ("Receiving file from server ...\r\n" + data.Length + " bytes copied");
}
catch (Exception ex)
{
textBox2.Text = ex.Message;
}
}
public static byte[] receiveData (Socket s)
{
int total = 0;
int recv;
byte[] datasize = new byte[4];
recv = s.Receive(datasize, 0, 4, 0);
int size = BitConverter.ToInt32(datasize, 0);
int dataleft = size;
byte[] data = new byte[size];
while (total < size)
{
recv = s.Receive(data, total, dataleft, 0);
if (recv == 0)
{
break;
}
total += recv;
dataleft -= recv;
}
return data;
}
private void button3_Click(object sender, EventArgs e)
{
textBox1.Clear();
textBox2.Clear();
textBox2.Text = "Connection closed";
socket.Shutdown(SocketShutdown.Both);
socket.Close();
}
Server codes
class Program
{
public const string SEND = "[SEND]";
public const string RECV = "[RECV]";
public const string QUIT = "[QUIT]";
static void Main(string[] args)
{
runServer();
}
static void runServer()
{
try
{
byte[] data = new byte[1024];
Socket server = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
IPEndPoint localEP = new IPEndPoint(IPAddress.Any, 9000);
server.Bind(localEP);
server.Listen(10);
Console.WriteLine("Waiting for Client ...");
Socket client = server.Accept();
Console.WriteLine("Client connected");
NetworkStream stream = new NetworkStream(client);
StreamReader reader = new StreamReader(stream);
StreamWriter writer = new StreamWriter(stream);
try
{
while(true)
{
string request = reader.ReadLine();
string filename = reader.ReadLine();
if (request == QUIT)
{
Console.WriteLine("Client disconnected");
break;
}
else if (request == SEND)
{
getFileFromClient(filename, client);
}
else if (request == RECV)
{
receiveFileFromClient(filename, client);
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
public static void getFileFromClient(string filename, Socket client)
{
try
{
FileStream output = File.OpenWrite(filename);
Console.WriteLine(filename + " created");
int count = 0;
while(true)
{
byte[] data = new byte[1024];
int size = client.Receive(data);
output.Write(data, 0, size);
count += size;
if(size<1024)
{
break;
}
}
output.Close();
Console.WriteLine(count + " bytes read from client");
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
public static void receiveFileFromClient(string filename, Socket client)
{
int count = 0;
FileStream input = File.OpenRead(filename);
Console.WriteLine("Reading " + filename);
while(true)
{
byte[] data = new byte[1024];
int bytesRead = input.Read(data, 0, 1024);
client.Send(data, bytesRead, SocketFlags.None);
count += bytesRead;
if(bytesRead < 1024)
{
break;
}
}
Console.WriteLine("Transferring file completed\r\n" + count + " bytes sent to Client");
input.Close();
}
}

Generally speaking
...try to approach the problem differently.
You can't just copy paste stuff from the Internet and hope for the best. You need to understand what you're doing thoroughly.
Regarding your exact problem
Take a look at the button2_Click method.
It contains a while loop which apparently never finishes.
while (true)
{
data = receiveData(socket);
MemoryStream ms = new MemoryStream(data);
break;
}
It does finish because of the break command. But this is all very hard to read.
When you copy paste code around and then apply quick fixes you end up with a pile of code which is very hard to debug.
It took me about 10 minutes to notice the fact that the client defines it's "message verbs" like so:
public const string SEND = "[SEND]";
public const string RECEIVE = "[RECEIVE]";
public const string QUIT = "[QUIT]";
while the server defines them like so:
public const string SEND = "[SEND]";
public const string RECV = "[RECV]";
public const string QUIT = "[QUIT]";
This is maybe not the only problem, but it is sufficient to create a deadlock,
because the server never executes the positive branch of this if statement:
else if (request == RECV)
{
receiveFileFromClient(filename, client);
}
so the client believes it is about to receive something, which proves to be false.
Also make sure you send the "SEND" and "RECEIVE" message verbs when you should and not mix them up.
Good luck!
PS: I would suggest you take a look at more simpler to use techniques for sending and receiving data, such as:
WCF
ASP.NET Web Services
Web API

Ignoring any logic errors that may occur in your programs, the way you are handling things in your client whenever it is doing an action it is doing it on the GUI thread. This will make your application seem like it is locking but instead it is executing your logic on the GUI thread.
The same problem is occurring on the server. It accepts a connection, and then goes on to receive the file. It will not be able to receive any other connection until it finished receiving the file.
The server is not without problems either because it is never checking if it receives 0 bytes from the socket. Which means that the client closed its end of the connection. You are simply assuming that if you receive less than 1024 you are receiving your last part of the file. This is simply not true for TCP. You only know you received the last part if you receive 0 bytes. TCP is a byte streaming protocol you cannot assume you will be receiving blocks of 1024 bytes. It is likely that this will be in fact the case, but you should not code it like that. Check for reception of 0 bytes. On the client you did check for 0 bytes, I am puzzled why you did not do the same on the server. The problematic part is this :
byte[] data = new byte[1024];
int size = client.Receive(data);
output.Write(data, 0, size);
count += size;
if(size<1024) //you can only break if the size is 0
{
break;
}
There are probably more bugs. As the other answer also indicated some other issues.

Named pipes async

Im trying to set up a named pipe server and client to send data between two programs.
My issue is that when i data is recived eg. BeginRead command om server triggers after i have serialized an object from the client it triggers the callback like 20 times for the same message. The goal is that the client program will send commands to the server program. And when the server processes tasks it will send status updates back to the client when there is one connected.
Here is my current test program.
class Program
{
static void Main(string[] args)
{
var server = new PipeServer();
server.Init();
var client = new PipeClient();
if (client.Connect())
{
Console.WriteLine("Connected to server.");
}
else
{
Console.WriteLine("Connection failed.");
return;
}
while (true)
{
Console.Write(" \\> ");
string input = Console.ReadLine();
if (string.IsNullOrEmpty(input)) break;
var arr = input.Split(new char[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);
int value = 0;
if (arr.Length != 2) break;
if (!int.TryParse(arr[1], out value)) break;
var obj = new PipeObject { Name = arr[0], Value = value };
client.Send(obj);
//string result = f.Deserialize(client) as string;
//Console.WriteLine(result);
}
}
}
internal class PipeServer
{
IFormatter Formatter = new BinaryFormatter();
public NamedPipeServerStream Instance { get; internal set; }
public bool IsConnected { get; internal set; }
byte[] buffer = new byte[65535];
public object Message { get; set; }
StreamReader sr;
StreamWriter sw;
internal PipeServer()
{
IsConnected = false;
}
public void Init()
{
var ps = new PipeSecurity();
ps.AddAccessRule(new PipeAccessRule(WindowsIdentity.GetCurrent().User, PipeAccessRights.FullControl, AccessControlType.Allow));
ps.AddAccessRule(new PipeAccessRule(new SecurityIdentifier(WellKnownSidType.AuthenticatedUserSid, null), PipeAccessRights.ReadWrite, AccessControlType.Allow));
Instance = new NamedPipeServerStream("Levscan4Pipe", PipeDirection.InOut, 1, PipeTransmissionMode.Message, PipeOptions.Asynchronous | PipeOptions.WriteThrough, 65535, 65535, ps);
sr = new StreamReader(Instance);
sw = new StreamWriter(Instance);
Instance.BeginWaitForConnection(OnClientConnected, Instance);
Thread t = new Thread(Run);
t.Start();
}
void Run()
{
int index = 0;
if (IsConnected)
{
try
{
Instance.BeginRead(buffer, 0, buffer.Length, OnRead_Completed, Instance);
//index += Instance.Read(buffer, 0, buffer.Length);
//try
//{
// using (var ms = new MemoryStream(buffer))
// {
// Message = Formatter.Deserialize(ms);
// index = 0;
// }
//}
//catch (Exception e)
//{
// Debug.WriteLine(e.Message);
// Debug.WriteLine(e.StackTrace);
//}
}
catch (IOException)
{
IsConnected = false;
Instance.Disconnect();
}
}
Thread.Sleep(Timeout.Infinite);
//Instance.WaitForConnection();
//Thread t = new Thread(Run);
//t.Start();
}
void OnClientConnected(IAsyncResult ar)
{
Instance.EndWaitForConnection(ar);
IsConnected = true;
}
void OnRead_Completed(IAsyncResult ar)
{
var bytes = Instance.EndRead(ar);
Debug.WriteLine("{1} > Read completed - bytes read: {0}".FormatWith(bytes, DateTime.Now.ToString()));
//try
//{
// using (var ms = new MemoryStream(buffer))
// {
// Message = Formatter.Deserialize(ms);
// }
//}
//catch (Exception e)
//{
// Debug.WriteLine(e.Message);
// Debug.WriteLine(e.StackTrace);
//}
}
}
internal class PipeClient
{
IFormatter f = new BinaryFormatter();
public NamedPipeClientStream Instance { get; internal set; }
StreamWriter sw;
StreamReader sr;
public PipeClient()
{
Instance = new NamedPipeClientStream(".", "Levscan4Pipe", PipeDirection.InOut, PipeOptions.Asynchronous | PipeOptions.WriteThrough);
sr = new StreamReader(Instance);
sw = new StreamWriter(Instance);
}
public bool Connect()
{
try
{
Instance.Connect(5000);
Instance.ReadMode = PipeTransmissionMode.Message;
Instance.WaitForPipeDrain();
return true;
}
catch
{
return false;
}
}
public void Send(object obj)
{
f.Serialize(Instance, obj);
Instance.Flush();
Instance.WaitForPipeDrain();
}
}
Edit
Changed the while loop to a if, to start the BeginRead. This solves the multiple callbacks but i stil dont get a complete message.

If the server is writing to the stream like:
write field 1
write field 2
write field 3
etc.
There is some time between the writes, and the receiver (your program) can be reading the first three fields while the server is still writing the others. The pipe stream doesn't know when the server is finished writing, so it can't buffer everything and send it to you all in one big chunk.
When the server writes everything to a memory stream first and then copies the memory stream to the pipe stream, your program can get it all at once. Maybe. If the server is sending a very large packet, you might read just part of it.
The pipe stream is just a stream of bytes. It doesn't impose any format on the data. It doesn't have any concept of records or anything like that. So you have to treat it like a stream of bytes and do your own composing of records, etc.
If you need to know the size of the record sent from the server, the server has to put that information in the stream for you. Typically, the server will write the length and then the data. The receiver can then read the length, convert it to an integer, and then read that many bytes from the stream. And, yes, it might take multiple reads in order to get all of the bytes. That's just the nature of a byte stream.
The other way to handle this is to have an end-of-record marker. So the server sends its data and your program reads until it finds the byte sequence that signifies the end of the record. You have to be careful, though, because the server could be sending multiple records and your read could grab the end of one record as well as the beginning of the next.
Working with byte streams can be a lot of work because you have to reconstruct records after reading the bytes. It's much easier to use an existing framework (like WCF, as mentioned in one of the comments) if you can.

TcpClient/TcpListener communication problems when not on localhost (multithreaded)

I'm working on a project where I communicate serialized commands over TCP/IP. I have it working when I'm on local host but when I run my listener on a different server, it fails randomly when trying to deserialize commands at the listeners side.
Exceptions thrown are: 'Attempting to deserialize an empty stream.' and 'End of Stream encountered before parsing was completed.' from the serializer
When I run a sequence of commands individually it works fine but when I create threads and runs several sequences simultaneously it fails.
The Listener create listeners on 4 different ports and the client runs 1 thread for each port. When one of the threads reaches the end of a sequence it terminates.
I tried to make my client singleton and also tried Mutex. But still the same problem.
Here is my client:
public class TcpIpCommunicator : ICommunicator, IDisposable
{
private Dictionary<int,TcpClient> clientSockets = new Dictionary<int,TcpClient>();
public IInverterCommand ReadAsyncCommand { set; get; }
private static TcpIpCommunicator tcpIpCommunicator;
private TcpIpCommunicator()
{
}
public static TcpIpCommunicator GetInstance()
{
if(tcpIpCommunicator == null)
tcpIpCommunicator = new TcpIpCommunicator();
return tcpIpCommunicator;
}
public void Send(IInverterCommand command, int id)
{
var serializer = new Serializer();
MemoryStream stream = serializer.SerializeMultipleObjects(command);
var _bytes = stream.GetBuffer();
var networkStream = clientSockets[id].GetStream();
networkStream.Write(_bytes, 0, _bytes.Length);
networkStream.Flush();
}
public IInverterCommand Read(int id)
{
var memoryStream = new MemoryStream();
byte[] buffer;
var networkStream = clientSockets[id].GetStream();
do
{
buffer = new byte[clientSockets[id].ReceiveBufferSize];
int sizeRead = networkStream.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, sizeRead);
} while (networkStream.DataAvailable);
networkStream.Flush();
memoryStream.Position = 0;
var serializer = new Serializer();
return serializer.DeSerializeMultipleObject(memoryStream);
}
public void ReadAsync(object id)
{
ReadAsyncCommand = Read((int)id);
}
public void Dispose()
{
foreach (var tcpClient in clientSockets.Values)
{
tcpClient.Close();
}
}
public int Connect(string ip, int port)
{
var tcpClient = new TcpClient();
tcpClient.ReceiveTimeout = int.MaxValue;
tcpClient.SendTimeout = int.MaxValue;
tcpClient.Connect(ip, port);
int key = findKey();
clientSockets.Add(key, tcpClient);
return key;
}
public void DestroyConnection(int id)
{
clientSockets[id].Close();
clientSockets.Remove(id);
}
private int findKey()
{
int key = 0;
while(clientSockets.ContainsKey(key))
{
key++;
}
return key;
}
}
And my server side code is here:
public class TCPListener : IDisposable
{
private readonly TcpListener _serverSocket;
private NetworkStream _networkStream;
private readonly TcpClient _clientSocket = default(TcpClient);
public TCPListener(int port)
{
_serverSocket = new TcpListener(port);
_serverSocket.Server.ReceiveTimeout = int.MaxValue;
_serverSocket.Server.SendTimeout = int.MaxValue;
_serverSocket.Start();
_clientSocket = _serverSocket.AcceptTcpClient();
}
public void Send(IInverterCommand message)
{
_networkStream = _clientSocket.GetStream();
var serialize = new Serializer();
var stream = serialize.SerializeMultipleObjects(message);
var _bytes = stream.GetBuffer();
if (_bytes.Length > _clientSocket.ReceiveBufferSize)
{
byte[] bytes = new byte[_clientSocket.ReceiveBufferSize];
for (int i = 0; i < _bytes.Length; i += _clientSocket.ReceiveBufferSize)
{
for (int j = 0; j < _clientSocket.ReceiveBufferSize && i + j != _bytes.Length; ++j)
{
bytes[j] = _bytes[i + j];
}
_networkStream.Write(bytes, 0, _clientSocket.ReceiveBufferSize);
}
}
else
{
_networkStream.Write(_bytes, 0, _bytes.Length);
}
Thread.Sleep(50);
_networkStream.Flush();
}
public IInverterCommand ReadCommand()
{
_networkStream = _clientSocket.GetStream();
var memoryStream = new MemoryStream();
do
{
var buffer = new byte[_clientSocket.ReceiveBufferSize];
int sizeRead = _networkStream.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, sizeRead);
Thread.Sleep(50);
} while (_networkStream.DataAvailable);
_networkStream.Flush();
memoryStream.Position = 0;
var serializer = new Serializer();
return serializer.DeSerializeMultipleObject(memoryStream);
}
public void Dispose()
{
_clientSocket.Close();
_serverSocket.Stop();
}
}
Here is typically calling code at client side:
IInverterCommand command = new SoftwareUpdateInverterCommand();
tcpClient.Send(command, tcpId);
var thread = new Thread(tcpClient.ReadAsync);
thread.Start(tcpId);
if (!thread.Join(timeout))
{
thread.Abort();
tcpClient.DestroyConnection(tcpId);
return;
}
And the server side calling code:
//Recieving CMD on software update
TcpListener = new TCPListener((int)port);
var command = TcpListener.ReadCommand();
//Sending OK back to server
command.Message = "OK";
TcpListener.Send(command);

I recommend you go read up on the subject of thread synchronization and what it means and how to use it. Also what happens when you share things between threads that are not thread safe.
You have no thread synchronization in your code at all, yet you read and write to a Dictionary (not safe) from at least 2 different threads without any synchronization.
That alone can lead to bugs that are very hard to trace and lots of different phenomenon you rather not deal with.
You really need to go reading on how to do threading properly.

Send a large file over tcp connection

I need to send a few large files to a computer over the internet. As a result I opened the ports that I plan to use on the router and forward the correspoinding ip addresses. anyways let me show you the classes that I have been working on in order to achieve this. This classes work great with small files but sometimes they fail with large files.
here is the code for the server: (it is a console application)
using System;
using System.IO;
using System.Net;
using System.Net.Sockets;
namespace ConsoleApplication20
{
//server
class Program
{
static Server s;
public static void mm()
{
s = new Server("192.168.0.196");
s.startServer();
Console.Read();
}
static void Main(string[] args)
{
// Thread t = new Thread(new ThreadStart(mm));
// t.Start();
mm();
Console.Read();
s.disconnect();
}
}
class MyTCP
{
protected const int MaxChunkSize = 4096;
protected Int32 port { get; set; }
protected string serverIP { get; set; }
protected TcpClient client { get; set; }
protected static NetworkStream stream { get; set; }
protected void sendData(NetworkStream stream, Byte[] data)
{
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
}
protected String receiveData(NetworkStream stream)
{
// Buffer to store the response bytes.
Byte[] data = new Byte[MaxChunkSize];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
return responseData;
}
protected static Byte[] textToBytes(string text)
{
return System.Text.Encoding.ASCII.GetBytes(text);
}
public virtual void disconnect() { }
public bool isServerConected { get { return client.Connected; } }
}
[Serializable]
public class FileProperties
{
public string FileName { get; set; }
public string DestPath { get; set; }
public double FileSize { get; set; }
public FileAttributes fileAttributes { get; set; }
public System.Security.AccessControl.FileSecurity FileSecurity { get; set; }
public DateTime creationTime { get; set; }
public DateTime lastAccessTime { get; set; }
public DateTime lastWriteTime { get; set; }
}
class Server: MyTCP
{
private System.IO.FileStream _FileStream;
private static TcpListener server;
private static bool disconect;
/// <summary>
/// Constructor
/// </summary>
/// <param name="localAddr">The ip address of the server</param>
/// <param name="port">on what port the server going to be listening to?</param>
/// <param name="autoStartServer">start listening for connections now? you may call the startserver() method latter...</param>
public Server(string localAddr, Int32 port = 13000, bool autoStartServer = false)
{
this.port = port;
this.serverIP = localAddr;
if (autoStartServer)
start();
}
/// <summary>
/// Start listening for connections
/// </summary>
public void startServer()
{
start();
}
public override void disconnect()
{
// Close everything.
stream.Close();
client.Close();
server.Stop();
disconect = true;
}
void start()
{
server = null;
try
{
// TcpListener server = new TcpListener(port);
server = new TcpListener(IPAddress.Parse(serverIP), port);
// Start listening for client requests.
server.Start();
// Buffer for reading data
Byte[] bytes = new Byte[MaxChunkSize];
String data = null;
// Enter the listening loop.
while (disconect==false)
{
Console.Write("Waiting for a connection... ");
// Perform a blocking call to accept requests.
// You could also user server.AcceptSocket() here.
client = server.AcceptTcpClient();
Console.WriteLine("Connected!");
// Get a stream object for reading and writing
stream = client.GetStream();
int i;
try
{
// Loop to receive all the data sent by the client.
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
// Translate data bytes to a ASCII string.
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
Console.WriteLine("Received: {0}", data);
if (data.ToUpper().Contains("<sendFile>".ToUpper()))
{
receiveFile(bytes);
}
continue;
}
}
catch { }
// Shutdown and end connection
client.Close();
}
}
catch (SocketException e)
{
Console.WriteLine("SocketException: {0}", e);
}
finally
{
// Stop listening for new clients.
server.Stop();
}
Console.WriteLine("\nHit enter to continue...");
Console.Read();
}
void receiveFile(Byte[] bytes)
{
// send 1
sendData(stream, textToBytes("<1>"));
// receive 2
int length = stream.Read(bytes, 0, bytes.Length);
byte[] tempA = new byte[length];
for (int k = 0; k < length; k++)
tempA[k] = bytes[k];
Stream ms = new MemoryStream(tempA);
FileProperties p = new FileProperties();
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(p.GetType());
try
{
p = (FileProperties)x.Deserialize(ms);
if (Directory.Exists(p.DestPath))
{
//send 3
sendData(stream, textToBytes("<3>"));
}
else
{
//send 3
sendData(stream, textToBytes("<no>"));
return;
}
}
catch
{
//send 3
sendData(stream, textToBytes("<no>"));
return;
}
int i;
string temp = Path.Combine(new string[]{ p.DestPath, p.FileName + ".temp"});
_FileStream = new System.IO.FileStream(temp, System.IO.FileMode.Create, System.IO.FileAccess.Write);
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
if (i == 11 & System.Text.Encoding.ASCII.GetString(bytes, 0, i).ToUpper().Equals("</sendFile>".ToUpper()))
{
_FileStream.Close();
Console.WriteLine("D!");
File.SetAttributes(temp, p.fileAttributes);
File.SetAccessControl(temp, p.FileSecurity);
File.SetCreationTime(temp, p.creationTime);
File.SetLastAccessTime(temp, p.lastAccessTime);
File.SetLastWriteTime(temp, p.lastWriteTime);
if(File.Exists(temp.Substring(0, temp.Length - 4)))
File.Delete(temp.Substring(0, temp.Length - 4));
File.Move(temp, temp.Substring(0, temp.Length - 4));
//sendData(stream, textToBytes("<done>"));
Console.WriteLine("Done!");
return;
}
_FileStream.Write(bytes, 0, i);
}
return;
}
}
}
and the code for my client is:
using System;
using System.Net.Sockets;
using System.Windows;
using System.IO;
namespace WpfApplication23sdfd
{
[Serializable]
public class FileProperties
{
public string FileName { get; set; }
public string DestPath { get; set; }
public double FileSize { get; set; }
public FileAttributes fileAttributes { get; set; }
public System.Security.AccessControl.FileSecurity FileSecurity { get; set; }
public DateTime creationTime { get; set; }
public DateTime lastAccessTime { get; set; }
public DateTime lastWriteTime { get; set; }
}
abstract class MyTCP
{
protected const int MaxChunkSize = 4096;
protected Int32 port { get; set; }
protected string serverIP { get; set; }
protected TcpClient client { get; set; }
protected static NetworkStream stream { get; set; }
protected void sendData(NetworkStream stream, Byte[] data)
{
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
// Receive the TcpServer.response.
}
protected String receiveData(NetworkStream stream)
{
// Buffer to store the response bytes.
Byte[] data = new Byte[MaxChunkSize];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
return responseData;
}
protected static Byte[] textToBytes(string text)
{
return System.Text.Encoding.ASCII.GetBytes(text);
}
public virtual void disconnect() { }
public bool isServerConected { get { return client.Connected; } }
}
//client
class Client: MyTCP
{
/// <summary>
/// Constructor
/// </summary>
/// <param name="serverIP">the ip address of the server</param>
/// <param name="port">through what port is the connection going to be established</param>
public Client(string serverIP, Int32 port = 13000, bool autoConnect = false)
{
this.port = port;
this.serverIP = serverIP;
if (autoConnect)
connect();
}
public bool connect()
{
Byte[] data = System.Text.Encoding.ASCII.GetBytes("connect");
// Create a TcpClient.
// Note, for this client to work you need to have a TcpServer
// connected to the same address as specified by the server, port
// combination.
try
{
client = new TcpClient(serverIP, port);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
stream = client.GetStream();
return true;
}
catch
{
return false;
}
}
public override void disconnect()
{
// Close everything.
stream.Close();
client.Close();
}
static void ConnectOld(String server, Byte[] data)
{
try
{
// Create a TcpClient.
// Note, for this client to work you need to have a TcpServer
// connected to the same address as specified by the server, port
// combination.
Int32 port = 13000;
TcpClient client = new TcpClient(server, port);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close();
client.Close();
}
catch (ArgumentNullException e)
{
Console.WriteLine("ArgumentNullException: {0}", e);
}
catch (SocketException e)
{
Console.WriteLine("SocketException: {0}", e);
}
Console.WriteLine("\n Press Enter to continue...");
Console.Read();
}
public void sendFile(string file, string destPath = "c:\\")
{
//let server know what you are going to be doing...
sendData(stream, textToBytes("<sendFile>"));
FileProperties p = new FileProperties {
creationTime = File.GetCreationTime(file),
fileAttributes = File.GetAttributes(file),
FileSecurity = File.GetAccessControl(file),
lastAccessTime = File.GetLastAccessTime(file),
lastWriteTime = File.GetLastWriteTime(file),
DestPath = destPath,
FileName = Path.GetFileName(file)
};
// receive 1
if (!receiveData(stream).ToUpper().Contains("<1>".ToUpper()))
{
MessageBox.Show("Error comunicating with server");
return;
}
// send object p to server
System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(p.GetType());
x.Serialize(stream, p); // send 2
//recieve 3
if (!receiveData(stream).ToUpper().Contains("<3>".ToUpper()))
{
MessageBox.Show("Error incorrect parameters sent to server");
return;
}
System.IO.FileStream streamFile = new System.IO.FileStream(file, System.IO.FileMode.Open, System.IO.FileAccess.Read);
while (true)
{
byte[] chunk = new byte[MaxChunkSize];
int index = 0;
// There are various different ways of structuring this bit of code.
// Fundamentally we're trying to keep reading in to our chunk until
// either we reach the end of the stream, or we've read everything we need.
while (index < chunk.Length)
{
int bytesRead = streamFile.Read(chunk, index, chunk.Length - index);
if (bytesRead == 0)
{
break;
}
if (bytesRead < MaxChunkSize)
{
byte[] temp = new byte[bytesRead];
for (var i = 0; i < bytesRead; i++)
temp[i] = chunk[i];
chunk = temp;
}
index += bytesRead;
}
if (index != 0) // Our previous chunk may have been the last one
{
sendData(stream,chunk); // index is the number of bytes in the chunk
}
if (index != chunk.Length) // We didn't read a full chunk: we're done
{
sendData(stream, textToBytes("</sendFile>".ToUpper()));
//receiveData(stream);//wait recall missing to check results
return;
}
}
}
}
}
the way I instantiate the client class is by providing the IP address of the server as:
Client c = new Client("192.168.0.196");
c.sendFile(#"A:\Users\Tono\Desktop\a.mp4");
the server has to be running first before executing that code.
I don't know why is so complicated to send a file over the internet using sockets. I don't konw WCF that's why I been loosing a lot of time creating this classes. Maybe there are already some built in classes that will enable me to send files over the internet to a different computer. I know just the basics of networking thereofore it will be nice if I could do it with a simple class. I don't undersatnd why my classes do not work all the time? if I increase the buffer size will my classes be more efficient? do I have to wait or pause my program a little bit before sending more bytes? It will be nice if someone can tell me what is wrong with this classes. They work nice with small files but with large files sometimes it does not work...

Here is my code to send a large file.
Some tips:
Check your buffer size. If it's too big it will fail.
Socket flags. The partial flag works the best.
Socket timeout is needed due to the transfer time.
Client :
string IPAddress = "";
int Port = 500;
string Filename = #"C:\Users\Ben\Desktop\TT.zip";
int bufferSize = 1024;
byte[] buffer = null;
byte[] header = null;
FileStream fs = new FileStream(Filename, FileMode.Open);
bool read = true;
int bufferCount = Convert.ToInt32(Math.Ceiling((double)fs.Length / (double)bufferSize));
TcpClient tcpClient = new TcpClient(IPAddress, Port);
tcpClient.SendTimeout = 600000;
tcpClient.ReceiveTimeout = 600000;
string headerStr = "Content-length:" + fs.Length.ToString() + "\r\nFilename:" + #"C:\Users\Administrator\Desktop\" + "test.zip\r\n";
header = new byte[bufferSize];
Array.Copy(Encoding.ASCII.GetBytes(headerStr), header, Encoding.ASCII.GetBytes(headerStr).Length);
tcpClient.Client.Send(header);
for (int i = 0; i < bufferCount; i++)
{
buffer = new byte[bufferSize];
int size = fs.Read(buffer, 0, bufferSize);
tcpClient.Client.Send(buffer,size,SocketFlags.Partial);
}
tcpClient.Client.Close();
fs.Close();
Server :
int Port = 500;
TcpListener listener = new TcpListener(IPAddress.Any, Port);
listener.Start();
Socket socket = listener.AcceptSocket();
int bufferSize = 1024;
byte[] buffer = null;
byte[] header = null;
string headerStr = "";
string filename = "";
int filesize = 0;
header = new byte[bufferSize];
socket.Receive(header);
headerStr = Encoding.ASCII.GetString(header);
string[] splitted = headerStr.Split(new string[] { "\r\n" }, StringSplitOptions.None);
Dictionary<string, string> headers = new Dictionary<string, string>();
foreach (string s in splitted)
{
if (s.Contains(":"))
{
headers.Add(s.Substring(0,s.IndexOf(":")), s.Substring(s.IndexOf(":") + 1));
}
}
//Get filesize from header
filesize = Convert.ToInt32(headers["Content-length"]);
//Get filename from header
filename = headers["Filename"];
int bufferCount = Convert.ToInt32(Math.Ceiling((double)filesize / (double)bufferSize));
FileStream fs = new FileStream(filename, FileMode.OpenOrCreate);
while(filesize > 0)
{
buffer = new byte[bufferSize];
int size = socket.Receive(buffer,SocketFlags.Partial);
fs.Write(buffer,0,size);
filesize -= size;
}
fs.Close();
Hope this will help someone.

There are a few issues that I can see immediately. The one that may be causing your program to only work some of the time is the fact that sending via TCP will not guarantee that every send will result in an identically-sized receive on the other side.
Your protocol seems to assume it will, because you're waiting for a read of exactly 11 bytes for the </sendFile>, whereas it could be received in multiple separate reads. E.g.: "[file data...]". If this happens, your code will not correctly finish.
It's also worth noting that the ASCII encoding is 7-bit, and so binary files (such as the MP4) will be received incorrectly (even if you fix the above). If it is binary data, you should not attempt to convert it to a string, but instead write it to file directly from the byte[].
If you wish to contine down this route (rather than using the many existing file transfer systems already available as mentioned in another answer) then you may also want to change your protocol so that instead of delimiting the file with <sendFile>...</sendFile>, you send the file's length initially, which will allow you to send files that may contain one of these special tags.

Sending files / directories over sockets is not trivial. I would recommend using some file transfer library (over sockets, or maybe higher level protocol, i.e. rsync, ftp, http, etc), rather than trying to code the thing from scratch.
After skimming over the code - try sending a large file containing some uniform content (filled with '0's or something). If it would pass through - your xml thing doesn't work.

Basically what you are doing is correct. The main point I could suggest for improvement is setting the MaxChunkSize to a larger value, as much as 65000. This will enable the socket code to handle any fragmentation, which will be much more efficient than any splitting you will do by yourself.
Also, you are aware to the fact that sending a large file should take some time. At a 100 Mbit LAN, the bandwidth would be (theoretically) 12.5 MByte per second. Thus sending a 700 MByte file will still take 56 seconds. Of course that the actual throughput depends and many factors in the network and the computers and network hardware involved, so it would be expected to be slower.
Finally, Wireshark (or any other sniffer program) is the most basic and invaluable tool in the networking programmer toolbox. Use it on both the server and the client to see how the TCP packets are transmitted, to see if you can identify any causes for the slow trasnfer.