My dotnet/c# server is running very well for my Unity game on my local machine.
But when I host it on an online cloud server (AWS EC2 instance) it will result in not all TCP messages arriving at the client side.
Server side method:
git: github.com/IRiViI/TCPmreServer
public void SendData(Packet _packet)
{
try
{
if (socket != null)
{
stream.BeginWrite(_packet.ToArray(), 0, _packet.Length(), null, null);
}
}
catch (Exception _ex)
{
Console.WriteLine($"Error sending data to player {id} via TCP: {_ex}");
}
}
Client side method:git: github.com/IRiViI/TCPmreClient
private void ReceiveCallback(IAsyncResult _result){
try {
int _byteLength = stream.EndRead(_result);
if (_byteLength <= 0){
client.Disconnect();
return;
}
byte[] _data = new byte[_byteLength];
Array.Copy(receiveBuffer, _data, _byteLength);
receivedData.Reset(HandleData(_data));
stream.BeginRead(receiveBuffer, 0, dataBufferSize, ReceiveCallback, null);
} catch(Exception _ex) {
Disconnect();
Debug.Log(_ex);
}
}
private bool HandleData(byte[] _data){
int _packetLength = 0;
receivedData.SetBytes(_data);
if(receivedData.UnreadLength() >= 4){
_packetLength = receivedData.ReadInt();
if (_packetLength <= 0){
return true;
}
}
while (_packetLength > 0 && _packetLength <= receivedData.UnreadLength()){
byte[] _packetBytes = receivedData.ReadBytes(_packetLength);
ThreadManager.ExecuteOnMainThread(() => {
using (Packet _packet = new Packet(_packetBytes)){
int _packetId = _packet.ReadInt();
client.packetHandlers[_packetId](_packet);
}
});
_packetLength = 0;
if(receivedData.UnreadLength() >= 4){
_packetLength = receivedData.ReadInt();
if (_packetLength <= 0){
return true;
}
}
}
if (_packetLength <= 1){
return true;
}
return false;
}
It only happens when I send many (100+) messages at the same time. below 100 it usally goes well.
I think the issue might be AWS related (firewall and such). But I would expect to get some error messages when a TCP message could not be send. I'm not able to find any errors. I would have expect this behaviour of UDP but not of TCP.
My question is: How can I increase the amount of messages that can be send or have some message not received callback inorder to handle failed TCP messages.
Update --
The problem seems to be on the client side. I tested my server code on my server at home and the same problem occures as when running my server code on an EC2 AWS instance.
The problem occures when there are multiple tcp packets handled.
At the moment when the tcp handler breaks, I get a packetsLength of 16777216 (which equals 2^24).
if(receivedData.UnreadLength() >= 4){
_packetLength = receivedData.ReadInt();
if (_packetLength <= 0){
return true;
}
Console.WriteLine($"_{_packetLength}");
}
Related
I have this code inside of a controller that is called from a view by multiple users. The purpose is to read data from a socket while the socket has information to deliver. Multiple calls can happen simultaneously. The problem is, for a same call the Socket will sometimes cut the message. We will not read it entirely. Is there something wrong in my call that could lead to an incomplete reading of the Socket ? I am not able to reproduce it on my dev environment I just know that on the production we have responses from the Socket being randomly cut / incomplete.
int byteCount = 1;
string response = "";
int total = 0;
bytesReceived = new Byte[8000];
bool bytesAreAvailable = true;
while (byteCount > 0 || bytesAreAvailable)
{
try {
byteCount = m_oSocket.Receive(bytesReceived, bytesReceived.Length, 0);
total += byteCount;
Debug.WriteLine("Bytecount: " + byteCount.ToString());
Debug.WriteLine("Avalable: " + m_oSocket.Available.ToString());
if (m_oSocket.Available > 0)
bytesAreAvailable = true;
else
bytesAreAvailable = false; //Everything was read
response += Encoding.UTF8.GetString(bytesReceived, 0, byteCount);
}
catch(Exception e)
{
Debug.WriteLine(e.Message);
}
}
Debug.WriteLine(response.ToString());
Thanks a lot for any help / tips you could have on solving my issue
I am trying to convert C# source I used in Unity to a Visual Studio Windows forms app and I can't figure out why my onreceive part won't work.
What I'm doing is making a TCP connection to the server (works) then my server sends a welcome package that I do receive but can't handle.
Sometimes it does run completely but does not trigger the trigger (explained later).
Since the source is originally from a unity project there I can wait for the update and I think that is why it works in there.
The code:
private void OnReceive(IAsyncResult ar)
{
try
{
int byteAmt = myStream.EndRead(ar);
byte[] myBytes = new byte[byteAmt];
Buffer.BlockCopy(asyncBuff, 0, myBytes, 0, byteAmt);
if (byteAmt == 0) return;
//UnityThread.executeInUpdate(() =>
//{
ClientHandleData.HandleData(myBytes);
//});
myStream.BeginRead(asyncBuff, 0, 8192, OnReceive, null);
}
catch
{
Console.WriteLine("we have an error in onreceive");
}
}
the Handledata
public static void HandleData(byte[] data)
{
byte[] Buffer;
Buffer = (byte[])data.Clone();
if (playerBuffer == null) playerBuffer = new ByteBuffer();
playerBuffer.WriteBytes(Buffer);
if (playerBuffer.Count() == 0)
{
playerBuffer.Clear();
return;
}
if (playerBuffer.Length() >= 8)
{
pLength = playerBuffer.ReadLong(false);
if (pLength <= 0)
{
playerBuffer.Clear();
return;
}
}
if (playerBuffer.Length() >= 8)
{
pLength = playerBuffer.ReadLong(false);
if (pLength <= 0)
{
playerBuffer.Clear();
return;
}
}
while (pLength > 0 & pLength <= playerBuffer.Length() - 8)
{
if (pLength <= playerBuffer.Length() - 8)
{
playerBuffer.ReadLong();
data = playerBuffer.ReadBytes((int)pLength);
HandleDataPackets(data);
}
pLength = 0;
if (playerBuffer.Length() >= 8)
{
pLength = playerBuffer.ReadLong(false);
if (pLength < 0)
{
playerBuffer.Clear();
return;
}
}
}
}
and last the HandleDataPackets
public static void HandleDataPackets(byte[] data)
{
long packetnum; ByteBuffer buffer; Packet_ packet;
buffer = new ByteBuffer();
buffer.WriteBytes(data);
packetnum = buffer.ReadLong();
buffer = null;
Form1.label1.Text = "reading packetnum";
if (packetnum == 0) return;
Form1.label1.Text = "packetnum read and is not 0";
if (packets.TryGetValue(packetnum, out packet))
{
Form1.label1.Text = "trigger";
packet.Invoke(data);
}
}
sometimes it runs all the way to the Form1.label1.Text = "packetnum read and is not 0"; line in HandleDataPacket but then it does not trigger the trigger line :(.
Can someone please tell me why this is?
Ok i found the problem.
I was trying to initialize the packages in the unity way but had to trigger it from the program itself.
i created a Networkmanager class that initializes the different packages and then makes the connection. this fixed it!
all code above is working as intended.
Just realized that john was telling me this!..
To bad i did not see that a few hours ago.
Post it as an answer and ill mark it like one.
I have got an embedded debian board with mono running an .NET 4.0 application with a fixed number of threads (no actions, no tasks). Because of memory issues I used CLR-Profiler in Windows to analyse memory heap.
Following diagram shows now, that IThreadPoolWorkItems are not (at least not in generation 0) collected:
Now, I really dont have any idea where this objects are possibly used and why they arent collected.
Where could the issue be for this behaviour and where would the IThreadPoolWorkItem being used?
What can I do to find out where they are being used (I couldnt find them through searching the code or looking in CLR-Profiler yet).
Edit
...
private Dictionary<byte, Telegram> _incoming = new Dictionary<byte, Telegram>();
private Queue<byte> _serialDataQueue;
private byte[] _receiveBuffer = new byte[2048];
private Dictionary<Telegram, Telegram> _resultQueue = new Dictionary<Telegram, Telegram>();
private static Telegram _currentTelegram;
ManualResetEvent _manualReset = new ManualResetEvent(false);
...
// Called from other thread (class) to send new telegrams
public bool Send(Dictionary<byte, Telegram> telegrams, out IDictionary<Telegram, Telegram> received)
{
try
{
_manualReset.Reset();
_incoming.Clear(); // clear all prev sending telegrams
_resultQueue.Clear(); // clear the receive queue
using (token = new CancellationTokenSource())
{
foreach (KeyValuePair<byte, Telegram> pair in telegrams)
{
_incoming.Add(pair.Key, pair.Value);
}
int result = WaitHandle.WaitAny(new[] { token.Token.WaitHandle, _manualReset });
received = _resultQueue.Clone<Telegram, Telegram>();
_resultQueue.Clear();
return result == 1;
}
}
catch (Exception err)
{
...
return false;
}
}
// Communication-Thread
public void Run()
{
while(true)
{
...
GetNextTelegram(); // _currentTelegram is set there and _incoming Queue is dequeued
byte[] telegramArray = GenerateTelegram(_currentTelegram, ... );
bool telegramReceived = SendReceiveTelegram(3000, telegramArray);
...
}
}
// Helper method to send and receive telegrams
private bool SendReceiveTelegram(int timeOut, byte[] telegram)
{
// send telegram
try
{
// check if serial port is open
if (_serialPort != null && !_serialPort.IsOpen)
{
_serialPort.Open();
}
Thread.Sleep(10);
_serialPort.Write(telegram, 0, telegram.Length);
}
catch (Exception err)
{
log.ErrorFormat(err.Message, err);
return false;
}
// receive telegram
int offset = 0, bytesRead;
_serialPort.ReadTimeout = timeOut;
int bytesExpected = GetExpectedBytes(_currentTelegram);
if (bytesExpected == -1)
return false;
try
{
while (bytesExpected > 0 &&
(bytesRead = _serialPort.Read(_receiveBuffer, offset, bytesExpected)) > 0)
{
offset += bytesRead;
bytesExpected -= bytesRead;
}
for (int index = 0; index < offset; index++)
_serialDataQueue.Enqueue(_receiveBuffer[index]);
List<byte> resultList;
// looks if telegram is valid and removes bytes from _serialDataQueue
bool isValid = IsValid(_serialDataQueue, out resultList, currentTelegram);
if (isValid && resultList != null)
{
// only add to queue if its really needed!!
byte[] receiveArray = resultList.ToArray();
_resultQueue.Add((Telegram)currentTelegram.Clone(), respTelegram);
}
if (!isValid)
{
Clear();
}
return isValid;
}
catch (TimeOutException err) // Timeout exception
{
log.ErrorFormat(err.Message, err);
Clear();
return false;
} catch (Exception err)
{
log.ErrorFormat(err.Message, err);
Clear();
return false;
}
}
Thx for you help!
I found out, like spender mentioned already, the "issue" is the communication over SerialPort. I found an interesting topic here:
SerialPort has a background thread that's waiting for events (via WaitCommEvent). Whenever an event arrives, it queues a threadpool work
item that may result in a call to your event handler. Let's focus on
one of these threadpool threads. It tries to take a lock (quick
reason: this exists to synchronize event raising with closing; for
more details see the end) and once it gets the lock it checks whether
the number of bytes available to read is above the threshold. If so,
it calls your handler.
So this lock is the reason your handler won't be called in separate
threadpool threads at the same time.
Thats most certainly the reason why they arent collected immediatly. I also tried not using the blocking Read in my SendReceiveTelegram method, but using SerialDataReceivedEventHandler instead led to the same result.
So for me, I will leave things now as they are, unless you bring me a better solution, where these ThreadPoolWorkitems arent kept that long in the Queue anymore.
Thx for your help and also your negative assessment :-D
this is going to sound silly but for some reason this code seems to skip over the read. Or maybe it is going too fast? I'm trying to get a telnet response from the server and I get the first line response but nothing more. It doesn't matter how many readline() I put or if I try to sleep the thread. What can I do to get the whole response from the server to be printed? I think I narrowed the problem down to exiting. In the sample program if you hard code the exit, it immediately ends the loop and displays nothing. I tried sleeping the thread but that just seemed to stop everything.
Output:
220 server-12.tower-558.messagelabs.com ESMTP
Expected Output:
250-server-11.tower-555.messagelabs.com says EHLO to iphere
250-PIPELINING
250-8BITMIME
250-STARTTLS
My Code:
//Telnet Start
IPHostEntry hostInfo = Dns.Resolve(list[j]);
TelnetConnection tc = new TelnetConnection(hostInfo.AddressList[0].ToString(), 25);
string prompt = "a";
string consoleout = "";
// while connected
while (tc.IsConnected && prompt != "exit")
{
// display server output
prompt = "ehlo a.com";
tc.WriteLine(prompt);
//I've tried adding a 2 or 5 second thread sleep here and I still get the same result.
Console.Write(tc.Read());
prompt = "exit";
}
//Telnet End
TelnetConnection Class:
// minimalistic telnet implementation
// conceived by Tom Janssens on 2007/06/06 for codeproject
//
// http://www.corebvba.be
using System;
using System.Collections.Generic;
using System.Text;
using System.Net.Sockets;
namespace MinimalisticTelnet
{
enum Verbs {
WILL = 251,
WONT = 252,
DO = 253,
DONT = 254,
IAC = 255
}
enum Options
{
SGA = 3
}
class TelnetConnection
{
TcpClient tcpSocket;
int TimeOutMs = 100;
public TelnetConnection(string Hostname, int Port)
{
try
{
tcpSocket = new TcpClient(Hostname, Port);
}
catch (SocketException e)
{
Console.Write(e);
}
}
public string Login(string Username,string Password,int LoginTimeOutMs)
{
int oldTimeOutMs = TimeOutMs;
TimeOutMs = LoginTimeOutMs;
string s = Read();
if (!s.TrimEnd().EndsWith(":"))
throw new Exception("Failed to connect : no login prompt");
WriteLine(Username);
s += Read();
if (!s.TrimEnd().EndsWith(":"))
throw new Exception("Failed to connect : no password prompt");
WriteLine(Password);
s += Read();
TimeOutMs = oldTimeOutMs;
return s;
}
public void WriteLine(string cmd)
{
Write(cmd + "\n");
}
public void Write(string cmd)
{
if (!tcpSocket.Connected) return;
byte[] buf = System.Text.ASCIIEncoding.ASCII.GetBytes(cmd.Replace("\0xFF","\0xFF\0xFF"));
tcpSocket.GetStream().Write(buf, 0, buf.Length);
}
public string Read()
{
if (!tcpSocket.Connected) return null;
StringBuilder sb=new StringBuilder();
do
{
ParseTelnet(sb);
System.Threading.Thread.Sleep(TimeOutMs);
} while (tcpSocket.Available > 0);
return sb.ToString();
}
public bool IsConnected
{
get { return tcpSocket.Connected; }
}
void ParseTelnet(StringBuilder sb)
{
while (tcpSocket.Available > 0)
{
int input = tcpSocket.GetStream().ReadByte();
switch (input)
{
case -1 :
break;
case (int)Verbs.IAC:
// interpret as command
int inputverb = tcpSocket.GetStream().ReadByte();
if (inputverb == -1) break;
switch (inputverb)
{
case (int)Verbs.IAC:
//literal IAC = 255 escaped, so append char 255 to string
sb.Append(inputverb);
break;
case (int)Verbs.DO:
case (int)Verbs.DONT:
case (int)Verbs.WILL:
case (int)Verbs.WONT:
// reply to all commands with "WONT", unless it is SGA (suppres go ahead)
int inputoption = tcpSocket.GetStream().ReadByte();
if (inputoption == -1) break;
tcpSocket.GetStream().WriteByte((byte)Verbs.IAC);
if (inputoption == (int)Options.SGA )
tcpSocket.GetStream().WriteByte(inputverb == (int)Verbs.DO ? (byte)Verbs.WILL:(byte)Verbs.DO);
else
tcpSocket.GetStream().WriteByte(inputverb == (int)Verbs.DO ? (byte)Verbs.WONT : (byte)Verbs.DONT);
tcpSocket.GetStream().WriteByte((byte)inputoption);
break;
default:
break;
}
break;
default:
sb.Append( (char)input );
break;
}
}
}
}
}
UPDATE:
Using the following loop I was able to get it to work once, but only once it would not iterate through the count of j when I verified that j was 2. My output, expected output and function are below.
Function:
for (int j = 0; j < list.Count; j++)
{
//Telnet Start
Console.WriteLine("On round #" + j);
IPHostEntry hostInfo = Dns.Resolve(list[j]);
TelnetConnection tc = new TelnetConnection(hostInfo.AddressList[0].ToString(), 25);
string prompt = "a";
string consoleout = "";
// while connected
while (tc.IsConnected && prompt != "exit")
{
// display server output
Console.Write(tc.Read());
// send client input to server
prompt = "ehlo a.com";
tc.WriteLine(prompt);
// display server output
consoleout = tc.Read();
Console.Write(consoleout);
//send exit input to server
prompt = "exit";
tc.WriteLine(prompt);
Console.Write(tc.Read());
}
Console.WriteLine("**DISCONNECTED**");
//Telnet End
if (consoleout.IndexOf("STARTTLS")>-1)
{
if (j == 0)
{
if (list[j].Contains(domains[i]))
dataGridView1.Rows.Add(domains[i], list[j], "Y", numberemployees, "Y");
else
dataGridView1.Rows.Add(domains[i], list[j], "Y", numberemployees, "N");
}
else
{
if (list[j].Contains(domains[i]))
dataGridView1.Rows.Add(null, list[j], "Y", null, "Y");
else
dataGridView1.Rows.Add(null, list[j], "Y", null, "N");
}
}
else
{
if (j == 0)
{
if (list[j].Contains(domains[i]))
dataGridView1.Rows.Add(domains[i], list[j], "N", numberemployees, "Y");
else
dataGridView1.Rows.Add(domains[i], list[j], "N", numberemployees, "N");
}
else
{
if (list[j].Contains(domains[i]))
dataGridView1.Rows.Add(null, list[j], "N", null, "Y");
else
dataGridView1.Rows.Add(null, list[j], "N", null, "N");
}
}
}
Output:
On round #0
True220 server-6.tower-95.messagelabs.com ESMTP
250-server-6.tower-95.messagelabs.com
250-STARTTLS
250-PIPELINING
250 8BITMIME
502 unimplemented (#5.5.1)
**DISCONNECTED**
On round #1
True220 server-14.tower-558.messagelabs.com ESMTP
**DISCONNECTED**
Expected Output:
On round #0
True220 server-6.tower-95.messagelabs.com ESMTP
250-server-6.tower-95.messagelabs.com
250-STARTTLS
250-PIPELINING
250 8BITMIME
502 unimplemented (#5.5.1)
**DISCONNECTED**
On round #1
True220 server-14.tower-558.messagelabs.com ESMTP
250-STARTTLS
250-PIPELINING
250 8BITMIME
**DISCONNECTED**
So for whatever reason, it simply won't do the second loop. But it also is starting to depend on the server. Some servers respond with a little information and some I get something like this, which makes it look like nothing happened at all.
On round #0
True220 SMTP Proxy Server Ready
**DISCONNECTED**
On round #1
True220 SMTP Proxy Server Ready
**DISCONNECTED**
On round #2
True220 SMTP Server Ready
**DISCONNECTED**
On round #3
True220 SMTP Server Ready
**DISCONNECTED**
On round #4
True220 SMTP Proxy Server Ready
**DISCONNECTED**
On round #5
True220 SMTP Proxy Server Ready
**DISCONNECTED**
Kyle,
It looks like you need to open the socket:
public string Read()
{
if (!tcpSocket.Connected)
{
throw new Exception("Socket is Closed.");
}
StringBuilder sb=new StringBuilder();
do
{
ParseTelnet(sb);
System.Threading.Thread.Sleep(TimeOutMs);
} while (tcpSocket.Available > 0);
return sb.ToString();
}
It isn't open, so it is returning.
[UPDATE]
Try looping through IPHostEntry hostInfo = Dns.Resolve(list[j]); before your //Telnet Start comment:
private const int PORT25 = 25; // I hate magic numbers
foreach (var item in list) {
var hostInfo = Dns.Resolve(item);
Console.WriteLine(hostInfo);
foreach (var address in hostInfo.AddressList) {
var tc = new TelnetConnection(address, PORT25);
Console.WriteLine("{0} TelnetConnection Connected: {1}", address, tc.IsConnected);
}
}
[Update 2]
This is really hard to debug, not knowing what all you are trying to connect to and what else is going on.
That said, let's try the following:
In your class MinimalisticTelnet, add this method:
public void Close() {
if (tcpSocket != null) {
tcpSocket.Close();
}
}
I don't see it anywhere else, and that could be causing some issues the next time you attempt your second connection.
Now, in your test code, add the new one line of code after the while loop:
while (tc.IsConnected && prompt != "exit") {
// display server output
Console.Write(tc.Read());
// send client input to server
prompt = "ehlo a.com";
tc.WriteLine(prompt);
// display server output
consoleout = tc.Read();
Console.Write(consoleout);
//send exit input to server
prompt = "exit";
tc.WriteLine(prompt);
Console.Write(tc.Read());
}
tc.Close();
Console.WriteLine("**DISCONNECTED**");
With any luck, the reason your 2nd connection was failing was because you still had an open connection.
I solve a so similar problem adding 3 reads, then 8 seconds timeout and finally reading again AND SHOWING IN MY RICHTEXBOX:
string r1 = conexion.Read();
string r2 = conexion.Read();
string r3 = conexion.Read();
System.Threading.Thread.Sleep(8000);
richtextbox.AppendText(r1 + r2 + r3);
I hope it can help you!
Tell me if it works to you.
I have a scale that connect to PC through RS232, I send "W" to receive the weight. The scale sends the weight all the time as it's read. How do I catch the weight that is being read?
Can i get any C# sample code?
Sending a W? Sounds like the Mettler Toledo scale that FedEx gives businesses. I happen to have some code that reads from such a scale:
// where this.port is an instance of SerialPort, ie
// this.port = new SerialPort(
// portName,
// 1200,
// Parity.None,
// 8,
// StopBits.One);
// this.port.Open();
protected override bool GetWeight(out decimal weightLB, out bool stable)
{
stable = false;
weightLB = 0;
try
{
string data;
this.port.Write("W\r\n");
Thread.Sleep(500);
data = this.port.ReadExisting();
if (data == null || data.Length < 12 || data.Substring(8, 2) != "LB")
{
return false;
}
if (decimal.TryParse(data.Substring(1, 7), out weightLB))
{
stable = (data[11] == '0');
return true;
}
}
catch (TimeoutException)
{
return false;
}
return false;
}
You need to use SerialPort component of .NET. The full description and examples are available on MSDN site: http://msdn.microsoft.com/en-us/library/system.io.ports.serialport.aspx