We Keep Coding

UDP SocketAsyncEventArgs mismatching data sent and received by server and client

I'm trying to learn Game Networking programming so I started a very simple async udp socket system using Unity, however when I added JSON serialization things got a little bit weird.
I can connect to the server and send packets with the SendPacket(string msg) method and it will receive it fine on the server side. And it will work fine as long as the size of the msg variable is the same size or bigger. So if I send a string "Hello" and then one "Hello World" will work fine and be able to print it on the other size. However if I was now to send another packet with the string "Hi" nothing would happen and no more connections will work on that socket neither sending new packets.
I have checked and I'm receiving the packet over the network and it's not empty it has the information however it seems to stop when it gets to the code:
Packet p = e.Buffer.FromJsonBinary<Packet>();
Which is on the OnConnect function on my server side.After that function It seems my server stops listening not accepting new connections and no other packets will be receive, I suspect it somehow stops my async process.
For the Json Serialization I'm using JsonUtility from Unity.
Any ideas what is happening?
Server Class
public class Server
{
private static string _protocolID = "hash";
private static ushort _port = 11000;
private Socket _socket;
private SocketAsyncEventArgs _event;
private List<Socket> _connections = new List<Socket>();
public void Start()
{
Listen();
}
private bool Listen()
{
// Create UDP Socket
_socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
// Set the socket into non-blocking mode
_socket.Blocking = false;
try
{
_socket.Bind(new IPEndPoint(IPAddress.Any, _port));
_event = new SocketAsyncEventArgs();
_event.Completed += OnConnect;
byte[] buffer = new byte[1024];
_event.SetBuffer(buffer, 0, 1024);
//_socket.ReceiveAsync(_event);
StartListening(_event);
}
catch (Exception e)
{
Debug.LogException(e);
return false;
}
return true;
}
private void StartListening(SocketAsyncEventArgs e)
{
//e.AcceptSocket = null;
_socket.ReceiveAsync(e);
}
private void OnConnect(object sender, SocketAsyncEventArgs e)
{
if (e.BytesTransferred > 0)
{
if (e.SocketError != SocketError.Success)
Debug.Log("ERROR"); // TODO: Close Socket
Packet p = e.Buffer.FromJsonBinary<Packet>();
if (p._protocolID != _protocolID)
Debug.Log("Protocol Error");
Debug.Log("Connect:" + p._msg);
if (!_socket.ReceiveAsync(e))
{
// Call completed synchonously
StartListening(e);
}
}
else
{
Debug.Log("No data");
}
}
}
Client Class
public class Client
{
private static string _protocolID = "hash";
private ushort _port = 11000;
private string _ip = "127.0.0.1";
private Socket _socket;
private SocketAsyncEventArgs _event;
public bool Connect()
{
// Create UDP Socket
_socket = new Socket(SocketType.Dgram, ProtocolType.Udp);
// Set the socket into non-blocking mode
_socket.Blocking = false;
IPEndPoint ip = new IPEndPoint(IPAddress.Parse(_ip), _port);
_event = new SocketAsyncEventArgs();
//_event.Completed += Callback;
try
{
_socket.Connect(ip);
Debug.Log($"Connection was to {_ip} was sucessfull");
}
catch(Exception e)
{
Debug.LogException(e);
Debug.Log("Couldn't connect Socket");
return false;
}
return true;
}
public void SendPacket<T>(T t)
{
try
{
if (_socket == null)
Debug.Log("Null socket");
// Encode the data string into a byte array.
byte[] buffer = t.ToJsonBinary();
_event.SetBuffer(buffer, 0, buffer.Length);
// Send the data through the socket.
//_socket.SendAsync(_event);
bool willRaiseEvent = _socket.SendAsync(_event);
//if (!willRaiseEvent)
//{
// SendPacket<T>(t);
//}
}
catch (SocketException e)
{a
Debug.LogException(e);
Debug.Log("Couldn't Send Packet");
}
}
public void SendPacket(string msg)
{
Packet packet = new Packet(_protocolID, msg);
SendPacket<Packet>(packet);
}
}
Json Serialization
public static byte[] ToJsonBinary(this object obj)
{
return Encoding.ASCII.GetBytes(JsonUtility.ToJson(obj));
}
public static T FromJsonBinary<T>(this byte[] data)
{
return JsonUtility.FromJson<T>(Encoding.ASCII.GetString(data));
}
Packet
[Serializable]
public struct Packet
{
public string _protocolID;
public string _msg;
public Packet(string protocolID, string msg)
{
_protocolID = protocolID;
_msg = msg;
}
}
Edit: I found out it's crashing due to Json Utility
System.ArgumentException: JSON parse error: The document root must not follow by other values.
Json string before being sent (top), Json string reiceived at the server (bottom)
It seems on the server the buffer isn't being cleared so it still has information in it.

Okay so it seems the problem was due to how SocketAsyncEventArgs works. Since I'm calling recursively and passing the same event it never clears it's buffer.
if (!_socket.ReceiveAsync(e))
{
OnConnect(null, e);
}
I found two solutions one was to do pass a new SocketAsyncEventArgs which will have a clear buffer. But I believe a better idea would be to make a pool to handle and reuse a set of predefined SocketAsyncEventArgs instead of constantly creating new ones.
Another solution I found was to simple set a new buffer as a way to clear the last one.
byte[] buffer = t.ToJsonBinary();
_event.SetBuffer(buffer, 0, buffer.Length);

Can a NetMQ router socket (server) detect if a req socket (client) has reached a timeout?

I am using NetMQ v3.3.3.4 to develop a simple server(RouterSocket) - client(RequestSocket) scenario. It works as expected except for one thing.
Sometimes the server takes too long to return a response and the client reaches a timeout. This behavior is desired. The issue here is that the server does not detect that the client is already gone and it continues processing the request. At some point the processing is done and the server tries to return a response. The server never realizes that the client is not listening anymore and thinks that a response has been successfully delivered. This leads to a misleading user feedback and wrong log file entries.
Is it possible for the server to detect if a client is still waiting for a response?
Server
private void Listen()
{
using (var poller = new NetMQPoller())
using (var responseSocket = new RouterSocket())
using (var poisonedSocket = new ResponseSocket())
{
responseSocket.Bind(_bindServerString);
poisonedSocket.Bind(_bindTerminationString);
var keepAliveTimer = new NetMQTimer(KeepAliveInterval);
keepAliveTimer.Elapsed += (s, a) =>
{
_monitorPublisher.Status["KeepAlive"] = Application.LocalDateTime.Now.ToString();
_monitorPublisher.SendStatus();
};
poisonedSocket.ReceiveReady += (s, a) =>
{
a.Socket.ReceiveFrameBytes();
poller.Stop();
};
responseSocket.ReceiveReady += (s, a) =>
{
try
{
bool header = true;
byte[] response = null;
while (a.Socket.TryReceiveFrameBytes(out byte[] message))
{
// Using a ROUTER socket...
if (header)
{
// We have to first, wait for the first frame,
// which is the address (header) of the REQ socket.
// Afterwards, we have to wait for the second frame,
// which is 0 and act as a delimiter between the REQ address
// and the actual body of the request.
a.Socket.SendFrame(message, true);
// Once we have the header delimiter, we can be sure that
// the next frame is the actual request.
header = !(message.Length == 0);
}
else
{
// Parse the request and return a response
BridgeRequestReceived?.Invoke(message, out response);
a.Socket.SendFrame(response);
}
}
}
catch (Exception e)
{
Logger.Instance.Log(e);
}
};
poller.Add(keepAliveTimer);
poller.Add(responseSocket);
poller.Add(poisonedSocket);
Logger.Instance.Log($"Receiving requests on: {_bindServerString}", Application.LogCategory.Info);
poller.Run();
responseSocket.Close();
poisonedSocket.Close();
}
}
Client
private string SendRequest(IBridgeMessage request, out Exception e)
{
e = null;
string response = null;
using (var socket = new RequestSocket())
{
try
{
socket.Connect(_endpoint);
byte[] encryptedRequest = CryptoAES.StandardCrypto.Encrypt(Encoding.UTF8.GetBytes(NMTBridgeMessageFactory.Serialize(request)));
socket.SendFrame(encryptedRequest);
if (socket.TryReceiveFrameBytes(_timeout, out byte[] reply))
{
response = Encoding.UTF8.GetString(CryptoAES.StandardCrypto.Decrypt(reply));
}
else
{
e = new TimeoutException("Timeout occured!");
response = null;
}
}
catch (Exception ex)
{
e = ex;
response = null;
}
}
return response;
}
I expect the server detects a client disconnection.

Using C# TcpClient to receive a JSON based protocol

I'm connecting to a server that talks in a JSON protocol. It sends each bit of information as a JSON object. Here are three examples.
{"from":"SERVER","code":"SERVER_OK"}
{"from":"SERVER","code":"CHAT_JOIN","data":{"room":"Lobby"}}
{"from":"SERVER","code":"PING","data":{"time":1405901428001}}
My C# code looks like this.
void Start () {
clientSocket = new TcpClient();
clientSocket.Connect("127.0.0.1", 5000);
serverStream = clientSocket.GetStream();
serverStream.BeginRead(buffer, 0, buffer.Length, ReadComplete, buffer);
}
void ReadComplete (IAsyncResult iar) {
buffer = (byte[])iar.AsyncState;
int bytesAvailable = serverStream.EndRead(iar);
string data = System.Text.Encoding.UTF8.GetString(buffer);
Array.Clear(buffer, 0, 4096);
serverStream.BeginRead(buffer, 0, buffer.Length, ReadComplete, buffer);
Debug.Log(data);
}
My debug log looks like this:
{"from":"SERVER","code":"SERVER_OK"}{"from":"SERVER","code":"CHAT_JOIN","data":{"room":"Lobby"}}
{"from":"SERVER","code":"PING","data":{"time":1405901428001}}
{"from":"SERVER","code":"PING","data":{"time":1405901433001}}
{"from":"SERVER","code":"PING","data":{"time":1405901438004}}
It looks like I can receive more than one JSON object at a time in each ReadComplete. I also assume I could receive a partial JSON object as well. What do I need to do to be able to process a single JSON object at a time? I'm guessing I have to concatenate each received chunk of data to a string and the chop off the front of it each object one at a time. I just have no idea how to go about doing that.

I ended up moving into a Thread and processing the stream one byte at a time looking for the JSON object boundaries. For each on I try to parse it and add it a Queue for the parent thread to process.
So far this seems to work without causing any threading issues with the rest of my application and has been working well with the overall performance for my needs.
// The network thread listening to the server
private void NetworkThread () {
Debug.Log("Connecting to server...");
clientSocket = new TcpClient();
clientSocket.Connect("127.0.0.1", 5000);
stream = clientSocket.GetStream();
int braces = 0;
bool inQ = false;
char lastB = ' ';
while (!stopThread) {
char b = (char)stream.ReadByte();
if (b < 0)
return;
buffer.Append((char)b);
if (b == '"' && lastB != '\\') {
inQ = !inQ;
}
else if (b == '{' && !inQ) {
braces += 1;
}
else if (b == '}' && !inQ) {
braces -= 1;
}
lastB = (char)b;
if (braces == 0) {
try {
JSONNode packet = JSONNode.Parse(buffer.ToString());
buffer = new StringBuilder();
lock (lockQueue) {
packetQueue.Enqueue(packet);
}
} catch (Exception e) {
}
}
}
}

Instat of manually reciving the data use Streamreader and it's .ReadLine() method. It looks like the server sends line for line, so it should not be a problem to read response for response.

How to send APNS push notification (iOS) from C# without queuing

It seems everyone uses PushSharp for sending push notifications to iOS devices from C#. But that library has a queue it uses instead of sending the notification directly, which then means you need a Windows Service or something to host it properly (per its own documentation) which is overkill for me. I have an incoming web request to my ASP.NET web service and as part of handling that, I want to immediately send a push notification. Simple as that.
Can anyone tell me either how to use PushSharp to send one immediately (bypassing its queue mechanism) or how to properly send the push notification myself? I already have the code that formulates the JSON message, but I don't know how to apply the .p12 file to the request. I can't find any Apple documentation for how to do that.

This is a old question, but the answer is not complete.
Here my code:
// private fields
private static readonly string _apnsHostName = ConfigurationManager.AppSettings["APNS:HostName"];
private static readonly int _apnsPort = int.Parse(ConfigurationManager.AppSettings["APNS:Port"]);
private static readonly string _apnsCertPassword = ConfigurationManager.AppSettings["APNS:CertPassword"];
private static readonly string _apnsCertSubject = ConfigurationManager.AppSettings["APNS:CertSubject"];
private static readonly string _apnsCertPath = ConfigurationManager.AppSettings["APNS:CertPath"];
private readonly ILogger _log;
private X509Certificate2Collection _certificatesCollection;
ctor <TAB key>(ILogger log)
{
_log = log ?? throw new ArgumentNullException(nameof(log));
// load .p12 certificate in the collection
var cert = new X509Certificate2(_apnsCertPath, _apnsCertPassword);
_certificatesCollection = new X509Certificate2Collection(cert);
}
public async Task SendAppleNativeNotificationAsync(string payload, Registration registration)
{
try
{
// handle is the iOS device Token
var handle = registration.Handle;
// instantiate new TcpClient with ApnsHostName and Port
var client = new TcpClient(_apnsHostName, _apnsPort);
// add fake validation
var sslStream = new SslStream(client.GetStream(), false, new RemoteCertificateValidationCallback(ValidateServerCertificate), null);
try
{
// authenticate ssl stream on ApnsHostName with your .p12 certificate
sslStream.AuthenticateAsClient(_apnsHostName, _certificatesCollection, SslProtocols.Tls, false);
var memoryStream = new MemoryStream();
var writer = new BinaryWriter(memoryStream);
// command
writer.Write((byte)0);
// first byte of the deviceId length (big-endian first byte)
writer.Write((byte)0);
// deviceId length (big-endian second byte)
writer.Write((byte)32);
// deviceId data (byte[])
writer.Write(HexStringToByteArray(handle.ToUpper()));
// first byte of payload length; (big-endian first byte)
writer.Write((byte)0);
// payload length (big-endian second byte)
writer.Write((byte)Encoding.UTF8.GetByteCount(payload));
byte[] b1 = Encoding.UTF8.GetBytes(payload);
// payload data (byte[])
writer.Write(b1);
writer.Flush();
byte[] array = memoryStream.ToArray();
await sslStream.WriteAsync(array, 0, array.Length);
// TIP: do not wait a response from APNS because APNS return a response only when an error occurs;
// so if you wait the response your code will remain stuck here.
// await ReadTcpResponse();
sslStream.Flush();
// close client
client.Close();
}
catch (AuthenticationException ex)
{
_log.Error($"Error sending APNS notification. Exception: {ex}");
client.Close();
}
catch (Exception ex)
{
_log.Error($"Error sending APNS notification. Exception: {ex}");
client.Close();
}
}
catch (Exception ex)
{
_log.Error($"Error sending APNS notification. Exception: {ex}");
}
}
private static byte[] HexStringToByteArray(string hex)
{
if (hex == null)
{
return null;
}
// added for newest devices (>= iPhone 8)
if (hex.Length % 2 == 1)
{
hex = '0' + hex;
}
return Enumerable.Range(0, hex.Length)
.Where(x => x % 2 == 0)
.Select(x => Convert.ToByte(hex.Substring(x, 2), 16))
.ToArray();
}
private static bool ValidateServerCertificate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
return true;
//if (sslPolicyErrors == SslPolicyErrors.None)
// return true;
//// do not allow this client to communicate with unauthenticated servers.
//return false;
}
private async Task<byte[]> ReadTcpResponse(SslStream sslStream)
{
MemoryStream ms = new MemoryStream();
byte[] buffer = new byte[2048];
int bytes = -1;
do
{
bytes = await sslStream.ReadAsync(buffer, 0, buffer.Length);
await ms.WriteAsync(buffer, 0, bytes);
} while (bytes != 0);
return ms.ToArray();
}
TIP: with iOS13, device token is received differently.
> iOS 12 (deviceToken as NSData).description -> "< your_token_here >"
> iOS 13 (deviceToken as NSData).description -> "{ length = 32, bytes = 0x321e1ba1c1ba...token_in_bytes }"
With iOS13 you must convert token to string or skip the method
'HexStringToByteArray' because you already have a byte[].
If you have question, I'm glad to answer.

I spent many hours trying to find a way to push notifications, then I found a piece of code that did it for me.
First of all make sure that you installed the certificates correctly, here is a link which will help you.
https://arashnorouzi.wordpress.com/2011/04/13/sending-apple-push-notifications-in-asp-net-%E2%80%93-part-3-apns-certificates-registration-on-windows/
Here is a code I used to push notifications:
public static bool ConnectToAPNS(string deviceId, string message)
{
X509Certificate2Collection certs = new X509Certificate2Collection();
// Add the Apple cert to our collection
certs.Add(getServerCert());
// Apple development server address
string apsHost;
/*
if (getServerCert().ToString().Contains("Production"))
apsHost = "gateway.push.apple.com";
else*/
apsHost = "gateway.sandbox.push.apple.com";
// Create a TCP socket connection to the Apple server on port 2195
TcpClient tcpClient = new TcpClient(apsHost, 2195);
// Create a new SSL stream over the connection
SslStream sslStream1 = new SslStream(tcpClient.GetStream());
// Authenticate using the Apple cert
sslStream1.AuthenticateAsClient(apsHost, certs, SslProtocols.Default, false);
PushMessage(deviceId, message, sslStream1);
return true;
}
private static X509Certificate getServerCert()
{
X509Certificate test = new X509Certificate();
//Open the cert store on local machine
X509Store store = new X509Store(StoreLocation.CurrentUser);
if (store != null)
{
// store exists, so open it and search through the certs for the Apple Cert
store.Open(OpenFlags.ReadOnly);
X509Certificate2Collection certs = store.Certificates;
if (certs.Count > 0)
{
int i;
for (i = 0; i < certs.Count; i++)
{
X509Certificate2 cert = certs[i];
if (cert.FriendlyName.Contains("Apple Development IOS Push Services"))
{
//Cert found, so return it.
Console.WriteLine("Found It!");
return certs[i];
}
}
}
return test;
}
return test;
}
private static byte[] HexToData(string hexString)
{
if (hexString == null)
return null;
if (hexString.Length % 2 == 1)
hexString = '0' + hexString; // Up to you whether to pad the first or last byte
byte[] data = new byte[hexString.Length / 2];
for (int i = 0; i < data.Length; i++)
data[i] = Convert.ToByte(hexString.Substring(i * 2, 2), 16);
return data;
}
Note that this code is for development certificates "Apple Development IOS Push Services".

C# Asynchronous read and write with NetworkStream

I have built a server that receives requests from a client and gives a response that depends on the request Type. If the request type is streaming, the server must send data array. While the server’s streaming data the client may send a stop request to stop the streaming. If the request and the response is transferred on the same TCP connection the server only receives the stop request when all the data has finished streaming to the client. I think I must use Asynchronous write to solve this problem. This is my code:
First I create a loop back to receive connection from clients:
while (!done)
{
try
{
Socket socket = listener.AcceptSocket();
ClientInteraction clIr = new ClientInteraction(socket, statusList);
Thread thread = new Thread(new ThreadStart(clIr.Process));
thread.Start();
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
In Process function of ClientInteraction class :
Public void Process()
{
ns = new NetworkStream(socket);
while (true)
{
try
{
this.myReadBuffer = new byte[socket.ReceiveBufferSize];
this.numberOfBytesRead = ns.Read(myReadBuffer, 0, myReadBuffer.Length);
}
catch
{
break;
}
if (numberOfBytesRead == 0)
{
break;
}
else
{
HandleRequest(myReadBuffer, numberOfBytesRead);
}
}
}
In HandleRequest Function, if request’s STREAM, I will send data in an array to client:
Public void HanldeRequest(……)
{
myCompleteMessage = "";
myCompleteMessage =
String.Concat(myCompleteMessage, Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));
If(myCompleteMessage == “Stream”)
{
//I get data and call SendData function
foreach(.......)
{
//get data
........
SendData(data);
}
}
}
public void SendData(byte[] data)
{
try
{
//ns.Write(data, 0, data.Length);
ns.BeginWrite(data, 0, data.Length, new AsyncCallback(StreamData), null);
}
catch
{
}
}
public void StreamData(IAsyncResult asynResult)
{
if(asynResult != null)
ns.EndWrite(asynResult);
}
With this code, I connected with client, send data to client. But I still can’t receive Stop request until all data is streamed. Please show me the correct way to fix my problem. Thank you.

I think you can try using multithread to solve this problem. I have met this circumstance and multithread is a good choice to solve. You can create a thread to write and a thread to read.