We Keep Coding

await causing web socket connections to close

I need to establish multiple web socket connections that will forever receive updates unless they quit through some case. I need to have these socket connections be awaited since this loop of polling for update events is limiting the number of threads that can be run at once thus limiting the number of connections I can open.
I attempted to solve this problem by using a semaphore as a way to detect events (the events are being observed through a IObserver interface)
static SemaphoreSlim send = new SemaphoreSlim(0,1);
static object newUpdate;
static numberOfClients = 0;
and the loop to poll and possibly receive updates or terminate
while(true)
{
await send.WaitAsync() //wait the thread so others can start
//do some things with the updated valu and send data or possibly quit
}
and in the OnNext implementation that detects events
public void OnNext(object value)
{
newUpdate = value;
for (int i = 0; i < numberOfClients; i++)
{
send.Release();
Thread.Sleep(100);
}
}
and the client implementation:
do
{
try
{
WebSocketReceiveResult result = await websocket.ReceiveAsync(buffer, CancellationToken.None);
byte[] receivedData = new byte[result.Count];
Buffer.BlockCopy(buffer.Array, 0, receivedData, 0, result.Count);
if (result.MessageType == WebSocketMessageType.Text)
{
Console.WriteLine("\nReceived message: " + Encoding.UTF8.GetString(receivedData));
}
}
catch (WebSocketException e)
{
if (websocket.State == WebSocketState.Aborted)
{
break;
}
continue;
}
if (result.EndOfMessage)
{
break;
}
}
while (websocket.State == WebSocketState.Open);
When I execute this it seems that the line:
await send.WaitAsync()
immediately makes the client code crash when it tries to call await .ReceiveAsync() and cries:
"Unable to read data from transport connection. An existing connection was forcibly closed by the remote host".
Any ideas how I can accomplish something like this? This is using OWIN WebSocket Extension.

Why I am getting program hang on the AcceptTcpClient?

Why I am getting program hang on the tcpClient = this.ss.AcceptTcpClient?
public virtual void Run()
{
if (this.tcpListener == null)
return;
TcpClient tcpClient = (TcpClient)null;
while (!this.m_Stop)
{
try
{
tcpClient = this.tcpListener.AcceptTcpClient();
ThreadPool.QueueUserWorkItem(new WaitCallback(this.handler.Handle), (object)tcpClient);
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
}
Here ss is the TcpListener.

I googled for TcpListener.AcceptTcpClient Method. This is what I found.
AcceptTcpClient is a blocking method that returns a TcpClient that you
can use to send and receive data. Use the Pending method to determine
if connection requests are available in the incoming connection queue
if you want to avoid blocking.
It seems that the method waits till there is a connection request available. So either use the Pending method call or do this on a background thread (not the UI thread).

Lock on dedicated lock-object causes deadlock

Yesterday I came accross a strange problem, which gave me quite some headaches. I have a server application with a Server class, which in turn is derived from a Connection class. The Connection class provides information about the connection state and the possibility to close the connection
public bool Connected
{
get
{
if (connection != null)
{
lock (lockObject)
{
bool blockingState = connection.Blocking;
try
{
connection.Blocking = false;
connection.Send(new byte[1], 1, 0);
}
catch (SocketException e)
{
if (!e.NativeErrorCode.Equals(10035))
{
return false;
}
//is connected, but would block
}
finally
{
connection.Blocking = blockingState;
}
return connection.Connected;
}
}
return false;
}
}
public virtual void CloseConnection()
{
if (Connected)
{
lock (lockObject)
{
connection.Close();
}
}
}
The Server class is resonsible for actually sending data
private void ConnectAndPollForData()
{
try
{
TcpListener listener = new TcpListener(Port);
listener.Start();
while (true)
{
connection = listener.AcceptSocket();
string currentBuffr = string.Empty;
const int READ_BUFFER_SIZE = 1024;
byte[] readBuffr = new byte[READ_BUFFER_SIZE];
while (Connected)
{
int bytesReceived;
lock (lockObject)
{
bytesReceived = connection.Receive(readBuffr, READ_BUFFER_SIZE, SocketFlags.None);
}
currentBuffr += ASCIIEncoding.ASCII.GetString(readBuffr, 0, bytesReceived);
//do stuff
}
}
catch(ThreadAbortException)
{
Thread.ResetAbort();
}
finally
{
}
}
public void SendString(string stringToSend)
{
stringToSend += "\r\n";
if(Connected)
{
lock(lockObject)
{
connection.Send(ASCIIEncoding.UTF7.GetBytes(stringToSend));
}
}
}
There is no other explicit access to the connection object. The ConnectAndPollForData function executes in a seperate thread. Whenever I ran the host in this version (I am currently using a non thread-safe version, which causes other problems) it hang after quite a few lines received via TCP. Pausing the debugger showed me, that one thread tried to execute the code withing the lock of Connected, while the other tried to receive data in the lock of ConnectAndPollForData. This behavior seems strange to me, for I would expect to execute the code within the first lock and then do the second. There do seem to be similar problems when using callbacks like in Deadlocking lock() method or 'Deadlock' with only one locked object? but the situation here is a bit different, for in my situation (I think) the code within the locks should not emit any events that themselves try to obtain a lock on the object.

Let's assume it gets the lock in the second method first. So it is holding the lock, and waiting for data. It is unclear whether this is directly receiving the data sent by the first method, or whether this is looking for a reply from an unrelated server - a reply to the message sent in the first method. But either way, I'm assuming that there won't be data incoming until the outbound message is sent.
Now consider: the outbound message can't be sent, because you are holding an exclusive lock.
So yes, you've deadlocked yourself. Basically, don't do that. There is no need to synchronize between inbound and outbound socket operations, even on the same socket. And since it makes very little sense to have concurrent readers on the same socket, or concurrent writers, I'm guessing you don't actually need those locks at all.

Efficient multiple tcp clients with intervals

I have been writing a command line program in C# that uses multiple tcp clients that all connect to the same server. Each client resides in it's own thread. At the moment I am trying to work out an effective method of spreading say 5 requests a second efficiently between let's say 4 threads.
My code currently looks like the following but I still end up with requests overlapping each other. Does anyone have any idea how to prevent these overlaps effectively?
// Max connections is 4, interval is 200
// Loop once to give tcp clients chance to connect
var tcpClients = new TcpClient[_maxConnections];
for(int i = 0; i < _maxConnections; i++)
{
tcpClients[i] = new TcpClient();
tcpClients[i].Connect(host, port);
}
// Loop again to setup tasks
for(int i = 0; i < _maxConnections; i++)
{
Task.Factory.StartNew(TcpHandler, tcpClients[i]);
// Sleep so every task starts separate from each other.
Thread.Sleep(_interval);
}
And then the TcpHandler code looks like:
public static void TcpHandler(Object o)
{
// active is already declared
while(_active)
{
var tcpClient = (TcpClient) o;
// .. do some send and receive...
Console.WriteLine("Something here..");
Thread.Sleep(_interval * _maxConnections);
}
}
So as you can see I am sleeping to provide sufficient space between each thread executing yet now and then they still overlap.
How can I make this threads run parallel without any overlap and limit to 5 times a second spread across all 4?
Or am I going about this all wrong?

Presuming each client requires a separate thread, and that only one thread may be communicating with the server at a given time (no overlap), a lock in the TcpHandler method should suffice:
// Max connections is 4, interval is 200
// Loop once to give tcp clients chance to connect
var tcpClients = new TcpClient[_maxConnections];
// dedicated lock object
static readonly object lockObject = new object();
And then in your TcpHandler method
public static void TcpHandler(Object o)
{
// active is already declared
while(_active)
{
//DO NON-SOCKET RELATED STUFF HERE
// ... code ...
//
//DO SOCKET RELATED STUFF HERE
lock(lockObject)
{
var tcpClient = (TcpClient) o;
// .. do some send and receive...
Console.WriteLine("Something here..");
Thread.Sleep(_interval * _maxConnections);
}
}
}

I am not quite sure why you are doing this but I have used System.Timers (actually an array of timers) in windows services and have staggered the start (intervals).
In the Elapse event maybe you could use a lock(myobject) { } so they don't overlap?
Gina

I think you are using sleep to manage connection times.. Why not instead setup a "Maximum connection delay" then use BeginConnect and a Timer to look after the connection.
eg.
//setup a timer variable
TCPClient connectionOpening;
_connecting = true;
_connected = false;
connectionOpening = tcpClient;
timer.change(5000, Infinite)
tcpClient.BeginConnect(ClientConnectCallback, tcpClient)
void ClientConnectCallback(iasyncresult ar)
{
_timer.change(infinite, infinite);
TCPClient tcp = (TCPClient)ar.AsyncState;
try
{
//if we have timed out because our time will abort the connect
tcp.EndConnect(ar);
_connected = true;
_connecting = false;
//we are now connected... do the rest you want to do.
//get the stream and BeginRead etc.
}
catch (Exception ex) // use the proper exceptions IOException , socketException etc
{
if (!_connecting)
{
//We terminated the connection because our timer ticked.
}
else
{
//some other problem that we weren't expecting
}
}
void TimerTick(object state)
{
_connecting = false;
_connected = false;
connectionOpening.Close();
}

fast parallel TcpClients with await BeginConnect new Thread Pulse

I am trying to establish a TCP connection with a number of IPs in parallel, and do that as fast as possible. I have converted some older code to use AsyncCTP for that purpose, introducing the parallelism.
Changes to Design and Speed, and Accessing Successful Connections?
My question is three-fold:
How bad is the following flow / what should I change?
i.e. the await starts a bunch of parallel TcpRequest threads,
but within each TcpRequest there is a tcpClient.BeginConnect
as well as another thread being spawn for reading (if connection is successful)
and the writing to the stream is done with a Wait / Pulse mechanism in a while loop.
Secondly, how could i make the process of connecting to a number of targets faster?
Currently, if the ip:port targets are not actually running any servers, then i get the "All Done" printed after about 18 seconds from the start, when trying to connect to about 500 local targets (that are not listening, and thus fail, on those ports).
How could i access the WriteToQueue method of successful connections, from the mothership?
Async Mothership Trying to Connect to All Targets in Parallel
// First get a bunch of IPAddress:Port targets
var endpoints = EndPointer.Get();
// Try connect to all those targets
var tasks = from t in topList select TcpRequester.ConnectAsync(t);
await TaskEx.WhenAll(tasks);
Debug.WriteLine("All Done");
Static Accessor for Individual TcpRequest Tasks
public static Task<TcpRequester> ConnectAsync(IPEndPoint endPoint)
{
var tcpRequester = Task<TcpRequester>.Factory.StartNew(() =>
{
var request = new TcpRequester();
request.Connect(endPoint);
return request;
}
);
return tcpRequester;
}
TcpRequester with BeginConnect TimeOut and new Thread for Reading
public void Connect(IPEndPoint endPoint)
{
TcpClient tcpClient = null;
Stream stream = null;
using (tcpClient = new TcpClient())
{
tcpClient.ReceiveTimeout = 1000;
tcpClient.SendTimeout = 1000;
IAsyncResult ar = tcpClient.BeginConnect(endPoint.Address, endPoint.Port, null, null);
WaitHandle wh;
wh = ar.AsyncWaitHandle;
try
{
if (!ar.AsyncWaitHandle.WaitOne(TimeSpan.FromMilliseconds(1000), false))
{
throw new TimeoutException();
}
if (tcpClient.Client != null)
{
// Success
tcpClient.EndConnect(ar);
}
if (tcpClient.Connected)
{
stream = tcpClient.GetStream();
}
// Start to read stream until told to close or remote close
ThreadStart reader = () => Read(stream);
// Reading is done in a separate thread
var thread = new Thread(reader);
thread.Start();
// See Writer method below
Writer(stream);
} finally
{
wh.Close();
}
}
} catch (Exception ex)
{
if (tcpClient != null)
tcpClient.Close();
}
}
}
Writing to Stream with Wait and Pulse
readonly Object _writeLock = new Object();
public void WriteToQueue(String message)
{
_bytesToBeWritten.Add(Convert(message));
lock (_writeLock)
{
Monitor.Pulse(_writeLock);
}
}
void Writer(Stream stream)
{
while (!_halt)
{
while (_bytesToBeWritten.Count > 0 && !_halt)
{
// Write method does the actual writing to the stream:
if (Write(stream, _bytesToBeWritten.ElementAt(0)))
{
_bytesToBeWritten.RemoveAt(0);
} else
{
Discontinue();
}
}
if (!(_bytesToBeWritten.Count > 0) && !_halt)
{
lock (_writeLock)
{
Monitor.Wait(_writeLock);
}
}
}
Debug.WriteLine("Discontinuing Writer and TcpRequester");
}

There are a few red flags that pop out at a cursory glance.
You have this Stream that is accepting reads and writes, but there is no clear indication that the operations have been synchronized appropriately. The documentation does state that a Stream's instance methods are not safe for multithreaded operations.
There does not appear to be synchronization around operations involving _bytesToBeWritten.
Acquiring a lock solely to execute Monitor.Wait and Monitor.Pulse is a little weird, if not downright incorrect. It is basically equivalent to using a ManualResetEvent.
It is almost never correct to use Monitor.Wait without a while loop. To understand why you have to understand the purpose of pulsing and waiting on a lock. That is really outside the scope of this answer.
It appears like the Writer and WriteToQueue methods are an attempt to generate a producer-consumer queue. The .NET BCL already contains the innards for this via the BlockingCollection class.
For what it is worth I see nothing flagrantly wrong with the general approach and usage of the await keyword.