Detect dead connections with SignalR - c#

SignalR version: SignalR 2.4.1
.Net Framework version: 4.8 (I am not using .Net Core)
SignalR transport: websockets
I am developing a background service for SignalR (PresenceMonitor) where I need to detect whether a connection with specific clientid is alive or not.
I am using the following code for Presence Monitor to start with what I want to achieve:
using System;
using System.Data.Entity.SqlServer;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using Microsoft.AspNet.SignalR.Transports;
namespace UserPresence
{
/// <summary>
/// This class keeps track of connections that the <see cref="UserTrackingHub"/>
/// has seen. It uses a time based system to verify if connections are *actually* still online.
/// Using this class combined with the connection events SignalR raises will ensure
/// that your database will always be in sync with what SignalR is seeing.
/// </summary>
public class PresenceMonitor
{
private readonly ITransportHeartbeat _heartbeat;
private Timer _timer;
// How often we plan to check if the connections in our store are valid
private readonly TimeSpan _presenceCheckInterval = TimeSpan.FromSeconds(10);
// How many periods need pass without an update to consider a connection invalid
private const int periodsBeforeConsideringZombie = 3;
// The number of seconds that have to pass to consider a connection invalid.
private readonly int _zombieThreshold;
public PresenceMonitor(ITransportHeartbeat heartbeat)
{
_heartbeat = heartbeat;
_zombieThreshold = (int)_presenceCheckInterval.TotalSeconds * periodsBeforeConsideringZombie;
}
public void StartMonitoring()
{
if (_timer == null)
{
_timer = new Timer(_ =>
{
try
{
Check();
}
catch (Exception ex)
{
// Don't throw on background threads, it'll kill the entire process
Trace.TraceError(ex.Message);
}
},
null,
TimeSpan.Zero,
_presenceCheckInterval);
}
}
private void Check()
{
using (var db = new UserContext())
{
// Get all connections on this node and update the activity
foreach (var trackedConnection in _heartbeat.GetConnections())
{
if (!trackedConnection.IsAlive)
{
continue;
}
Connection connection = db.Connections.Find(trackedConnection.ConnectionId);
// Update the client's last activity
if (connection != null)
{
connection.LastActivity = DateTimeOffset.UtcNow;
}
else
{
// We have a connection that isn't tracked in our DB!
// This should *NEVER* happen
// Debugger.Launch();
}
}
// Now check all db connections to see if there's any zombies
// Remove all connections that haven't been updated based on our threshold
var zombies = db.Connections.Where(c =>
SqlFunctions.DateDiff("ss", c.LastActivity, DateTimeOffset.UtcNow) >= _zombieThreshold);
// We're doing ToList() since there's no MARS support on azure
foreach (var connection in zombies.ToList())
{
db.Connections.Remove(connection);
}
db.SaveChanges();
}
}
}
}
The issue I am facing is here:
// Get all connections on this node and update the activity
foreach (var trackedConnection in _heartbeat.GetConnections())
{
Scanning all the connections when there are large number of connections is deeply affecting the performance of my application and is giving lot of CPU spikes.
In my database, I already have the mapping for connection ids per user. Based on that I already a have field in my cache per user whether that user has any connection in db or not. Those mappings are are already cached. I would scan each of those mappings and would check whether the connection (connection id) for that specific user is is alive or not. I tried looking for ITransportHeartbeat Interface for the same but unfortunately, that interface gives us just these four methods:
//
// Summary:
// Manages tracking the state of connections.
public interface ITransportHeartbeat
{
//
// Summary:
// Adds a new connection to the list of tracked connections.
//
// Parameters:
// connection:
// The connection to be added.
//
// Returns:
// The connection it replaced, if any.
ITrackingConnection AddOrUpdateConnection(ITrackingConnection connection);
//
// Summary:
// Gets a list of connections being tracked.
//
// Returns:
// A list of connections.
IList<ITrackingConnection> GetConnections();
//
// Summary:
// Marks an existing connection as active.
//
// Parameters:
// connection:
// The connection to mark.
void MarkConnection(ITrackingConnection connection);
//
// Summary:
// Removes a connection from the list of tracked connections.
//
// Parameters:
// connection:
// The connection to remove.
void RemoveConnection(ITrackingConnection connection);
}
Ther is no method where I can get the state of connection by connectionid. Is there any way where I can get a specific connection information without scannig all the connetcions. I am aware of the traditional way to get that which could be using this: _heartbeat.GetConnections().Select(b => b.ConnectionId). But that code also will scan all the connections.
I am aware of OnDisconnected event also which we could use on a Hub itself but the OnDisconnected even doesn't guarantee to fire always (browser can close, internet shut down, site restart).
Is there any code which I could hook in my Hub itself to detect the ping done by the Heartbeat API? I could store the last pings per connection (kind of denormalize the way of detecting last ping) and can detect whether that connection is dead or not?
SignalR for .Net Core has something like that:
var heartbeat = Context.Features.Get<IConnectionHeartbeatFeature>();
heartbeat.OnHeartBeat(MyAction,
but I am looking for a similar feature like that in SignalR for .NET Framework.

Related

Azure Cosmos db throwing Socket Exceptions

I am using azure cosmos db with .net core 2.1 application. I am using gremlin driver with this. It's working fine but after every few days it start throwing socket exception on server and we have to recycle IIS pool. Average per day hits are 10000.
Now we are using default gateway mode. Should we have to switch to direct mode as it might be a firewall issue ?
Here is the implementation:
private DocumentClient GetDocumentClient( CosmosDbConnectionOptions configuration)
{
_documentClient = new DocumentClient(
new Uri(configuration.Endpoint),
configuration.AuthKey,
new ConnectionPolicy());
//create database if not exists
_documentClient.CreateDatabaseIfNotExistsAsync(new Database { Id = configuration.Database });
return _documentClient;
}
and in startup.cs:
services.AddSingleton(x => GetDocumentClient(cosmosDBConfig));
and here is how we are communicating with cosmos db:
private DocumentClient _documentClient;
private DocumentCollection _documentCollection;
private CosmosDbConnectionOptions _cosmosDBConfig;
public DocumentCollectionFactory(DocumentClient documentClient, CosmosDbConnectionOptions cosmosDBConfig)
{
_documentClient = documentClient;
_cosmosDBConfig = cosmosDBConfig;
}
public async Task<DocumentCollection> GetProfileCollectionAsync()
{
if (_documentCollection == null)
{
_documentCollection = await _documentClient.CreateDocumentCollectionIfNotExistsAsync(
UriFactory.CreateDatabaseUri(_cosmosDBConfig.Database),
new DocumentCollection { Id = _cosmosDBConfig.Collection },
new RequestOptions { OfferThroughput = _cosmosDBConfig.Throughput });
return _documentCollection;
}
return _documentCollection;
}
and then:
public async Task CreateProfile(Profile profile)
{
var graphCollection = await _graphCollection.GetProfileCollectionAsync();
var createQuery = GetCreateQuery(profile);
IDocumentQuery<dynamic> query = _documentClient.CreateGremlinQuery<dynamic>(graphCollection, createQuery);
if(query.HasMoreResults)
{
await query.ExecuteNextAsync();
}
}
I'm assuming that for communication with CosmosDB you are using HttpClient. The application should share a single instance of HttpClient.
Every time you make a connection after HttpClient disposal there are still a bunch of connections in the state of TIME_WAIT. This means that the connection was closed on one side ( OS ) but it is in "waiting for additional packets" state.
By default, Windows may hold this connection in this state for 240 seconds. There is a limit to how quickly OS can open new sockets. All this may lead to System.Net.Sockets.SocketException exception.
Very good article that explains in details why and how this problem appears digging into TCP diagram and explaining with more details.
UPDATED
Possible solution.
You are using the default ConnectionPolicy object. That object has a property called IdleTcpConnectionTimeout which controls the amount of idle time after which unused connections are closed. By default, idle connections are kept open indefinitely. The value must be greater than or equal to 10 minutes.
So the code could look like:
private DocumentClient GetDocumentClient( CosmosDbConnectionOptions configuration)
{
_documentClient = new DocumentClient(
new Uri(configuration.Endpoint),
configuration.AuthKey,
new ConnectionPolicy() {
IdleTcpConnectionTimeout = new TimeSpan(0,0,10,0)
});
//create database if not exists
_documentClient.CreateDatabaseIfNotExistsAsync(new Database { Id = configuration.Database });
return _documentClient;
}
Here is a link to ConnectionPolicy Class documentation

How to check database connection in MongoDB [duplicate]

I use MongoDB drivers to connect to the database. When my form loads, I want to set up connection and to check whether it is ok or not. I do it like this:
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var server = client.GetServer();
var database = server.GetDatabase("reestr");
But I do not know how to check connection. I tried to overlap this code with try-catch, but to no avail. Even if I make an incorrect connectionString, I still can not get any error message.
To ping the server with the new 3.0 driver its:
var database = client.GetDatabase("YourDbHere");
database.RunCommandAsync((Command<BsonDocument>)"{ping:1}")
.Wait();
There's a ping method for that:
var connectionString = "mongodb://localhost";
var client = new MongoClient(connectionString);
var server = client.GetServer();
server.Ping();
full example for 2.4.3 - where "client.GetServer()" isn't available.
based on "Paul Keister" answer.
client = new MongoClient("mongodb://localhost");
database = client.GetDatabase(mongoDbStr);
bool isMongoLive = database.RunCommandAsync((Command<BsonDocument>)"{ping:1}").Wait(1000);
if(isMongoLive)
{
// connected
}
else
{
// couldn't connect
}
I've had the same question as the OP, and tried every and each solution I was able to find on Internet...
Well, none of them worked to my true satisfaction, so I've opted for a research to find a reliable and responsive way of checking if connection to a MongoDB Database Server is alive. And this without to block the application's synchronous execution for too long time period...
So here are my prerequisites:
Synchronous processing of the connection check
Short to very short time slice for the connection check
Reliability of the connection check
If possible, not throwing exceptions and not triggering timeouts
I've provided a fresh MongoDB Installation (version 3.6) on the default localhost URL: mongodb://localhost:27017. I've also written down another URL, where there was no MongoDB Database Server: mongodb://localhost:27071.
I'm also using the C# Driver 2.4.4 and do not use the legacy implementation (MongoDB.Driver.Legacy assembly).
So my expectations are, when I'm checking the connection to the first URL, it should give to me the Ok for a alive connection to an existing MongoDB server, when I'm checking the connection to the second URL it should give to me the Fail for a non-existing MongoDB server...
Using the IMongoDatabase.RunCommand method, queries the server and causes the server response timeout to elapse, thus not qualifying against the prerequisites. Furthermore after the timeout, it breaks with a TimeoutException, which requires additional exception handling.
This actual SO question and also this SO question have delivered the most of the start information I needed for my solution... So guys, many thanks for this!
Now my solution:
private static bool ProbeForMongoDbConnection(string connectionString, string dbName)
{
var probeTask =
Task.Run(() =>
{
var isAlive = false;
var client = new MongoDB.Driver.MongoClient(connectionString);
for (var k = 0; k < 6; k++)
{
client.GetDatabase(dbName);
var server = client.Cluster.Description.Servers.FirstOrDefault();
isAlive = (server != null &&
server.HeartbeatException == null &&
server.State == MongoDB.Driver.Core.Servers.ServerState.Connected);
if (isAlive)
{
break;
}
System.Threading.Thread.Sleep(300);
}
return isAlive;
});
probeTask.Wait();
return probeTask.Result;
}
The idea behind this is the MongoDB Server does not react (and seems to be non-existing) until a real attempt is made to access some resource on the server (for example a database). But retrieving some resource alone is not enough, as the server still has no updates to its state in the server's Cluster Description. This update comes first, when the resource is retrieved again. From this time point, the server has valid Cluster Description and valid data inside it...
Generally it seems to me, the MongoDB Server does not proactivelly propagate its Cluster Description to all connected clients. Rather then, each client receives the description, when a request to the server has been made. If some of you fellows have more information on this, please either confirm or deny my understandings on the topic...
Now when we target an invalid MongoDB Server URL, then the Cluster Description remains invalid and we can catch and deliver an usable signal for this case...
So the following statements (for the valid URL)
// The admin database should exist on each MongoDB 3.6 Installation, if not explicitly deleted!
var isAlive = ProbeForMongoDbConnection("mongodb://localhost:27017", "admin");
Console.WriteLine("Connection to mongodb://localhost:27017 was " + (isAlive ? "successful!" : "NOT successful!"));
will print out
Connection to mongodb://localhost:27017 was successful!
and the statements (for the invalid URL)
// The admin database should exist on each MongoDB 3.6 Installation, if not explicitly deleted!
isAlive = ProbeForMongoDbConnection("mongodb://localhost:27071", "admin");
Console.WriteLine("Connection to mongodb://localhost:27071 was " + (isAlive ? "successful!" : "NOT successful!"));
will print out
Connection to mongodb://localhost:27071 was NOT successful!
Here a simple extension method to ping mongodb server
public static class MongoDbExt
{
public static bool Ping(this IMongoDatabase db, int secondToWait = 1)
{
if (secondToWait <= 0)
throw new ArgumentOutOfRangeException("secondToWait", secondToWait, "Must be at least 1 second");
return db.RunCommandAsync((Command<MongoDB.Bson.BsonDocument>)"{ping:1}").Wait(secondToWait * 1000);
}
}
You can use it like so:
var client = new MongoClient("yourConnectionString");
var database = client.GetDatabase("yourDatabase");
if (!database.Ping())
throw new Exception("Could not connect to MongoDb");
This is a solution by using the try-catch approach,
var database = client.GetDatabase("YourDbHere");
bool isMongoConnected;
try
{
await database.RunCommandAsync((Command<BsonDocument>)"{ping:1}");
isMongoConnected = true;
}
catch(Exception)
{
isMongoConnected = false;
}
so when it fails to connect to the database, it will throw an exception and we can handle our bool flag there.
If you want to handle connection issues in your program you can use the ICluster.Description event.
When the MongoClient is created, it will continue to attempt connections in the background until it succeeds.
using MongoDB.Driver;
using MongoDB.Driver.Core.Clusters;
var mongoClient = new MongoClient("localhost")
mongoClient.Cluster.DescriptionChanged += Cluster_DescriptionChanged;
public void Cluster_DescriptionChanged(object sender, ClusterDescriptionChangedEventArgs e)
{
switch (e.NewClusterDescription.State)
{
case ClusterState.Disconnected:
break;
case ClusterState.Connected:
break;
}
}

JIT SignalR Hub Sending and Receiving

Up till now for the past 3 months, I still have 0 clue how SignalR works at the JIT (Just-in-time) level. I'm trying to build a Hub that sends data to the client just in time, and the client will then receive the data and work along with it.
EDIT: Incase you have no idea what I mean by JIT Sending and
Receiving,
I meant it by the server being able to send connected socket clients data when there is new data available. The socket connection will only be closed either when the server is shutdown/has an issue OR the client disconnects from the socket. So in short, no matter what, when new data arises from the server, it will always send that data ONE BY ONE to connection clients.
So here's what I'm missing out/confused about:
Is the SubscribeToAll (Check out TickerHub.cs below) Method the place where I call when I have new data to notify and beep to the clients or where is it?
I know how the asynchronous WriteToChannel works. Basically it sends a collection, item by item to the client. Key issue is, how do I convert this entire function to JIT? And where do I handle the list of clients subscribed to this hub?
Currently, TickerHub.cs keeps retrieving a dataset (named CurrencyPairs) and then broadcasts it to the clients indefinitely. I have a background service that syncs and updates the CurrencyPairs 24/7. I just need a SignalR expert's help to explain/show how I can invoke the Hub from the background service and then allow the hub to broadcast that new data to the connected clients.
TickerHub.cs
public class TickerHub : Hub, ITickerHubClient
{
private IEnumerable<CurrencyPair> _currencyPairs;
private readonly ICurrencyPairService _cpService;
public TickerHub(ICurrencyPairService cpService)
{
_cpService = cpService;
}
public async Task<NozomiResult<CurrencyPair>> Tickers(IEnumerable<CurrencyPair> currencyPairs = null)
{
var nozRes = new NozomiResult<CurrencyPair>()
{
Success = true,
ResultType = NozomiResultType.Success,
Data = currencyPairs
};
return nozRes;
}
// We can use this to return a payload
public async Task<ChannelReader<NozomiResult<CurrencyPair>>> SubscribeToAll()
{
// Initialize an unbounded channel
//
// Unbounded Channels have no boundaries, allowing the server/client to transmit
// limitless amounts of payload. Bounded channels have limits and will tend to
// drop the clients after awhile.
var channel = Channel.CreateUnbounded<NozomiResult<CurrencyPair>>();
_ = WriteToChannel(channel.Writer); // Write all Currency Pairs to the channel
// Return the reader
return channel.Reader;
// This is a nested method, allowing us to write repeated methods
// with the same semantic conventions while maintaining conformity.
async Task WriteToChannel(ChannelWriter<NozomiResult<CurrencyPair>> writer)
{
// Pull in the latest data
_currencyPairs = _cpService.GetAllActive();
// Iterate them currency pairs
foreach (var cPair in _currencyPairs)
{
// Write one by one, and the client receives them one by one as well
await writer.WriteAsync(new NozomiResult<CurrencyPair>()
{
Success = (cPair != null),
ResultType = (cPair != null) ? NozomiResultType.Success : NozomiResultType.Failed,
Data = new[] {cPair}
});
}
// Beep the client, telling them you're done
writer.Complete();
}
}
}
In case you want to find out if my client sided code doesn't work well, here it is
using Microsoft.AspNetCore.SignalR.Client;
using Newtonsoft.Json;
using Nozomi.Client.Data.Interfaces;
using Nozomi.Data;
using Nozomi.Data.CurrencyModels;
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace Nozomi.Client
{
public class NozomiClient
{
private CancellationToken _tickerStreamCancellationToken;
private string ServerPath;
private HubConnection _hubConnection;
public NozomiClient(string serverPath)
{
ServerPath = serverPath;
_hubConnection = new HubConnectionBuilder()
.WithUrl(serverPath)
.Build();
}
public async Task InitializeAsync()
{
await _hubConnection.StartAsync();
}
public async Task StreamTickers()
{
// Setup the channel for streaming
var streamTickerChannel = await _hubConnection.StreamAsChannelAsync<NozomiResult<CurrencyPair>>("SubscribeToAll", CancellationToken.None);
// Setup the asynchronous data stream
// https://learn.microsoft.com/en-us/aspnet/core/signalr/streaming?view=aspnetcore-2.1#net-client
//while (await streamTickerChannel.WaitToReadAsync())
//{
// while (streamTickerChannel.TryRead(out var cp))
// {
// Console.WriteLine(JsonConvert.SerializeObject(cp));
// }
//}
_hubConnection.On<CurrencyPair>("SubscribeToAll", cp =>
{
Console.WriteLine(cp);
});
while (!_tickerStreamCancellationToken.IsCancellationRequested)
{
if (await streamTickerChannel.WaitToReadAsync())
{
while (streamTickerChannel.TryRead(out var cp))
{
Console.WriteLine(JsonConvert.SerializeObject(cp));
}
}
Console.WriteLine("Processing");
Thread.Sleep(1000);
}
}
public ICurrencyPair CurrencyPairs { get; }
public ISource Sources { get; }
}
}

WCF TimeoutException despite stepping through showing successful return

I have two self hosted services running on the same network. The first is sampling an excel sheet (or other sources, but for the moment this is the one I'm using to test) and sending updates to a subscribed client.
The second connects as a client to instances of the first client, optionally evaluates some formula on these inputs and the broadcasts the originals or the results as updates to a subscribed client in the same manner as the first. All of this is happening over a tcp binding.
My problem is occuring when the second service attempts to subscribe to two of the first service's feeds at once, as it would do if a new calculation is using two or more for the first time. I keep getting TimeoutExceptions which appear to be occuring when the second feed is subscribed to. I put a breakpoint in the called method on the first server and stepping through it, it is able to fully complete and return true back up the call stack, which indicates that the problem might be some annoying intricacy of WCF
The first service is running on port 8081 and this is the method that gets called:
public virtual bool Subscribe(int fid)
{
try
{
if (fid > -1 && _fieldNames.LeftContains(fid))
{
String sessionID = OperationContext.Current.SessionId;
Action<Object, IUpdate> toSub = MakeSend(OperationContext.Current.GetCallbackChannel<ISubClient>(), sessionID);//Make a callback to the client's callback method to send the updates
if (!_callbackList.ContainsKey(fid))
_callbackList.Add(fid, new Dictionary<String, Action<Object, IUpdate>>());
_callbackList[fid][sessionID] = toSub;//add the callback method to the list of callback methods to call when this feed is updated
String field = GetItem(fid);//get the current stored value of that field
CheckChanged(fid, field);//add or update field, usually returns a bool if the value has changed but also updates the last value reference, used here to ensure there is a value to send
FireOne(toSub, this, MakeUpdate(fid, field));//sends an update so the subscribing service will have a first value
return true;
}
return false;
}
catch (Exception e)
{
Log(e);//report any errors before returning a failure
return false;
}
}
The second service is running on port 8082 and is failing in this method:
public int AddCalculation(string name, string input)
{
try
{
Calculation calc;
try
{
calc = new Calculation(_fieldNames, input, name);//Perform slow creation before locking - better wasted one thread than several blocked ones
}
catch (FormatException e)
{
throw Fault.MakeCalculationFault(e.Message);
}
lock (_calculations)
{
int id = nextID();
foreach (int fid in calc.Dependencies)
{
if (!_calculations.ContainsKey(fid))
{
lock (_fieldTracker)
{
DataRow row = _fieldTracker.Rows.Find(fid);
int uses = (int)(row[Uses]) + 1;//update uses of that feed
try
{
if (uses == 1){//if this is the first use of this field
SubServiceClient service = _services[(int)row[ServiceID]];//get the stored connection (as client) to that service
service.Subscribe((int)row[ServiceField]);//Failing here, but only on second call and not if subscribed to each seperately
}
}
catch (TimeoutException e)
{
Log(e);
throw Fault.MakeOperationFault(FaultType.NoItemFound, "Service could not be found");//can't be caught, if this timed out then outer connection timed out
}
_fieldTracker.Rows.Find(fid)[Uses] = uses;
}
}
}
return id;
}
}
catch (FormatException f)
{
Log(f.Message);
throw Fault.MakeOperationFault(FaultType.InvalidInput, f.Message);
}
}
The ports these are on could change but are never shared. The tcp binding used is set up in code with these settings:
_tcpbinding = new NetTcpBinding();
_tcpbinding.PortSharingEnabled = false;
_tcpbinding.Security.Mode = SecurityMode.None;
This is in a common library to ensure they both have the same set up, which is also a reason why it is declared in code.
I have already tried altering the Service Throttling Behavior for more concurrent calls but that didn't work. It's commented out for now since it didn't work but for reference here's what I tried:
ServiceThrottlingBehavior stb = new ServiceThrottlingBehavior
{
MaxConcurrentCalls = 400,
MaxConcurrentSessions = 400,
MaxConcurrentInstances = 400
};
host.Description.Behaviors.RemoveAll<ServiceThrottlingBehavior>();
host.Description.Behaviors.Add(stb);
Has anyone had similar issues of methods working correctly but still timing out when sending back to the caller?
This was a difficult problem and from everything I could tell, it is an intricacy of WCF. It cannot handle one connection being reused very quickly in a loop.
It seems to lock up the socket connection, though trying to add GC.Collect() didn't free up whatever resources it was contesting.
In the end the only way I found to work was to create another connection to the same endpoint for each concurrent request and perform them on separate threads. Might not be the cleanest way but it was all that worked.
Something that might come in handy is that I used the svc trace viewer to monitor the WCF calls to try and track the problem, I found out how to use it from this article: http://www.codeproject.com/Articles/17258/Debugging-WCF-Apps

Maintaining an open Redis connection using BookSleeve

Does anyone have a solid pattern fetching Redis via BookSleeve library?
I mean:
BookSleeve's author #MarcGravell recommends not to open & close the connection every time, but rather maintain one connection throughout the app. But how can you handle network breaks? i.e. the connection might be opened successfully in the first place, but when some code tries to read/write to Redis, there is the possibility that the connection has dropped and you must reopen it (and fail gracefully if it won't open - but that is up to your design needs.)
I seek for code snippet(s) that cover general Redis connection opening, and a general 'alive' check (+ optional awake if not alive) that would be used before each read/write.
This question suggests a nice attitude to the problem, but it's only partial (it does not recover a lost connection, for example), and the accepted answer to that question draws the right way but does not demonstrate concrete code.
I hope this thread will get solid answers and eventually become a sort of a Wiki with regards to BookSleeve use in .Net applications.
-----------------------------
IMPORTANT UPDATE (21/3/2014):
-----------------------------
Marc Gravell (#MarcGravell) / Stack Exchange have recently released the StackExchange.Redis library that ultimately replaces Booksleeve. This new library, among other things, internally handles reconnections and renders my question redundant (that is, it's not redundant for Booksleeve nor my answer below, but I guess the best way going forward is to start using the new StackExchange.Redis library).
Since I haven't got any good answers, I came up with this solution (BTW thanks #Simon and #Alex for your answers!).
I want to share it with all of the community as a reference. Of course, any corrections will be highly appreciated.
using System;
using System.Net.Sockets;
using BookSleeve;
namespace Redis
{
public sealed class RedisConnectionGateway
{
private const string RedisConnectionFailed = "Redis connection failed.";
private RedisConnection _connection;
private static volatile RedisConnectionGateway _instance;
private static object syncLock = new object();
private static object syncConnectionLock = new object();
public static RedisConnectionGateway Current
{
get
{
if (_instance == null)
{
lock (syncLock)
{
if (_instance == null)
{
_instance = new RedisConnectionGateway();
}
}
}
return _instance;
}
}
private RedisConnectionGateway()
{
_connection = getNewConnection();
}
private static RedisConnection getNewConnection()
{
return new RedisConnection("127.0.0.1" /* change with config value of course */, syncTimeout: 5000, ioTimeout: 5000);
}
public RedisConnection GetConnection()
{
lock (syncConnectionLock)
{
if (_connection == null)
_connection = getNewConnection();
if (_connection.State == RedisConnectionBase.ConnectionState.Opening)
return _connection;
if (_connection.State == RedisConnectionBase.ConnectionState.Closing || _connection.State == RedisConnectionBase.ConnectionState.Closed)
{
try
{
_connection = getNewConnection();
}
catch (Exception ex)
{
throw new Exception(RedisConnectionFailed, ex);
}
}
if (_connection.State == RedisConnectionBase.ConnectionState.Shiny)
{
try
{
var openAsync = _connection.Open();
_connection.Wait(openAsync);
}
catch (SocketException ex)
{
throw new Exception(RedisConnectionFailed, ex);
}
}
return _connection;
}
}
}
}
With other systems (such as ADO.NET), this is achieved using a connection pool. You never really get a new Connection object, but in fact get one from the pool.
The pool itself manages new connections, and dead connections, independently from caller's code. The idea here is to have better performance (establishing a new connection is costy), and survive network problems (the caller code will fail while the server is down but resume when it comes back online). There is in fact one pool per AppDomain, per "type" of connection.
This behavior transpires when you look at ADO.NET connection strings. For example SQL Server connection string (ConnectionString Property) has the notion of 'Pooling', 'Max Pool Size', 'Min Pool Size', etc. This is also a ClearAllPools method that is used to programmaticaly reset the current AppDomain pools if needed for example.
I don't see anything close to this kind of feature looking into BookSleeve code, but it seems to be planned for next release: BookSleeve RoadMap.
In the mean time, I suppose you can write your own connection pool as the RedisConnection has an Error Event you can use for this, to detect when it's dead.
I'm not a C# programmer, but the way I'd look at the problem is the following:
I'd code a generic function that would take as parameters the redis connection and a lambda expression representing the Redis command
if trying to execute the Redis command would result in an exception pointing out a connectivity issue, I've re-initialize the connection and retry the operation
if no exception is raised just return the result
Here is some sort of pseudo-code:
function execute(redis_con, lambda_func) {
try {
return lambda_func(redis_con)
}
catch(connection_exception) {
redis_con = reconnect()
return lambda_func(redis_con)
}
}

Categories