In an clustered environment I have a seed node and node1 and node2.
From node1 I want to send a message to an Actor which has been created on node2. The local path to this node on node2 is akka:MyAkkaSystem/user/AnActor.
Now I want to send a message from an Actor from node1 to this specific actor by using an ActorSelection like that:
var actorSystem = ActorSystem.Create("MyTestSystem");
var c = actorSystem.ActorSelection("/user/ConsoleReceiver");
c.Tell("Hello World");
On node2 the actor has been created like that:
var actorSystem = ActorSystem.Create("MyTestSystem");
var r = actorSystem.ActorOf(Props.Create<MessageReceiver>(), "ConsoleReceiver");
Console.WriteLine(r.Path);
Console.ReadLine();
actorSystem.Terminate().Wait();
Unfortunately this does not work out since the attempt ends in dead letters.
The HOCON configuration on node2 looks like that:
akka {
actor {
provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
deployment {
}
}
remote {
log-remote-lifecycle-events = DEBUG
log-received-messages = on
helios.tcp {
transport-class = "Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote"
applied-adapters = []
transport-protocol = tcp
hostname = "127.0.0.1"
port = 0
}
}
cluster {
#will inject this node as a self-seed node at run-time
seed-nodes = ["akka.tcp://webcrawler#127.0.0.1:4053"] #manually populate other seed nodes here, i.e. "akka.tcp://lighthouse#127.0.0.1:4053", "akka.tcp://lighthouse#127.0.0.1:4044"
roles = [crawler]
}
}
As seed node I am using lighthouse. From connection point of view everything seems to work out. The seed has been found and each node got has received a welcome message.
I thought I had location transparency on a Cluster and could reach remote resources as if they where local.
I thought I had location transparency on a Cluster and could reach remote resources as if they where local.
This is not so easy. Consider following scenario: What if you've created an actor on both nodes under the same path. If you'll try to use relative path - without showing which node you have in mind - which of the actor's should receive the message?.
Using basic cluster capabilities you can choose node easily using Context.ActorSelection(_cluster.ReadView.Members.Single(m => /* which node you want to choose */).Address + "/user/ConsoleReceiver");. Cluster extension gives you a read view data with info about all members visible from current node.
There are many ways to send message to another actor, without having to know on which node it lives.
First approach is to use Akka.Cluster.Tools cluster singleton feature - it allows you to create at most one instance of an actor present in the cluster. In case of node failures it will migrate to another node. Be aware that this solution shouldn't be used, if you want to have many actors working that way. It's more for distinct, special-case actors.
Second approach is to use Akka.Cluster.Tools Distributed Pub/Sub feature to broadcast cluster-wide events across actors in the cluster subscribed to specific topic without worrying of their actual location. This is good choice for message broadcasting scenarios.
Last approach is to use Akka.Cluster.Sharding feature which manages actors lifecycle automatically - you don't need to create actors explicitly - it's also able to route messages to them from anywhere in the cluster and can rebalance them across many cluster nodes when needed.
Related
I am using the example on the Akka.NET github to play with some basic remoting.
In the remoting example available from GitHub there is the following section in the configuration string for Akka.NET.
deployment {
/localactor {
router = round-robin-pool
nr-of-instances = 5
}
/remoteactor {
router = round-robin-pool
nr-of-instances = 5
remote = ""akka.tcp://system2#localhost:666""
}
}
remote {
dot-netty.tcp {
port = 1234
hostname = localhost
}
What does the forward slash / indicate? is this a comment or is this just the format of the files?
What does the router option 'round-robin-pool' control? I can see that it maps to the following class but I am hoping someone can explain what akka.routing actually means in the context of a remoting scenario? I am assuming this has something to do with how urls or ips are mapped?
Any clarification would be appreciated.
Consider the following snippet from the example:
/localactor {
router = round-robin-pool
nr-of-instances = 5
}
What does the forward slash / indicate? is this a comment or is this just the format of the files?
The forward slash is not a comment; it indicates the name of an actor. The code in the example refers to the actor named localactor in the following way:
var local = system.ActorOf(Props.Create(() => new SomeActor("hello", 123)).WithRouter(FromConfig.Instance), "localactor");
What does the router option 'round-robin-pool' control? I can see that it maps to the following class but I am hoping someone can explain what akka.routing actually means in the context of a remoting scenario? I am assuming this has something to do with how urls or ips are mapped?
round-robin-pool is used to define a router. localactor in the above configuration snippet is a router actor that creates a pool of five routee instances to which it routes messages in round-robin order. A router has no special meaning in a remoting context; it is essentially no different from a router in a non-remoting scenario. You can read more about routers in the linked documentation.
We have created a RabbitMQ cluster with two nodes (rabbit and rabbit1). We have 4 queues which are configured to be highly available queues by following http://www.rabbitmq.com/clustering.html and http://www.rabbitmq.com/ha.html
Before clustering, we used to connect to the node using the snippet below.
var factory = new ConnectionFactory(){ HostName = _rabbitMQ_Hostname, UserName = _rabbitMQ_Username, Password = _rabbitMQ_Password};
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: _autoCancellationPNS_QueueName,
durable: true,
exclusive: false,
autoDelete: false,
arguments: null);
string message = appointmentId.ToString();
var body = Encoding.UTF8.GetBytes(message);
IBasicProperties properties = channel.CreateBasicProperties();
properties.DeliveryMode = 2;
channel.BasicPublish(exchange: _rabbitMQ_Exchange,
routingKey: _autoCancellationPNS_RoutingKey,
basicProperties: properties,
body: body);
returnMessage.ShortMessage = "Added to queue";
returnMessage.LongMessage = "Added to queue";
logger.Debug("|Added to queue");
}
How should we deal with cluster?
The RabbitMQ.Client have been supporting connecting to multiple hosts for over a year. It was fixed in pull request #92. You should be able to do something like the following
using (var connection = connectionFactory.CreateConnection(hostList))
using (var channel = connection.CreateModel())
{
}
However, with this approach you would need to perform all the recovery etc. yourself. About a year ago, we had massive problem with stability in the EasyNetQ client, but since we started using RawRabbit our clustered environment and never really had a problem with it.
Disclaimer: I am the creator of RawRabbit.
You can connect to the node you prefer.
Exchanges and queues are visible across the cluster.
Using a load-balancer in front of the nodes is common practice, so the clients have to know only the balancer IP/DNS.
clients ----> balancer -----> RabbitMQ cluster
The .Net client does (to my knowledge) not offer any support for this. You build something yourself to select and connect to a node on the cluster.
For example, if you want to implement a round-robin strategy, the pseudo code would be something like
Get list of hostname/port combinations that form the cluster
do {
try {
connect to next hostname in the list
} catch (rabbit connection failed) {
maybe log a warning
}
} while not connected
Of course you now need to think about connection strategies, retries, number of connection attempts, exponential backoff, ...
... which is why I would strongly recommend to look for a library that already provides this kind of functionality (and much more). One such library is EasyNetQ (available on nuget), maybe NServiceBus (with RabbitMq Transport) or MassTransit could also be interesting.
Another approach could be to set up an intelligent loadbalancer in front of the individual nodes (so myrabbitcluster.mycompany.com load balances between the cluster nodes and should then be responsible to detect node failures and take faulty nodes out of the cluster).
I am trying to use to Azure Service Bus to broadcast a message from a Web Role to all the instances of a single worker role. This is the code I use to receive messages:
// Create the topic if it does not exist already
string connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
// Configure Topic Settings
TopicDescription td = new TopicDescription("CommandTopic");
td.MaxSizeInMegabytes = 5120;
td.DefaultMessageTimeToLive = new TimeSpan(0, 0, 1);
if (!namespaceManager.TopicExists("CommandTopic"))
{
namespaceManager.CreateTopic(td);
}
Random rand = new Random();
double randNum = rand.Next();
if (!namespaceManager.SubscriptionExists("CommandTopic", "CommandSubscription"+randNum))
{
namespaceManager.CreateSubscription("CommandTopic", "CommandSubscription" + randNum);
}
Client = SubscriptionClient.CreateFromConnectionString(connectionString, "CommandTopic", "CommandSubscription" + randNum, ReceiveMode.ReceiveAndDelete);
Trace.WriteLine("SUBSCRIPTION: COMMANDSUBSCRIPTION"+randNum);
In order to create a separate subscription for each worker role instance (so that all instances receive the message in the topic) I had to use a random number. Is there a way of using some Id of the instance instead of the random number. There is Instance.Id however it is too long to be used as a parameter for the subscription name. Is there a shorter version without using substring? Also, is creating a separate subscription for each instance the proper approach? Previously all instances subscribed to the same subscription and so only 1 instance was getting the message and deleting it from the subscription.
Try adding a bogus InternalEndpoint for the worker role to your configuration. This ensures the list of instances of a role gets populated.
please find the following link, I hope it can help you make the intercommunication between your roles:
http://msdn.microsoft.com/en-us/library/windowsazure/hh180158.aspx
and also please have a look on the following link, I think it has exactly what you are looking for:
http://windowsazurecat.com/2011/08/how-to-simplify-scale-inter-role-communication-using-windows-azure-service-bus/
I think what you are talking about would be like the Scenario 4 in the following link where a role can communicate to several other roles. I am not sure if what you asked for is possible, but try working with the Windows Azure Worker Role with Service Bus Queue, I think this might help you a lot and might also be a better solution than the topic and subscription in this case.
I have a SPItemEventReceiver that does nothing else than notify another HTTP server at a given IP and Port abouth the events using POST requests.
The HTTP server runs on the same computer as sharepoint, so I used to send the notification at localhost and a fixed Port number. But since the eventreceiver can be called in other servers in the serverfarm, localhost:PORT will not be available then.
So, everytime my HTTP server starts, it needs to save its IP address and Port somewhere in SharePoint where all EventReceivers have access, no matter on what server they are called.
What would be a good place to store such globally available information?
I tought about SPWebService.ContentService.Properties , but I'm not really sure if that's a good idea. What do you think?
Well, if you are using Sharepoint 2010 I would consider store those values in the property bag. Using client object model or even Javascript/ECMAScript Client Object Model. These codes maybe help you.
using (var context = new ClientContext("http://localhost"))
{
var allProperties = context.Web.AllProperties;
allProperties["testing"] = "Hello there";
context.Web.Update();
context.ExecuteQuery();
}
Or using javascript:
function getWebProperty() {
var ctx = new SP.ClientContext.get_current();
var web = ctx.get_site().get_rootweb();
this.props = web.get_allProperties();
this.props.set_item(“aProperty”, “aValue”);
ctx.load(web);
ctx.executeQueryAsync(Function.createDelegate(this, gotProperty), Function.createDelegate(this, failedGettingProperty));
}
function gotProperty() {
alert(this.props.get_item(“aProperty”));
}
function failedGettingProperty() {
alert("failed");
}
Sources:
https://sharepoint.stackexchange.com/questions/49299/sharepoint-2010-net-client-object-model-add-item-to-web-property-bag
https://www.nothingbutsharepoint.com/sites/devwiki/articles/Pages/Making-use-of-the-Property-Bag-in-the-ECMAScript-Client-Object-Model.aspx
There are actually several ways of saving configuration values in SharePoint:
Property Bags of SharePoint objects SPWebApplication, SPFarm,SPSite, SPWeb, SPList, SPListItem`
A "configuration" list in SharePoint - just a regular list you might set to Hidden = TRUE
The web.config file - specifically the <AppSettings>
Wictor Wilen actually explains the 6 ways to store settings in SharePoint.
As you are talking about an external process trying to save its settings somewhere, generally I would recommend the web.config, but each change in the web.config would lead to an IISRESET making it not a good option. I would strongly advise to use either a property bag (e.g. the SPWebApplication.Properties bag) or a hidden list in your favorite web site. You would set the property bag like so:
SPWebApplication webApplication = ...
object customObject = ...
// set value in hashtable
webApp.Add("MySetting", customObject);
// persist the hashtable
webApp.Update();
See what is cool about this? You can actually store an object with the web application which could contain multiple settings as long as you keep your object serializable.
I have a webpage that uses ajax polling to get stock market updates from the server. I'd like to use SignalR instead, but I'm having trouble understanding how/if it would work.
ok, it's not really stock market updates, but the analogy works.
The SignalR examples I've seen send messages to either the current connection, all connections, or groups. In my example the stock updates happen outside of the current connection, so there's no such thing as the 'current connection'. And a user's account is associated with a few stocks, so sending a stock notification to all connections or to groups doesn't work either. I need to be able to find a connection associated with a certain userId.
Here's a fake code example:
foreach(var stock in StockService.GetStocksWithBigNews())
{
var userIds = UserService.GetUserIdsThatCareAboutStock(stock);
var connections = /* find connections associated with user ids */;
foreach(var connection in connections)
{
connection.Send(...);
}
}
In this question on filtering connections, they mention that I could keep current connections in memory but (1) it's bad for scaling and (2) it's bad for multi node websites. Both of these points are critically important to our current application. That makes me think I'd have to send a message out to all nodes to find users connected to each node >> my brain explodes in confusion.
THE QUESTION
How do I find a connection for a specific user that is scalable? Am I thinking about this the wrong way?
I created a little project last night to learn this also. I used 1.0 alpha and it was Straight forward. I created a Hub and from there on it just worked :)
I my project i have N Compute Units(some servers processing work), when they start up they invoke the ComputeUnitRegister.
await HubProxy.Invoke("ComputeUnitReqisted", _ComputeGuid);
and every time they do something they call
HubProxy.Invoke("Running", _ComputeGuid);
where HubProxy is :
HubConnection Hub = new HubConnection(RoleEnvironment.IsAvailable ?
RoleEnvironment.GetConfigurationSettingValue("SignalREndPoint"):
"http://taskqueue.cloudapp.net/");
IHubProxy HubProxy = Hub.CreateHubProxy("ComputeUnits");
I used RoleEnviroment.IsAvailable because i can now run this as a Azure Role , a Console App or what ever in .NET 4.5. The Hub is placed in a MVC4 Website project and is started like this:
GlobalHost.Configuration.ConnectionTimeout = TimeSpan.FromSeconds(50);
RouteTable.Routes.MapHubs();
public class ComputeUnits : Hub
{
public Task Running(Guid MyGuid)
{
return Clients.Group(MyGuid.ToString()).ComputeUnitHeartBeat(MyGuid,
DateTime.UtcNow.ToEpochMilliseconds());
}
public Task ComputeUnitReqister(Guid MyGuid)
{
Groups.Add(Context.ConnectionId, "ComputeUnits").Wait();
return Clients.Others.ComputeUnitCameOnline(new { Guid = MyGuid,
HeartBeat = DateTime.UtcNow.ToEpochMilliseconds() });
}
public void SubscribeToHeartBeats(Guid MyGuid)
{
Groups.Add(Context.ConnectionId, MyGuid.ToString());
}
}
My clients are Javascript clients, that have methods for(let me know if you need to see the code for this also). But basicly they listhen for the ComputeUnitCameOnline and when its run they call on the server SubscribeToHeartBeats. This means that whenever the server compute unit is doing some work it will call Running, which will trigger a ComputeUnitHeartBeat on javascript clients.
I hope you can use this to see how Groups and Connections can be used. And last, its also scaled out over multiply azure roles by adding a few lines of code:
GlobalHost.HubPipeline.EnableAutoRejoiningGroups();
GlobalHost.DependencyResolver.UseServiceBus(
serviceBusConnectionString,
2,
3,
GetRoleInstanceNumber(),
topicPathPrefix /* the prefix applied to the name of each topic used */
);
You can get the connection string on the servicebus on azure, remember the Provider=SharedSecret. But when adding the nuget packaged the connectionstring syntax is also pasted into your web.config.
2 is how many topics to split it about. Topics can contain 1Gb of data, so depending on performance you can increase it.
3 is the number of nodes to split it out on. I used 3 because i have 2 Azure Instances, and my localhost. You can get the RoleNumber like this (note that i hard coded my localhost to 2).
private static int GetRoleInstanceNumber()
{
if (!RoleEnvironment.IsAvailable)
return 2;
var roleInstanceId = RoleEnvironment.CurrentRoleInstance.Id;
var li1 = roleInstanceId.LastIndexOf(".");
var li2 = roleInstanceId.LastIndexOf("_");
var roleInstanceNo = roleInstanceId.Substring(Math.Max(li1, li2) + 1);
return Int32.Parse(roleInstanceNo);
}
You can see it all live at : http://taskqueue.cloudapp.net/#/compute-units
When using SignalR, after a client has connected to the server they are served up a Connection ID (this is essential to providing real time communication). Yes this is stored in memory but SignalR also can be used in multi-node environments. You can use the Redis or even Sql Server backplane (more to come) for example. So long story short, we take care of your scale-out scenarios for you via backplanes/service bus' without you having to worry about it.