I am trying to use to Azure Service Bus to broadcast a message from a Web Role to all the instances of a single worker role. This is the code I use to receive messages:
// Create the topic if it does not exist already
string connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
// Configure Topic Settings
TopicDescription td = new TopicDescription("CommandTopic");
td.MaxSizeInMegabytes = 5120;
td.DefaultMessageTimeToLive = new TimeSpan(0, 0, 1);
if (!namespaceManager.TopicExists("CommandTopic"))
{
namespaceManager.CreateTopic(td);
}
Random rand = new Random();
double randNum = rand.Next();
if (!namespaceManager.SubscriptionExists("CommandTopic", "CommandSubscription"+randNum))
{
namespaceManager.CreateSubscription("CommandTopic", "CommandSubscription" + randNum);
}
Client = SubscriptionClient.CreateFromConnectionString(connectionString, "CommandTopic", "CommandSubscription" + randNum, ReceiveMode.ReceiveAndDelete);
Trace.WriteLine("SUBSCRIPTION: COMMANDSUBSCRIPTION"+randNum);
In order to create a separate subscription for each worker role instance (so that all instances receive the message in the topic) I had to use a random number. Is there a way of using some Id of the instance instead of the random number. There is Instance.Id however it is too long to be used as a parameter for the subscription name. Is there a shorter version without using substring? Also, is creating a separate subscription for each instance the proper approach? Previously all instances subscribed to the same subscription and so only 1 instance was getting the message and deleting it from the subscription.
Try adding a bogus InternalEndpoint for the worker role to your configuration. This ensures the list of instances of a role gets populated.
please find the following link, I hope it can help you make the intercommunication between your roles:
http://msdn.microsoft.com/en-us/library/windowsazure/hh180158.aspx
and also please have a look on the following link, I think it has exactly what you are looking for:
http://windowsazurecat.com/2011/08/how-to-simplify-scale-inter-role-communication-using-windows-azure-service-bus/
I think what you are talking about would be like the Scenario 4 in the following link where a role can communicate to several other roles. I am not sure if what you asked for is possible, but try working with the Windows Azure Worker Role with Service Bus Queue, I think this might help you a lot and might also be a better solution than the topic and subscription in this case.
Related
I am trying to get a messaging system up and running betwen multiple applications. I have an instance of RabbitMQ running and that appears to be fine. I can connect multiple subscribers/publisher to the RabbitMQ instance and they appear to be fine. I can then publish a message from one publisher but only one subscriber is getting the message.
I believe it has to do with the the way I am establishing the queues. I've looked at the tutorial on Rabbit, https://www.rabbitmq.com/tutorials/tutorial-three-dotnet.html, but I dont know hopw this translates into the Masstransit library.
For the life of me I am having trouble working out what I am doing wrong.
NuGets:
MassTransit.Extensions.DependencyInjection 5.3.2
MassTransit.RabbitMQ 5.3.2
Can anyone help?
// Register MassTransit
services.AddMassTransit(mtCfg =>
{
mtCfg.AddConsumer<DomainMessageConsumer>();
mtCfg.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(rbCfg =>
{
var host = rbCfg.Host(settings.RabbitMq.Host, settings.RabbitMq.VirtualHost, h =>
{
h.Username(settings.RabbitMq.Username);
h.Password(settings.RabbitMq.Password);
});
rbCfg.ReceiveEndpoint(host, settings.RabbitMq.ConnectionName, ep =>
{
ep.PrefetchCount = 16;
ep.UseMessageRetry(x => x.Interval(2, 100));
ep.ConfigureConsumer<DomainMessageConsumer>(provider);
});
}));
});
The problem you are having is because you are using the same queuename on all consumers. If you want broadcasting to all consumers, you should make all queuenames unique. In your code example, it's the settings.RabbitMq.ConnectionName variable that you should make unique for each consumer.
Check the below picture and imagine Subscription B is the queue settings.RabbitMq.ConnectionName you've set. What you'll get is the left part on the picture, only Subscriber B1 receives (actually it's round-robin balancing, but this is going offtopic). If you want broadcasting, you can create separate subscriptions (or settings.RabbitMq.ConnectionName in your example).
I've been working on an application that starts some worker roles based on messaging.
This is the way I want the application to work:
Client sends a request for work (RPC).
One of the worker roles accepts the work, generates a random id, and responds to the RPC with the new id.
The worker will post its debug logs on a log channel with the id.
The client will subscribe to this channel so users can see what's going on.
The RPC is working fine, but I can't seem to figure out how to implement the log-sending.
This is the code that accepts work (simplified)
var bus = Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://xxxxxx.nl"), h =>
{
h.Username("xxx");
h.Password("xxxx");
});
sbc.ReceiveEndpoint(host, "post_work_item", e =>
{
e.Consumer<CreateWorkItemCommand>();
});
sbc.ReceiveEndpoint(host, "list_work_items", e =>
{
e.Consumer<ListWorkItemsCommand>();
});
});
The CreateWorkItemCommand will create the thread, do the work, etc. Now, how would I implement the log-sending with Masstransit? I was thinking something like:
bus.Publish(
obj: WorkUpdate{ Message = "Hello world!" },
channel: $"work/{work_id}"
)
And the client will do something this:
bus.ReceiveFromEvented($"work/{rpc.work_id}").OnMessage += { more_psuedo_code() }
I can't seem to find out how to do this.
Can anyone help me out?
Thanks!
It looks both like a saga and turnout. Current Turnout implementation is monitoring the job itself and I doubt you can really subscribe to that message flow. And it is still not really done.
You might solve this using the saga. Some external trigger (a command) will start the first saga, which will use Request/Response to start the process, which will do the work, and get its correlation id (job id). The long job can publish progress reports using the same correlation id and the saga will consume them, doing what it needs to do.
The "work/{rpc.work_id}" will be then replaced by the correlation.
We use Azure Notifications Hub to manage notifications registrations. Every time user launches application, we call PushNotificationChannelManager.CreatePushNotificationChannelForApplicationAsync and then RegisterNativeAsync of NotificationHub to register channel uri, returned by first with some tags like "Username" and "InstallId" - that is unique per app installation. Then from back-end we send notifications using these tags.
But we have noticed problem - when user hard-resets device, the previous channel registration stays active in notification hub. In that case user receives duplicate notifications by his "Username" tag. "InstallId" doesn't help in that case, as it is changing with new app installation.
We have thought about managing channels server-side. But that will not solve the problem.
Maybe anyone has some suggested work-around?
Also, we don't know what information does PushNotificationChannelManager use when creating new or returning existing channel? Does it use some device information?
I think you can send the backend the device unique Id along with the installation Id. The device id will not change upon hard reset.
private string GetDeviceUniqueID()
{
HardwareToken token = HardwareIdentification.GetPackageSpecificToken(null);
IBuffer hardwareId = token.Id;
HashAlgorithmProvider hasher = HashAlgorithmProvider.OpenAlgorithm("MD5");
IBuffer hashed = hasher.HashData(hardwareId);
string hashedString = CryptographicBuffer.EncodeToHexString(hashed);
return hashedString;
}
In an clustered environment I have a seed node and node1 and node2.
From node1 I want to send a message to an Actor which has been created on node2. The local path to this node on node2 is akka:MyAkkaSystem/user/AnActor.
Now I want to send a message from an Actor from node1 to this specific actor by using an ActorSelection like that:
var actorSystem = ActorSystem.Create("MyTestSystem");
var c = actorSystem.ActorSelection("/user/ConsoleReceiver");
c.Tell("Hello World");
On node2 the actor has been created like that:
var actorSystem = ActorSystem.Create("MyTestSystem");
var r = actorSystem.ActorOf(Props.Create<MessageReceiver>(), "ConsoleReceiver");
Console.WriteLine(r.Path);
Console.ReadLine();
actorSystem.Terminate().Wait();
Unfortunately this does not work out since the attempt ends in dead letters.
The HOCON configuration on node2 looks like that:
akka {
actor {
provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
deployment {
}
}
remote {
log-remote-lifecycle-events = DEBUG
log-received-messages = on
helios.tcp {
transport-class = "Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote"
applied-adapters = []
transport-protocol = tcp
hostname = "127.0.0.1"
port = 0
}
}
cluster {
#will inject this node as a self-seed node at run-time
seed-nodes = ["akka.tcp://webcrawler#127.0.0.1:4053"] #manually populate other seed nodes here, i.e. "akka.tcp://lighthouse#127.0.0.1:4053", "akka.tcp://lighthouse#127.0.0.1:4044"
roles = [crawler]
}
}
As seed node I am using lighthouse. From connection point of view everything seems to work out. The seed has been found and each node got has received a welcome message.
I thought I had location transparency on a Cluster and could reach remote resources as if they where local.
I thought I had location transparency on a Cluster and could reach remote resources as if they where local.
This is not so easy. Consider following scenario: What if you've created an actor on both nodes under the same path. If you'll try to use relative path - without showing which node you have in mind - which of the actor's should receive the message?.
Using basic cluster capabilities you can choose node easily using Context.ActorSelection(_cluster.ReadView.Members.Single(m => /* which node you want to choose */).Address + "/user/ConsoleReceiver");. Cluster extension gives you a read view data with info about all members visible from current node.
There are many ways to send message to another actor, without having to know on which node it lives.
First approach is to use Akka.Cluster.Tools cluster singleton feature - it allows you to create at most one instance of an actor present in the cluster. In case of node failures it will migrate to another node. Be aware that this solution shouldn't be used, if you want to have many actors working that way. It's more for distinct, special-case actors.
Second approach is to use Akka.Cluster.Tools Distributed Pub/Sub feature to broadcast cluster-wide events across actors in the cluster subscribed to specific topic without worrying of their actual location. This is good choice for message broadcasting scenarios.
Last approach is to use Akka.Cluster.Sharding feature which manages actors lifecycle automatically - you don't need to create actors explicitly - it's also able to route messages to them from anywhere in the cluster and can rebalance them across many cluster nodes when needed.
I have a webpage that uses ajax polling to get stock market updates from the server. I'd like to use SignalR instead, but I'm having trouble understanding how/if it would work.
ok, it's not really stock market updates, but the analogy works.
The SignalR examples I've seen send messages to either the current connection, all connections, or groups. In my example the stock updates happen outside of the current connection, so there's no such thing as the 'current connection'. And a user's account is associated with a few stocks, so sending a stock notification to all connections or to groups doesn't work either. I need to be able to find a connection associated with a certain userId.
Here's a fake code example:
foreach(var stock in StockService.GetStocksWithBigNews())
{
var userIds = UserService.GetUserIdsThatCareAboutStock(stock);
var connections = /* find connections associated with user ids */;
foreach(var connection in connections)
{
connection.Send(...);
}
}
In this question on filtering connections, they mention that I could keep current connections in memory but (1) it's bad for scaling and (2) it's bad for multi node websites. Both of these points are critically important to our current application. That makes me think I'd have to send a message out to all nodes to find users connected to each node >> my brain explodes in confusion.
THE QUESTION
How do I find a connection for a specific user that is scalable? Am I thinking about this the wrong way?
I created a little project last night to learn this also. I used 1.0 alpha and it was Straight forward. I created a Hub and from there on it just worked :)
I my project i have N Compute Units(some servers processing work), when they start up they invoke the ComputeUnitRegister.
await HubProxy.Invoke("ComputeUnitReqisted", _ComputeGuid);
and every time they do something they call
HubProxy.Invoke("Running", _ComputeGuid);
where HubProxy is :
HubConnection Hub = new HubConnection(RoleEnvironment.IsAvailable ?
RoleEnvironment.GetConfigurationSettingValue("SignalREndPoint"):
"http://taskqueue.cloudapp.net/");
IHubProxy HubProxy = Hub.CreateHubProxy("ComputeUnits");
I used RoleEnviroment.IsAvailable because i can now run this as a Azure Role , a Console App or what ever in .NET 4.5. The Hub is placed in a MVC4 Website project and is started like this:
GlobalHost.Configuration.ConnectionTimeout = TimeSpan.FromSeconds(50);
RouteTable.Routes.MapHubs();
public class ComputeUnits : Hub
{
public Task Running(Guid MyGuid)
{
return Clients.Group(MyGuid.ToString()).ComputeUnitHeartBeat(MyGuid,
DateTime.UtcNow.ToEpochMilliseconds());
}
public Task ComputeUnitReqister(Guid MyGuid)
{
Groups.Add(Context.ConnectionId, "ComputeUnits").Wait();
return Clients.Others.ComputeUnitCameOnline(new { Guid = MyGuid,
HeartBeat = DateTime.UtcNow.ToEpochMilliseconds() });
}
public void SubscribeToHeartBeats(Guid MyGuid)
{
Groups.Add(Context.ConnectionId, MyGuid.ToString());
}
}
My clients are Javascript clients, that have methods for(let me know if you need to see the code for this also). But basicly they listhen for the ComputeUnitCameOnline and when its run they call on the server SubscribeToHeartBeats. This means that whenever the server compute unit is doing some work it will call Running, which will trigger a ComputeUnitHeartBeat on javascript clients.
I hope you can use this to see how Groups and Connections can be used. And last, its also scaled out over multiply azure roles by adding a few lines of code:
GlobalHost.HubPipeline.EnableAutoRejoiningGroups();
GlobalHost.DependencyResolver.UseServiceBus(
serviceBusConnectionString,
2,
3,
GetRoleInstanceNumber(),
topicPathPrefix /* the prefix applied to the name of each topic used */
);
You can get the connection string on the servicebus on azure, remember the Provider=SharedSecret. But when adding the nuget packaged the connectionstring syntax is also pasted into your web.config.
2 is how many topics to split it about. Topics can contain 1Gb of data, so depending on performance you can increase it.
3 is the number of nodes to split it out on. I used 3 because i have 2 Azure Instances, and my localhost. You can get the RoleNumber like this (note that i hard coded my localhost to 2).
private static int GetRoleInstanceNumber()
{
if (!RoleEnvironment.IsAvailable)
return 2;
var roleInstanceId = RoleEnvironment.CurrentRoleInstance.Id;
var li1 = roleInstanceId.LastIndexOf(".");
var li2 = roleInstanceId.LastIndexOf("_");
var roleInstanceNo = roleInstanceId.Substring(Math.Max(li1, li2) + 1);
return Int32.Parse(roleInstanceNo);
}
You can see it all live at : http://taskqueue.cloudapp.net/#/compute-units
When using SignalR, after a client has connected to the server they are served up a Connection ID (this is essential to providing real time communication). Yes this is stored in memory but SignalR also can be used in multi-node environments. You can use the Redis or even Sql Server backplane (more to come) for example. So long story short, we take care of your scale-out scenarios for you via backplanes/service bus' without you having to worry about it.