I am looking for a simple .NET library that implements a concept of async message passing similar to Erlang OTP platform. So far, I have only found RetLang to be somewhat similar, but
It seems to be abandoned, and
It only supports message passing within one process.
You can try with MSMQ .you can use for single or a group of messages you want to put in the queue and read from it later asynchronously. :)
In my opinion the easiest way to do this in .net (aside from F# ;) ) is the TPL dataflow - lib
Use MSMQ which is quite simple to implement. It is exactly what you need - asynchronous messaging system. WCF is also good but is more complex to manage (config files) and adds a bit of overhead. MSMQ is a standard (and free) Windows component but to use it you need to enable it. MSMQ can be used for local communication (same process or any 2 processes within the same Windows domain)
Read this answer for more details and code examples.
Microsoft research developed and proved out (with EA/Bioware) a project called Orleans. It has been used in production and is being actively developed.
Quick Summary
Intro on Pluralsight
https://github.com/dotnet/orleans
You can use WCF to send and receive asynchronous messages. you can read more at msdn
Please have a look at spring messaging, this might be useful for you.
Related
I'm working on a large project (teams are working separately) and now that most of the apps are finished, we need to integrate everything together. A lot of apps/forms need to communicate information to other apps/forms or solicit information from others; I wonder if there are standard ways of doing this. The way we are planning to do that is through the use of shared files (e.g. one app writes info to a file and the other reads it). Any suggestions?
Yes, as #Uwe Keim said in a comment, you should definitely use a database for everything. A SO answer is not the place to start teaching you about relational databases and how to use them properly from C# applications, though. You should tell the person managing the teams that's the way to go, and have them figure out who to hire to solve this.
As regards your question, I think that there is an obvious drawback in using the file-shared solution, which is the delay in your operation because of IO operations.
Moreover, there are other options, please see below:
To fully implement your requirements, you should separate the communication between applications and the communication among forms inside a application.
To implement communication between applications, I would suggest that you use the subscriber/publisher pattern using WCF. The implementation is quite easy. You can google it. One example is here. By doing that, you could control exactly when something happens and the other application should re-acts as you expects.
To implement communication amongst forms in an application, it'd better to create an event aggregator using the Pub/Sub pattern, and then inject it into your application. You can have a look at how it implemented in SCSF.
Another solution is to implement a message protocol to make the apps communicate each other.
To avoid bottlenecks, a message queue service can manage the messages exchange.
References:
wikipedia
msdn
Vaguely remember seeing some discussions on this quite a while back but haven't heard anything since. So basically are you able to subscribe to an IObservable on a remote machine?
You can use IObservable.Remotable to use observables directly from other machines via .NET Remoting.
Another possible solution could be to use named pipes.
There is an excellent NuGet package NamedPipeWrapper, see the source on GitHub. It would be easy to write a thin RX wrapper over this, i.e. subscribe to the RX stream and push the messages out to other .NET listening processes using this library.
As this solution uses named pipes, it would be a true pub/sub solution which would support multiple subscribers in different processes.
Update
It is indeed very easy to write simple RX bridge code over the named pipes library. Use an RX Subject and insert the RX bridge code into the event handlers. Its not more than 4 lines of additional code at both ends. I can post the code if anyone is interested.
Update
For more information on named pipes, see .NET 3.5 Adds Named Pipes Support and Interprocess Communication Using .NET 3.5 Named Pipes IO. The aforementioned NuGet package NamedPipeWrapper is a much nicer version of the built-in support for named pipes that .NET 3.5 introduced.
Found this cool video on Channel 9 which an example of using IObservable.Remotable as Paul pointed out:
http://channel9.msdn.com/posts/J.Van.Gogh/Whats-different-about-the-3-versions-of-Rx-Part-3-NET-35-SP1/
Very interesting stuff, gonna spend a bit of time playing around with it now! :-D
Yes.
RX has built in support for using .NET Remoting to cross process boundaries.
If you install the NuGet package rx-remoting, it will install the assembly System.Reactive.Runtime.Remoting.dll which provides support for cross-process RX messages.
For demo code from Microsoft, see RX Across Processes. I've just tested the code on this page, and it works nicely. In order to get it to compile, you will need to add the following references:
NuGet: Reactive Extensions - Main Library (search for reactive extensions main)
NuGet: Reactive Extensions - .NET Remoting Support (search for reactive extensions remoting)
System.Runtime.Remoting (add as a normal reference, this assembly ships with .NET)
The Channel 9 video mentioned by #theburningmonk is also interesting to watch.
Update
Unfortuantely, this solution has one large limitation: you can only have one client process listening (all subsequent clients cannot connect). Pushqa solves this problem (see my other answer). Essentially, any library which implements RX on top of a pub/sub signalling bus should do the trick.
Yes.
Check out Pushqa.
Its easy to use. I was up and running in about 5 minutes.
It works with C# .NET, WPF, ASP.NET or Javascript. SignalR is built into ASP.NET, but it works for any C# .NET project if you add the right NuGet package.
It is superior to RX over .NET remoting (see my other answer), as we can have one server and many subscribers (it is a true pub/sub model, just like RX).
The queries are compiled into expression trees, and executed on the server (which minimizes network traffic, as only relevant results are returned from the server).
If we want queries to be filtered client side, then its easy - just do a client side filter on the results returned from pushqa.
Its literally 1% of the pain, 1% of the boilerplate code, and 10x the usability of Tibco. I wrote RX wrappers for Tibco and it was a nightmare to get it correct (Tibco has more corner cases than a tub of dodecahedrons). Unless you need to connect to legacy mainframe clients, or want to multicast to hundreds of clients over UDP, or want to waste a kings random in licensing fees, this solution is far superior to Tibco.
Its free.
Its open source.
There's no reason that a framework couldn't be devised for doing that. The framework would have to provide a means to address remote objects, generate proxies for them, then marshal the activity of the remote object across the application boundaries (i.e. through socket communication). .NET Remoting may be a suitable option for implementing this. WCF would be even better.
Are you specifically bound to using Rx as the solution to your problem? WCF provides duplex services, which have the ability for clients to register callback endpoints to a service. The service may then initiate calls back to its clients as necessary. It is effectively a remoted observer pattern. If RX is a must, then it should be fairly strait forward to wrap WCF duplex services with an RX support framework, allowing your clients to "transparently" observe service behavior with IObservable.
Does anyone know of a good resource for open-source libraries for asynchronous C# (or native stuff to the language). I'm interested in anything on this topic, but I'm specifically looking for stuff pertaining to HTTP and DB calls. Maybe an event-driven framework with plugs for HTTP and DB?
Unfortunately I can't use a non-C# solution or anything that does not work on mono, unless it is planned to run on mono soon.
For HTTP, it kinda depends on whether you're talking about client or server. Assuming client, you could just use the *Async methods in WebClient
http://www.go-mono.com/docs/index.aspx?link=T:System.Net.WebClient/*
For DB, the sqlcommand (or similar) class exposes BeginExecute* methods for async calls
http://www.go-mono.com/docs/index.aspx?link=T:System.Data.SqlClient.SqlCommand/*
You would probably have to use some kind of queue system. There are lots of queue engines. MSMQueue is the "standard" Microsoft solution.
Have you looked at the Linxter Internet Service Bus system? You can find some details at http://www.linxter.com and some sample apps that show how to perform database transactions distributed over the Internet.
The question is quite generic, both HTTP, DB and asynchronous could mean a lot of different things, depending on the requirements:
On codeplex you can find a more standards compliant HTTP server implementation which is event driven, compatible with Mono and has been used by others successfully.
There are a number of Asp.Net providers, as well as NHibernate
For Linq to Sql, your best option will probably be to use DbLinq although DbLinq is being included in the Mono namespace.
You probably should check NServiceBus. If it runs on mono it offers you a good framework for asynchronous calls (based on messaging).
It doesn't offer DB or HTTP connectivity by default, but this should be fairly easy to integrate.
Is there a product (ideally open source, but not necessary), that would enable a zero dependency deployment? every service bus or queue library I've been able to find has a dependency on one of the queue apps (like msmq), or a database. I would like a very lightweight solution that I can just add a reference to my application, build it, and deploy it with as little configuration as possible.
In an ideal world, the queue/service bus would run on IIS, and allow web and rich clients to talk to it.
Such a tool would be ideal for fast prototyping of large distributed systems on a local development machine.
Rhino Queues from Ayende is exactly what you are looking for, this is the blog post introducing it:
http://ayende.com/Blog/archive/2008/08/01/Rhino-Queues.aspx
I think that all of the limitations mentioned in this post have been fixed since then.
From the blog post, what rhino queues is:
XCopyable, Zero Administration, Embedded, Async queuing service
Robust in the face of networking outages
System.Transactions support
Fast
Works over HTTP
In a similar vein to ShuggyCoUk's suggestion, you could rig up a queue (or queues) using the Windows built-in ESENT database (comes already installed with Windows). There is a managed code access library (open source): http://www.codeplex.com/ManagedEsent. If you stick with writing / reading CLOBs or BLOBs, it should work just fine. If you want to be really clever, you can use NServiceBus and write (contribute?) ESENT-flavored subscription storage and transports. There are some forays into using ESENT on Ayende's blog as well (you'll have to poke around his SVN repository for the juicy bits).
If you're happy to be:
Windows specific
Limited to the local domain
Seriously limited in the message size supported
Wrap the underlying win32 calls in P/Invoke
Deal with the polling yourself
Deal with the hacks needed to allow back and forth communication
Deal with the shared config needed to keep the names in sync
Then a quick wrapper around the windows MailSlot API might be sufficient.
This simple example is a reasonable basis to start.
This article has some further information but assumes the use case is via a control (rather than a Component as it should be) as well as some poor WinForms integration so should be considered for incidental reading rather than a basis for any library.
This article is C++ but is of a higher standard (and a commenter has extended it to support the batching of larger messages into several smaller ones).
You get 424 bytes (so with .Net 212 chars) you may want to drop to ASCII to double your useful message length if you are talking text.
Note that despite its simplicity, limitations and lack of features it does provide multicast delivery, something often complex to layer on a point to point protocol yourself.
This ayende post provides and interesting comparison of three service buses. We use NServiceBus and think if it's not clear that Udi Dahan would respond to how you'd plug in non-dependent queue.
We work using MSMQ happily but there are other options and in theory it should be open to practically anything, given that you may lose some reliability and durability depending on your choice.
Why not Amazon's message service Simple Queue Service?
We moved our projects from MSMQ to ActiveMQ. its really better :)
ActiveMQ is open source queue ,based on Apache web server.
We used him in production on high frequently data workflow, where msmq have a lot of problem (we work with msmq a year)
The csharp implementation is nms
I'm currently working on an open source WCF based service bus. You can find it here: http://rockbus.codeplex.com/. It supports dynamic (#run-time) subscriptions, subcription repository (database), pluggable transports, XPath based content-based routing, transactional delivery over wcf protocols, roundrobin delivery, pluggable subscription evaluation, and more. Have a look!
Have you thought about using a service like IronMQ by http://Iron.io?
You wouldn't have any dependencies, could quickly prototype apps without setting up any queue infrastructure, and it's highly available and fast.
There is not currently a locally installable version but it's based on the upcoming OpenStack protocol so there will be.
Btw I work for Iron.
Try https://github.com/mcintyre321/PieQ - this is my attempt to write a threadsafe, persistent, zero-config, embedded work queue. It probably needs a little love, but I think it might be the kind of tool you are looking for.
I have developed an InMemory JMS library which can be used to in testing JMS applications without really connecting to JMS providers/server (Think of hsqldb). You don't have to deal with connection or protocol or anything, all you need to do is to send and receive messages.
https://github.com/Dhana-Krishnasamy/InMemoryJMS
I have used IPC in Win32 code a while ago - critical sections, events, and semaphores.
How is the scene in the .NET environment?
Are there any tutorial explaining all available options and when to use and why?
Most recent Microsoft's stuff in IPC is Windows Communication Foundation. Actually there is nothing new in the lower level (tcp, upd, named pipes etc) But WCF simplifies IPC development greatly.
Useful resource:
Interprocess Communication with WCF on Dr. Dobb's portal
WCF Communication Options in the .NET Framework 3.5
and of course MSDN on WCF
Apart from the obvious (WCF), there is a ZeroMQ binding for C#/CLR which is pretty good:
http://www.zeromq.org/bindings:clr
Does message-oriented IPC, pub/sub and various other strategies with much less code and config than WCF.
It's also at least an order of magnitude faster than anything else and has less latency if you require low latency comms.
With respects to semaphores, locks, mutexes etc. If you share by communicating rather than communicate by sharing, you'll have a whole load less hassle than the traditional paradigm.
I tend to use named pipes or Unix sockets (depending on whether I'm targetting MS.NET or Mono -- I have a class that abstracts it away) since it's easy to use, portable, and allows me to easily interoperate with unmanaged code. That said, if you're only dealing with managed code, go with WCF or remoting -- the latter if you need Mono support, since their WCF support simply isn't there yet.
I would recommend using Memory Mapped Files if you need to use on the machine domain not communication through network. See the following link.
http://techmikael.blogspot.com/2010/02/blazing-fast-ipc-in-net-4-wcf-vs.html
There is also .NET Remoting, which I found quite cool, but I guess they are obsoleting it now that they have WCF.
It sounds as though you're interested in synchronization techniques rather than communication. If so, you might like to start here, or perhaps this more concise overview.