Is there a product (ideally open source, but not necessary), that would enable a zero dependency deployment? every service bus or queue library I've been able to find has a dependency on one of the queue apps (like msmq), or a database. I would like a very lightweight solution that I can just add a reference to my application, build it, and deploy it with as little configuration as possible.
In an ideal world, the queue/service bus would run on IIS, and allow web and rich clients to talk to it.
Such a tool would be ideal for fast prototyping of large distributed systems on a local development machine.
Rhino Queues from Ayende is exactly what you are looking for, this is the blog post introducing it:
http://ayende.com/Blog/archive/2008/08/01/Rhino-Queues.aspx
I think that all of the limitations mentioned in this post have been fixed since then.
From the blog post, what rhino queues is:
XCopyable, Zero Administration, Embedded, Async queuing service
Robust in the face of networking outages
System.Transactions support
Fast
Works over HTTP
In a similar vein to ShuggyCoUk's suggestion, you could rig up a queue (or queues) using the Windows built-in ESENT database (comes already installed with Windows). There is a managed code access library (open source): http://www.codeplex.com/ManagedEsent. If you stick with writing / reading CLOBs or BLOBs, it should work just fine. If you want to be really clever, you can use NServiceBus and write (contribute?) ESENT-flavored subscription storage and transports. There are some forays into using ESENT on Ayende's blog as well (you'll have to poke around his SVN repository for the juicy bits).
If you're happy to be:
Windows specific
Limited to the local domain
Seriously limited in the message size supported
Wrap the underlying win32 calls in P/Invoke
Deal with the polling yourself
Deal with the hacks needed to allow back and forth communication
Deal with the shared config needed to keep the names in sync
Then a quick wrapper around the windows MailSlot API might be sufficient.
This simple example is a reasonable basis to start.
This article has some further information but assumes the use case is via a control (rather than a Component as it should be) as well as some poor WinForms integration so should be considered for incidental reading rather than a basis for any library.
This article is C++ but is of a higher standard (and a commenter has extended it to support the batching of larger messages into several smaller ones).
You get 424 bytes (so with .Net 212 chars) you may want to drop to ASCII to double your useful message length if you are talking text.
Note that despite its simplicity, limitations and lack of features it does provide multicast delivery, something often complex to layer on a point to point protocol yourself.
This ayende post provides and interesting comparison of three service buses. We use NServiceBus and think if it's not clear that Udi Dahan would respond to how you'd plug in non-dependent queue.
We work using MSMQ happily but there are other options and in theory it should be open to practically anything, given that you may lose some reliability and durability depending on your choice.
Why not Amazon's message service Simple Queue Service?
We moved our projects from MSMQ to ActiveMQ. its really better :)
ActiveMQ is open source queue ,based on Apache web server.
We used him in production on high frequently data workflow, where msmq have a lot of problem (we work with msmq a year)
The csharp implementation is nms
I'm currently working on an open source WCF based service bus. You can find it here: http://rockbus.codeplex.com/. It supports dynamic (#run-time) subscriptions, subcription repository (database), pluggable transports, XPath based content-based routing, transactional delivery over wcf protocols, roundrobin delivery, pluggable subscription evaluation, and more. Have a look!
Have you thought about using a service like IronMQ by http://Iron.io?
You wouldn't have any dependencies, could quickly prototype apps without setting up any queue infrastructure, and it's highly available and fast.
There is not currently a locally installable version but it's based on the upcoming OpenStack protocol so there will be.
Btw I work for Iron.
Try https://github.com/mcintyre321/PieQ - this is my attempt to write a threadsafe, persistent, zero-config, embedded work queue. It probably needs a little love, but I think it might be the kind of tool you are looking for.
I have developed an InMemory JMS library which can be used to in testing JMS applications without really connecting to JMS providers/server (Think of hsqldb). You don't have to deal with connection or protocol or anything, all you need to do is to send and receive messages.
https://github.com/Dhana-Krishnasamy/InMemoryJMS
Related
Alright, so I've asked several questions on StackOverflow about .NET Remoting, and there is always at least one person who just has to chime in, ".NET Remoting is deprecated, use WCF instead." I understand that it's deprecated and there is no guarantee of future support with new versions of the .NET Framework. But what are some other good reasons we would want to move over to WCF? I have seen a few mostly minor annoyances with .NET Remoting, however, this is not enough to change the minds the powers that be who believe firmly in "if it ain't broke, don't fix it". At this time, the only reason that attitude will change is if .NET Remoting is removed from a future version of the .NET Framework, so who knows how long that will be?
Does anybody have any insight as why exactly WCF is "better" than .NET Remoting, or why Remoting is inferior to WCF? What are the pros and cons of each technology? Are there additional things you can do with WCF and not with Remoting?
I mean, it would be great if I could convince them to let us migrate our software over to WCF just to allow a configurable TcpChannel timeout to be set on the client side (this seems to have been broken for a while, no matter what steps or troubleshooting I try), and when this happens, it makes our software look like absolute shite.
Thanks in advance for helping to shed some light on this.
There are plenty of reasons to ditch remoting; a few might include:
lack of transport flexibility
versioning requirements are huge pain
platform dependent (no sensible chance of cross-platform usage)
no chance of usage from the growing mobile market
lack of future development: whatever feature you want added - it won't be
however, I would disagree that WCF is the automatic replacement; WCF itself is a pretty versatile tool, but can be pretty complex, and has restrictions of its own. I haven't used it myself, but I have seen lots of praise for Service Stack, essentially with users describing it as "WCF done right", i.e. the good bits of WCF, without the pain points. However, there are plenty of other options too. One nice thing about the idea of Service Stack, though, is that it iterates pretty quickly, and if it lacks something you want you can change it.
.NET Remoting is now a legacy technology, quoted from MSDN:
This topic is specific to a legacy technology that is retained for backward compatibility with existing applications and is not recommended for new development. Distributed applications should now be developed using the Windows Communication Foundation (WCF).
And here is a performance comparison between WCF and .NET Remoting done in 2007: http://msdn.microsoft.com/en-us/library/bb310550.aspx
To summarize the results, WCF is 25%—50% faster than ASP.NET Web
Services, and approximately 25% faster than .NET Remoting.
So I guess speed is a good reason to drop .NET Remoting.
While the given reasons are probably the driving considerations there are other non-trivial reasons:
Transport independence
IDE tooling
Ease of testing
Maintainability
When you use WCF you can change transport merely by editing your config file. This can be very handy when some sanctimonious system admin won't open a port and you need to use HTTP on port 80 to get through the corporate firewall.
The WCF tooling in Visual Studio is phenomenal. The hardest part is figuring out the URL you need. After that it's just point and click for code generation. There are one or two gotchas with serialisation of collections but broadly speaking if you tell both ends to use arrays it will just plain work. If you need a collection at the destination you can always construct one around the received array, and since LINQ will happily operate on arrays you can fold this into other transformations.
I'm not sure what Stephan P means by pain points. Editing the config can be tricky but Microsoft provides an excellent GUI tool that takes all the guesswork out of it by providing a full tree of options yet generating a sparse config file.
WCF services are easy to test because they have a published interface to which you can connect a test harness. This is more a virtue of SoA in general rather than WCF in particular, but it's still desirable.
WCF makes things a great deal simpler in my code, since neither application nor service is polluted with "routing" code (to determine what ought to process the message content); it looks like simple method calls or implementations. I mostly use WCF as a wrapper for MSMQ, and the only visible consequence of the transport selection is that these method calls must all be void functions because it's a OneWay transport. But that's hardly surprising when the point was persistent queueing.
This all speaks to maintainability. Even for in-house applications, maintenance is a dominant cost, and when you're supporting your software at customer sites poor maintainability can be crippling.
Then there's interoperability with otherwise incompatible platforms. In this case I'm thinking of using HTTP/XML or HTTP/JSON to provide service to web apps written in (eg) PHP.
Going the other way isn't quite so easy but it's fairly straightforward.
I give points for WCF with respect to logging and security.
Logging
WCF has an integrated Logging mechanism that helps you log traces that become a boon during maintanence. In other technologies, developer has to do some work to achieve this but in WCF, all that we have to do is to enable trace by changing the config file and WCF starts providing traces for you.
Security
Security mechanism in WCF is fairly simple and out of box when you look from the implementors perspective but is very robust and highly secure. The best part is that for the highy used and recommended bindings, WCF provides default security which can be trusted to the core. The message security on WSHTTPbinding is an example on these lines.
.NET Framework remoting does not do authentication or encryption by default. Therefore, it is recommended that you take all necessary steps to make certain of the identity of clients or servers before interacting with them remotely
Moreover WCF is an framework to develop Service Oriented applications under microsoft platform mixing both message and rpc style of programming. Which was not in the remoting. Remoting is basically oriented to rpc only.
I am working on a C# application that contains multiple windows services that will need to communicate with each other to pass data around. These services may be on the same machine but they could be remote. I looked into using WCF for this purpose but it seems like WCF is too heavy and has a lot of extra configuration that, to me, seems unnecessary (.NET 3.5 is a requirement here, I know that .NET 4 simplified this)
So my question is, what would be the best replacement to WCF, besides the deprecated .NET Remoting that provide this functionality?
I have been using PInvoke to access the Windows RPC runtime for nearly 8 years. It's wicked fast and very reliable as far as a transport goes. When combined with a fast serializer like protobuf-csharp-port the resulting communications are rock solid and very fast.
So to build this from the ground-up this requires three parts:
Google's Protocol Buffers (protobuf-csharp-port) for serialization.
My own CSharpTest.Net.RpcLibrary for the transport.
A bit of glue code to put them together from protobuf-csharp-rpc.
These are all available on NuGet in the following packages: Google.ProtocolBuffers, CSharpTest.Net.RpcLibrary, and Google.ProtocolBuffers.Rpc.
The following is a quick run-down on getting started:
define a set of messages and a service using the Google Protocol Buffer Language.
Once you have that defined you will run ProtoGen.exe to generate the service stubs and messages in C#. Be sure to add the "-service_generator_type=IRPCDISPATCH" to generate the correct service code.
Now that you have the generated source files add them to a project and reference the three assemblies from the packages listed above.
Lastly take a look at the sample client/server code on the protobuf-csharp-rpc project page. Replace the "SearchService" with your service name, and you should be ready to run.
Optionally change the configuration of the RPC client/server. The example shows the use of LRPC which is local-host only; however the DemoRpcLibrary.cs source file show TCP/IP and Named Pipes as well.
You can always email me (roger # my user name) for any further information or examples.
Update
I wrote a quick startup guide: WCF replacement for cross process/machine communication.
You may want to look into ZeroMQ, it's very lightweight and effective and comes with good C# bindings. (Typing this on my mobile so you'll have to google for it yourself for now, sorry).
Look at NFX Glue.
It is way faster than WCF for coupled systems.
Interprocess communication with Glue Blog
Benchmark
Code: https://github.com/aumcode/nfx
I am looking for a simple .NET library that implements a concept of async message passing similar to Erlang OTP platform. So far, I have only found RetLang to be somewhat similar, but
It seems to be abandoned, and
It only supports message passing within one process.
You can try with MSMQ .you can use for single or a group of messages you want to put in the queue and read from it later asynchronously. :)
In my opinion the easiest way to do this in .net (aside from F# ;) ) is the TPL dataflow - lib
Use MSMQ which is quite simple to implement. It is exactly what you need - asynchronous messaging system. WCF is also good but is more complex to manage (config files) and adds a bit of overhead. MSMQ is a standard (and free) Windows component but to use it you need to enable it. MSMQ can be used for local communication (same process or any 2 processes within the same Windows domain)
Read this answer for more details and code examples.
Microsoft research developed and proved out (with EA/Bioware) a project called Orleans. It has been used in production and is being actively developed.
Quick Summary
Intro on Pluralsight
https://github.com/dotnet/orleans
You can use WCF to send and receive asynchronous messages. you can read more at msdn
Please have a look at spring messaging, this might be useful for you.
Vaguely remember seeing some discussions on this quite a while back but haven't heard anything since. So basically are you able to subscribe to an IObservable on a remote machine?
You can use IObservable.Remotable to use observables directly from other machines via .NET Remoting.
Another possible solution could be to use named pipes.
There is an excellent NuGet package NamedPipeWrapper, see the source on GitHub. It would be easy to write a thin RX wrapper over this, i.e. subscribe to the RX stream and push the messages out to other .NET listening processes using this library.
As this solution uses named pipes, it would be a true pub/sub solution which would support multiple subscribers in different processes.
Update
It is indeed very easy to write simple RX bridge code over the named pipes library. Use an RX Subject and insert the RX bridge code into the event handlers. Its not more than 4 lines of additional code at both ends. I can post the code if anyone is interested.
Update
For more information on named pipes, see .NET 3.5 Adds Named Pipes Support and Interprocess Communication Using .NET 3.5 Named Pipes IO. The aforementioned NuGet package NamedPipeWrapper is a much nicer version of the built-in support for named pipes that .NET 3.5 introduced.
Found this cool video on Channel 9 which an example of using IObservable.Remotable as Paul pointed out:
http://channel9.msdn.com/posts/J.Van.Gogh/Whats-different-about-the-3-versions-of-Rx-Part-3-NET-35-SP1/
Very interesting stuff, gonna spend a bit of time playing around with it now! :-D
Yes.
RX has built in support for using .NET Remoting to cross process boundaries.
If you install the NuGet package rx-remoting, it will install the assembly System.Reactive.Runtime.Remoting.dll which provides support for cross-process RX messages.
For demo code from Microsoft, see RX Across Processes. I've just tested the code on this page, and it works nicely. In order to get it to compile, you will need to add the following references:
NuGet: Reactive Extensions - Main Library (search for reactive extensions main)
NuGet: Reactive Extensions - .NET Remoting Support (search for reactive extensions remoting)
System.Runtime.Remoting (add as a normal reference, this assembly ships with .NET)
The Channel 9 video mentioned by #theburningmonk is also interesting to watch.
Update
Unfortuantely, this solution has one large limitation: you can only have one client process listening (all subsequent clients cannot connect). Pushqa solves this problem (see my other answer). Essentially, any library which implements RX on top of a pub/sub signalling bus should do the trick.
Yes.
Check out Pushqa.
Its easy to use. I was up and running in about 5 minutes.
It works with C# .NET, WPF, ASP.NET or Javascript. SignalR is built into ASP.NET, but it works for any C# .NET project if you add the right NuGet package.
It is superior to RX over .NET remoting (see my other answer), as we can have one server and many subscribers (it is a true pub/sub model, just like RX).
The queries are compiled into expression trees, and executed on the server (which minimizes network traffic, as only relevant results are returned from the server).
If we want queries to be filtered client side, then its easy - just do a client side filter on the results returned from pushqa.
Its literally 1% of the pain, 1% of the boilerplate code, and 10x the usability of Tibco. I wrote RX wrappers for Tibco and it was a nightmare to get it correct (Tibco has more corner cases than a tub of dodecahedrons). Unless you need to connect to legacy mainframe clients, or want to multicast to hundreds of clients over UDP, or want to waste a kings random in licensing fees, this solution is far superior to Tibco.
Its free.
Its open source.
There's no reason that a framework couldn't be devised for doing that. The framework would have to provide a means to address remote objects, generate proxies for them, then marshal the activity of the remote object across the application boundaries (i.e. through socket communication). .NET Remoting may be a suitable option for implementing this. WCF would be even better.
Are you specifically bound to using Rx as the solution to your problem? WCF provides duplex services, which have the ability for clients to register callback endpoints to a service. The service may then initiate calls back to its clients as necessary. It is effectively a remoted observer pattern. If RX is a must, then it should be fairly strait forward to wrap WCF duplex services with an RX support framework, allowing your clients to "transparently" observe service behavior with IObservable.
I need to monitor several systems going thru stress testing. I want to make a client-server .NET(C#) application that will ping the systems to check temperature, memory usage etc.
The client will provide system info (cpu-mem-hdd configuration) at start then it will undergo through several benchmark/stress tests. The server will keep track of what is been executed and also will be able to detect system crashes. I pretty much have all the code for the client (have been running on the system, using WMI)
I have no experience with .NET remoting nor WCF, but I think this is a great opportunity to learn them.
What technology would you use?
WCF is meant to unify .net remoting with a handful of other Microsoft technologies. WCF gives you a lot of flexibility to change the design of your client-server architecture simply by changing a few .net attributes.
I recommend you proceed with caution and make sure you have a good WCF reference at hand. You will spend a lot of time spinning your wheels in the mud without one.
Snmp, maybe based on http://www.snmpsharpnet.com/.
Nothing fancy, new and shiny, but a protocol that is meant to be used for exactly the stuff you describe. Would definitely be my first choice.
If that is not an option for you: Second choice for me would be WCF, because that can be used from different platforms (you are more flexible in the future).
i will choose service bus.