I'm working on a large project (teams are working separately) and now that most of the apps are finished, we need to integrate everything together. A lot of apps/forms need to communicate information to other apps/forms or solicit information from others; I wonder if there are standard ways of doing this. The way we are planning to do that is through the use of shared files (e.g. one app writes info to a file and the other reads it). Any suggestions?
Yes, as #Uwe Keim said in a comment, you should definitely use a database for everything. A SO answer is not the place to start teaching you about relational databases and how to use them properly from C# applications, though. You should tell the person managing the teams that's the way to go, and have them figure out who to hire to solve this.
As regards your question, I think that there is an obvious drawback in using the file-shared solution, which is the delay in your operation because of IO operations.
Moreover, there are other options, please see below:
To fully implement your requirements, you should separate the communication between applications and the communication among forms inside a application.
To implement communication between applications, I would suggest that you use the subscriber/publisher pattern using WCF. The implementation is quite easy. You can google it. One example is here. By doing that, you could control exactly when something happens and the other application should re-acts as you expects.
To implement communication amongst forms in an application, it'd better to create an event aggregator using the Pub/Sub pattern, and then inject it into your application. You can have a look at how it implemented in SCSF.
Another solution is to implement a message protocol to make the apps communicate each other.
To avoid bottlenecks, a message queue service can manage the messages exchange.
References:
wikipedia
msdn
Related
I am currently involved in a simple to medium complex IOT project. The main purpose of our application is gathering data from our devices and analyzing that data as well as calculating statistics.
On the server side we run a MVC application. Up until now we used Hangfire to schedule the calculations. Hangfire is an amazing tool for scheduling emails and other simple stuff, for more advanced things it's too slow. The calculations can take up a lot of time and are processor-intensive (we are trying to optimize them though), so we need to call them in a background task, a simple API call won't be enough.
I thought about splitting the application into multiple parts, the website, the core and a windows service.
The problem is, I never tried that before and I have no idea what the best practice is to achieve that kind of thing. I searched for examples and articles, but all I found were suggestions to use Hangfire and/or Quartz.NET.
Does anyone have any resources on what the best practice is to build a MVC application, a Windows service and how they could communicate (probably through a queue)? What is the best practice in such a situation?
Although there may be many different possible ways to connect a site with a windows service, I'd probably chose one of the following two, based on your statements:
Direct communication
One way of letting your site send data to your backend windows service would be to use WCF. The service would expose an endpoint. For simplicity's sake this could be a basicHttpBinding or a netTcpBinding. The choice should be made based on your specific requirements; if the data is small then basicHttp may be "sufficient".
The advantage of this approach is that there's relatively little overhead needed: You'll just have to setup the windows service (which you'll have to do anyway) and open a port for the WCF binding. The site acts as client, the service as server. There's nothing special with it, just because the client being a MVC site. You can take almost any WCF tutorial as a starting point.
Note that instead of WCF you could use another technology like .NET Remoting or even sockets just as well. Personally, I often use WCF because I'm quite used to it, but this choice is pretty opinion based.
Queued communication
If reliability and integrity is crucial for your project, then using a queue might be a good idea. Again: depending on your needs, there may come diffeent products into consideration. If you don't need much monitoring and out-of-the-box management goodies, then even a very simplistic technology like MSMQ may be sufficient.
If your demands to the aforementioned points are more relevant, then maybe you should look for something else. Just recently I got in touch with Service Bus for Windows Server (SBWS). It's the Azure Service Bus's little brother which can be used on premises locally on your windows server. The nice thing about it is, that it comes at no extra charge as it's already licensed with your windows server licence.
As with the first point: MSMQ and SBWS are just two examples. There may be a lot of other products like NServiceBus, ZeroMQ or others usable, you name it.
I am looking for a simple .NET library that implements a concept of async message passing similar to Erlang OTP platform. So far, I have only found RetLang to be somewhat similar, but
It seems to be abandoned, and
It only supports message passing within one process.
You can try with MSMQ .you can use for single or a group of messages you want to put in the queue and read from it later asynchronously. :)
In my opinion the easiest way to do this in .net (aside from F# ;) ) is the TPL dataflow - lib
Use MSMQ which is quite simple to implement. It is exactly what you need - asynchronous messaging system. WCF is also good but is more complex to manage (config files) and adds a bit of overhead. MSMQ is a standard (and free) Windows component but to use it you need to enable it. MSMQ can be used for local communication (same process or any 2 processes within the same Windows domain)
Read this answer for more details and code examples.
Microsoft research developed and proved out (with EA/Bioware) a project called Orleans. It has been used in production and is being actively developed.
Quick Summary
Intro on Pluralsight
https://github.com/dotnet/orleans
You can use WCF to send and receive asynchronous messages. you can read more at msdn
Please have a look at spring messaging, this might be useful for you.
I'm interested in gaining a better understanding of WCF.
Of course, I can read books and tutorials about it, but it seems that a better way would be to actually come up with some project idea (either open-source or a startup) which would actually benefit from using WCF, and then build it using WCF.
What are your ideas for small-scale projects which might benefit from WCF?
I'm not sure it is really a matter of scale that drives a decision to use WCF. If a learning project is all you are interested in, then take a normal idea for a project, and turn the entire data access layer into WCF calls.
This should give you a fair understanding of all the little nooks and crannies of WCF, and allow you to fail in a controlled manner. That way you can make decisions in the future about when are where it is best to apply a service boundary using WCF.
As was already mentioned, anything to do with the web can benefit tremendously from WCF. Heck, you could build a pure JavaScript and HTML 5 application using WCF without ever touching ASP.Net.
A hosted service that a mobile device (such as a WP7 or iPhone) could connect with to retrieve data
WCF is great for setting up non-ASPX endpoints for Ajax clients. See for example this article. There are many more out there.
Any project involving .NET and communication is likely to benefit from WCF. WCF is the replacement for ASMX web services and for .NET Remoting. There's no one particular type of application that it is suited for. For instance, it's not like it's suitable for Enterprise applications but not for small ones.
WCF data contracts are very easy and handy for storing application configuration, settings and state. Write a library/application to take care of serialisation and editing.
Is there a product (ideally open source, but not necessary), that would enable a zero dependency deployment? every service bus or queue library I've been able to find has a dependency on one of the queue apps (like msmq), or a database. I would like a very lightweight solution that I can just add a reference to my application, build it, and deploy it with as little configuration as possible.
In an ideal world, the queue/service bus would run on IIS, and allow web and rich clients to talk to it.
Such a tool would be ideal for fast prototyping of large distributed systems on a local development machine.
Rhino Queues from Ayende is exactly what you are looking for, this is the blog post introducing it:
http://ayende.com/Blog/archive/2008/08/01/Rhino-Queues.aspx
I think that all of the limitations mentioned in this post have been fixed since then.
From the blog post, what rhino queues is:
XCopyable, Zero Administration, Embedded, Async queuing service
Robust in the face of networking outages
System.Transactions support
Fast
Works over HTTP
In a similar vein to ShuggyCoUk's suggestion, you could rig up a queue (or queues) using the Windows built-in ESENT database (comes already installed with Windows). There is a managed code access library (open source): http://www.codeplex.com/ManagedEsent. If you stick with writing / reading CLOBs or BLOBs, it should work just fine. If you want to be really clever, you can use NServiceBus and write (contribute?) ESENT-flavored subscription storage and transports. There are some forays into using ESENT on Ayende's blog as well (you'll have to poke around his SVN repository for the juicy bits).
If you're happy to be:
Windows specific
Limited to the local domain
Seriously limited in the message size supported
Wrap the underlying win32 calls in P/Invoke
Deal with the polling yourself
Deal with the hacks needed to allow back and forth communication
Deal with the shared config needed to keep the names in sync
Then a quick wrapper around the windows MailSlot API might be sufficient.
This simple example is a reasonable basis to start.
This article has some further information but assumes the use case is via a control (rather than a Component as it should be) as well as some poor WinForms integration so should be considered for incidental reading rather than a basis for any library.
This article is C++ but is of a higher standard (and a commenter has extended it to support the batching of larger messages into several smaller ones).
You get 424 bytes (so with .Net 212 chars) you may want to drop to ASCII to double your useful message length if you are talking text.
Note that despite its simplicity, limitations and lack of features it does provide multicast delivery, something often complex to layer on a point to point protocol yourself.
This ayende post provides and interesting comparison of three service buses. We use NServiceBus and think if it's not clear that Udi Dahan would respond to how you'd plug in non-dependent queue.
We work using MSMQ happily but there are other options and in theory it should be open to practically anything, given that you may lose some reliability and durability depending on your choice.
Why not Amazon's message service Simple Queue Service?
We moved our projects from MSMQ to ActiveMQ. its really better :)
ActiveMQ is open source queue ,based on Apache web server.
We used him in production on high frequently data workflow, where msmq have a lot of problem (we work with msmq a year)
The csharp implementation is nms
I'm currently working on an open source WCF based service bus. You can find it here: http://rockbus.codeplex.com/. It supports dynamic (#run-time) subscriptions, subcription repository (database), pluggable transports, XPath based content-based routing, transactional delivery over wcf protocols, roundrobin delivery, pluggable subscription evaluation, and more. Have a look!
Have you thought about using a service like IronMQ by http://Iron.io?
You wouldn't have any dependencies, could quickly prototype apps without setting up any queue infrastructure, and it's highly available and fast.
There is not currently a locally installable version but it's based on the upcoming OpenStack protocol so there will be.
Btw I work for Iron.
Try https://github.com/mcintyre321/PieQ - this is my attempt to write a threadsafe, persistent, zero-config, embedded work queue. It probably needs a little love, but I think it might be the kind of tool you are looking for.
I have developed an InMemory JMS library which can be used to in testing JMS applications without really connecting to JMS providers/server (Think of hsqldb). You don't have to deal with connection or protocol or anything, all you need to do is to send and receive messages.
https://github.com/Dhana-Krishnasamy/InMemoryJMS
What is the best and easiest method that can be used for inter-process communication in a very large project?
My requirement is to communicate between a normal Windows Forms Application and Windows Services.
Methods that are easy to maintain and implement are preferred.
Thanks
From the tags I understand that we are talking about .NET. Perhaps you should try Microsoft WCF. It unifies this issue, abstracting specific inter-process (inter-service) communication technology from actual code. So generally you'll design and write the interfaces that your processes will use to talk to each other and then you'll configure a specific communication technology in XML config file. That is, you have rather clear separation between "what do the processes talk about" and "how is this communication implemented specifically".
WCF supports SOAP, TCP\IP communication, MSMQ etc., you processes can be IIS-hosted web-services, usual Windows services, console applications etc. - all this under unified framework. I think, this is exactly what you are looking for.
It really depends on the project as there are a large number of methods.
This may depend on where the different portions of the project run (They could run on different servers, or different technology stacks altogether.).
The most common method is probably web services. Although these come with an overhead, so it may be worth looking at a simple interface API via a DLL.
Whatever you do it should probably be thought about and designed carefully, considering security and performance, and how you will extend or modify it in the future.
Not necessarily the best or the easiest....
In the .NET world try MSMQ or IBM MQ message queue middle ware.
If the communication is mostly 1-way, then consider using WCF services, which are both good and easy if you let the code generators in Visual Studio do most of the work for you.