I am currently involved in a simple to medium complex IOT project. The main purpose of our application is gathering data from our devices and analyzing that data as well as calculating statistics.
On the server side we run a MVC application. Up until now we used Hangfire to schedule the calculations. Hangfire is an amazing tool for scheduling emails and other simple stuff, for more advanced things it's too slow. The calculations can take up a lot of time and are processor-intensive (we are trying to optimize them though), so we need to call them in a background task, a simple API call won't be enough.
I thought about splitting the application into multiple parts, the website, the core and a windows service.
The problem is, I never tried that before and I have no idea what the best practice is to achieve that kind of thing. I searched for examples and articles, but all I found were suggestions to use Hangfire and/or Quartz.NET.
Does anyone have any resources on what the best practice is to build a MVC application, a Windows service and how they could communicate (probably through a queue)? What is the best practice in such a situation?
Although there may be many different possible ways to connect a site with a windows service, I'd probably chose one of the following two, based on your statements:
Direct communication
One way of letting your site send data to your backend windows service would be to use WCF. The service would expose an endpoint. For simplicity's sake this could be a basicHttpBinding or a netTcpBinding. The choice should be made based on your specific requirements; if the data is small then basicHttp may be "sufficient".
The advantage of this approach is that there's relatively little overhead needed: You'll just have to setup the windows service (which you'll have to do anyway) and open a port for the WCF binding. The site acts as client, the service as server. There's nothing special with it, just because the client being a MVC site. You can take almost any WCF tutorial as a starting point.
Note that instead of WCF you could use another technology like .NET Remoting or even sockets just as well. Personally, I often use WCF because I'm quite used to it, but this choice is pretty opinion based.
Queued communication
If reliability and integrity is crucial for your project, then using a queue might be a good idea. Again: depending on your needs, there may come diffeent products into consideration. If you don't need much monitoring and out-of-the-box management goodies, then even a very simplistic technology like MSMQ may be sufficient.
If your demands to the aforementioned points are more relevant, then maybe you should look for something else. Just recently I got in touch with Service Bus for Windows Server (SBWS). It's the Azure Service Bus's little brother which can be used on premises locally on your windows server. The nice thing about it is, that it comes at no extra charge as it's already licensed with your windows server licence.
As with the first point: MSMQ and SBWS are just two examples. There may be a lot of other products like NServiceBus, ZeroMQ or others usable, you name it.
Related
I'm interested in gaining a better understanding of WCF.
Of course, I can read books and tutorials about it, but it seems that a better way would be to actually come up with some project idea (either open-source or a startup) which would actually benefit from using WCF, and then build it using WCF.
What are your ideas for small-scale projects which might benefit from WCF?
I'm not sure it is really a matter of scale that drives a decision to use WCF. If a learning project is all you are interested in, then take a normal idea for a project, and turn the entire data access layer into WCF calls.
This should give you a fair understanding of all the little nooks and crannies of WCF, and allow you to fail in a controlled manner. That way you can make decisions in the future about when are where it is best to apply a service boundary using WCF.
As was already mentioned, anything to do with the web can benefit tremendously from WCF. Heck, you could build a pure JavaScript and HTML 5 application using WCF without ever touching ASP.Net.
A hosted service that a mobile device (such as a WP7 or iPhone) could connect with to retrieve data
WCF is great for setting up non-ASPX endpoints for Ajax clients. See for example this article. There are many more out there.
Any project involving .NET and communication is likely to benefit from WCF. WCF is the replacement for ASMX web services and for .NET Remoting. There's no one particular type of application that it is suited for. For instance, it's not like it's suitable for Enterprise applications but not for small ones.
WCF data contracts are very easy and handy for storing application configuration, settings and state. Write a library/application to take care of serialisation and editing.
I am starting on a fairly basic Server/Client application (logic wise), but I am a bit confused as to what I should use for my needs. It looks like there a few options, but basically I am going to have a Master Server, and X amount of client applications (one per dedicated machine). The main purpose of this setup is so that I can basically do the following...
-Issue command to server (console app) via an ASP front end to install software on one of the remote clients.
- Server tells client to download zip package (from a various FTP site) to location and extract it to specific path.
I am not positive, but it looks like C# has Sockets and then some sort of WebClient type of deal. I am assuming Sockets would be the best route to take, and to use asynchronous (each remote client is connected in its own thread, dealing with the server individually of others).
Any information on this would be great!
Without going into too much detail for your specific requirements, I would definitely look at WCF.
It encompasses a lot of the existing remoting, client / server, web services scenarios in a very complete and secure framework.
Client Server Programming with WCF
WebClient allows you to make HTTP requests, so I don't think it's very relevant here.
There are many approaches you can take for this app.
One is of course going with WCF, which provides about a million time more options than you will need. However, WCF does have a learning curve and in particular it's hard to understand what exactly is hidden behind all the abstractions without prior experience. Furthermore, this solution is not available if you are targeting .NET 2.0.
You can also implement a simple TCP client/server model using sockets. While you can program against raw sockets, .NET also offers the convenience classes System.Net.Sockets.TcpListener for the server and System.Net.Sockets.TcpClient for the clients. This approach is much closer to the metal, but this is a tradeoff: it's much easier to understand what exactly you are doing, but you will have to implement a fair bit of functionality yourself.
I'm writing a simple little game for my kids - it doesn't really matter what it does, though I couldn't tell you anyway, since I/they haven't quite decided yet! However, I think it will have a server component and a number of client components, and I'm looking at ways that the clients can communicate with the server.
ALL my previous experience... my entire career in fact... has involved the server element either being a database, a web server, or the two in tandem. Neither are appropriate in this case, so I'm curious as to what means I could & should use to communicate between the two.
Obviously, it would be preferable to adopt a technology or technique that I can re-use in my work, where I'm increasingly working with Windows Forms. I imagine there are 1001 different approaches I could adopt; it's a question of sorting the wheat from the chaff.
I've literally just started reading about WCF, but its unclear as yet, if this service-oriented approach is what I'm looking for.
I'm being deliberately vague about what the applications will do; I expect the client will announce their presence to the server, will feed user choices up to the server, and in return, the server will periodically update the client with what is going on in the wider game. The game will be turn-based rather than real-time... and quite low-tech really!
Suggestions? Ideally, with links to good learning resources if any are known.
Conclusion:
I actually thought there might be more viable alternatives; there is Remoting (now depracated), but the consensus says that WCF is the way to go - in my case, self-hosting looks appealing.
Thanks for the responses.
It looks like you're interested in WCF and that is a reasonable technology to use in this case.
When writing a network game the easiest approach is to use the client server approach here too. With WCF you have some different hosting possibilities, hosting in IIS or self hosting. I would go for self hosting to avoid the need for IIS on your home computers.
The service could be hosted in either a windows service or actually in one of the clients. I recommend running as a windows service. The service could very well run on one the same machine as one of the clients.
Edit:
If you want to host the server in one of the clients you could provide a menu option "start service" that starts a self hosted service on that computer (and automatically connect the client part to localhost). After the service has started you can present the computer name that you enter on the "other" computers to connect.
I would suggest to separate the service part into a separate project, then you can easily break out the service to a windows service later on if you like.
Edit2:
By the way, since WCF is driven by calls from the client you need to poll the server for changes. You can google wcf long polling or just long polling for methods to "push" messages from the server to the client.
Check out this article on creating a chat application using WCF -- http://www.codeproject.com/KB/IP/WCFWPFChatRoot.aspx
I've played around with this code in the past and the project was pretty simple to launch locally.
Here is a good book on WCF, which can help get you started. It has quite a bit of a learning curve (I still have barely scratched the surface myself), but I get the sense that it's a very powerful way to set up a service-based application. On the page I linked, check out the Examples link, which includes a bunch of code from the book, including Juval's ServiceModelEx, which has a wide variety of useful classes for working with WCF.
I'm going to go against the crowd here and suggestion NOT using WCF. WCF is a great technology for service oriented architecture, however it is largely written on the idea that clients will not stay connected to the server, but rather connect, send a message, perhaps receive a result, and disconnect.
There is a mechanism that can be used for long connected clients, but frankly, it doesn't work very well, has all kinds of issues and quirks, and is not very reliable in terms of knowing when clients have disconnected or not, or whether the clients know if they are still connected to the server. It's more of a bolted on solution that tries to shoehorn itself into the WCF model.
The other issue is one of firewalls. There can't be a firewall between the initiator and the receiver (or there must be an open port). That means you can't easily have two clients behind firewalls that want to talk to each other. You need some kind of exposed intermediate server that is open. While this problem applies to all solutions, it's easier to manage with straight TCP connections than WCF.
I'm not a huge proponent of rolling your own solution, but WCF really is overkill for simple two way communication between client apps.
I have not yet found a good open source network library for .NET. I know lots of people will chime in with various libraries, but every one i've seen is old and has various flaws. Most seem to be someone that ripped out their library from an app and tried to package it as a generic one, but leaving in all the assumptions of their app.
The problem is that recent versions of .NET have added lots of new network functionality, particularly in terms of asynchronous support. As of yet, i've not seen any libraries which implement these functions.
Anyone want to help me build a good, from scratch, network library based on .net 3.5+?
I would look at Remoting if I was you. It is fairly easy to impliment and you can definatly use it at work.
There is a tutorial on it here:
http://generally.wordpress.com/2007/05/31/a-simple-remoting-example-in-c/
I'm writing a windows service to do some daily processing, and I want to have a user-friendly way to interact with it. I'll just be doing basic things like checking its status and viewing logs, though I may decide I want to throw in a function call or two as well. After doing some research, it sounds like I need a separate application to perform these functions, since the service will run independently of any user that's logged into the host machine. My idea is to have this application interact with the service through some kind of interface, but I'm not sure where to begin.
What would be the simplest way to have an application communicate with a separate service? Would I use COM, WCF, a message queue, or something else entirely? I know there are probably a few ways to do this, so I would love to hear some pros and cons if possible.
Edit: The service and the application will both be running on the same machine.
Use WCF with NetNamedPipeBinding (allows only IPC on the same machine) or .NET Remoting. If you want to do it quickly choose the technology you are more familiar with. If you are not familiar with any of these technologies choose WCF because it is newer one and you will more probably use it again in the future - so exeprience with it will be useful.
Ideally you would create a separate application and use WCF to communicate between your service and this application.
But there is a 'cheaper' way which is to implement your own simple Web server using HttpListener. See http://msdn.microsoft.com/en-us/magazine/cc163879.aspx
This makes it easy to accept a few simple commands and you can send them using any web browser.
For viewing logs why not just tail the log files (using e.g. baretail)?
Skip WCF, and just use plain .NET Remoting. So much easier. Why they call it deprecated, God knows.
Edit: Seeing it runs on the same PC, the transport would be Named Pipes, IIRC WCF supports this too.
More of a design/conceptual question.
At work the decision was made to have our data access layer be called through webservices. So our website would call the webservices for any/all data to and from the database. Both the website & the webservices will be on the same machine(so no trip across the wire), but the database is on a separate machine(so that would require a trip across the wire regardless). This is all in-house, the website, webservice, and database are all within the same company(AFAIK, the webservices won't be reused by another other party).
To the best of my knowledge: the website will open a port to the webservices, and the webservices will in turn open another port and go across the wire to the database server to get/submit the data. The trip across the wire can't be avoided, but I'm concerned about the webservices standing in the middle.
I do agree there needs to be distinct layers between the functionality(such as business layer, data access layer, etc...), but this seems overly complex to me. I'm also sensing there will be some performance problems down the line.
Seems to me it would be better to have the (DAL)assemblies referenced directly within the solution, thus negating the first port to port connection.
Any thoughts(or links) both for and against this idea would be appreciated
P.S. We're a .NET shop(migrating from vb to C# 3.5)
Edit/Update
Marked Dathan as answer, I'm still not completely sold(I'm still kind of on the fence, though leaning it may not be as bad as I feared), he provided a well thought out answer. I appreciated all the feedback.
Both designs (app to web service to db; app to db via DAL) are pretty standard. Web services are often used when interfacing with clients to standardize the semantics of data access. The web service is usually able to more accurately represent the semantics of your data model than the underlying persistence store, and thus helps the maintainability of the system by abstracting and encapsulating IO-specific concerns. Web services also serve the additional purpose of providing a public interface (though "public" may still mean internal to your company) to your data via a protocol that's commonly accessible across firewalls. When using a DAL to connect directly to the DB, it's possible to encapsulate the data IO concerns in a similar way, but ultimately your client has to have direct access to the database. By restricting IO to well-defined semantics (usually CRUD+Query), you add an additional layer of security. This isn't such a big deal for you, since you're running a web app, though - all DB access is already done from trusted code. The web service does provide an increase in robustness against SQL injection, though.
All web service justifications aside, the real questions are:
How much will it be used? The website/web service/database format does impose slightly higher overhead on the web server - if the website gets hammered, you want to consider long and hard before putting another service on the same machine. Otherwise, the added small inefficiency is probably not a big deal. On the other hand, if the site is getting hammered, you probably want to scale horizontally anyway, and you should be able to scale the web service at the same time.
How much to you gain? One of the big reasons for having a web service is to provide data accessibility to client code - particularly when multiple possible application versions need to be supported. Since your web app is the only client to use the web service, this isn't a concern - it's probably actually less effort to version the app by itself.
Are you looking to expand? You say it probably won't ever be used by any client other than the single web app, but these things have a way of gaining in size. If there's any chance your web app might grow in scope or popularity, consider the web service. By designing around a web service, you're already targeting a modular, multi-host solution, so your app will probably scale with fewer growing pains.
In case you couldn't guess, I'm a web service fan. But the above are also my honest (if somewhat biased) opinions on the subject. If you do go the web service route, be sure to make it simple - keep application logic in the app and service logic in the service, and try to draw a bright line between them when extending the two. And do design your service for efficiency and configure the hosting to keep it running as smoothly as possible.
This is a questionable design, but your shop isn't the only one using it.
Since you're using .NET 3.5 and running on the same machine, you should use WCF with the netNamedPipesBinding, which uses binary data transfer over named pipes, only on the same machine. That should mitigate the performance issue somewhat.
I like the idea because it gives you flexibility. We use a very similar approach because we can have more than 1 type of database storing our data (MSSQL or Oracle) depending on our customer install choices.
It also gives customers the ability to hook into our database if they choose not to use our front end web site. As a result we get an open API for little to no extra effort.
If speed is your most critical issue than you have to lessen your layers. However in most cases the time it takes for your web Service to process the request from the database does not add allot of time. (This is assuming you do your Web Service Layer Correctly, you can easily make it slow if you don't watch it.)