I need to create a service that will expose data and contain business logic that different clients will use.
The problem is that if I develop it as a single WCF service I will need to shutdown it for some time to update it. This is not acceptable.
I was thinking of creating core WCF service that will be watching a directory with plugins and load them dynamically if the file changes. Is this the correct way of doing this? And do I organize such system with WCF?
Is load balancing acceptable to you? Then you can have two servers and update them one at a time. There would be no interruption, and no need for a complex plugin architecture.
Multiple instances on Azure would be an easy (though perhaps not cost effective?) way of achieving this.
It would update one instance at a time with no down time between. And with >1 instance you get 99.9% uptime in the SLA
Host it in IIS (a normal web application). On update, if you copy your new dlls to the 'bin' directory, the application will be recycled. Practically without downtime.
Related
I'm curious about the best way for a C# gui to access the functions of a Windows Service, be it WCF or ServiceController or some other option.
I'll explain what I'm doing: on a regulated time interval the service will be zipping one hours worth of datafiles from location A and sending the zipped file to location B, this will be done in the background 24/7 or until the service is stopped by the user and runs even when no one is logged in (hence the need for service) I would like the user to be able to pull up a GUI program that will allow them several options:
1) change the location to zip from
2) change the location to zip to
3) manually start the zipping process for a DateTime range specified
Now most of the functions for zipping and timers is all stored within the service. SO im wondering if a ServiceController in the GUI program would allow me to send variables to/from the service (ie folderpath names as strings, various other data) or if I'll need to spend the time making a WCF and treat the GUI as the Client and the Windows service as a source.
It should be noted the GUI will likely recieve data from the service, but only to signify if it is currently busy zipping.
One option is to tave a WCF service embedded on your windows service. With this WCF you can control the behaviour without restarting the service.
But the best option IMO is to have this in a config file. You can add some keys, but you would have to restart the service when you update the config.
In this case you can try a workaround, as in this thread.
The config is a good place for this kind of detail, because it can be easily modified and, unlike a database, it will be always avaiable.
I don't fully understand what you're trying to say, but you define what the interface to your service is when you make it. If the operations you define take in variables, then you can pass data from your application to the service.
From what you described, just make opertaions in the service to do those 3 things you listed then call those from a button click in your UI code.
WCF would be fine for this here's a basic introduction to it http://msdn.microsoft.com/en-us/library/bb332338.aspx.
I am developing a asp.net site that needs hit a few social media sites daily for blanket friend/follower data. I have chosen arvixe business class as my hosting. In the future if we grow, I'd love to get onto a dedicated server and run a windows service, however since that is not in the cards at this point I need another reliable way of running scheduled tasks. I am familiar with running a thread timer from the app_code(global.aspx). However the app pool recycling will cause some problems with the timer. I have never used task scheduling like quartz but have read a lot about it on stackoverflow. I was looking for some advise as to how to approach my goal. One big problem I have using either method is that I will need the crawler threads to sleep for up to an hour regularly due to api call limits. My first thoughts were to use the db to save the starting and ending of a job. When the app pool recycles I would clear out any parts not completed and only start parts that do not have a record of running on that day. What do the experts here think? any good links to sample architecture of this type of scheduling?
It doesn't really matter what method you use, whether you roll your own or use Quartz. You are at the mercy of ASP.NET/IIS because that's where you want to host it.
Do you have a spare computer laying around that can just run a scheduled task and upload data to a hosted database? To be honest, it's possibly safer (depending on your use case) to just do it that way then try to run a scheduler in ASP.NET.
Somewhat along the lines of Bryan's post;
Find a spare computer.
Instead of allowing DB access have it call up a web service on your site. This service call should be the initiator of the process you are trying to do. Don't try to put params into it, just something like "StartProcess()" should work fine.
As far as going to sleep and resuming later take a look at Workflow Foundation. There are some nice built in features to persist state.
Don't expose your DB to the outside world, instead expose that page or web service and wraps some security around that. WCF has some nice built in security features for that.
The best part is when you decide to move off, you can keep your web service and have it called from a Windows Service in the same manner.
As long as you use a persistent job store (like a database) and you write and schedule your jobs so that they can handle things like being killed half way through, having IIS recycle your process is not that big a deal.
The bigger issue is that IIS shuts your site down if it doesn't have traffic. If you can keep your site up, then just make sure you set the misfire policy appropriately and that your jobs store any state data needed to pick up where they left off, you should be able to pull it off.
If you are language-agnostic and don't mind writing your "job-activation-script" in your favourite, Linux-supported language...
One solution that has worked very well for me is:
Getting relatively cheap, stable Linux hosting(from reputable
companies),
Creating a WCF service on your .Net hosted platform that will contain the logic you want to run regularly (RESTfully or SOAP or XMLRPC... whichever suits you),
Handling the calls through your Linux hosted cron jobs, written in your language of choice(I use PHP).
Working very well, like I said. No VPS expense,configurable and externally activated. I have one central place where my jobs are activated, with 99 to 100% uptime(never had any failures).
I am in the process of creating an application which will communicate with a single server where WCF Web Service(s) would be installed. I am a little new to this process and was wondering which of these two options would be better in the long run to handle the load for a significant amount of users:
1- Create and install a single Web Service on a multi-core server for all of the client applications to communicate with.
2- Create and install multiple Web Services on a multi-core server, each to communicate with different modules inside of the client application.
All-in-all I'm just trying to figure out whether in processing time and with a large number of users whether there is a significant difference between options 1 and 2, or if option 2 would just create an unnecessary programming headache.
Thanks,
Patrick
The advantage of having multiple web services would be that each can have their own application pool (i.e. worker process) in IIS. So you can recycle one application pool for one web service without affecting the others.
The advantage of having a single web service would be potentially easier maintenance, since the code is in one file, etc. Of course, if it's a lot of code, this can make maintenance harder too.
So the question is, what's the right level of granularity?
You can split the web services up per business function, and I've found that this is a good approach. For example, if you have some business methods that deal with invoicing, you could put those into an Invoicing web service.
If you have other business methods that deal with shipping orders, you could put those into a Shipping web service.
This creates a nice split, in my opinion, and also lets you leverage the application pool advantages discussed earlier.
Example
You can see a real world example of this type of split with FedEx. Note how they split their web services up by shipping, tracking and visibility, etc.
I have a little experience with WCF and would like to get your opinion/suggestion on how the following problem can be solved:
A web service needs to be accessible from multiple clients simultaneously and service needs to return a result from a shared data set. The concrete project I'm working on has to store a list of IP addresses/ranges. This list will be queried by a bunch of web servers for a validation purposes and we speak of a couple of thousand or more queries per minute.
My initial draft approach was to use Windows service as a WCF host with service contract implementing class that is decorated with ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple) that has a list object and a custom locking for accessing it. So basically I have a WCF service singleton with a list = shared data -> multiple clients. What I do not like about it is that data and communication layers are merged into one and performance wise this doesn't feel "right".
What I really really (- want is Windows service running an instance of IP list holding container class object, a second service running WCF service contract implementation and a way the latter querying the former in a nice way with a minimal blocking. Using another WCF channel would not really take me far away from the initial draft implementation or would it?
What approach would you take? Project is still in a very early stage so complete design re-do is not out of question.
All ideas are appreciated. Thanks!
UPDATE: The data set will be changed dynamically. Web service will have a separate method to add IP or IP range and on top of that there will be a scheduled task that will trigger data cleanup every 10-15 minutes according to some rules.
UPDATE 2: a separate benchmark project will be kicked up that should use MySQL as a data backend (instead on in-memory list).
It depends how far it has to scale. If a single server will suffice, then fine; keep it conveniently in memory (as long as you can recreate the data if the server gets restarted). If the data-volume is low, then simple blocking (lock) should work fine to synchronize the data, or for higher throughput a ReaderWriterLockSlim. I would probably not store it directly in the WCF class instance, though.
I would avoid anything involving sessions (if/when this ties into the WCF life-cycle); this is rarely helpful to simple services.
For distributed load (over multiple servers) I would give consideration to a separate dedicated backend. A database or memcached / AppFabric / etc would be worth consideration.
I've got a C# service that currently runs single-instance on a PC. I'd like to split this component so that it runs on multiple PCs. Each PC should be assigned a certain part of the work. If one PC fails, its work should be moved to a backup machine.
Data synchronization can be done by the DB, so that should not be much of an issue. My current idea is to use some kind of load balancer that splits and sends the incoming requests to the array of PCs and makes sure the work is actually processed.
How would I implement such a functionality? I'm not sure if I'm asking the right question. If my understanding of how this goal should be achieved is wrong, please give me a hint.
Edit:
I wonder if the idea given above (load balancer splitswork packages to PCs and checks for result) is feasible at all. If there is some kind of already implemented solution so this seemingly common problem, I'd love to use that solution.
Availability is a critical requirement.
I'd recommend looking at a Pull model of load-sharing, rather than a Push model. When pushing work, the coordinating server(s)/load-balancer must be aware of all the servers that are currently running in your system so that it knows where to forward requests; this must either be set in config or dynamically set (such as in the Publisher-Subscriber model), then constantly checked to detect if any servers have gone offline. Whilst it's entirely feasible, it can complicate the scaling-out of your application.
With a Pull architecture, you have a central work queue (hosted in MSMQ, Sql Server Service Broker or similar) and each processing service pulls work off that queue. Expose a WCF service to accept external requests and place work onto the queue, safe in the knowledge that some server will do the work, even though you don't know exactly which one. This has the added benefits that each server monitors it's own workload and picks up work as-and-when it is ready, and you can easily add or remove servers to/from this model without any change in config.
This architecture is supported by NServiceBus and the communication between Windows Azure Web & Worker roles.
From what you said each PC will require a full copy of your service -
Each PC should be assigned a certain
part of the work. If one PC fails, its
work should be moved to a backup
machine
Otherwise you won't be able to move its work to another PC.
I would be tempted to have a central server which farms out work to individual PCs. This means that you would need some form of communication between each machine and and keep a record back on the central server of what work has been assigned where.
You'll also need each machine to measure it's cpu loading and reject work if it is too busy.
A multi-threaded approach to the service would make good use of those multiple processor cores that are ubiquitoius nowadays.
How about using a server and multi-threading your processing? Or even multi-threading on a PC as you can get many cores on a standard desktop now.
This obviously doesn't deal with the machine going down, but could give you much more performance for less investment.
you can check windows clustering, and you have to handle set of issues that depends on the behaviour of the service (you can put more details about the service itself so I can answer)
This depends on how you wanted to split your workload, this usually done by
Splitting the same workload by multiple services
Means same service being installed on
different servers and will do the
same job. Assume your service is reading huge data from the db servers and processing them to produce huge client specific datafiles and finally this datafile is been sent to the clients. In this approach all your services installed in diff servers will do the same work but they split the work to increaese the performance.
Splitting the part of the workload by multiple services
In this approach each service will be assigned to the indivitual jobs and works on different goals. in above example one serivce is responsible for reading data from db and generating huge data files and another service is configured only to read the data file and send it to clients.
I have implemented the 2nd approach in one of my work. Because this let me isolate and debug the errors in case of any failures.
The usual approach for load balancer is to split service requests evenly between all service instances.
For each work item (request) you can store relative information in database. Then each service should also have at least one background thread checking database for abandoned work items.
I would suggest that you publish your service through WCF (Windows Communication Foundation).
Then implement a "central" client application which can keep track of available providers of your service and dish out work. The central app will act as scheduler and load balancer of the tasks to be performed.
Check out Juwal Lövy's book on WCF ("Programming WCF Services") for a good introduction on this topic.
You can have a look at NGrid : http://ngrid.sourceforge.net/
or Alchemi : http://www.gridbus.org/~alchemi/index.html
both are grid computing framework with load balancers that will get you started in no time.
Cheers,
Florian