Transfer data between websites - c#

I need some architectural advice on how to build a background service application.
Background:
I have 2 websites, and I need to transfer some data from website A to website B. Service
would have to run in a background (as windows service) and should connect (every 5 minutes)
to websites's A database directly (MSSQL) grab some data and insert this data through
websites's B API (API is build on MVC Web Api). Both websites are hosted on a same virtual
machine (Windows Server 2008 R2 Datacenter), but this might change (website B can be switched to another virtual server or cloud hosting as Windows Azure or Amazon AWS).
Question:
What do you suggest (best practices) and what guidelines you can give me? I want this to
be scalable and fast as possible and that service will receive multiple requests.
Thank you,
Jani

If it is important to know what data was transferred, then:
Add logs - log4net for instance
Issue tickets if the process stops, and close the ticket when it restarts, this way you will know if a process fails. Depending on the amount of data use you could use Redis/Riak.
Put monitoring on each service A and B, and you might also consider restarting the service via IIS on fail down.

Related

Deploy a backend application on a Windows Server

I'm making an application in C# with VS 2012 that checks a database every 15 seconds and perform some actions when it finds data. Right now I've created a Console Application so I can debug it easely but during relese this application needs to run in a IIS server.
How can I do that? I've read this question but it looks like some sort of workaround because to run it I need to perform these steps. Right now I'm reading the docs about Windows Service Application, Is this the right way?
EDIT Sorry but I've never used Windows server before, so as people pointed out IIS is only a web server, the thing I need to do is run my application in a Windows Server environment
IIS is a web-server and accordingly it should be used for hosting web applications.
Develop a windows service which does the job of checking the database in intervals and invoke a web service (which you can host in IIS)
If your application is performing some data query and manipulation on the server then I would recommend the approach to host it in a windows service.
Some advantages to this are:
The service will start and run independently of a user logging into the server.
You can configure the service to recover should it experience an exception (ideally not!).
The service will start automatically (if configured) when the server restarts.
You can configure which user group (or user) the service should run under so you have a more granual approach to security.
As it's running as a seperate process, you can monitor its memory and processor utilisation.
Debugging is slightly more cumbersome but not difficult, one approach I've used is to install the service locally, start it and then attach to it via the debugger. This question describes another approach I've also used.

Permanent data gathering

I have database and mvc application hosted on iis. I periodicaly gather data from internet and save them in sql database. And i calculate statistic and graphs from these data and publish them in mvc application.
Problem is that iis have recycling period about 1 hour -> meaning that my timer(function) which gather data from interenet is stoped whenever there is server restart, recycling or there is no request on the web page.
solutions i have found are:
turn of recycling - i don't own srv can't do that.
windows service - 99% hostings don't allow host ws...
So is there any solution, service, framework, which purpose is to gather data and i can be sure that it will not stop after some inactivity time or server restart? or is my logic completely wrong and i need to gather data diferently? can it be done on hosting which i don't own? can it be done using iis?
can it be done using iis?
If the IIS in question has app fabric installed, then that supports an auto start feature, which effectively lets you write 'service like' code which will keep running in the background.
Quick overview here

How to run a program in WCF?

I am new to WCF and i am designing a project in which i want to run a crawler program (coded in c#) which crawlers some websites and it stores the crawled data in the tables of database (sql server db). I want that crawler runs repeatedly after 30 minutes and updated the database.
I want to then use the service on my hosted platform so that i can use the data from tables in web form (i.e. .aspx page)
Is it okay to use WCF for this purpose ?
Please suggest me how to move on ?
Thanks
Windows Communication Foundation (WCF) is responsible for communication between 2 points with different channel technology. you will use WCF if you want to send/receive some data between two point regardless channel technology (TCP/UDP/NetPipe/MSMQ , ...)
But you first need to design you crawler application which is configured to fetch data from your target web sites, then you need to design a schedular application using
http://quartznet.sourceforge.net/
to run your crawlers.
after running and storing your web pages you can use WCF if you need to do replication or synchronization with center server but it is optional
You could use a WCF service to do this but I would go for another setup:
I'd build a Windows application that is scheduled to run every 30 minutes by the Windows Task Scheduler. A simple console application might be fine.
I'd use a Web application (possibly ASP MVC) to query the database.
As you can see there is no need to use WCF at all.
An exception can/must be made when the server is not yours but you are using a hosting provider who doesn't allow you to schedule a Windows task. In that case you might want to run the crawling process by hand through the web application and have it repeat itself after 30 minutes.
Some hosting providers do allow the scheduling of tasks but in a different way so it might be worth to investigate.

IIS vs Windows Service?

I have a C# application that needs to always be running. I originally planned on making this a windows service but I now have a requirement to make the application host a web admin console.
I haven't played with IIS in quite a few years so my question is this:
What would you recommend I use?
I've thought about making a windows service and embedding a web server such as Cassini but so far I'm not very happy with the open source web servers I've looked at.
Can IIS handle this? Do people use it for this type of scenario, and if so how?
This sounds like a job for two separate projects.
One is the original Windows Service. Windows Services are well suited for what you're doing.
The second is the Web Project that will be used to administer the Windows Service. This is the part that runs in IIS.
It depends on what you mean by always running. An ASP.NET web application deployed in IIS could very well be unloaded by the web server if there aren't any requests for certain amount of time killing all background threads. So if you want an ever running background thread it would be better suited to use a Windows Service. As far as the web admin is concerned, well, here you don't have much choice: ASP.NET in IIS. In order to do something useful those two applications should be able to find a common language to talk. So you could use a database to store the results into which could be used by both applications.
IIS will run your app on first request, not on server boot. So you will still need to run a service to ensure your app is always running.
You can use IIS as a webserver for your web admin part, and link your ASP.net app with your service by means of a configuration database (easy) or webservices (a little more tricky).
Windows and Web services are two very different creatures. A web service will expose external methods that you can implement against an application, while a windows service is an entity within itself. If you're planning on using this service on a timed interval to perform an operation, a Windows service would be the right way to go. If you use a web service, you will need to invoke the method you wish to run from a secondary application.
If you need to queue commands against your windows service, you could always create a database that was accessible by both your website and your windows service. This way you could send commands and query data between the two. Placing a web service in to serve as an intermidary between the two may be overkill.

partially connected application using asp.net 3.5 (not mobile apps)

We had a requirement to build a ASP.NET 3.5 web application using web forms, WCF, ADO.NET and SQL Server. The users would connect via Internet.
Recently we understood that it is possible that users would often remain disconnected and would have Internet access intermittently.
I need to understand if we can create occasionally connected web application using asp.net 3.5 - what all technologies/features we need to use? Is MS Sync Framework the answer to the problem - is it a viable option to use with web application?
Is windows application the right approach instead of web applications - where the business logic would be run at the client itself, using local SQL Express editions with data then been synced up with Enterprise SQL server at server end when connection is established using replication and/or MS Sync framework. In that case is there a need to use WCF?
Does Silverlight applications help in this context -building paritally connected web apps?
Really appreciate if you can give pointers to how to go about this task of creating .net partially connected apps (not mobile apps)?
It looks to me as if you'll need to store your client data locally when not connected.
If you use wcf you can determine what type of protocol to use according to connectivity without affecting your main code e.g. tcp/ip for LAN, http for internet and msmq for storing up data when disconnected.
If data for transfer is stored up using msmq, as soon as a connection is remade then the data will be passed to your main server.
If you write your wcf, or communications code to run as a service (assuming windows functionality here) then it is up to you whether to retain the asp code or write a new windows app.
edit
Setup MSMQ at both ends, its part of windows setup and can be installed on a client machine, just the same as IIS is, it's on the installation disk but not installed by default.
I wouldn't use it to get web pages, have those available on the local machine, but instead use it to queue up data that MUST get back to the server. Your data access layer should be separated from your GUI layer anyway. I assume that your using the MVC pattern or similar.
I don't know what your application is requried to do but here is the example that I've worked on.
A mobile user who visits clients. He has a replicated copy of a company product database on his laptop. When he visits client sites he may not be able to connect to his company server, but still wants to place client orders. This he does using his laptop based application and database. Order data is queued up in MSMQ on the laptop.
As soon as he is able to connect to his company server MSMQ automatically sends the order data. The server has queued up MSMQ messages of changes to pricing and stock etc. that took place whilst he was disconnected. These are now received and the local database is updated.
The choice of TCP/IP, HTTP or MSMQ all happens seemlessly to the main application, the WCF code copes with the choice.
From what I know, you have two options:
Use Gears (abandoned) or Web Storage to store and sync local data, combined with heavily javascripted web pages that can detect loss of connection and work against the local data store.
Use the Sync Framework with a rich client (WinForms, WPF or possibly Silverlight OOB if it gets supported). The Sync Framework does not require a local installation of a database, instead it uses SQL Server Compact, which is simply a local file.
At this stage using Sync franework with probably rich client seems to be better option. Thanks a lot Guys for taking your time out and trying to answer my queries. I will let you know the technologies used after i manage to deploy the app!

Categories