I have some C# code and i want that this C# should run exactly at 5 in the evening for that i need to create a job on a server. For creating job i need an ETL package.
How to make ETL of C# code?
The obvious answer would be to use a script task in the control flow. You can put your C# code there and have it execute. The tricky bit is if your code needs access to non-standard libraries in which case you'll need those deployed to the GAL on the server.
You could also use the execute process task if you have a compiled executable but in that case I would imagine you could just copy it to the server and use windows scheduler.
You can also directly run executables using the SQL agent: How execute exe file from sql agent or job?
Related
I just started my internship as dev student.
I need to write a program in C# that will read a CSV productfeed, then parse it, map it and insert OR update this into a SQL Server database.
The program will run on a Windows VPS and has to be executed daily as the CSV will also be updated daily. I have to automate this process.
SO to come to my questions:
1. Can I go with a console application? (because I don't need any GUI)
2. Will a program like this with nested loops for insert/update crash or be too heavy on the DB as the CSV can contain +100k records?
Any guidance in the right direction is much apreciated
YES. You can develop a Console Application indeed. Doing something with Windows Forms or similar (i.e a C# app with a form and a single button with code.) would be a bad idea as you don't need a GUI and would be much harder to automate.
What options do you have?
Console application: Easy to develop. You can schedule it using Windows Task Scheduler.
Windows Service: A little bit harder to develop. It is much easier to control if it is running or not.
WCF or Restful Web Service: Harder to develop but more open if it needs to evolve in the future. Easier to interconnect with other servers as execution can be triggered externally.
NO, potentially no, although with bad code you can manage to create problems in SQL Server.
My advice: If you don't have much time/skills, go for Console App. If you can, create an asynchronous RESTFUL web service implementing GET to see the status of last/current execution, POST to trigger a new execution and return correct HTTP Response codes.
1. Can I go with a console application? (because i don't need any GUI)
Yes you can you Console Application and Task schedule for run this program daily.
Also you can write Windows Service - in this case your program will run in background.
2. Will a program like this with nested loops for insert/update crash or be too heavy on the DB as the CSV can contain +100k records?
100K+ rows its nothing for MS SQL. But sure you need write code correct. I will parse its and then Insert in db in multi thread. But it depends on what you need to do.
Technically you could use any application type you want, but If I'm gonna write this application I will pick Windows Service Application , because you want the application to run repeatedly in an automated way.
and for the performance, I don't think that hitting the database for every record it the best practice here may be you should look for Bulk Insert.
I'm in the process of building a Scheduler in C sharp. The scheduler should ping the DataBase Queue and if anything new is written in the Database it should pick that up and send it to the remote server for processing. (The files will be uploaded to the database by the client using a WCF webservice) how do you guys think i should handle this situation? any type of help would be highly appreciated. I'm still in the design phase.
Thanks.
As this is a periodic task, I'd consider writing an app that does the job when run (with no scheduling). Then add it to the Windows Task Scheduler.
Take a look at Quartz.NET. It easily lets you set up a scheduler and plug in Jobs (i.e. Send the files to the remote server) and Triggers (How often do you want it to run..) It has an excellent Fluent API as well as support for CRON style expressions.
There are also tutorials on the site, and google will find you a bunch more.
#spender's idea is easy to implement. Another option is using a database trigger to call CLR proc
http://msdn.microsoft.com/en-us/library/ms131094%28v=SQL.90%29.aspx
Can I call a C# function by a SQL Server trigger?
I need to write a program in .NET which runs every few hours and does some task. It does not require user interaction but it needs a configuration file like App.config where I can put that configuration information.
So should I write console program in C# ( like php or perl script) and schedule it in task manager ( like crontab) or write a Windows Service or something else?
If you want scheduling capabilities and that schedule needs to be very configurable, a windows scheduled task does that pretty well. You would write that as a console app.
However, I would suggest a service for most things. Here is a great tutorial that talks about how to create windows services. Its much easier than you think.
http://www.switchonthecode.com/tutorials/creating-a-simple-windows-service-in-csharp
My suggestion would be to go for the console app and use the task manager.
It saves you from having to implement a custom timer
It gives you elaborate control over timing
It saves you from having to implement an UI
It gives you automation options, like piping in- and output at the command line
Depending on the complexity of your task the best option would be to create an application which can be run via a task scheduler. If you need to run something every few hours a windows service would also work, but remember if you create a windows service then you will have to use a timer control within your application which might hang up without giving any clues. There are lots of examples for both the concepts and instead of using windows service you can use Window Communication Foundation (WCF).
I am writing a software for a company in c# which is intended to run on windows platform.
One of my requirements is to allow the user to schedule back ups.
That is, the user will set a time where the database will be backed up automatically by the computer.
On the linux platform I would have use crons but I am a bit lost on the windows platform. I do not want the software itself to be actually opened for the back up to run. I want it to be carried out even if the software itself is not running.
My best bet is to use windows scheduler or create a custom service which will run at start up.
Can anyone point me to how to actually achieve this? Any constructive suggestions are welcomed.
Thanks.
For info the Windows "AT" command is somewhat similar to cron. You can get help from the command line thus:
AT /?
I wouldn't necessarily recommend it for a db backup. Either create a Windows scheduled task, or to backup a SQL Server database, use SQL server's built in scheduler.
Another alternative would be to create a windows service to handle the task. Then you could write any code needed (e.g. Backup / Email logs, etc) quickly and easily, and it would work w/o your application running.
There are ways to accomplish the same task with Task Scheduler built into windows, but just an alternative that I would prefer.
Here's an interesting problem. I have an ETL script written in c# that I have been running manually on a somewhat regular basis. It is used to update my web app's database.
I want to automate the ETL process AND create an interface for the web app admins to manually start the ETL process.
I could have sql server kick off the ETL process on a schedule and implement a button or something on a web page that will do the same thing, but I don't want to put my code in 2 different places because I don't want to update it in 2 places when it changes. But I don't know how to make my web app tell SQL server to manually start a scheduled process. Can this be done?
OR
I could somehow implement the scheduling in the web app itself, but by now most people are familiar with the problems that are faced when trying that (app may not be running at certain times, must wait on request to start a process (without some trickery)). Also, since the ETL process takes a while, I don't want to make some poor end user wait on a response, so it would definitely have to use a new thread.
What else could I do? What would you do?
You mentioned you are using SQL Server. What version is it? Is SQL Server Integration Services (which is the full-blown ETL tool for the platform) an option for you? If you want scheduled Extraction, Transformation and Load jobs, SSIS is a great tool on the Microsoft platform and is included in most SQL Server licenses.
Or, check out Quartz.