We Keep Coding

How to disable requeue jobs on Hangfire dashboard

Currently the Hangfire dashboard offers an option to requeue jobs (either succeed or failed) and in my case running twice a job can cause problems.
I have tried to add AutomaticRetry attribute...
[AutomaticRetry(Attempts = 0)]
Which solves the problem when jobs fails, jobs are not requeued automatically, but the button is still on the dashboard and they can be manually requeued.

Currently there is no way to make Hangfire stop running the jobs. For example when you have issue with one of services that are used in jobs, and you want to stop hangfire until issue resolution.
The idea is to have your job raise an exception. It will then go into the failed state and depending on your AutomaticRetry setting attempt to rerun the job if needed automatically (up to the defined number of retry attempts) or stay there so that once the problem is solved you can manually requeue the job from the dashboard.
Having the job sit there waiting for a service to come back online does not sound advisable (speaking in general, I obviously don’t know your specific scenario that well).
On the whole I find I am even extremely careful of even doing automatic retries. I only even consider doing those if I have a guarantee that whatever the job does is idempotent (i.e. running the same actions multiple times does not cause issues).
Imagine a job that adds 100 $ to the salary of every employee in a company (i.e. set salary = salary+100). You run the job updating the DB but halfway through the DB server connection drops. Half the employees have had the salary increase, the other half did not get it yet. Running the same job again should not apply the 100$ increase a second time to those employees done in the first run.
Stopping the whole server also seems a bit drastic. I believe the advised mechanism is to just delete the job if you don’t want the job (if it is a recurring one and not a fire and forget) to enqueue new runs for a while. Then when the issue is solved you just reschedule it. I do agree that a pause feature would be a nice to have. You could extend hangfire yourself to do this using the jobfilters and IElectStateFilters. Just have a boolean (i.e. IsHangfirePaused=true) somewhere that you can check in the OnStateElection event and prevent the job from transitioning to the EnqueuedState when it is set to true.
this is according to https://discuss.hangfire.io/t/ability-to-stop-running-jobs/4215/2

How about deleting the job?
RecurringJob.RemoveIfExists("myJobID");

You can set date for that task from hangfire database.
UPDATE [HangFire].[Hash]
SET [Value] = '2050-01-01T00:00:00.0000000Z'
where [Key]='Job_Name' and [Field]='NextExecution'

As odinserj suggests on Hangfire Discussion.
You can simply use a condition inside a recurring job and store it in your database to prevent job running:
public void MyMethod()
{
if (someCondition) { return; }
/* ... */
}

Build a job scheduler in C#

I would like to build a job scheduler.
So this job scheduler allows the user to configure:
Job start time. (datetime value)
Job frequency: minutes, hours, days, months, years (any integer value)
I have 2 options on how to build this scheduler:
Use the C# Timer class. But this also means that I have to create a new Timer object for every scheduled job. I am planning to expose an API endpoint for the user to call and POST the starttime and frequency info. So when the endpoint is called, I will need to create a new Timer object.
Will this even scale? How do I manage the Timer objects? I need to allow user to create, update and delete their jobs.
Use the Azure Scheduler. However, I have a very large user database. I had a look at the pricing, the maximum total jobs that can run on 1 instance is 5 million jobs only. Plus, it is difficult for me to manage the running instances if I have more than 1 instance running. How do I decide to load balance multiple instances of schedulers?

You could use Azure Worker roles, one worker role per job type, and different schedules for each job, users can create their own separate schedule for each job, picking from a list of predefined "job types".

For this you can use RabbitMQ. Basically, every scheduler is just producer-consumer concept over standard messenger. I recommend you to use some messenger for this task, because they can guarantee that your task executed if it was scheduled, even if your system was shutted down. It is self-balanced, stable and pretty much everything you need already implemented here.
More here: https://www.rabbitmq.com/tutorials/tutorial-one-dotnet.html
For your task you simply specify producer side which if running will enqueue tasks into RabbitMQ with some schedule without care about how they execute in messenger. This way you can add, delete, edit, read schedules in your producer. Also, a little advice, instead of frequency terms use cron-expressions. It is widely support, easy to understand concept for specifying human-readable schedules.
More here: https://crontab.guru/

There are existing libraries that you can use, e.g. Quartz or Hangfire. The first one is rather simple to use library that I have used successfully, the latter has a UI in addition of running tasks, etc. that you can serve.
I am sure there are plenty of other libraries if those are not good enough.

Derek here from Azure Scheduler.We have enterprise customers using Scheduler similar to the scenarios you described, depending on what your user/job profile looks like, it should be easy to come up with a good way to partition them into different job collections.
How many jobs do you expect tot have? Sounds like you need much more than the 5 million we support in a single job collection with P20 plan. I'd like to better understand your scenario and help you decided whether Azure Scheduler is the right solution.
You can reach me at Derek.Li (at) microsoft dot com.

Hangfire with multiple servers

I am working on an application where I have multiple servers on different machines doing long operations for me. There is a windows service running on those machines written with hangfire/topshelf. Only one operation can run at a time per machine. Additionally I want to do some status check and cleaning jobs periodically on each server, so I can't just queue them as jobs.
Is there a way to do that in hangfire? Also, is there a way to send a follow-up job to the same server as an earlier job?
ADD-ON: I know one possibility would be to add another hangfire layer: Make each of the services a hangfire client with own DB and serve themselves, and then schedule recurring jobs for them, but that seems awfully complicated - especially when scaling out and adding servers.

If your task is to run some scheduled task on each server, I think, the best option is to implement it yourself, Hangfire don't support events handling, only command handling. I think, you reached the point of Hangfire possibilities and need to switch to more powerful and general tool.
For events and their handling you can use other systems, for example RabbitMQ. You just specify event generator and subscribe all your machines for this event.

I know this is a bit late, but the way we handle this sort of thing is just to write a simple console application and schedule it with Windows Task Scheduler.

You've probably resolved this by now, but
1 - one job per server - as you have it - worker count - probably the best as you can have multiple queues per server and the filters won't help you there.
2 - should the cleanup run after each processing job?
if yes, you can create the cleanup job from within your process job execution (ok maybe not perfect design but it works just fine) and assign to a queue on the same server, just add some logic in filters to ensure processing job is followed by a cleanup job and you're sorted.
alternatively you can use Continuation jobs (as on the site https://www.hangfire.io/) - Haven't used these but sounds like it might do the trick.
if you just want to periodically run the cleanup code then just schedule the job as recurring on each of the servers

Perform Tasks at Different Intervals in Azure Worker Role

I have a simple Azure Worker role running that performs a task every few seconds. Below is the code that accomplishes this.
public override void Run()
{
try
{
while (true)
{
DoSomething();
System.Threading.Thread.Sleep(3000);
}
}
catch (Exception ex)
{
Log.Add(ex, true);
}
}
What I'd like to do now is add a second task DoSomethingElse() that fires once and only once per day. I've thought of a couple of ways to accomplish this:
Add a counter that calls the new task every nth loop
Add conditional logic to the new task that compares the current time to a prescribed time of day
Use some TBD scheduler library (such as Quartz.NET)
The first two solutions strike me as very brittle without additional code to deal with situations where the service is stopped and restarted. The third solution strikes me as potentially overkill.
My question is, what is the best practice for scheduling tasks at different intervals within an Azure Worker Role? I have a slight preference for sticking with straight .NET and not using a third-party library (though I'm not ruling it out).
Note, #3 above comes from this older question Recommend a C# Task Scheduling Library

The first two options are the simplest but they are brittle - especially in the cloud where roles can be recycled/load balanced etc... If the persistence is in memory or even disk based in the cloud, then it will be brittle.
Outside of other third party options, you could look at persisting the schedule and execution data into external storage (table services, sql azure, etc...). On a periodic timer, the worker role can query for the jobs that are due to be performed, record starting and then run the job. That also allows you to potentially scale out the number of worker roles since it's persistence is external.
This can get complicated in a hurry but if you keep it simple with frequency and recording run times, it can be fairly straight forward.

Steve Marx wrote a nice couple of blog entries on how to build a task scheduler on Windows Azure using blob leases, I think you will find this very useful.

How do you set scheduled task properly in a webapplication

I have the following problem.
A customer of ours has an application that is used by multiple users. Now they want to notify the users that are inactive for more than 30 days.
I'm using Spring.Quartz to solve this problem. But now this stuff is running within a windows service (which communicates with the website's database).
I was wondering if it isn't possible to use the Quartz library within the web application.
I know this works as long as the application is active, but what if the application recycles? Or is inactive for some time (ex 2 days).
Edit: Regular inactivity is possible. But the notifications should still work.
Are there other methods to do this?
Any help is welcome.
Cheers

The Windows Service approach is best in this case. You can also create a Windows Schedule, which will call your page (e.g. http://[your-site]/[yourapp]/notifyusers.aspx), which will do what is necessary. Or, if you expect the application to be visited pretty often (so you're sure that it is not just recycled), place to Application_Start of global.asax the QueueWorkingItem to start the thread, which will have something like below:
private void MyPingThread(object state)
{
ThreadExitState lstate = state as ThreadExitState;
EventWaitHandle handle = new EventWaitHandle(false, EventResetMode.ManualReset);
while (true)
{
if (lstate != null)
{
if (!lstate.Active)
return;
}
handle.WaitOne(1000 * 60 * 60 * 4); // Run the task each 4 hours
//Do your work here
}
}

Just kick of a thread when the application starts (you can use the Global.asax). Wrap it in a while(true) and a try/catch, make sure you have some nice long sleeps in place even if errors occur (i.e. outside the try/catch block), and you're good to go. Probably not appropriate if you need down-to-the-second notification, but it should work for what you're doing.

I usually create a new 'Class Library' project and add it to my solution.
Then I create a new task in Task Scheduler.
I never mix scheduled jobs with website. If scheduled job is unresponsive, it may slow down or even crash your website.