I am very new to using Quartz and I have a question regarding triggers. Is it possible to trigger based on file existence? I would like to have Quartz run a job until a certain file is found, then stop running that job and perhaps move on to a different one.
For example, I would like to do something like this:
(1) Job1 checks if File.txt exists in a given directory every 60 seconds.
(2) If File.txt is found, trigger Job2 to start. Job1 stops checking for file existence.
Right now, I have:
// Job definitions
var Job1 = JobBuilder.Create<TestEmail>().WithIdentity("job1", "group1").Build();
var Job2 = JobBuilder.Create<TestFileTrigger>().WithIdentity("job2", "group2").Build();
// Triggers
ITrigger trigger1 = TriggerBuilder.Create()
.WithIdentity("trigger1", "group1").StartNow()
.WithSimpleSchedule(x => x.WithIntervalInSeconds(5).RepeatForever())
.Build();
ITrigger trigger2 = TriggerBuilder.Create()
.WithIdentity("trigger2", "group2").StartNow()
.Build();
// Schedule jobs
scheduler.ScheduleJob(Job1, trigger1);
if (TestFileTrigger.fileExistence == true)
{
scheduler.ScheduleJob(Job2, trigger2);
}
but it seems like Job2 never starts.
TestEmail and TestFileTrigger simply print to console at the moment. The boolean TestFileTrigger.fileExistence comes from checking if a file exists at a given location (which it does).
Edit:
TestFileTrigger.fileExistence is a boolean. Added definitions of Job1/Job2 if that helps.
Edit:
I found that if I put Thread.Sleep(TimeSpan.FromSeconds(x)); before the if statement, the if statement will run if the condition is met. (Where x is some number of seconds.) Why does it work in this case, but not otherwise? I cannot always know how many seconds it will take for the condition to be met.
What type of application is this?
If this is, for example, a Windows service - to keep the scheduler alive so that it hangs around to execute the jobs according to your triggers, you need to do something like:
ThreadStart start = SetupSchedules()
var thread = new Thread(start) { Name = "mysvc" }
thread.Start();
.. this would go into the override void OnStart(string[] args) method of the Windows service.
The SetupSchedules method would be the thing that hooks into Quartz jobs and would be something like (The code you've written above in the OP would make a good start):
ISchedulerFactory factory = new StdSchedulerFactory();
JobScheduler = factory.GetScheduler();
JobScheduler.ScheduleJob(job1, trigger1);
This should keep it alive so that it executes the jobs. I've omitted a bunch of stuff here, but hopefully this should give you a few pointers to help weave it into your app.
You will also need something like this:
private void ManageThread()
{
var _thread = Thread.CurrentThread;
while (!_threadMustStop) // false by default, set this to true in a 'shutdown' process
{
Thread.Sleep(10000);
}
}
...which you call from your SetupSchedules method
It looks like you don't understand concurrency and threading that is involved here.
The issue is as follows.
Your MAIN thread does the following.
Create two jobs
Give each job a trigger
Schedule Job1 to start
Check TestFileTrigger.fileExistence and if true, start Job2 (it is false so it doesn't run).
THEN a threadpool thread will start Job1. Most likely setting TestFileTrigger.fileExistence = true. But Main thread has already completed it work.
FIN.
At no point do you go back and check if TestFileTrigger.fileExistence is true. So its result is irrelevant. You are in fact checking the result BEFORE you get a result.
By adding a Thread.Sleep you give the job1 enough time to complete and give you a result (job1 runs asynchronously and concurrently, and it is clear you expected it to run synchronously). Imagine for example you tell your friend Fred to go to the shop to buy Pizza and place it on your desk (asynchronous), then turn around straight away and wonder why there is no pizza on your desk.
Synchronous would be if you yourself went to the shop, bought a pizza and took it home and placed it on your desk, THEN eating pizza from your desk.
JobScheduler.ScheduleJob(job1, trigger1); does work asynchronously.
You should create a job to wrap up step 4 and schedule that to run periodically, OR you use the built in FileScanJob instead.
Related
I have a method which can be called many times
public void DoSomething(){
IJobDetail job = JobBuilder
.Create<NotificationResendJob>()
.UsingJobData(newJobDataMap)
.Build();
ITrigger trigger = TriggerBuilder.Create()
.WithIdentity("HelloWorldJob-trigger")
.StartNow()
.WithSimpleSchedule(x => x.WithIntervalInSeconds(5).RepeatForever())
.Build();
var scheduler = await _schedulerFactory.GetScheduler();
await scheduler.ScheduleJob(job, trigger);
}
It works fine, but if it is invoked more than once, then an exception is thrown because there is an instance already running.
My question is, it is possible to create a method that will build a new job and trigger for every method invocation, supposing that this method will not be called often
For example user clicks a button and a job is triggered, another user after some time clicks this button too, and another job is triggered for him
Unable to store Trigger: 'DEFAULT.HelloWorldJob-trigger', because one already exists with this identification.'
You probably want to make a distinction between job and a trigger. Usually job is a logic that can be run multiple times and even multiple parameters.
Say job is GreetPerson, so you want to run this every morning, so once a day, but also for different people.
You can define an IJob implementation that does the Console.Writeline using data from job data map. Job data map consists of values from both Job (more global) and trigger (per invocation, there can be multiple different triggers).
In this case you could call
string personToGreet = context.User.DisplayName;
JobDataMap dataMap = BuildJobDataMapForThisInvocation(personToGreet);
IScheduler.TriggerJob(theJobKey, myJobParameters);
Then the job would retrieve invocation (trigger) specific data from map and would be reusable among different use cases.
I notice the timer is not correct.
This is a very simple C# code: it will print current date/time every 1 minute.
My expected result is: let it run at 3:30 PM then we will have: 3:31 PM, 3:32 PM, 3:33 PM, ...
But sometime don't receive above result: sometime it is 3:31 PM, 3:32 PM, 3:34 PM, ...
So it lost 1 row.
Could anyone point me what is problem?
class Program
{
static Timer m_Timer;
static int countDown;
static void Main(string[] args)
{
countDown = 60;
m_Timer = new Timer(TimerCallback, null, 0, 1000);
while (true) { System.Threading.Thread.Sleep(10); };
}
static void TimerCallback(Object o)
{
countDown -= 1;
if (countDown <= 0)
{
Console.WriteLine(" ===>>>>>" + System.DateTime.Now.ToString());
countDown = 60;
}
System.Threading.Thread.Sleep(10000); //long running code demo
}
}
System.Threading.Timer runs on threads from thread pool. You run callback function which runs on one thread in pool every 1s and block it for 10s using sleep. Depending on how many threads you have in thread pool at some timepoints they all may be blocked and wait or .NET should allocate new thread up to the maximum of threads in pool for you.
From comments extended answer.
Each function is independent and it does not wait until another processing finish. A simple task is: call a function to do something every 1 minutes. "do something" in my case is saving local variables into SQL server. This process is fast not slow. I use 1 timer for many functions because each function is schedule in different cycle. For example, function 1 is triggered every 1 minute, function 2 is triggered every 10 seconds ... That why I use the timer 1 second.
Your use case seems to be more complex as I read it from initial question. You have different tasks and try to implement sort of scheduler. Maybe each particular tasks is fast but all together some runs may be longer and blocking. Not sure how this logic was well implemented but there could be a lot of edge cases e.g. some run was missed etc.
How I would approach it?
I would not try to implement on my own if scheduler can be more complex. I would pick ready solution, e.g. Quartz.NET. They consider edge cases and help to scale on cluster with needed and help with config.
In any case I would refactor bigger schedule to have each task to run on its schedule based on configuration (custom implementation or Quartz) as smaller tasks
I would scale your "queue" of tasks first locally by introducing some queue, for example using ConcurrentQueue or BlockingCollection or any produce-consumer to limit number of threads and if performance of such execution is not good scale on cluster. By doing so you can at least guarantee that N tasks can be scheduled and executed locally and everything beyond is queued. Maybe having some priorities for tasks can also help because there might be execution which could be missed but there are execution which must run on schedule.
I doubt it is a good idea to start from thread timer execution other threads or tasks if most likely you already have problems with threading.
You problem is not with System.Threading.Timer, it does its job well. Your use case is more complex.
Windows - is not real time operating system. So, if you expect that timer waits ecactly 1 second - it's wrong. There are many reasonsm when timer can wait more time. Because of timer resolution or other high load operations.
If you like newer .NET TPL syntax yo can write it like this:
using System;
using System.Threading.Tasks;
namespace ConsoleApp1
{
internal class Program
{
private static void Main(string[] args)
{
Repeat(TimeSpan.FromSeconds(10));
Console.ReadKey();
}
private static void Repeat(TimeSpan period)
{
Task.Delay(period)
.ContinueWith(
t =>
{
//Do your staff here
Console.WriteLine($"Time:{DateTime.Now}");
Repeat(period);
});
}
}
}
The above code causes, that every second you run 10-second "demo" (sleep). You will run 10 worker threads simultanously.
Are you sure, this is what you are trying to achieve?
To see what really happens in your app, simply add:
Console.WriteLine($"Time:{DateTime.Now.ToString("hh:mm:ss.fff tt")},Thread:{Thread.CurrentThread.ManagedThreadId},countDown:{countDown}");
in the beginning of TimerCallback. You will notice, that timespan between following callbacks are not exactly 1000ms (usually it is a little bit more). This is perfectly normal in non-rtc OS, and, in most cases - it's not a problem. Just keep in mind, that Timer is not exact.
Moreover, if you are trying to use Timer that way, and trying to count ticks - these little errors cumulates in following ticks.
I just post what found here for people that have problem like me.
I found the answer from another thread.
I use "HighResolutionTimer.cs" and it works perfect:
https://gist.github.com/DraTeots/436019368d32007284f8a12f1ba0f545
I'm creating a scheduler to fire events at specific times of the day, and to do this I'm spinning up Tasks (one at a time, i.e. the 'next' schedule only) with a Task.Delay of anything up to a few days delay. For example, after the last event fires on a Friday afternoon, I'll set up the next one which will be some time on Monday, so it could potentially be a TimeSpan of up to 3 days (~260,000,000 milliseconds).
Is this acceptable practice? I'm concerned that this won't be stable/robust enough for a production environment.
Here's some snippets of code to describe what I've put together:
private void SetNextEvent()
{
TimeModel next = GetNextScheduledTime();
Debug.WriteLine($"Next schedule [{next.TimeType}]: {next.Time.ToString("yyyy-MM-dd HH:mm:ss")}");
TimeSpan delay = next.Time.Subtract(DateTime.Now);
Task.Run(async () =>
{
await Task.Delay(delay);
FireEvent(next);
});
}
private void FireEvent(TimeModel time)
{
Debug.WriteLine($"Event fired [{time.TimeType}]: {DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss")}");
OnSchedulerEvent?.Invoke(this, new SchedulerEventArgs { ScheduleType = time.TimeType });
if (_running)
SetNextEvent();
}
This is totally reliable. .NET timers are very efficient. The biggest problem is that you must assume that your production apps can exit at any time. The reason that is easiest to understand is a bug that kills the process. Other reasons include reboots, app pool recycles, deployments, ....
So if you can recover your state after being killed this is fine. If you add a comment about specific concerns I'll address them.
It looks like you have a way to recover timers because you apparently can compute the next due time. Under those circumstances it's very safe to do this. You need to ensure that your code is always running e.g. right after a reboot or a crash.
Note, that IIS apps need to tolerate running multiple times concurrently. Otherwise, IIS is a great host for your scenario.
If you're running Windows, I'd use the TaskScheduler to do what you're trying to do.
run taskschd.msc you can use that program to schedule periodic tasks.
There should be an "Create Task..." button in the panel on the right.
I agree that the Windows Task Scheduler is probably the best approach is you know ahead of time the schedule to run the next task.
If you do not know this ahead of time(i.e. the time to wait for the next task can vary) then I would suggest using a timer and an event handler method when the timer expires.
Each time you can set the Interval property of the timer for the time to wait. When the time expires the timer event handler will run and it can execute the ask and reset the Interval for the timer. This seems a little cleaner then a Task.Delay.
Is there a way to tell Quartz.NET not to fire two triggers at the same time? That means if trigger A and trigger B have the exact same schedule, then trigger B waits a certain amount of time and then fire?
I have seen in my program that this can cause an issue when both my jobs are reading from the same file and executing the same .exe file. This causes an uncaught exception that I am yet to figure out.
I am not sure how Quartz.NET handles this. But is there a way to delay such triggers (even if it's just for a few seconds)?
You can use DisallowConcurrentExecutionAttribute for your job.
[DisallowConcurrentExecutionAttribute]
class DisallowConcurrentJob : IJob
{
//Implementation goes here
}
It prevents multiple instances of a job WITH THE SAME KEY from running
at the same time.
A very good explanation can be found here.
UPDATE:
If you want to be sure that the trigger/job always runs you can use misfire instructions:
IJobDetail job1 = JobBuilder.Create<InheritedJob1>()
.WithIdentity("DisallowConcurrentJob", "MYGROUP")
.RequestRecovery(true)
.Build();
//Schedule this job to execute every second, a maximum of 5 times
ITrigger trigger1 = TriggerBuilder.Create()
.WithSchedule(SimpleScheduleBuilder.RepeatSecondlyForTotalCount(5)
.WithMisfireHandlingInstructionFireNow())
.StartNow()
.WithIdentity("DisallowConcurrentJobTrigger", "MYGROUP")
.Build();
Scheduler.ScheduleJob(job1, trigger1);
WithMisfireHandlingInstructionFireNow
The job is executed immediately after the scheduler discovers misfire situation.
When I do this:
testScheduler.Schedule("Hello world",(scheduler, state) => Console.WriteLine(state));
testScheduler.AdvanceTo(testScheduler.Now);
I hit this code in VirtualTimeSchedulerBase:
public void AdvanceTo(TAbsolute time)
{
int num = this.Comparer.Compare(time, this.Clock);
if (num < 0)
throw new ArgumentOutOfRangeException("time");
if (num == 0)
return;
num == 0 is true, and I exit the method.
I can call testScheduler.Start() and my action will execute. But then TestScheduler will carry on executing everything in its queue. Whereas I want it to stop executing actions at the current time.
I can't see any other methods on TestScheduler that will get me the behaviour I want.
Is this a bug, or is it the correct behaviour but I'm missing something?
Edit:
I misunderstood. TestScheduler doesn't execute actions until after the date at which they are scheduled.
Scheduling an action immediately schedules it for the current value of testScheduler.Now. So it won't be executed until Now + 1.
var testScheduler = new TestScheduler();
var due = new DateTime();
testScheduler.Schedule("Hello world", due, (scheduler, s) =>
{
Console.WriteLine(s);
return Disposable.Empty;
});
testScheduler.AdvanceTo(due.Ticks);
// Nothing has happened
testScheduler.AdvanceTo(due.Ticks+1);
// -> "Hello world"
This still isn't the behaviour I would like, but there you go.
You might want to consider how you are using the TestScheduler:
It will in general execute at the due time. For example, this code will write to the Console:
var scheduler = new TestScheduler();
scheduler.Schedule(
TimeSpan.FromTicks(100),
() => Console.WriteLine("Hello"));
scheduler.AdvanceTo(100);
However, TestScheduler will only inspect it's queue when time is moved. So if you schedule an action, you need to move time via AdvanceBy, AdvanceTo or Start to get it to process the queue. When it does, it will process everything up to the current time. E.g. Even this will output to the console despite scheduling "in the past":
var scheduler = new TestScheduler();
scheduler.AdvanceTo(TimeSpan.FromTicks(100).Ticks);
scheduler.Schedule(
DateTime.MinValue + TimeSpan.FromTicks(50),
() => Console.WriteLine("Hello"));
Console.WriteLine("No output until the scheduler is given time...");
scheduler.AdvanceBy(1);
Idiomatic use of TestScheduler usually involves queuing up all your work and then running to completion via a Start() call and then checking for expected state. Use of AdvanceBy, AdvanceTo tends to be for more demanding test scenarios where you need to test some intermediate state - and even then you generally queue everything up first with appropriate due times and then AdvanceBy or AdvanceTo your time of interest, check state, and then progress again with AdvanceBy, AdvanceTo or Start.
What you don't want to do generally is queue work, run a bit, queue more work, run a bit - especially if you are scheduling without a due time. Then you will hit the problem you have here.
That's not to say in your specific scenario this may be unavoidable - if you need to make decisions about what to schedule after a particular time for example - but consider if you can set everything up in advance as it's probably going to result in cleaner test code that more closely follows the Arrange Act Assert pattern.
I try to Arrange by scheduling, then Act by moving time, then Assert the results.
use AdvanceBy(1) to advance the scheduler by 1 tick. The scheduler only executes events when the clock actually advances.
Different schedulers behave differently when you schedule something for immediate execution. Some of them really do execute it immediately. Some put it in a queue to be executed at the first available opportunity.
There's not a good way for the TestScheduler to behave in this situation unless the API is modified to let you tell it which way it should behave.
The Start method will execute everything scheduled. You can schedule a Stop method call to pause the execution at a given point.
var testScheduler = new TestScheduler();
var due = new DateTime();
testScheduler.Schedule("Hello world", due, (scheduler, s) =>
{
Console.WriteLine(s);
return Disposable.Empty;
});
testScheduler.Schedule(due.Ticks + 1, (scheduler, s) => testScheduler.Stop());
testScheduler.Schedule("Do more stuff", due.AddMinutes(1), (scheduler, s) => Console.WriteLine(s));
testScheduler.Start();
Assert.IsFalse(testScheduler.IsEnabled);
Assert.AreEqual(due.Ticks + 1, testScheduler.Clock);