Quartz.net - Issues with Adjusting and Speeding up SystemTime causing Misfires - c#

For testing reasons I want to be able to adjust what time Quartz.Net currently thinks it is so I do not necessarily have to wait hours, days, or weeks in order to check that my code is working.
For this purpose I created the following simple function (it is in F# but could be easily be done in C# or another language) :
let SimulateTime = fun () ->
currentTime <- DateTimeOffset.UtcNow
timeDifferenceInSeconds <- (currentTime - lastCheckedTime).TotalSeconds
simulatedTime <- simulatedTime.AddSeconds((timeDifferenceInSeconds *scaleTimeBy))
lastCheckedTime <- currentTime
simulatedTime
Where currentTime, lastCheckedTime, and simulatedTime would all be of type DateTimeOffset and both timeDifferenceInSeconds and scaleTimeBy are of type float.
I then change SystemTime.Now and SystemTime.UtcNow to use the above function as follows :
SystemTime.Now <-
Func<DateTimeOffset>(
fun () -> SimulateTime())
SystemTime.UtcNow <-
Func<DateTimeOffset>(
fun () -> SimulateTime())
Which was shown by Mark Seemann in a previous question of mine that can find here.
Now this mostly works except it seems like the longer function causes it to be off by a decently wide margin. What I mean by this is that all of my triggers will misfire. For example if I have a trigger set to occur every hour and set scaleTimeBy to 60.0 so that every second passed counts as a minute, it will never actually trigger on time. If I have a misfire policy, the trigger can then go off but the time it lists for when it activated will be as late as the half hour mark (so takes a full 30 seconds slower than what it should have been in this example).
However I can do this :
Console.WriteLine(SimulateTime())
Thread.Sleep(TimeSpan.FromSeconds(60.0))
Console.WriteLine(SimulateTime())
And the difference between the two times output to the screen in this example will be exactly an hour so the call doesn't seem like it should be adding as much of a time difference than it does.
Anyone have any advice on how to fix this issue or a better way of handling this problem?
Edit :
So the C# version of the SimulateTime function would be something like this :
public DateTimeOffset SimulateTime() {
currentTime = DateTimeOffset.UtcNow;
double timeDifference = (currentTime - lastCheckedTime).TotalSeconds;
simulatedTime = simulatedTime.AddSeconds(timeDifference * scaleTimeBy);
lastCheckedTime = currentTime
return simulatedTime;}
If that helps anyone with solving this problem.

So this issue is misfires caused by the fact that Quartz.net will idle and wait when it thinks it doesn't have any triggers occurring any time soon to avoid making too many calls. By default it waits about 30 seconds give or take if it doesn't have any triggers occurring in the time span. The idleWaitTime variable is a Timespan set in the QuartzSchedulerThread. Now when checking for triggers that might occur soon it also uses the BatchTimeWIndow from QuartzSchedulerResources.
Both idleWaitTime and BatchTimeWindow can be set in configuration/properties files where they'd be called "org.quartz.scheduler.idleWaitTime" and "org.quartz.scheduler.batchTriggerAcquisitionFireAheadTimeWindow."
Based off what it is called in BatchTimeWindow I thought it was just a bit of look ahead for grabbing a variable (which would like since if I'm speeding things up, I'd want a small idleWaitTime but I would want it to look further ahead for triggers because the few seconds your waiting is actually minutes so will trigger sooner than it thinks), but the description of "org.quartz.scheduler.batchTriggerAcquisitionFireAheadTimeWindow" on pages going over configuration properties implies that it can cause things to fire early and be less accurate. So to start here is the code for just modifying idleWaitTime
let threadpool = Quartz.Simpl.SimpleThreadPool()
let jobstore = Quartz.Simpl.RAMJobStore()
let idleWaitTime = TimeSpan.FromSeconds(30.0/scaleTimeBy)
let dbfailureretryinverval = TimeSpan(int64 15000)
Quartz.Impl.DirectSchedulerFactory.Instance.CreateScheduler("TestScheduler","TestInstance",threadpool,jobstore,idleWaitTime,dbfailureretryinverval)
let scheduler = Quartz.Impl.DirectSchedulerFactory.Instance.GetScheduler("TestScheduler")
You can create a Scheduler that has the idleWaitTime you want by using the DirectSchedulerFactory which probably could use a little bit better documentation. It takes also a bunch of stuff you may or may not want to modify depending on what you are working on. For threadpool I just use Quartz.net's default SimpleThreadPool because I do not care about messing with the threading at this time and would not want to explain how you go about doing so unless that was the whole point of the question. Information on jobstores is available here. I am using RAMJobStore here because it is simpler than AdoJobStore but it shouldn't matter for this example. The dbfailureretryinterval is another value that don't care about for this example so I just looked up what it is set to by default. Its value should matter the least for this example because not connecting to a database. For idleWaitTime might want to do more tests to figure out what is a good value for it, but I chose to go with just scaling its default value of 30 seconds by scaleTimeBy since that is what I'm using to scale how fast things are going by. So this should make it so if I am having the program simulate time going by at a much faster rate, then it should only remain idle for smaller periods of time. One important thing to note is that when create a scheduler in this way, it is not returned as well so need to make a separate call to get the scheduler I just created. I have no idea why this is this way, I'm guessing that if you are creating several Schedulers and not necessarily using all of them it is better this way.
Now after all that you are likely to still get a bit of a misfire rate. While it is now idling for much smaller units of time (only a few seconds so potentially an acceptable margin depending on what your use case is), it still has the issue of it is only then checking to see if it has a coming trigger in the next few fractions of a second.
So lets see if adding time to BatchTimeWindow helps matters?
let threadpool = Quartz.Simpl.SimpleThreadPool()
let threadexecutor = Quartz.Impl.DefaultThreadExecutor()
let jobstore = Quartz.Simpl.RAMJobStore()
let schedulepluginmap = System.Collections.Generic.Dictionary<String,Quartz.Spi.ISchedulerPlugin>()
let idleWaitTime = TimeSpan.FromSeconds(30.0/timeScale)
let maxBatchSize = 1
let batchTimeWindow = TimeSpan.FromSeconds(timeScale)
let scheduleexporter = Quartz.Simpl.RemotingSchedulerExporter()
Quartz.Impl.DirectSchedulerFactory.Instance.CreateScheduler("TestScheduler","TestInstance",threadpool,threadexecutor,jobstore,schedulepluginmap,idleWaitTime,maxBatchSize,batchTimeWindow,scheduleexporter)
let scheduler = Quartz.Impl.DirectSchedulerFactory.Instance.GetScheduler("TestScheduler")
Now this has even more variables that don't really care about for the purposes of this example and won't even bother going over because adjusting batchTimeWindow actually makes it worse. Like getting you back to misfiring by 30 minutes. So no, batchTimeWindow while looks like might be useful is not. Only modify idleWaitTime.
Ideally for this use would want a small wait time and a larger look ahead time, but the option for that does not seem like its available.

Related

Automatic update custom DateTime object based on system time

I think my question is a bit confusing as I'm not really sure how should I give it a proper title.
Anyway, my question is because I'm getting current date on my server instead of system time (or device's time). How can I make it to act like a "clock"?
For example:
// I have a string of date retrieved from server, let's call it as "stringFromServer"
myDate = Convert.ToDateTime(stringFromServer);
So after the above line of codes, I need myDate to update every seconds so that I can use it anytime I want.
Let say the current time (get from myDate) is 09:00 and after 5 seconds I expect it to be updated to 09:05.
One of the solution I have in mind is in my Update() function,
I'll do something like this:
myDate.Value.AddSeconds(deltaTime);
But I'm looking for better solution and hopefully I can get some help here.
PS: The reason I'm not using system time is to prevent user changing the time in device's settings. So I have to update myDate whenever the user exit or reopen the app.
In Unity3D, if you want an event to happen every X seconds, a good way to do that is to use an IEnumerator function. So for example, if you want to keep track of a custom time variable you could have an IEnumerator like this:
IEnumerator UpdateServerTime()
{
while(true)
{
myDate.Value.AddSeconds(1);
yield return new WaitForSeconds(1);
}
}
This example would continue to update your time varable by one second, every second. As other users have mentioned in the comments, this is not the perfect way to keep track of time. But it should achieve what you want, if you are looking to track time in unity.
It is important to remember that in order to use this method properly, you need to call it like so:
StartCoroutine(UpdateServerTime());

How can i minimize performance hit for Coded ui when using nested if statement in test method

Is there a way for me to minimize the performance hit when i'm either running or debugging my coded U.I test. Currently its taking me a long time to run my coded UI test because it takes to long to execute. I"ve timed it and too long means that for checking if a screen exist and doing an action it takes over 1min plus, so its taking me to long to debug and finish it out.
To give some more background. These if statements are all inside one test method, where i'm checking for different screens. Its very dynamic but takes to long to run. I've read i can do ordered test but i didn't think i can create ordered test with these dynamic screens(reason being i dont think ordered test can act as if statements to account for dynamic dialog and screens) and plus i think its too late in the process to go to that architecture.
I've tried the following playback settings with little or no improvements.
Here are my current playback settings
Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.Disabled;
//Playback.PlaybackSettings.SmartMatchOptions = SmartMatchOptions.None;
Playback.PlaybackSettings.MaximumRetryCount = 10;
Playback.PlaybackSettings.ShouldSearchFailFast = false;
Playback.PlaybackSettings.DelayBetweenActions = 1000;
Playback.PlaybackSettings.SearchTimeout = 2000;
None of these setting have helped either turning off smart options.
I could have sworn that i've read somewhere that if i replace my if statements
with try catch that this would help, but i maybe totally wrong since i'm just grabbing at straws to try to atleast increase performance by 40% or so.
Would anyone have any tips or tricks when dealing with ifs statements that you had to code in your coded ui code.
I'm guessing your if statements are of a kind:
if (uTtestControl.exists)
{
do something
}
if that's the case - your delays are a result of codedui searching for the control - a time costly operation - especially when searching for a control that doesn't exists.
there are a number of ways to handle this - if my guess is in the ball park - please confirm and i'll detail the options.
Updtae:
the main reason for delay is the MaximumRetryCount =10. in addition try the following settings:
Playback.PlaybackSettings.MaximumRetryCount = 3;
Playback.PlaybackSettings.DelayBetweenActions = 100;
Playback.PlaybackSettings.SearchTimeout = 15000;
when waiting for control to exists use the:
uiTtestControl.WaitForControlExist(5000)
this will tell the playback to search for the control for a max of 5 sec.
in addition - you should reduce the Playback.PlaybackSettings.SearchTimeout before searching for a control that you know might not exists:
var defaultTimeout = Playback.PlaybackSettings.SearchTimeout;
Playback.PlaybackSettings.SearchTimeout = 5000;
and after you finish searching return it to the default value:
Playback.PlaybackSettings.SearchTimeout = defaultTimeout;
this should do the trick

.NET: How to get time for keypress as precisely as possible?

I am trying to build an application that does not have to be super performant in general, except for the timestamp of a keyboard press (or external controller button press). I would like the timestamp to be as precise as possible, preferably down to +/- 5ms. This is for a scientific application. What is the best programming paradigm to achieve this with minimal latency? Which of the following is preferable?
(a) Create a worker thread that runs in a higher priority and loops to see if a key was pressed. Use Sleep(x) where x is less than 5.
(b) Create an keyboard hook, which has an asynchronous callback.
(c) Another option.
In addition to an answer, any code (or link to sample) would be much appreciated since I am a reasonably new dev.
EDIT: Nomenclature. Should have been more careful. By timestamp, I mean time in general, not necessarily the full Day Month Year Hour Minute Second Millisecond. I was planning on using the StopWatch class from the beginning, because all I need is the time between the start of an event in the program and the time of the button press. Sorry if this was not clear.
From comments:
I am looking for the difference between the start of an event and the key press.
It's important to understand that that's very different to trying to get the actual timestamp of a key press.
This situation is exactly what Stopwatch is for. Yes, use that and be grateful you're not really trying to get a timestamp.
Just use Stopwatch.StartNew when the event starts, and Stopwatch.Stop on the key press event. There's the slight latency of the event handler firing, but that will be tiny compared with anything else - I'd be astonished if it caused you any problems here.
If you are looking for exact timestamp of keypress, DateTime mightn't be what you want to use:
Get DateTime.Now with milliseconds precision
DateTime has a lot of precision, but is fairly coarse in terms of
accuracy. Generally speaking, you can't. Usually the system clock
(which is where DateTime.Now gets its data from) has a resolution of
around 10-15ms. See Eric Lippert's blog post about precision and
accuracy for more details.
If you need more accurate timing than this, you may want to look into
using an NTP client.
And the referenced link from that post:
http://blogs.msdn.com/b/ericlippert/archive/2010/04/08/precision-and-accuracy-of-datetime.aspx
Now, the question “how much time has elapsed from start to finish?” is
a completely different question than “what time is it right now?” If
the question you want to ask is about how long some operation took,
and you want a high-precision, high-accuracy answer, then use the
StopWatch class. It really does have nanosecond precision and accuracy
that is close to its precision.
Remember, you don’t need to know what time it is to know how much time
has elapsed. Those can be two different things entirely.
DateAndTime.Timer is the most exact. It returns a double precision float, which represents number of seconds passed from midnight.
Usage (in VB):
x# = DateAndTime.Timer
'... some code...
msgbox((DateAndTime.Timer - x).ToString + " seconds elapsed")
In case you end up using a loop, here's one that outperforms all the others that I've tested ->
<DllImport("user32.dll", EntryPoint:="GetInputState")>
Private Shared Function Chk4AnyKey() As Boolean
End Function
Sub HighPerfKybdRead(maxtime#)
maxtime += DateAndTime.Timer
Process.GetCurrentProcess.PriorityBoostEnabled = True
Process.GetCurrentProcess.PriorityClass = 256
while DateAndTime.Timer < maxtime
If Chk4AnyKey() Then
t# = DateAndTime.Timer
' Do further processing of the pressed key
' You might wanna call the PeekMessage function here
' with the option to ignore the repeated keystrokes
' and use the "t#" variable accordingly
End If
End While
Process.GetCurrentProcess.PriorityBoostEnabled = False
Process.GetCurrentProcess.PriorityClass = ProcessPriorityClass.Normal
End Sub
But if you're using an external controller instead of keyboard, then I guess you should be calling GetQueueStatus and not GetInputState...

Prevent context-switching in timed section of code (or measure then subtract time not actually spent in thread)

I have a multi-threaded application, and in a certain section of code I use a Stopwatch to measure the time of an operation:
MatchCollection matches = regex.Matches(text); //lazy evaluation
Int32 matchCount;
//inside this bracket program should not context switch
{
//start timer
MyStopwatch matchDuration = MyStopwatch.StartNew();
//actually evaluate regex
matchCount = matches.Count;
//adds the time regex took to a list
durations.AddDuration(matchDuration.Stop());
}
Now, the problem is if the program switches control to another thread somewhere else while the stopwatch is started, then the timed duration will be wrong. The other thread could have done any amount of work before the context switches back to this section.
Note that I am not asking about locking, these are all local variables so there is no need for that. I just want the timed section to execute continuously.
edit: another solution could be to subtract the context-switched time to get the actual time done doing work in the timed section. Don't know if that's possible.
You can't do that. Otherwise it would be very easy for any application to get complete control over the CPU timeslices assigned to it.
You can, however, give your process a high priority to reduce the probability of a context-switch.
Here is another thought:
Assuming that you don't measure the execution time of a regular expression just once but multiple times, you should not see the average execution time as an absolute value but as a relative value compared to the average execution times of other regular expressions.
With this thinking you can compare the average execution times of different regular expressions without knowing the times lost to context switches. The time lost to context switches would be about the same in every average, assuming the environment is relatively stable with regards to CPU utilization.
I don't think you can do that.
A "best effort", for me, would be to put your method in a separate thread, and use
Thread.CurrentThread.Priority = ThreadPriority.Highest;
to avoid as much as possible context switching.
If I may ask, why do you need such a precise measurement, and why can't you extract the function, and benchmark it in its own program if that's the point ?
Edit : Depending on the use case it may be useful to use
Process.GetCurrentProcess().ProcessorAffinity = new IntPtr(2); // Or whatever core you want to stick to
to avoid switch between cores.

Set Expiring Date To .NET Output (Without Hardware Dongles)

My first solution for this problem is to use OS/BIOS timer and check it with encrypted date file ( see below pseudocode )
public void CheckFrequently()
{
DateTime registeredDate = ReadFromBiosOrOSTimer();
DateTime readEncryptedDate = ReadFromEncryptedFile();
if(registeredDate >= readEncryptedDate)
{
ShowExpireDateForm();
CloseProgram();
}
}
In this case its obvious that user could change OS/BIOS timer easily and my method not works.
my questions are :
Is there any way to fix user OS/BIOS timer change problem?
Is there any better way to set expiration date to .Net projects?
(My answer is assuming you want to have an "expiring" program of some sort.)
The end-all-be-all big brother answer would be to retrieve a trusted time from an external source, say, a web service. Of course, connectivity (or lack thereof) may make this impossible.
Other than that, knowing that if someone is going to cheat the clock, they would likely do it very close to the expiration, periodically write, somewhere, a timestamp of the current time. If you ever encounter a case where the retrieved (via system call) time is less than the last timestamp, someone might be trying to trick the clock and you can invalidate the session/instance with the appropriate error message. Once you've detected the "expired" case, it's simple to flip a switch and refuse to run anymore.
All of that said, a countermeasure like this will most likely always be beaten by an adversary who is determined enough.

Categories