I'm trying to control access to an object so that it may only be accessed a certain number of times over a given timespan. In one unit test that I have, access is limit to once per second. So 5 accesses should take just over 4 seconds. However, the test is failing on our TFS server, taking only 2 seconds. A stripped down version of my code to do this is here:
public class RateLimitedSessionStrippedDown<T>
{
private readonly int _rateLimit;
private readonly TimeSpan _rateLimitSpan;
private readonly T _instance;
private readonly object _lock;
private DateTime _lastReset;
private DateTime _lastUse;
private int _retrievalsSinceLastReset;
public RateLimitedSessionStrippedDown(int limitAmount, TimeSpan limitSpan, T instance )
{
_rateLimit = limitAmount;
_rateLimitSpan = limitSpan;
_lastUse = DateTime.UtcNow;
_instance = instance;
_lock = new object();
}
private void IncreaseRetrievalCount()
{
_retrievalsSinceLastReset++;
}
public T GetRateLimitedSession()
{
lock (_lock)
{
_lastUse = DateTime.UtcNow;
Block();
IncreaseRetrievalCount();
return _instance;
}
}
private void Block()
{
while (_retrievalsSinceLastReset >= _rateLimit &&
_lastReset.Add(_rateLimitSpan) > DateTime.UtcNow)
{
Thread.Sleep(TimeSpan.FromMilliseconds(10));
}
if (DateTime.UtcNow > _lastReset.Add(_rateLimitSpan))
{
_lastReset = DateTime.UtcNow;
_retrievalsSinceLastReset = 0;
}
}
}
While running on my computer, in both Debug and Release, it works fine. However, I have a unit test that fails once I commit to our TFS build server. This is the test:
[Test]
public void TestRateLimitOnePerSecond_AssertTakesAtLeastNMinusOneSeconds()
{
var rateLimiter = new RateLimitedSessionStrippedDown<object>(1, TimeSpan.FromSeconds(1), new object());
DateTime start = DateTime.UtcNow;
for (int i = 0; i < 5; i++)
{
rateLimiter.GetRateLimitedSession();
}
DateTime end = DateTime.UtcNow;
Assert.GreaterOrEqual(end.Subtract(start), TimeSpan.FromSeconds(4));
}
I wonder if the loop in the test is being optimised in a way that it runs each iteration of the loop on a separate thread (or something similar), which means that the test completes quicker than it should because Thread.Sleep only blocks the thread that it is being called on?
Your problem is inside of the Block method, and now that I look at the comments, it appears that Henk Holterman has already brought this up.
It will only fail when _lastReset.Add(_rateLimitSpan) and DateTime.UtcNow are equal. This doesn't happen very often, hence the reason why it fails intermittently. A fix would be to change > to >= on this line:
if (DateTime.UtcNow > _lastReset.Add(_rateLimitSpan))
It's not intuitive why, unless you understand that each call to DateTime.UtcNow doesn't necessarily return a new value one each call.
Even though DateTime.UtcNow is precise up to 100 nanoseconds, its precision is not the same as its accuracy. It relies on the machine's timer interval, which ranges from 1-15ms, but more often set to 15.25ms, unless you're doing something with multimedia.
You can see this in action with this dotnetfiddle. Unless you have a program open that is setting the timer to a different value, like 1ms, you'll notice that the difference between the ticks is about 150000 ticks, about 15ms, or the normal system timer interval.
We can also validate this by lifting out the calls to DateTime.UtcNow into temporary variables and comparing them at the end of the method:
private void Block()
{
var first = DateTime.UtcNow;
while (_retrievalsSinceLastReset >= _rateLimit &&
_lastReset.Add(_rateLimitSpan) > first)
{
Thread.Sleep(TimeSpan.FromMilliseconds(10));
first = DateTime.UtcNow;
}
var second = DateTime.UtcNow;
if (second > _lastReset.Add(_rateLimitSpan))
{
_lastReset = DateTime.UtcNow;
_retrievalsSinceLastReset = 0;
}
if (first == second)
{
Console.WriteLine("DateTime.UtcNow returned same value");
}
}
On my machine, all five calls to Block printed out DateTime.UtcNow as being equal.
Related
I've been working on a hobby project being developed in C# + Xamarin Forms + Prism + EF Core + Sqlite, debugging in UWP app.
I've written the following code to store tick data received from broker to Sqlite.
First, the OnTick call back that receives the ticks (approx. 1 tick per sec per instrument):
private void OnTick(Tick tickData)
{
foreach (var instrument in IntradayInstruments.Where(i => i.InstrumentToken == tickData.InstrumentToken))
{
instrument.UpdateIntradayCandle(tickData);
}
}
And the UpdateIntradayCandle method is:
public void UpdateIntradayCandle(Tick tick)
{
if (LastIntradayCandle != null)
{
if (LastIntradayCandle.Open == 0m)
{
LastIntradayCandle.Open = tick.LastPrice;
}
if (LastIntradayCandle.High < tick.LastPrice)
{
LastIntradayCandle.High = tick.LastPrice;
}
if (LastIntradayCandle.Low == 0m)
{
LastIntradayCandle.Low = tick.LastPrice;
}
else if (LastIntradayCandle.Low > tick.LastPrice)
{
LastIntradayCandle.Low = tick.LastPrice;
}
LastIntradayCandle.Close = tick.LastPrice;
}
}
The LastIntradayCandle is a property:
object _sync = new object();
private volatile IntradayCandle _lastIntradayCandle;
public IntradayCandle LastIntradayCandle
{
get
{
lock (_sync)
{
return _lastIntradayCandle;
}
}
set
{
lock (_sync)
{
_lastIntradayCandle = value;
}
}
}
Now, the LastIntradayCandle is changed periodically, say, 5 minutes, and a new candle is put in place for updating, from a different thread coming from a System.Threading.Timer which is scheduled to run every 5m.
public void AddNewIntradayCandle()
{
if (LastIntradayCandle != null)
{
LastIntradayCandle.IsClosed = true;
}
var newIntradayCandle = new IntradayCandle { Open = 0m, High = 0m, Low = 0m, Close = 0m };
LastIntradayCandle = newIntradayCandle;
IntradayCandles.Add(newIntradayCandle);
}
Now, the problem is, I'm getting 0s in those Open, High or Low but not in Close, Open having the most number of zeroes. This is happening very randomly.
I'm thinking that if any of the Open, High, Low or Close values is getting updated, it means the tick is having a value to be grabbed, but somehow one or more assignments in UpdateIntradayCandle method are not running. Having zeroes is a strict NO for the purpose of the app.
I'm neither formally trained as a programmer nor an expert, but a self-learning hobbyist and definitely never attempted at multi-threading before.
So, I request you to please point me what I am doing wrong, or better still, what should I be doing to make it work.
Multithreading and EF Core is not compatible things. EF Core context is not a thread safe. You have to create new context for each thread. Also making your object thread safe is wasting time.
So, schematically you have to do the following and you can remove locks from your object.
private void OnTick(Tick tickData)
{
using var ctx = new MyDbContext(...);
foreach (var instrument in ctx.IntradayInstruments.Where(i => i.InstrumentToken == tickData.InstrumentToken))
{
instrument.UpdateIntradayCandle(tickData);
}
ctx.SaveChanges();
}
I have a scenario, where I have a worker method (DoWork()) which is getting called by the web service continuously.
This worker method writes the data to a file using the (writeToFile()) and I need to write this to a file under 2 conditions
a) when the number of records reached in 500 OR
b) 2 minutes has been passed from the previous file been written
my sample code is as follows:
public override void DoWork()
{
//-- This worker method is called continuously by the webservice
List<string> list1 = new List<string>();
List<string> list2 = new List<string>();
int count =0;
//--- Some code that writes the RawData
list1.Add(rawData);
If(list1.Count<=500)
{
list2=list1;
count = WriteToFile(list2);
}
}
public static int WriteToFile(List<string> list)
{
//-- The writing of File Should happen when 500 records reached OR 2 mins passed.
// ---- Logic for writing the list file using the Streamwriter is working fine
}
I need a logic check if
500 records reached in the List OR
2 mins passed from the previous
file generated,
only then the file writing should happen.
Thanks
:)
To make it a little more readable, I'd use a little helper, here...
Time check:
class TimeCheck{
private TimeSpan timeoutDuration;
private DateTime nextTimeout;
// Default = 2 minutes.
public TimeCheck(): this(TimeSpan.FromMinutes(2))
{}
public TimeCheck( TimeSpan timeout ){
this.timeoutDuration= timeout;
this.nextTimeout = DateTime.Now + timeoutDuration;
}
public bool IsTimeoutReached => DateTime.Now >= nextTimeout;
public void Reset(){
nextTimeout = DateTime.Now + timeoutDuration;
}
}
Usage:
// on class level
const int MAXITEMS = 500;
private Lazy<TimeCheck> timecheck = new Lazy<TimeCheck>( () => return new TimeCheck() );
private List<string> list1 = new List<string>();
private readonly object Locker = new object();
public override void DoWork()
{
lock(Locker){
// ... add items to list1
// Write if a) list1 has 500+ items or b) 2 minutes since last write have passed
if( list1.Count >= MAXITEMS || timecheck.Value.IsTimeoutReached )
{
WriteToFile(list1);
list1.Clear(); // assuming _all_ Items are written.
timecheck.Value.Reset();
}
}
}
Attention:
If code is called by multiple threads, you need to make sure it's thread safe. I used lock which will create a bottleneck. You may want to figure out a more sophisticated manner, but this answer concentrates on the condition requirements.
Above snippet assumes, list1 is not accessed elsewhere.
I have a service that handles many (~100K) requests per second. Before each request, it checks (for example) if it's started raining, and if so, the behavior changes:
if(IsRaining())
return "You probably shouldn't go anywhere today.";
//... otherwise proceed
IsRaining Version 1 (Slowest)
public bool IsRaining() => ExternalService.IsRaining();
In trying to speed up my service I discovered that checking Service.IsRaining is the performance bottleneck.
I decided I didn't care if the status only just changed to "raining", I could cache the result for a small time. (With a slight exception - if it stops raining, I want to know immediately).
I solved that using the following approach:
IsRaining Version 2 (Faster)
bool isRainingCache;
public bool IsRaining()
{
DateTime now = DateTime.UTCNow;
// If the last time we checked, it was raining, make sure it still is. OR
// If it hasn't been raining, only check again if we haven't checked in the past second.
if (isRainingCache || (now - lastChecked) > TimeSpan.FromSeconds(1))
{
isRainingCache = ExternalService.IsRaining();
lastChecked = now;
}
return isRainingCache;
}
This made things a lot faster and worked for a long time. Then, my service got even faster, it started getting called hundreds of thousands of times per second, and benchmarking informed me that calling DateTime.Now so much makes up 50% of all CPU time.
I know what you're thinking:
Is calling DateTime.Now really your bottleneck?
I'm pretty sure it is. I'm calling it hundreds of thousands of times per second. My real service is just a wrapper for a hash-map lookup - calls are meant to be very fast.
My next thought is that rather than checking how long it's been every single call, some timer could asynchronously expire the cached result after some time:
IsRaining Version 3 (Fastest?)
bool? isRainingCache = null;
public bool IsRaining()
{
// Only check for rain if the cache is empty, or it was raining last time we checked.
if (isRainingCache == null || isRainingCache == true)
{
isRainingCache = ExternalService.IsRaining();
// If it's not raining, force us to check again after 1 second
if(!isRainingCache)
Task.Run(() => Task.Delay(1000).ContinueWith(() => { isRainingCache = null; }));
}
return false;
}
The above (untested) would speed things along, but I feel like this leaves me with several new problems:
It feels abusive to "fire and forget" a Task like this (especially as often as once per second).
If my service is disposed or finalized, I would be leaving queued tasks lying around. I feel like I need to hold on to the task or a cancellation token.
I'm generally inexperienced with TPL, but I feel like it's not appropriate to use Timers or Threads here, which in my experience can lead to a myriad of other shutdown and cleanup issues.
If anyone has any suggestions for a better approach, I would be very appreciative.
I've got several cases like this, I'm thinking it would be nice to abstract the solution into it's own wrapper class, something like:
// Calls the getter at most once per 1000 ms, returns a cached value otherwise.
public Throttled<bool> IsRaining = new Throttled<bool>(() => Service.IsRaining, 1000);
If you change your code to use Environment.TickCount, you should notice a speedup. This is probably going to be the cheapest timer you can check.
#Fabjan's answer may be better, though, if you truly are seeing this method hit 100,000 times a second.
bool isRainingCache;
int lastChecked = Environment.TickCount - 1001;
public bool IsRaining()
{
int now = Environment.TickCount;
// If the last time we checked, it was raining, make sure it still is. OR
// If it hasn't been raining, only check again if we haven't checked in the past second.
if (isRainingCache || unchecked(now - lastChecked) > 1000)
{
isRainingCache = ExternalService.IsRaining();
lastChecked = now;
}
return isRainingCache;
}
A simple rewrite to use Stopwatch instead of DateTime.Now reduces the overhead (for this isolated part) quite significantly.
(since another answer here posted Environment.TickCount I added it for completeness and it has the lowest overhead of them all, note that this value has a turnover rate around 24-25 days before it goes negative so any solution would need to take that into account, note that the answer by #Cory Nelson does that, it uses unchecked to make sure the subtraction works.)
void Main()
{
BenchmarkSwitcher.FromAssembly(GetType().Assembly).RunAll();
}
public class Benchmarks
{
private DateTime _Last = DateTime.Now;
private DateTime _Next = DateTime.Now.AddSeconds(1);
private Stopwatch _Stopwatch = Stopwatch.StartNew();
private int _NextTick = Environment.TickCount + 1000;
[Benchmark]
public void ReadDateTime()
{
bool areWeThereYet = DateTime.Now >= _Last.AddSeconds(1);
}
[Benchmark]
public void ReadDateTimeAhead()
{
bool areWeThereYet = DateTime.Now >= _Next;
}
[Benchmark]
public void ReadStopwatch()
{
bool areWeThereYet = _Stopwatch.ElapsedMilliseconds >= 1000;
}
[Benchmark]
public void ReadEnvironmentTick()
{
bool areWeThereYet = Environment.TickCount > _NextTick;
}
}
Output:
Method | Mean | Error | StdDev |
-------------------- |-----------:|----------:|----------:|
ReadDateTime | 220.958 ns | 4.3334 ns | 4.8166 ns |
ReadDateTimeAhead | 214.025 ns | 0.8364 ns | 0.7414 ns |
ReadStopwatch | 25.365 ns | 0.1805 ns | 0.1689 ns |
ReadEnvironmentTick | 1.832 ns | 0.0163 ns | 0.0153 ns |
So a simple change to this should reduce the overhead for this isolated part of your code:
bool isRainingCache;
Stopwatch stopwatch = Stopwatch.StartNew();
public bool IsRaining()
{
DateTime now = DateTime.Now;
// If the last time we checked, it was raining, make sure it still is. OR
// If it hasn't been raining, only check again if we haven't checked in the past second.
if (isRainingCache || stopwatch.ElapsedMilliseconds > 1000)
{
isRainingCache = ExternalService.IsRaining();
stopwatch.Restart();
}
return isRainingCache;
}
The fact that DateTime.Now call is a bottleneck of application indicates that something might be wrong with the architecture. What's possible wrong here is that we're updating the cache inside of a method that should only get the latest value and return it. If we split up updating the cache and method to get the latest value we'd get something along the lines of:
const int UpdateCacheInterval = 300;
// we use keyword volatile as we access this variable from different threads
private volatile bool isRainingCache;
private Task UpdateCacheTask { get; set; }
// Use it to cancel background task when it's requred
private CancellationTokenSource CancellationTokenSource = new CancellationTokenSource();
private void InitializeCache()
{
UpdateCacheTask = Task.Run(async () =>
{
while(!CancellationTokenSource.Token.IsCancellationRequested)
{
await Task.Delay(UpdateCacheInterval);
isRainingCache = ExternalService.IsRaining();
}
}, CancellationTokenSource.Token);
}
public bool IsRaining()
{
// set the UpdateCacheInterval to a short interval where it's not possible
// that one second has expired from the time of the last check
return isRainingCache;
}
// To stop the task execution
public async Task Stop()
{
CancellationTokenSource.Cancel();
await UpdateCacheTask;
}
I'm generally inexperienced with TPL, but I feel like it's not
appropriate to use Timers or Threads here, which in my experience can
lead to a myriad of other shutdown and cleanup issues
It's perfectly fine to use timers and threads here because we need some backgroundworker to update the cache.
Thanks for the different approaches. If anyone is curious, I did end up abstracting this functionality into a re-usable class, so I can go:
private static readonly Throttled<bool> ThrottledIsRaining =
new Throttled<bool>(ExternalService.IsRaining, 1000);
public static bool IsRaining()
{
bool cachedIsRaining = ThrottledIsRaining.Value;
// This extra bit satisfies my special case - bypass the cache while it's raining
if (!cachedIsRaining) return false;
return ThrottledIsRaining.ForceGetUpdatedValue();
}
/// <summary>Similar to <see cref="Lazy{T}"/>. Wraps an expensive getter
/// for a value by caching the result and only invoking the supplied getter
/// to update the value if the specified cache expiry time has elapsed.</summary>
/// <typeparam name="T">The type of underlying value.</typeparam>
public class Throttled<T>
{
#region Private Fields
/// <summary>The time (in milliseconds) we must to cache the value after
/// it has been retrieved.</summary>
private readonly int _cacheTime;
/// <summary>Prevent multiple threads from updating the value simultaneously.</summary>
private readonly object _updateLock = new object();
/// <summary>The function used to retrieve the underlying value.</summary>
private readonly Func<T> _getValue;
/// <summary>The cached result from the last time the underlying value was retrieved.</summary>
private T _cachedValue;
/// <summary>The last time the value was retrieved</summary>
private volatile int _lastRetrieved;
#endregion Private Fields
/// <summary>Get the underlying value, updating the result if the cache has expired.</summary>
public T Value
{
get
{
int now = Environment.TickCount;
// If the cached value has expired, update it
if (unchecked(now - _lastRetrieved) > _cacheTime)
{
lock (_updateLock)
{
// Upon acquiring the lock, ensure another thread didn't update it first.
if (unchecked(now - _lastRetrieved) > _cacheTime)
return ForceGetUpdatedValue();
}
}
return _cachedValue;
}
}
/// <summary>Construct a new throttled value getter.</summary>
/// <param name="getValue">The function used to retrieve the underlying value.</param>
/// <param name="cacheTime">The time (in milliseconds) we must to cache the value after
/// it has been retrieved</param>
public Throttled(Func<T> getValue, int cacheTime)
{
_getValue = getValue;
_cacheTime = cacheTime;
_lastRetrieved = unchecked(Environment.TickCount - cacheTime);
}
/// <summary>Retrieve the current value, regardless of whether
/// the current cached value has expired.</summary>
public T ForceGetUpdatedValue()
{
_cachedValue = _getValue();
_lastRetrieved = Environment.TickCount;
return _cachedValue;
}
/// <summary>Allows instances of this class to be accessed like a normal
/// <typeparamref name="T"/> identifier.</summary>
public static explicit operator T(Throttled<T> t) => t.Value;
}
I decided to minimize expiry check time using #CoryNelson's unckecked TickCount method. While using an asynchronous expiry mechanism should be faster, I found it not worth the complexity of maintaining additional disposable resources and worrying about additional threading and cleanup issues.
I also took into account #Servy's warning about race conditions that might arise when multiple threads access the same throttled value. The addition of a lock avoids unnecessarily updating the value more than once within the expiry window.
Let me know if you think I missed anything. Thanks everyone.
I'm using Hangfire version "1.6.8".
var datetime = DateTime.Now;
var cron = Cron.Monthly(datetime.Day,datetime.Hour);
RecurringJob.AddOrUpdate<IService>( recurringId, x =>x.CreateRecurring(id), cron);
How can I end this Recurring Job after 'n' times executing it?
The simplest way to do this could be to pass in a specific number of times when the method that is called and prevent it from being executed once that number has been reached:
public class MyService : IService
{
public int runCount = 0;
public void CreateRecurring(id, int? maxTimes = null)
{
if (maxTimes.HasValue && (runCount >= maxTimes))
{
// Has run enough times now, don't do it again
return;
}
// do something...
}
}
// Run a max of 5 times
RecurringJob.AddOrUpdate<IService>( recurringId, x =>x.CreateRecurring(id, 5), cron);
It has to be trivial, but I just cannot get through it.
I have to limit amount of tasks (let's say connections, emails sent or clicks in the button) per amount of time. So e.g. I can send 1000 emails per hour.
How can I do that in c#? I don't know and don't care how much time each operation will take. I just want to make sure that for last hour, only 1000 will be executed.
class EventLimiter
{
Queue<DateTime> requestTimes;
int maxRequests;
TimeSpan timeSpan;
public EventLimiter(int maxRequests, TimeSpan timeSpan)
{
this.maxRequests = maxRequests;
this.timeSpan = timeSpan;
requestTimes = new Queue<DateTime>(maxRequests);
}
private void SynchronizeQueue()
{
while ((requestTimes.Count > 0) && (requestTimes.Peek().Add(timeSpan) < DateTime.UtcNow))
requestTimes.Dequeue();
}
public bool CanRequestNow()
{
SynchronizeQueue();
return requestTimes.Count < maxRequests;
}
public void EnqueueRequest()
{
while (!CanRequestNow())
Thread.Sleep(requestTimes.Peek().Add(timeSpan).Subtract(DateTime.UtcNow));
// Was: System.Threading.Thread.Sleep(1000);
requestTimes.Enqueue(DateTime.UtcNow);
}
}
Assuming a rolling hour window:
Maintain a list of when actions were done.
Each time you want to do your action, remove all in the list not within the hour.
If there are fewer than 1000 then do the action and add a record to your list.
Assuming hourly:
Create a proxy method and a variable that is incremented for every action, and reduced to zero on the hour.
Do your action if the counter is < 1000.
The above solution looked fine. Here is my trimmed down version:
public class EmailRateHelper
{
private int _requestsPerInterval;
private Queue<DateTime> _history;
private TimeSpan _interval;
public EmailRateHelper()
: this(30, new TimeSpan(0, 1, 0)) { }
public EmailRateHelper(int requestsPerInterval, TimeSpan interval)
{
_requestsPerInterval = requestsPerInterval;
_history = new Queue<DateTime>();
_interval = interval;
}
public void SleepAsNeeded()
{
DateTime now = DateTime.Now;
_history.Enqueue(now);
if (_history.Count >= _requestsPerInterval)
{
var last = _history.Dequeue();
TimeSpan difference = now - last;
if (difference < _interval)
{
System.Threading.Thread.Sleep(_interval - difference);
}
}
}
}
You can use Rx extensions (How to use the new BufferWithTimeOrCount in Rx that returns IObservable<IObservable<T>> instead of IObservable<IList<T>>), but I would implement the buffering manually by adding an appropriate proxy object.
You may also consider storing {action, time, user} information in a database and get number of actions in a last hour fomr the DB (or similar persisted storager) if you need to handle Application pool restarts / crashes. Otherwise clever user may circumvent your in-memory protection with overloading your server.
You can create a persistent counter for every user. Every time you receive a request (for sending an email) you need to check the value of the counter and the date of the counter creation.
If the count is greater than the limit you refuse the request
If the date is older than an hour you reset the counter and set the new creation date
If the date is correct and the count is under the limit you increase the counter
Only in the last two cases the request is executed.