We Keep Coding

gRPC - Limit client requests (Throttle)

Is there a way to add a throttle in gRPC Unary calls? My goal is to limit each user to 10 requests per second.
In my researches I found that (maybe) it would be placed in a class that inherits Interceptor, like the following:
public class LimitClientRequestsInterceptor : Interceptor
{
public override UnaryServerHandler<TRequest, TResponse>(
TRequest request,
ServerCallContext context,
UnaryServerMethod<TRequest, TResponse> continuation)
{
// Add code to limit 10 requests per second for each user.
return await base.UnaryServerHandler(request, context, continuation);
}
}

I found a way to solve my problem, since gRPC does not have a built-in method (in c#) for rate limits or throttling. However, I was able to do it as shown below.
My interceptor class:
public class LimitClientRequestsInterceptor : Interceptor
{
private readonly ThrottleGauge _gauge;
public override UnaryServerHandler<TRequest, TResponse>(
TRequest request,
ServerCallContext context,
UnaryServerMethod<TRequest, TResponse> continuation)
{
var username = context.GetHttpContext().User.GetUserNameInIdentity();
if (ThrottlingAttribute.UserHasReachedMaxRateLimit(username))
{
throw new RpcException(new Status(
StatusCode.Cancelled,
Newtonsoft.Json.JsonConvert.SerializeObject(new
{ Code = 429, Detail = $"Throttle: {username} exceeded
{_gauge.MaxMessagesPerTimeSlice} messages in {_gauge.TimeSlice}" })));
}
return await base.UnaryServerHandler(request, context, continuation);
}
}
My Throttle class:
internal class ThrottlingAttribute
{
private static Dictionary<string, ThrottleGauge> _byUser;
private static TimeSpan _defaultThrottle_TimeSliceInMilliseconds = TimeSpan.FromMilliseconds(ServiceSettings.Instance.DefaultThrottle_TimeSliceInMilliseconds);
private static int _defaultThrottle_MaxMessagesPerTimeSlice = ServiceSettings.Instance.DefaultThrottle_MaxMessagesPerTimeSlice;
public static bool UserHasReachedMaxRateLimit(string username)
{
ThrottleGauge gauge;
if (!_byUser.TryGetValue(username, out gauge))
{
gauge = new ThrottleGauge(
_defaultThrottle_TimeSliceInMilliseconds,
_defaultThrottle_MaxMessagesPerTimeSlice);
_byUser[username] = gauge;
}
return gauge.WillExceedRate();
}
}
And my ThrottleGauge class:
internal class ThrottleGauge
{
private readonly object _locker = new object();
private Queue<DateTime> _Queue = new Queue<DateTime>();
public TimeSpan TimeSlice { get; private set; }
public int MaxMessagesPerTimeSlice { get; private set; }
public ThrottleGauge(TimeSpan timeSlice, int maxMessagesPerTimeSlice)
{
TimeSlice = timeSlice;
MaxMessagesPerTimeSlice = maxMessagesPerTimeSlice;
}
// returns true if sending a message now message exceeds limit rate
public bool WillExceedRate()
{
lock (_locker)
{
var now = DateTime.Now;
if (_Queue.Count < MaxMessagesPerTimeSlice)
{
_Queue.Enqueue(now);
return false;
}
DateTime oldest = _Queue.Peek();
if ((now - oldest).TotalMilliseconds < TimeSlice.TotalMilliseconds)
return true;
_Queue.Dequeue();
_Queue.Enqueue(now);
return false;
}
}
}

DelegatingHandler not getting invoked if exception caught by ExceptionHandler

I am trying to understand how Web API Http pipeline works!
In my Web API project, I am using the following technique to log/handle exceptions:
ExceptionHandler -> Handle exceptions at the global level
ExceptionFilterAttribute -> Handle custom exception thrown by user
DelegatingHandler -> log request and response data
Sample code for each implementation:
ExceptionFilter:
public class CustomExceptionFilter : ExceptionFilterAttribute
{
public override void OnException(HttpActionExecutedContext context)
{
var request = context.ActionContext.Request;
if (context.Exception is ItemNotFoundException)
{
context.Response = request.CreateResponse(HttpStatusCode.NotFound, context.Exception.Message);
}
else if (context.Exception is InvalidRequestException)
{
context.Response = request.CreateResponse(HttpStatusCode.BadRequest, context.Exception.Message);
}
}
}
Exception Handler:
public class GlobalExceptionHandler : ExceptionHandler
{
public override void Handle(ExceptionHandlerContext context)
{
var result = new HttpResponseMessage(HttpStatusCode.InternalServerError)
{
Content = new StringContent(Constant.ErrorMessage.InternalServerError)
};
context.Result = new ErrorMessageResult(context.Request, result);
}
}
public class ErrorMessageResult : IHttpActionResult
{
private readonly HttpRequestMessage _request;
private readonly HttpResponseMessage _httpResponseMessage;
public ErrorMessageResult(HttpRequestMessage request, HttpResponseMessage httpResponseMessage)
{
_request = request;
_httpResponseMessage = httpResponseMessage;
}
public Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken)
{
return Task.FromResult(_httpResponseMessage);
}
}
DelegatingHandler:
public class LogRequestAndResponseHandler : DelegatingHandler
{
private readonly ILoggingService _loggingService;
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
string requestBody = await request.Content.ReadAsStringAsync();
_loggingService.FirstLevelServiceLog(requestBody);
var result = await base.SendAsync(request, cancellationToken);
if (result.Content != null)
{
var responseBody = await result.Content.ReadAsStringAsync();
_loggingService.FirstLevelServiceLog(responseBody);
}
return result;
}
}
Observation:
When there is an custom exception CustomExceptionFilter is getting invoked and later the response is logged in LogRequestAndResponseHandler.
However, if the exception is not handled, it goes in GlobalExceptionHandler then the response DOES NOT come to LogRequestAndResponseHandler for logging.
Could anyone let me know, what code change have to be done in CustomExceptionFilter/GlobalExceptionHandler in order to receive the response in DelegatingHandler?
Solution: (Updated 10/09/2018)
Okay, so i found the solution here
By modifying ExceptionHandler code, i am able to catch the response in DelegatingHandler
Key was to inherit from IExceptionHandler rather than ExceptionHandler
Code:
public class GlobalExceptionHandler : IExceptionHandler
{
public Task HandleAsync(ExceptionHandlerContext context, CancellationToken cancellationToken)
{
var httpResponse = context.Request.CreateResponse(HttpStatusCode.InternalServerError, Constant.ErrorMessage.InternalServerError);
context.Result = new ResponseMessageResult(httpResponse);
return Task.FromResult(0);
}
}
Question:
I am still not able to understand how it's working? What is the difference between IExceptionHandler & ExceptionHandler?
Could anyone shed some light on this?

ExceptionHandler implements IExceptionHandler like this:
Task IExceptionHandler.HandleAsync(ExceptionHandlerContext context, CancellationToken cancellationToken)
{
if (context == null)
throw new ArgumentNullException(nameof (context));
ExceptionContext exceptionContext = context.ExceptionContext;
if (!this.ShouldHandle(context))
return TaskHelpers.Completed();
return this.HandleAsync(context, cancellationToken);
}
Where I suspect you're seeing the difference is in that ShouldHandle check, which is implemented like this:
public virtual bool ShouldHandle(ExceptionHandlerContext context)
{
if (context == null)
throw new ArgumentNullException(nameof (context));
return context.ExceptionContext.CatchBlock.IsTopLevel;
}
I'm not intimately familiar with the pipeline, but from what I've seen it appears that exceptions can be handled at various points, and the ExceptionHandler base class assumes you probably only want to handle exceptions at the top level of the execution stack. I've seen cases where other handlers like CORS get in the way of this, and the catch block never ends up being at the top level.
If this is what you're seeing, you can still extend ExceptionHandler, and override the ShouldHandle method to just always return true. Or you could be more surgical and specifically detect whether CORS is likely to get in the way of the top-level check as suggested in this comment.

How to wait for async method to finish when calling a external service in .net core?

I have seen some of the existing questions regarding async waiting for completion , However for me none of the solution work.
I am using a C# wrapper for connecting to sales force https://github.com/developerforce/Force.com-Toolkit-for-NET/
In the below method i want to wait for the method UsernamePasswordAsync to complete execution so that i can get the values from the auth object.
public async Task<Token> GetTokenForSalesForce()
{
Token token = null;
try
{
var auth = new AuthenticationClient();
await auth.UsernamePasswordAsync(configuration.Value.ClientId, configuration.Value.ClientSecert,
configuration.Value.SFUsername, configuration.Value.SFPassword,
configuration.Value.SFBaseUrl);
if (!string.IsNullOrEmpty(auth.AccessToken) && !string.IsNullOrEmpty(auth.InstanceUrl))
{
token = new Token
{
BearerToken = auth.AccessToken,
InstanceURL = auth.InstanceUrl,
ApiVersion = auth.ApiVersion
};
}
}
catch (Exception ex)
{
throw ex;
}
return token;
}
public async Task<List<SFDashboardResponse>> GetOrderCountFromSalesForce(Token token)
{
List<SFDashboardResponse> sFDashboardResponses = new List<SFDashboardResponse>();
try
{
var client = new ForceClient(token.InstanceURL, token.BearerToken, token.ApiVersion);
var response = await client.QueryAsync<SFDashboardResponse>("SELECT something ");
var records = response.Records;
}
catch(Exception e)
{
}
return sFDashboardResponses;
}
The signature in the library is
public async Task WebServerAsync(string clientId, string clientSecret, string redirectUri, string code, string tokenRequestEndpointUrl)
{
}
The problem is while the method wait for await to be first execute another thread executes the other part of the orignal caller.
I call it from here
public IActionResult post()
{
var authtoken = _salesForceService.GetTokenForSalesForce();
var response = _salesForceService.GetOrderCountFromSalesForce(authtoken.Result);
DashboardModel dashboardModel = null;
if (authtoken.Status == TaskStatus.RanToCompletion)
{
fill the object
}
return Ok(dashboardModel);
}

You can wrap the IActionResult with a Task and await on the tasks below.
public async Task<IActionResult> post()
{
var authtoken = await _salesForceService.GetTokenForSalesForce();
var response = await _salesForceService.GetOrderCountFromSalesForce(authtoken);
DashboardModel dashboardModel = //fill the object
return Ok(dashboardModel);
}
At least this is what you are asking for as far as I understand, if its another problem let me know.
EDIT 1:
This is just my suggestion/opinion.
Personally I dont really like having the code wrapped in try-catch everywhere, this way the code can be hard to read and maintain. You really should consider centralizing exception handling in one place, you could have a base controller or just a middleware like this one:
public class ErrorHandlingMiddleware
{
private readonly RequestDelegate _next;
public ErrorHandlingMiddleware(RequestDelegate next)
{
this._next = next;
}
public async Task Invoke(HttpContext context, ILogger logger)
{
try
{
await _next(context);
}
catch (Exception ex)
{
await HandleExceptionAsync(context, ex, logger);
}
}
private static Task HandleExceptionAsync(HttpContext context, Exception exception, ILogger logger)
{
logger.Log(exception);
//do something
return context.Response.WriteAsync(... something ...); //Maybe some JSON message or something
}
}
The you just register it as a middleware in the Configure method like below:
app.UseMiddleware<ErrorHandlingMiddleware>();

Creating a simple Async Workflow Activity

I'm trying to get in to workflow foundation but apparently i can't seem to get even the most basic implementation of an async activity working.
Could anyone point me in the right direction with this activity I have put together in order to make an async OData request using HttpClient ...
Firstly I created a base type extending from AsyncCodeActivity ...
public abstract class ODataActivity<TResult> : AsyncCodeActivity<TResult>, IDisposable
{
protected HttpClient Api =
new HttpClient(
new HttpClientHandler() { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate }
)
{ BaseAddress = new Uri(new Config().ApiRoot) };
bool disposed = false;
public void Dispose()
{
Dispose(disposed);
}
public virtual void Dispose(bool disposed)
{
if (!disposed)
{
Api.Dispose();
Api = null;
}
}
}
Next I inherit that to provide my implementation ...
public class ODataFetchActivity<TResult> : ODataActivity<TResult>
{
public string Query { get; set; }
protected override IAsyncResult BeginExecute(AsyncCodeActivityContext context, AsyncCallback callback, object state)
{
var task = Api.GetAsync(Query)
.ContinueWith(t => t.Result.Content.ReadAsAsync<TResult>())
.ContinueWith(t => callback(t));
context.UserState = task;
return task;
}
protected override TResult EndExecute(AsyncCodeActivityContext context, IAsyncResult result)
{
var response = ((Task<TResult>)result).Result;
context.SetValue(Result, response);
return response;
}
}
... the idea being that this activity can do only get requests, i could then implement a post, put and delete to get full crud in the same manner on top of my base type above.
The problem comes when i add this to a workflow and try to execute the flow using a re-hosted designer in a new wpf app resulting in the following exception ...
Edit:
So I did a little bit more tinkering and have something that appears to not complain but i'm not convinced this is a "good" way to handle this as Task implements IAsyncResult directly and it feels like i'm jumping through a bunch of hoops I perhaps don't need to.
public class ODataFetchActivity<TResult> : ODataActivity<TResult>
{
public string Query { get; set; }
Func<TResult> work;
protected override IAsyncResult BeginExecute(AsyncCodeActivityContext context, AsyncCallback callback, object state)
{
work = () => Api.Get<TResult>(Query).Result;
context.UserState = work;
return work.BeginInvoke(callback, state);
}
protected override TResult EndExecute(AsyncCodeActivityContext context, IAsyncResult result)
{
TResult response = work.EndInvoke(result);
Result.Set(context, response);
return response;
}
}
This appears to compile and run but i can't help but feel like there's a cleaner way to handle this.

Hmm apparently this works fine ...
public class ODataFetchActivity<TResult> : ODataActivity<TResult>
{
public string Query { get; set; }
Func<TResult> work;
protected override IAsyncResult BeginExecute(AsyncCodeActivityContext context, AsyncCallback callback, object state)
{
work = () => Api.Get<TResult>(Query).Result;
context.UserState = work;
return work.BeginInvoke(callback, state);
}
protected override TResult EndExecute(AsyncCodeActivityContext context, IAsyncResult result)
{
TResult response = work.EndInvoke(result);
Result.Set(context, response);
return response;
}
}
I was getting some odd behavior from the designer re-hosting where it would run the previous version until a save made (no idea why)

base.SendAsync -How symmetrical execution was done?

In asp.net - using Message Handlers - I can customize the request/response by adding message handlers.
So, a request comes in , going through multiple message handlers and then the response is back through the same handlers( in opposite direction).
So, for example : if I attach 2 message handlers : (yes I know, async/await is preferred, but that's from a book)
public class CustomMessageHandler1 : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
Debug.WriteLine("CustomMessageHandler1 request invoked");
return base.SendAsync(request, cancellationToken)
.ContinueWith(task =>
{
Debug.WriteLine("CustomMessageHandler1 response invoked");
var response = task.Result;
return response;
});
}
}
public class CustomMessageHandler2 : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
Debug.WriteLine("CustomMessageHandler2 request invoked");
return base.SendAsync(request, cancellationToken)
.ContinueWith(task =>
{
Debug.WriteLine("CustomMessageHandler2 response invoked");
var response = task.Result;
return response;
});
}
}
Let's not forget to register those in global.asax :
var config = GlobalConfiguration.Configuration;
config.MessageHandlers.Add(new CustomMessageHandler1());
config.MessageHandlers.Add(new CustomMessageHandler2());
And the result is :
As you can see , like I said and like this article says : The execution is symmetrical.
Great.
But then I thought to myself - how did they do that symmetrical execution?
So I succeed creating my own demo of symmetrical execution using continuation:
void Main()
{
Method1() ;
}
public async Task Method1 ()
{
Console.WriteLine ("Method_1"); //alias to request
await Method2();
Console.WriteLine ("Finished Method_1"); //alias to response
}
public async Task Method2 ()
{
Console.WriteLine ("Method_2"); //alias to request
await Task.FromResult("...");//dummy
Console.WriteLine ("Finished Method_2"); //alias to response
}
And the result was indeed symetrical :
Method_1
Method_2
Finished Method_2
Finished Method_1
But in my code Method1 called Method2 and that's why it worked !.
But in the first code above - they do NOT call each other ! it's like something is invoking only the first part ( before the ContinueWith) from each method , and then run the second part( after the ContinueWith) from each method.
Something like :
So i've look at the reference source for base.Sendasync : But couldn't find how base.Sendasync is doing this symmetrical execution
Question
How does base.Sendasync is doing that symmetrical execution without having one method calling the other?

Here is the console-appified Web API pipeline for you.
abstract class BaseHandler // HttpHandler
{
public abstract Task MyMethodAsync();
}
abstract class Handler : BaseHandler // MessageHandler
{
public Handler InnerHandler { get; set; }
public override Task MyMethodAsync()
{
if (this.InnerHandler != null)
return this.InnerHandler.MyMethodAsync();
else
return Task.FromResult<object>(null);
}
}
class Handler1 : Handler
{
public override async Task MyMethodAsync()
{
Console.WriteLine("Method_1"); //alias to request
await base.MyMethodAsync();
Console.WriteLine("Finished Method_1"); //alias to response
}
}
class Handler2 : Handler
{
public override async Task MyMethodAsync()
{
Console.WriteLine("Method_2"); //alias to request
await base.MyMethodAsync();
Console.WriteLine("Finished Method_2"); //alias to response
}
}
class LastHandler : Handler
{
public override async Task MyMethodAsync()
{
// Does nothing
await base.MyMethodAsync();
}
}
class Program
{
static void Main(string[] args)
{
List<Handler> handlers = new List<Handler>();
// You do this when you add the handler to config
handlers.Add(new Handler1());
handlers.Add(new Handler2());
// This part is done by HttpClientFactory
Handler pipeline = new LastHandler();
handlers.Reverse();
foreach (var handler in handlers)
{
handler.InnerHandler = pipeline;
pipeline = handler;
}
pipeline.MyMethodAsync().Wait();
}
}

Each delegating handler is aware of its "next" handler, and DelegatingHandler.SendAsync does call SendAsync on the next ("inner") handler. You can think of it like a linked list, as such:
public abstract class MyDelegatingHandler
{
private readonly MyDelegatingHandler _next;
public MyDelegatingHandler(MyDelegatingHandler next = null)
{
_next = next;
}
public virtual Task SendAsync()
{
if (_next == null)
return Task.FromResult(0);
return _next.SendAsync();
}
}