Exception: Microsoft.Azure.Documents.RequestRateTooLargeException while querying from DocumentDB - c#

y query is something like
this.ProcessRequestSync(() => this.Client.CreateDocumentQuery<Model>(this.DocumentDBCollectionLink).Where(d => d.name.Equals(name) && d.code.Equals(code) && d.Type.Equals(this.documentType) && d.CreatedBy.Equals(myName).ToList<Model>());
public dynamic ProcessRequestSync(Func<dynamic> getRequest)
{
var delay = TimeSpan.Zero;
var minDelayTime = new TimeSpan(0, 0, 1);
for (;;)
{
try
{
Thread.Sleep(delay);
return getRequest();
}
catch (DocumentClientException documentClientException)
{
var statusCode = (int)documentClientException.StatusCode;
if (statusCode == 429 || statusCode == 503)
{
string errorMessage = string.Format("failed at DocumentDB with {0} status and {1} retry time", statusCode, documentClientException.RetryAfter);
this.Logger.Log(errorMessage );
// Back off if the request rate is too large or the service is temporarily unavailable
delay = TimeSpan.Compare(documentClientException.RetryAfter, minDelayTime) >= 0 ? documentClientException.RetryAfter: minDelayTime;
}
else
{
throw;
}
}
}
}
This is the method for retry logic when requestRateTooLarge exception raise.
I am not sure, whether it is working fine or not,
I am getting Exception: Microsoft.Azure.Documents.RequestRateTooLargeException while querying and inserting around 4000 records at a time,
I used the same retry logic for inserting, its working fine.
I am not getting any error and also successfully inserted all records but unable to get query data.

Based on #aravind Ramachandra and #Ryan CrawCour answers above/below, this is what I am using to get round the issue.
public async Task SaveToDocDb(dynamic jsonDocToSave)
{
using (var client = new DocumentClient(endpoint, authKey))
{
var queryDone = false;
while (!queryDone)
{
try
{
await client.CreateDocumentAsync(docCollectionlink, jsonDocToSave);
queryDone = true;
}
catch (DocumentClientException documentClientException)
{
var statusCode = (int)documentClientException.StatusCode;
if (statusCode == 429 || statusCode == 503)
Thread.Sleep(documentClientException.RetryAfter);
else
throw;
}
catch (AggregateException aggregateException)
{
if(aggregateException.InnerException.GetType() == typeof(DocumentClientException)){
var docExcep = aggregateException.InnerException as DocumentClientException;
var statusCode = (int)docExcep.StatusCode;
if (statusCode == 429 || statusCode == 503)
Thread.Sleep(docExcep.RetryAfter);
else
throw;
}
else
throw;
}
}
}
}

You also need a catch block for AggregateException, and check if the AggregateException.InnerException is a DocumentClientException and performs the same check for StatusCode == 429. Since the query execution is asynchronous, you might be getting the throttle exception wrapped inside an AggregateException.
If you could post a full repro, we might able to definitively identify the problem.

Related

ASP.NET Core Web API - Queries performing 'LastOrDefault' operation must have a deterministic sort order

In ASP.NET Core-6 Web API using Entity Framework, I have this code:
public async Task<Response<NotificationCredentialListDto>> CreateNotificationCredentialAsync(CreateNotificationCredentialDto requestDto)
{
var userName = _currentUserService.UserName;
var response = new Response<NotificationCredentialListDto>();
using (var transaction = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
if (userName != null)
{
try
{
var notification = _mapper.Map<NotificationCredential>(requestDto);
var existingNotification = _dbContext.NotificationCredentials.LastOrDefault(e => e.MerchantId == notification.MerchantId && e.IsModified == false);
if (existingNotification != null)
{
existingNotification.IsModified = true;
_unitOfWork.MerchantNotificationCredentials.Update(notification);
await _unitOfWork.Save();
}
requestDto.IsModified = false;
notification.IsModified = requestDto.IsModified;
notification.UserName = requestDto.UserName;
requestDto.BasicAuth = encoded;
notification.BasicAuth = requestDto.BasicAuth;
notification.CreatedBy = _currentUserService.UserName;
await _unitOfWork.MerchantNotificationCredentials.InsertAsync(notification);
await _unitOfWork.Save();
return response;
}
catch (Exception ex)
{
_logger.Error("An error occured: " + ex);
transaction.Dispose();
return response;
}
}
_logger.Error("Registration failed");
transaction.Dispose();
response.StatusCode = (int)HttpStatusCode.BadRequest;
response.Successful = false;
response.Message = "Registration failed. Please try again";
return response;
}
}
What I want to achieve is that if record exist, it should update the last record as:
IsModified = true
and also insert a new one.
But I git this error:
An error occured: System.InvalidOperationException: Queries performing 'LastOrDefault' operation must have a deterministic sort order. Rewrite the query to apply an 'OrderBy' operation on the sequence before calling 'LastOrDefault'.
How do I resolve this?
Thanks
First, I think your code does not work correctly because after checking the condition existingNotification != null and updating the entity MerchantNotificationCredentials you are inserting it in next lines.
Second, I suggest change code to this:
public async Task<Response<NotificationCredentialListDto>> CreateNotificationCredentialAsync(CreateNotificationCredentialDto requestDto)
{
var userName = _currentUserService.UserName;
var response = new Response<NotificationCredentialListDto>();
using (var transaction = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
if (userName != null)
{
try
{
var notification = _mapper.Map<NotificationCredential>(requestDto);
var existingNotification = await _dbContext.NotificationCredentials.Where(e => e.MerchantId == notification.MerchantId && e.IsModified == false).OrderByDescending(x=>x.CreatedAt).FirstOrDefaultAsync();
if (existingNotification != null)
{
existingNotification.IsModified = true;
_unitOfWork.MerchantNotificationCredentials.Update(notification);
await _unitOfWork.Save();
return response;
}
requestDto.IsModified = false;
notification.IsModified = requestDto.IsModified;
notification.UserName = requestDto.UserName;
requestDto.BasicAuth = encoded;
notification.BasicAuth = requestDto.BasicAuth;
notification.CreatedBy = _currentUserService.UserName;
await _unitOfWork.MerchantNotificationCredentials.InsertAsync(notification);
await _unitOfWork.Save();
return response;
}
catch (Exception ex)
{
_logger.Error("An error occured: " + ex);
transaction.Dispose();
return response;
}
}
_logger.Error("Registration failed");
transaction.Dispose();
response.StatusCode = (int)HttpStatusCode.BadRequest;
response.Successful = false;
response.Message = "Registration failed. Please try again";
return response;
}
}
Like #Rafalon mentioned, you need to sort the collection using OrderBy() before using LastOrDefault().
var existingNotification = _dbContext.NotificationCredentials.OrderBy(e => e.CreatedAt).LastOrDefault(e => e.MerchantId == notification.MerchantId && e.IsModified == false);
But be careful because multiple records could have been added before CreateNotificationCredentialAsync is called. Maybe you should search for all records where MerchantId = notification.MerchantId and e.IsModified = false:
var existingNotifications = _dbContext.NotificationCredentials.Where(e => e.MerchantId == notification.MerchantId && e.IsModified == false).ToList();
and update IsModified property to all of them.

In app billing plugin fails on Xamarin.Android with a vague error message ("Something went wrong on our end. Please try again")

When using James Montemagno's In App Billing Plugin for Xamarin.Android, a vague error message appears at the purchase screen when attempting to purchase products from Google Play:
Something went wrong on our end. Please try again
I am more or less completely using sample code from the plugin's documentation. The plugin works fine with reserved billing IDs for testing, such as android.test.purchased. This support forum states that the issue is due to "existing security measures", which is not particularly clear- https://forums.xamarin.com/discussion/153166/in-app-purchase-error-something-went-wrong-on-our-end-please-try-again.
The error code that results is Plugin.InAppBilling.Abstractions.PurchaseError.UserCancelled.
Here is the code for the purchasing function:
class ProVersionUnlocker
{
string ProductID = "XXXXXXXXXXXX";
public async Task<bool> WasProVersionPurchased()
{
/*TESTING
*
* TESTING
*
* TESTING
*/
#if DEBUG
//return true;
#endif
var billing = CrossInAppBilling.Current;
try
{
var connected = await billing.ConnectAsync(ItemType.InAppPurchase);
if (!connected)
{
//Couldn't connect
return false;
}
//check purchases
var purchases = await billing.GetPurchasesAsync(ItemType.InAppPurchase);
//check for null just incase
if (purchases?.Any(p => p.ProductId == ProductID) ?? false)
{
//Purchase restored
return true;
}
else
{
//no purchases found
return false;
}
}
catch (InAppBillingPurchaseException purchaseEx)
{
//Billing Exception handle this based on the type
Log.Debug("IAP", "Error: " + purchaseEx);
}
catch (Exception ex)
{
//Something has gone wrong
}
finally
{
await billing.DisconnectAsync();
}
return false;
}
public async Task<bool> PurchaseProVersion()
{
if (await WasProVersionPurchased())
return true;
string Payload = "ProVersionProVersion";
var billing = CrossInAppBilling.Current;
try
{
var connected = await billing.ConnectAsync(ItemType.InAppPurchase);
if (!connected)
{
//we are offline or can't connect, don't try to purchase
return false;
}
//check purchases
var purchase = await billing.PurchaseAsync(ProductID, ItemType.InAppPurchase, Payload, new Verify());
//Consume for further testing
//var consumedItem = await CrossInAppBilling.Current.ConsumePurchaseAsync(purchase.ProductId, purchase.PurchaseToken);
//possibility that a null came through.
if (purchase == null)
{
//did not purchase; do nothing and fall through to failure
}
else if (purchase.State == PurchaseState.Purchased)
{
return true;
}
}
catch (InAppBillingPurchaseException purchaseEx)
{
//Billing Exception handle this based on the type
//Maybe it was an Already Purchased Error, in which case we return true
if (purchaseEx.PurchaseError == PurchaseError.AlreadyOwned)
return true;
Log.Debug("IAP", "Error: " + purchaseEx);
}
catch (Exception ex)
{
//Something else has gone wrong, log it
Log.Debug("IAP", "Issue connecting: " + ex);
}
finally
{
await billing.DisconnectAsync();
}
return false;
}
public class Verify : IInAppBillingVerifyPurchase
{
const string key1 = #"XOR_key1";
const string key2 = #"XOR_key2";
const string key3 = #"XOR_key3";
public Task<bool> VerifyPurchase(string signedData, string signature, string productId = null, string transactionId = null)
{
//This is only used for testing things at the moment.
#if false
var key1Transform = Plugin.InAppBilling.InAppBillingImplementation.InAppBillingSecurity.TransformString(key1, 1);
var key2Transform = Plugin.InAppBilling.InAppBillingImplementation.InAppBillingSecurity.TransformString(key2, 2);
var key3Transform = Plugin.InAppBilling.InAppBillingImplementation.InAppBillingSecurity.TransformString(key3, 3);
return Task.FromResult(Plugin.InAppBilling.InAppBillingImplementation.InAppBillingSecurity.VerifyPurchase(key1Transform + key2Transform + key3Transform, signedData, signature));
#else
return Task.FromResult(true);
#endif
}
}
Here is the calling code, which is within a PreferenceFragment:
var Unlocker = new ProVersionUnlocker();
var unlock = this.FindPreference("unlock");
unlock.PreferenceClick += async (sender, e) =>
{
//Toast.MakeText(Plugin.CurrentActivity.CrossCurrentActivity.Current.Activity, "Ayy lmao", ToastLength.Short).Show();
unlock.SetSummary(Resource.String.PressAgain);
bool Success = await Unlocker.PurchaseProVersion();
//Toast.MakeText(Plugin.CurrentActivity.CrossCurrentActivity.Current.Activity, Success.ToString(), ToastLength.Short).Show();
UnlockStuff();
};

Site not working Error if the function is not working

I have a function which checks and authenticates the User and on that basis the data is displayed to the respective User. And the function name is Get_AuthenticateUser_Ums(strUserName);
I call this function on Page_load. This function contains a web service. Now what I want is whenever the service is not working or has some issue, I want that the site should not be displayed to the user and message should prompt as The service is down, so couldnt load the site.
Below is my code
if (!IsPostBack)
{
Get_AuthenticateUser_Ums(strUserName); }
And function
private void Get_AuthenticateUser_Ums(string strUserName)
{
try
{
strReturnMessage = string.Empty;
Boolean bolReturn = ObjUMS.AuthenticateApplicationAccess(strUserName, strAppUrl, out strReturnMessage);
if (bolReturn)
{
DataSet dsUserGroups = new DataSet();
dsUserGroups = ObjUMS.GetUserAppDetailsbyUserNameApplicationUrl(strUserName, strAppUrl, out strReturnMessage);
if (dsUserGroups.Tables[1] != null && dsUserGroups.Tables[1].Rows.Count > 0)
{
string strSubGroupName = dsUserGroups.Tables[1].Rows[0]["SUBGROUP_NAME"].ToString();
if (strSubGroupName == "UBR Requester")
{
if (dsUserGroups.Tables[2] != null && dsUserGroups.Tables[2].Rows.Count > 0)
{
string[] allStates = dsUserGroups.Tables[2].AsEnumerable().Select(r => r.Field<string>("BOUNDARY_VALUE")).ToArray();
ViewState["States"] = string.Join(",", allStates);
}
}
else
{
Response.Redirect("~/NotAuthorize.aspx", false);
}
}
else
{
Response.Redirect("~/NotAuthorize.aspx", false);
}
}
else
{
Response.Redirect("~/NotAuthorize.aspx", false);
}
}
catch (Exception ex)
{
throw ex;
}
}
you can create a Method to check connection using the url to svc , and which returns a boolean based on that you can able to see whether the Service is up or not
public bool checkConnection(){
var url = "http://nvmbd1bkh150v02/UMSService/UserProvider.svc";
bool tosend = false;
try
{
var myRequest = (HttpWebRequest)WebRequest.Create(url);
var response = (HttpWebResponse)myRequest.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
tosend = true ;
// it's at least in some way responsive
// but may be internally broken
// as you could find out if you called one of the methods for real
Debug.Write(string.Format("{0} Available", url));
}
else
{
tosend = false;
// well, at least it returned...
Debug.Write(string.Format("{0} Returned, but with status: {1}",
url, response.StatusDescription));
}
}
catch (Exception ex)
{
// not available at all, for some reason
Debug.Write(string.Format("{0} unavailable: {1}", url, ex.Message));
}
return tosend;
}

Operation is not valid due to the state of the object

When making multiple requests in quick succession, I am sometimes hit with an InvalidOperationException which is marked as the operation is not valid due to the state of the object. I'm not entirely sure what's wrong with the following code:
this.Abort();
this.request = (HttpWebRequest)WebRequest.Create(uri);
this.result = new WebClientAsyncResult(this.request);
if (data != null)
{
this.request.ContentType = "application/x-www-form-urlencoded";
this.request.Method = "POST";
}
// BeginGetResponse does have a synchronous component
// See: http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse.aspx
Action asyncAction = () =>
{
try
{
if (this.request == null)
{
Debug.WriteLine("Request is null");
return;
}
if (request.Method == "POST")
{
GetRequest(result, data, responseAction, errorAction);
}
else
{
GetResponse(result, responseAction, errorAction);
}
}
catch (InvalidOperationException ex)
{
Debug.WriteLine("Invalid");
if (this.request == null)
{
errorAction(new Error(ex.InnerException) {WasAborted = true});
}
}
};
Dispatcher.BeginInvoke(asyncAction, null, null);
return this.result;
This was solve by using a local variable outside of the lambda as it wasn't capturing the value of the field until execution which was causing issues.

Converting async/await to Task.ContinueWith

This question was triggered by comments to this one:
How to back-port a non-linear async/await code to .NET 4.0 without Microsoft.Bcl.Async?
In the linked question, we have a WebRequest operation we want to retry for a limited number of times, if it keeps failing. The Async/await code could look like this:
async Task<HttpWebResponse> GetResponseWithRetryAsync(string url, int retries)
{
if (retries < 0)
throw new ArgumentOutOfRangeException();
var request = WebRequest.Create(url);
while (true)
{
WebResponse task = null;
try
{
task = request.GetResponseAsync();
return (HttpWebResponse)await task;
}
catch (Exception ex)
{
if (task.IsCanceled)
throw;
if (--retries == 0)
throw; // rethrow last error
// otherwise, log the error and retry
Debug.Print("Retrying after error: " + ex.Message);
}
}
}
From the first thought, I'd use TaskCompletionSource, as something like this (untested):
Task<HttpWebResponse> GetResponseWithRetryAsync(string url, int retries)
{
if (retries < 0)
throw new ArgumentOutOfRangeException();
var request = WebRequest.Create(url);
var tcs = new TaskCompletionSource<HttpWebResponse>();
Action<Task<WebResponse>> proceesToNextStep = null;
Action doStep = () =>
request.GetResponseAsync().ContinueWith(proceedToNextStep);
proceedToNextStep = (prevTask) =>
{
if (prevTask.IsCanceled)
tcs.SetCanceled();
else if (!prevTask.IsFaulted)
tcs.SetResult((HttpWebResponse)prevTask.Result);
else if (--retries == 0)
tcs.SetException(prevTask.Exception);
else
doStep();
};
doStep();
return tcs.Task;
}
The question is, how to do this without TaskCompletionSource?
I've figured out how to do it without async/await or TaskCompletionSource, using nested tasks and Task.Unwrap instead.
First, to address #mikez's comment, here's GetResponseAsync implementation for .NET 4.0:
static public Task<WebResponse> GetResponseTapAsync(this WebRequest request)
{
return Task.Factory.FromAsync(
(asyncCallback, state) =>
request.BeginGetResponse(asyncCallback, state),
(asyncResult) =>
request.EndGetResponse(asyncResult), null);
}
Now, here's GetResponseWithRetryAsync implementation:
static Task<HttpWebResponse> GetResponseWithRetryAsync(string url, int retries)
{
if (retries < 0)
throw new ArgumentOutOfRangeException();
var request = WebRequest.Create(url);
Func<Task<WebResponse>, Task<HttpWebResponse>> proceedToNextStep = null;
Func<Task<HttpWebResponse>> doStep = () =>
{
return request.GetResponseTapAsync().ContinueWith(proceedToNextStep).Unwrap();
};
proceedToNextStep = (prevTask) =>
{
if (prevTask.IsCanceled)
throw new TaskCanceledException();
if (prevTask.IsFaulted && --retries > 0)
return doStep();
// throw if failed or return the result
return Task.FromResult((HttpWebResponse)prevTask.Result);
};
return doStep();
}
It's been an interesting exercise. It works, but I think its the way more difficult to follow, than the async/await version.

Categories