How to add extra logging for working with Sybase DB in .Net Core - c#

I am working with a .Net Core application that is receiving data from a Kafka Message. We are taking the data and then sending it to a Stored Proc so that the Tables in the db can be updated. There is a Policy retry handler that is being used that when the Stored Proc is called and if there is an error, then we put the message into a Dead Letter Queue and retry if need be.
However, as I previously said, I am wanting to add an extra message on top of the error message that comes back from Sybase. The following is the code that is being used to execute the function that eventually calls the Stored Proc:
try
{
.Handle<TException>(ex => !ex.IsInvalidDataException())
.WaitAndRetryAsync(Settings.MaxRetryAttempts,
retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt))
, (exception, timeSpan, retryCount, context) =>
{
// Logging to Azure
});
})
.ExecuteAsync(func);
}
catch (Exception exception)
{
// Extra code to determine what needs to happen to the message that was received
}
If the data that is sent to the Stored Proc causes an error for some reason then it shows a message from Sybase of what the error is.
However I am wanting to add a custom message to the message that I am already receiving and was wondering how that might be done.

Related

SignalR disconnect doesn't fire immediately

I am trying to handle disconnect error with reconnection so I used the nextRetryDelayInMilliseconds method for reconnection, but in this case, if the connection is lost or there are some problems with the connection, the onclose handler does not work.
I am using "#microsoft/signalr": "^7.0.2" package in Angular project
I used "withAutomaticReconnect" option, but it doesn't fire immediately. I get a callback after some timeout.
Connection builder:
this.hubconnection = new HubConnectionBuilder()
.withUrl(url)
.withAutomaticReconnect({
nextRetryDelayInMilliseconds: () => {
this._errorState$.next(true);
return 1000;
}
})
.build();
Error handler (which doesn't fire if I use "withAutomaticReconnect"):
this.hubConnection.onclose(error => callback(error)
Expecting to handle the error immediately when there are some errors related to connection.
After test, I found the correct format should be like below.
connection.onclose(() => {
});
In your scenario, code should be
this.hubConnection.onclose((error) => {
console.error(`Something went wrong: ${error}`);
});
After testing, if you restart the signalr server, you may not get the error information, which should be normal. I don’t know if you can see more in the production environment. This is the standard usage, you can also refer to the following Link.
onclose event in signalr client
Suggestion:
The test above I test in signalr javascript client, and I also reproduce the issue, if it not works, please use #microsoft/signalr 7.0.0 version to troubleshoot problems caused by the new version.
I also using reconnectDelay property when build the connection.
var connection = new signalR.HubConnectionBuilder().withUrl("/mainHub")
.withAutomaticReconnect({
nextRetryDelayInMilliseconds: () => {
this._errorState$.next(true);
return 1000;
},
reconnectDelay: 500 // set the reconnect delay to 500ms
})
.configureLogging(signalR.LogLevel.Trace).build();

Detect and handle when StrawberryShake Subscription connection wasn't successful

I have two asp.net applications running locally in development mode. One is running HotChocolate as the GraphQL server and working well. The second application is attempting to run StrawberryShake GraphQL client and access a subscription.
The StrawberyShake documentation is pretty sparse, especially with subscriptions and I'm unable to figure out how to know when a connection error happens. I've pasted below the code that shows what I've tried so far with no success. I've added comments to show what doesn't happen.
try
{
// client is the client auto-generated by StrawberryShake.
session = client.KeyRequest
.Watch(new ServerInfoInput
{
Name = "test",
MachineIdentifier = "machine",
}).Subscribe(result =>
{
// A breakpoint in here never gets hit.
var data = result.Data;
}, () =>
{
// This writeline never gets hit.
Console.WriteLine("Complete");
});
}
catch (Exception e)
{
// An exception is never thrown.
Console.WriteLine(e.ToString());
}
In conclusion, is there a way to know if my connection to the server was successful?

Should I Have to Wait After Creating Team with Graph

I am using the MS Graph API from our web app to create an MS Teams Team in clients' systems and setup a few folders. But I will randomly get errors if I don't impose a hard-coded wait after creating the team. I call the following endpoints in the order shown:
//Create new Team and get basic info
POST teams
GET teams/{team-id}/primaryChannel
GET teams/{team-id}
GET teams/{team-id}/channels/{channel-id}/filesFolder
//Sometimes unknown users must be invited to join org as guest
POST invitations
//Everyone but the owner is added as a guest
POST teams/{team-id}/members
//This is done in a batch, because there is one folder per team guest + one for owner
POST groups/{team-id}/drive/items/{channel-folder-id}/children
//Team members' folders are permitted to them only. So all permissions are deleted and a single user added back
GET groups/{folder-id}/drive/items/{folder-id}/permissions
DELETE groups/{team-id}/drive/items/{folder-id}/permissions/{permission-id}
POST groups/{folder-id}/drive/items/{item-id}/invite
I will sporadically get Forbidden and/or Bad Request responses from:
POST teams/{team-id}/members
DELETE - groups/{team-id}/drive/items/{item-id}/permissions/{permission-id}
Obviously the return statuses of 403 are bugs, because the app definitely has permission to perform the action.
Imposing a 60 second wait after creating the Team seems to resolve this. However, I am currently testing on our Teams environment and am concerned that clients with larger Teams setups will require a longer wait period. I've seen other areas where the documentation says you should wait up to 15 minutes before using a Team that was created from a Group (I am not sure if this applies to creating a normal Team though).
Does anyone know what kind of latency I should be prepared for generally, and if there is any endpoint I can ping to see if the Team is ready for use?
Azure AD, Teams and Exchange are all different systems and need some kind of synchronization that sometimes needs some time.
Whenever you're going to create something in one of these systems, be prepared that it takes some time to access it.
One of the most awkward behaviour I came across is, when you create a group through Exchange Remote Powershell you'll get instantly the group object back. This object has an Azure Object ID. But if you immediately go to Graph and make a request for that group you'll get a 404. Also a look into Azure Portal shows nothing. But if you wait some time (minimum 30 secs, but up to 20!! minutes) the group suddenly appears.
The same also applies if you create a user in Azure through Graph. If you do this, you'll get back an object with the azure id. If you immediately try to add this user to a group or a directory role, it can also happen to get an error, but the timeout here is normally somewhere below 2 sec and I've never seen something above 10 secs.
So for everything, where I'm going to create something in Graph and immediately try to use it, I build some helper method, that tries it multiple times with some smaller timeout between each call:
internal static class Multiple
{
public static Task Try<TException>(int maxRetries, TimeSpan interval, Func<Task> task)
where TException : Exception
{
return Try<TException>(maxRetries, interval, task, exception => true);
}
public static async Task Try<TException>(int maxRetries, TimeSpan interval, Func<Task> task, Func<TException, bool> isExpectedException)
where TException : Exception
{
do
{
try
{
await task().ConfigureAwait(false);
return;
}
catch (Exception ex) when (ex.GetType() == typeof(TException) && isExpectedException((TException)ex))
{
maxRetries--;
if (maxRetries <= 0)
throw;
await Task.Delay(interval);
}
} while (true);
}
}
The usage of the class is as follows:
await Multiple.Try<ServiceException>(20, TimeSpan.FromSeconds(1), async () =>
{
educationClass = await serviceClient.Education.Classes[groupId.ToString()].Request().GetAsync();
}, ex => ex.Error.Code == "Request_ResourceNotFound");
This helper will call the inner method up to 20 times with a timeout of one second. Also the thrown exception must have the given error code. If the number of retries is exceeded or a different error is thrown, the call will rethrow the original exception and must be handled on a higher level.
Simply be aware that behind the Graph interface a highly distributed system works and it sometimes needs some time to get everything in sync.
I test it in my side and met same issues with yours. The 403 error should be a bug as you mentioned because I also have the permission to do the operation. But you mentioned that add guest user to owner, I test it with bad request response, I think it is by design.
Since you can request success after waiting 60 seconds, I think the solution is add a while loop in your code to request the graph api multiple times. In the while loop, if request fail, wait 10 seconds then request again(as Flydog57 mentioned in comments). But you also need to add a mechanism to break loop when request always fail in your code to avoid infinite loops.

MessageReceiver.RegisterMessageHandler throws exceptions continuously if network is down

I have successfully implemented a connection to ServiceBus with MessageReceiver using RegisterMessageHandler that starts a pump (from this example) and all seems to work just fine.
But in case of exception like e.g. when I turn off network connection the pump throws exceptions continuously to the ExceptionHandler. Every second or even faster. I am wondering if this is supposed default behavior and more importantly if it's possible to change, so that e.g. connection retries can happen every 1 minute. Or am I supposed to do Thread.Sleep or something to achieve that?
receiver.RegisterMessageHandler(
async (message, cancellationToken1) => await HandleMessage(receiver, message),
new MessageHandlerOptions(HandleException)
{
AutoComplete = false,
MaxConcurrentCalls = 1
});
P.S. This is how I solved it now, but not sure if it's a proper way:
private Task HandleException(ExceptionReceivedEventArgs args)
{
_logger.Error(...);
return Task.Delay(60000);
}
P.S Here is the RetryPolicy.Default dump:
Azure Service Bus has a default retry policy (RetryPolicy.Default), but given the transport is trying to receive messages and the broker is not available, will raise exceptions.
ExceptionReceivedContext provides a context, ExceptionReceivedContext which has an action that has failed, and the original exception. You can evaluate the action and decide what needs to be done. You could also check if the exception is transient or not. For transient errors, based on the action, you could just wait for the message to be retried again later (Receive action). In other cases you could either log an error or take a more specific action.
Try to configure the "RetryExponential" on your "SubscriptionClient" like this:
var receiver = new Microsoft.Azure.ServiceBus.SubscriptionClient(_serviceBusConnString, _topic, _subscription, this._receiveMode, new RetryExponential(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(10), _retryPolicyMaximumRetryCount));
This is the parameters descriptions:
https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.retryexponential?view=azure-dotnet
Here other post about what the properties means:
ServiceBus RetryExponential Property Meanings

Consuming _error queue in masstransit

For each queue masstransit has consumers, it automatically creates a [queuename]_error queue, and moves messages that could not be processed there (after retrials, etc.)
I´m trying to create a consumer, that takes errors from that queue, and writes it to a database.
In order to consume those messages, I had to create a handler/consumer for the error queue, receiving the original message.
cfg.ReceiveEndpoint(host, "myqueuename", e =>
{
e.Handler<MyMessage>(ctx =>
{
throw new Exception ("Not expected");
});
});
cfg.ReceiveEndpoint(host, "myqueuename_error", e =>
{
e.BindMessageExchanges = false;
e.Handler<MyMessage>(ctx =>
{
Console.WriteLine("Handled");
// do whatever
return ctx.CompleteTask;
});
});
All that works fine, the problem to retrieve the actual exception that occurred.
I was actually able to do that, with some serious hack....
e.Handler<MyMessage>(m =>
{
var buffer = m.ReceiveContext.TransportHeaders
.GetAll().Single(s => s.Key == "MT-Fault-Message").Value as byte[];
var errorText = new StreamReader(new MemoryStream(buffer)).ReadToEnd();
Console.WriteLine($"Handled, Error={errorText}");
return m.CompleteTask;
});
That just fells wrong though.
PS: I Know i could subscribe to a Fault event, but in this particular case, it is a RequestClient (request-response) pattern, and MT redirects FaultAddress back to the client, and I can´t garantee it is still running.
Request/reply should only be used for getting the data. It means that if the requestor goes down - there are no more reasons to reply with data or with fault and you do not have interest in consuming faults.
So, the reason for the request client to use a temporary (non-durable) queue instead of the receive endpoint queue is by design. It encourages you not to understand that the scope of your replies is only within the request waiting time.
If you send commands and need to be informed if the command has been processed - you should publish events to inform about the outcome of the command processing. Using message metadata (initiator id and conversation id) allows you to find out, how events correlate with commands.
So, only use request/reply for requesting information (queries) using decoupled invocation SOA pattern, where the reply only have a meaning in correlation with request and if the requestor goes down, the reply is no longer needed, no matter if it was a success of failure.

Categories