Bug with Azure Batch, initializing job object from taskitem - c#

To add a task, as shown in the official tutorial from Microsoft, I have to make a chain of initialization. Here is the code.
var cred = new BatchCredentials(Credentials.AzureBatch.Name, Credentials.AzureBatch.AccountKey);
var batchClient = BatchClient.Connect(Credentials.AzureBatch.Uri, cred);
var workItemManager = batchClient.OpenWorkItemManager();
_job = workItemManager.GetJob(Credentials.AzureBatch.Name, "job-0000000001");
Problem is that the code execution stops on the next line.
_job = workItemManager.GetJob(Credentials.AzureBatch.Name, "job-0000000001");
Then throws an exception with the description {"The remote server returned an error: (404) Not Found."}.
I assume, job with the same name is not found on the server. But according to the tutorial, the name given job at its automatic creation, together with the creation of workitem.
What's wrong?

Your code doesn't show the workitem creation part, I assume you have already done so. If not, you need to create the workitem first.
Workitem and job creation are not synchronize. So, it's possible that your workitem has been created but not the job. Just catch the exception and retry until you find the job.
#ccoxton is right that you can download the Batch Explorer from https://code.msdn.microsoft.com/windowsazure/Azure-Batch-Explorer-c1d37768. This should give you a view on what's happening on the server.

Download the Azure Batch Explorer application, and connection your account to it. This will show you the running pools, work items, and jobs. You must have a running work item for that code to work. There could have been a problem with the code you used to create the work item.

download the batch explorer code from here..
https://github.com/Azure/azure-batch-samples/tree/master/CSharp/BatchExplorer

Related

Datastax C# driver 3.3.0 deadlocking on connect to cluster?

To Datastax C# driver engineers:
C# driver 3.3.0 is deadlocking while calling to Connect(). The following code snippet on Windows Forms will deadlock trying to connect:
public void SimpleConnectTest()
{
const string ip = "127.0.0.1";
const string keyspace = "somekeyspace";
QueryOptions queryOptions = new QueryOptions();
queryOptions.SetConsistencyLevel(ConsistencyLevel.One);
Cluster cluster = Cluster.Builder()
.AddContactPoints(ip)
.WithQueryOptions(queryOptions)
.Build();
var cassandraSession = cluster.Connect(keyspace);
Assert.AreNotEqual(null, cassandraSession);
cluster.Dispose();
}
Deadlocking happens here:
Cluster.cs ->
private void Init()
{
...
TaskHelper.WaitToComplete(_controlConnection.Init(), initialAbortTimeout);
...
}
I have tested this on Cassandra 3.9.0, CQL spec 3.4.2 on local machine.
Everything deadlocks on calling this method _controlConnection.Init() here:
task = Id = 11, Status = WaitingForActivation, Method = "{null}", Result = "{Not yet computed}"
This then just runs for 30000ms and throws this:
throw new TimeoutException(
"Cluster initialization was aborted after timing out. This mechanism is put in place to" +
" avoid blocking the calling thread forever. This usually caused by a networking issue" +
" between the client driver instance and the cluster.", ex);
Running same test on 3.2.0 has no such problems. Can anyone else test this? Maybe this just happens to me.
Edit:
Here is the screenshot for the deadlock:
Thanks to the details in your comments, we were able to identify the underlying issue.
Similar to what was proposed by Luke, there were some missing ConfigureAwait() calls.
This issue impacts users that are calling Cluster.Connect() on environments with SynchonizationContext which is not a common use case:
For Windows Forms, its unlikely to communicate directly to a database (without a service in the middle). Furthermore, users should call Connect() before creating a form (where there is no SynchonizationContext) to share the same Session instance across all forms.
For ASP.NET, users should call Connect() outside of any endpoint action, before the HttpContext is created (where there is no SynchonizationContext).
Note that this issue affects only Connect() calls. Other blocking calls like Execute() don't have this issue.
In any case, this issue could be a showstopper for users getting started with the driver, for example, users creating a simple windows forms app to try a concept.
I've submitted a pull request with the fix, which also contains a test that looks into the source code for the usage of await without ConfigureAwait() calls to avoid having this issue in the future:
https://github.com/datastax/csharp-driver/pull/309
You can expect the fix to land in the next patch release.
I can't reproduce the problem, but I suspect the problem might be with a recent change to make the connection process asynchronous internally. I don't know for sure, but tracing through the Connect code, I suspect it might be a missing ConfigureAwait(false). In particular, it looks like the Reconnect method (which could definitely get hit as part of that Init code path) is missing one after that commit. It's possible that I'm not able to reproduce it because I'm not hitting the Reconnect code path while for some reason you are in your environment.
I'm not 100% sure that's the culprit, but I opened a PR to fix it. Stephen Cleary wrote a great explanation on why this can happen in Forms/Web apps. You could try building the driver from my fork to see if that change fixes the problem, or wait and see what happens with the PR and a new release. If it's still happening, I'd suggest opening an issue on the JIRA.
Hope that helps!
Issue has been opened here with workaround:
https://datastax-oss.atlassian.net/projects/CSHARP/issues/CSHARP-579
For anyone experiencing the same - just wrap your connection code into a new task.
Task.Run(() =>
{
SimpleConnectTest();
});

How to programmatically mark an Azure WebJob as failed?

Is there a way to mark a WebJob (triggered, not continuous) as failed, without throwing an exception? I need to check that certain conditions are true to mark the job as successful.
According to Azure WebJob SDK, Code from TriggeredFunctionExecutor class.
public async Task<FunctionResult> TryExecuteAsync(TriggeredFunctionData input, CancellationToken cancellationToken)
{
IFunctionInstance instance = _instanceFactory.Create((TTriggerValue)input.TriggerValue, input.ParentId);
IDelayedException exception = await _executor.TryExecuteAsync(instance, cancellationToken);
FunctionResult result = exception != null ?
new FunctionResult(exception.Exception)
: new FunctionResult(true);
return result;
}
We know that the WebJobs status depends on whether your WebJob/Function is executed without any exceptions or not. We can't set the finial status of a running WebJob programmatically.
I need to check that certain conditions are true to mark the job as successful.
Throw an exception is the only way I found. Or you could store the webjob execute result in an additional place(For example, Azure Table Storage). We can get the current invocation id by ExecutionContext class. In your webjob, you could save the current invocation id and the status you wanted to an Azure Table Storage. You could query the status later if you needed from Azure Table Storage based on the invocation id.
public static void ProcessQueueMessage([QueueTrigger("myqueue")] string message, ExecutionContext context, TextWriter log)
{
log.WriteLine(message);
SaveStatusToTableStorage(context.InvocationId, "Fail/Success");
}
To use ExecutionContext as parameter, you need to install Azure WebJobs SDK Extensions using NuGet and invoke UserCore method before your run your WebJob.
var config = new JobHostConfiguration();
config.UseCore();
var host = new JobHost(config);
host.RunAndBlock();
Throwing an unmanaged exception will result in a Failed execution.
But i have noticed that it will also result with a bad management of your message: i.e. your message will be dequeued but not moved to your poison queue regarding your configuration (but maybe it was due to my SDK version).
#Jean NETR-VALERE the newer versions of the WebJobs packages do act as you say and if an exception is thrown the job will fail, and will continue to be run over and over and over until you finally clear your queue. This is absolutely horrible behavior and I have no clue why they changed this.
Yes they did change it to make it work this way, because I use an older version of the webjobs package just for this reason. About 3 months ago I upgraded to the newer version, and shortly after could not understand why the above behavior was happening . Once I reverted back to the older version, it started working correctly again and after failing 5 times is moved to poison queue and never ran again. My point is that if you want the correct (IMO) behavior, see if you can go back to using version 1.1.0 and you will be happy. Hope that helps.
To mark a triggered web job as failed you just need to set process exit code to non-zero.
System.Environment.ExitCode = 1;
When you throw an unhandled exception it also sets the exit code, that is how Azure determines failure.

Hangfire is not working in real server

I have to schedule two task every 5 minutes in my ASP.NET C# project, create a zip file and add a log. I used hangfire to schedule my tasks, and it's working fine in local server. When I deploy it, none of them are working (zip or log is not created). When I looked into the hangfire dashboard, I saw the create zip is under Scheduled, create log is under Failed and the error message is System.UnauthorizedAccessException Access to the path is denied. I looked into this question and created a app.manifest and added level="requireAdministrator". But this did not help.
Check the AppPool that your HangFire instance is running. Make sure the AppPool identity has the permission to access the resource.
Try this, first write all the methods you need to run in a new aspx page and run that page, in your case write the log function in a page and run the page you created (say writelog.aspx). If this prints the log, then you can try this.
Inside your schedule call function, call the page you created
WebClient client = new WebClient();
client.DownloadData("yourhostaddress/writelog.aspx");

Converting Microsoft EWS StreamingNotification Example to a service

I've been working to try and convert Microsoft's EWS Streaming Notification Example to a service
( MS source http://www.microsoft.com/en-us/download/details.aspx?id=27154).
I tested it as a console app. I then used a generic service template and got it to the point it would compile, install, and start. It stops after about 10 seconds with the ubiquitous "the service on local computer started and then stopped."
So I went back in and upgraded to C# 2013 express and used NLog to put a bunch of log trace commands to so I could see where it was when it exited.
The last place I can find it is in the example code, SynchronizationChanges function,
public static void SynchronizeChanges(FolderId folderId)
{
logger.Trace("Entering SynchronizeChanges");
bool moreChangesAvailable;
do
{
logger.Trace("Synchronizing changes...");
//Console.WriteLine("Synchronizing changes...");
// Get all changes since the last call. The synchronization cookie is stored in the
// _SynchronizationState field.
// Only the the ids are requested. Additional properties should be fetched via GetItem
//calls.
logger.Trace("Getting changes into var changes.");
var changes = _ExchangeService.SyncFolderItems(folderId, PropertySet.IdOnly, null, 512,
SyncFolderItemsScope.NormalItems,
_SynchronizationState);
// Update the synchronization cookie
logger.Trace("Updating _SynchronizationState");
the log file shows the trace message ""Getting changes into var changes." but not the "Updating _SynchronizationState" message.
so it never gets past var changes = _ExchangeService.SyncFolderItems
I cannot for the life figure out why its just exiting. There are many examples of EWS streaming notifications. I have 3 that compile and run just fine but nobody as far as I can tell has posted an example of it done as a service.
If you don't see the "Updating..." message it's likely the sync threw an exception. Wrap it in a try/catch.
OK, so now that I see the error, this looks like your garden-variety permissions problem. When you ran this as a console app, you likely presented the default credentials to Exchange, which were for your login ID. For a Windows service, if you're running the service with one of the built-in accounts (e.g. Local System), your default credentials will not have access to Exchange.
To rectify, either (1) run the service under the account you did the console app with, or (2) add those credentials to the Exchange Service object.

How to set Header when restoring Background Downloads

I'm trying to make some downloads using cookie authentication doing:
var downloader = new BackgroundDownloader();
downloader.SetRequestHeader("Cookie", "JSESSIONID=" + App.LoginGateway.JSESSIONID);
downloader.SetRequestHeader("Cookie", "JSESSIONID=" + App.LoginGateway.JSESSIONID);
Until here everything works perfectly, the problem begins when I try to restore my downloads and my JSESSIONID is expired
IReadOnlyList<DownloadOperation> downloads = null;
downloads = await BackgroundDownloader.GetCurrentDownloadsAsync();
I tried to find where could I set the request Header again but I was not capable. If I create a new BackgroundDownloader where could I set it for my download Operation?? Some Help is very appreciated
As of Windows 8.1, BackgroundTransfer does not support updating the headers associated with a DownloadOperation/UploadOperation after the operation has been created, even if the operation is Paused/Resumed. You'll need to abort the old download and create a new download with an updated JSESSIONID header.
When your application is launched, it should be using BackgroundDownloader.GetCurrentDownloadsAsync() to query for all the DownloadOperations that might have been occurring in the background while your app was suspended/terminated. Your application should then call AttachAsync on each DownloadOperation in order to attach progress and completion handlers. In this case, you should implement logic in your completion handler that can identify this particular error case and create a new download (with the new JSESSIONID) for the same content.
As an aside, if you go searching for other folks that need to post-process failed downloads like this, you may run across some people who are performing these checks in Background Tasks that they register to run on a periodic timer. While this may sound like a good idea (since it means you could retry your download without waiting for the next time the user brings your application to the foreground), keep in mind that BackgroundTransfer may hang if you attempt to call AttachAsync in a Background Task for an operation which was started in your foreground application code.

Categories