Related
We have an application "App1" in Azure will all the required permissions and have 7 instances of a Windows service in C# that uses the same Azure Application "App1" to connect to Outlook Mailboxes. The code is same across all the 7 windows service instances with the difference in config i.e. each instance connecting to a unique Mailbox.
Question:
We do authentication using the code below and each service is running on its own Server(Windows), when this authentication code is executed for lets say all the services will there be issue with the token overlapping since it uses the same Azure "App1" or since its connecting to a unique Mailbox for each service instance it wont be a problem and have unique access token per service instance.
AuthenticationResult authResult = null;
GraphServiceClient graphClient = null;
IMailFolderMessagesCollectionPage inboxMessage = null;
try
{
string tenantId = "TenantId";
string clientId = "ClientId";
string clientSecret = "ClientSecret";
var authority = $"https://login.microsoftonline.com/{tenantId}";
var app = ConfidentialClientApplicationBuilder
.Create(clientId)
.WithClientSecret(clientSecret)
.WithAuthority(new Uri(authority))
.Build();
var scopes = new[] { "https://graph.microsoft.com/.default" };
authResult = await app.AcquireTokenForClient(scopes).ExecuteAsync();
}
catch (Exception ex)
{
//return;
}
if (authResult != null)
{
graphClient = new GraphServiceClient(
new DelegateAuthenticationProvider(requestMessage =>
{
requestMessage.Headers.Authorization =
new AuthenticationHeaderValue("bearer", authResult.AccessToken);
return Task.FromResult(0);
}));
We get below error frequently and want to know if this setup could be affecting our applications. If not what is the reason we are getting such errors.
Message: System.Threading.Tasks.TaskCanceledException: A task was canceled.
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.Graph.HttpProvider.<SendRequestAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.Graph.HttpProvider.<SendAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.Graph.BaseRequest.<SendRequestAsync>d__40.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.Graph.BaseRequest.<SendAsync>d__34`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at Microsoft.Graph.MessageAttachmentsCollectionRequest.<GetAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at GraphPOC.EmailMain.<GetDataAsync>d__63.MoveNext()
Category: Error
Priority: -1
EventId: 0
Severity: Error
Title:
App Domain: GraphPOC.exe
ProcessId: 988
Process Name: D:\Program Files\GraphPOC.exe
Thread Name:
Win32 ThreadId:4884
Extended Properties:
Callstack= at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
at System.Environment.get_StackTrace()
at System.Diagnostics.TraceEventCache.get_Callstack()
at System.Diagnostics.TraceListener.WriteFooter(TraceEventCache eventCache)
at Microsoft.Practices.EnterpriseLibrary.Logging.TraceListeners.FormattedTextWriterTraceListener.TraceData(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, Object data)
at Microsoft.Practices.EnterpriseLibrary.Logging.TraceListeners.FlatFileTraceListener.TraceData(TraceEventCache eventCache, String source, TraceEventType eventType, Int32 id, Object data)
at Microsoft.Practices.EnterpriseLibrary.Logging.LogSource.TraceData(TraceEventType eventType, Int32 id, LogEntry logEntry, TraceListenerFilter traceListenerFilter)
at Microsoft.Practices.EnterpriseLibrary.Logging.LogWriter.ProcessLog(LogEntry log)
at Microsoft.Practices.EnterpriseLibrary.Logging.LogWriter.Write(LogEntry log)
at GraphPOC.EmailMain.<GetDataAsync>d__63.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception)
at Microsoft.Graph.MessageAttachmentsCollectionRequest.<GetAsync>d__3.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception)
at Microsoft.Graph.BaseRequest.<SendAsync>d__34`1.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception)
at Microsoft.Graph.BaseRequest.<SendRequestAsync>d__40.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception)
at Microsoft.Graph.HttpProvider.<SendAsync>d__18.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetException(Exception exception)
at Microsoft.Graph.HttpProvider.<SendRequestAsync>d__19.MoveNext()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord, Object cancellationException)
at System.Threading.Tasks.Task`1.TrySetCanceled(CancellationToken tokenToRecord)
at System.Threading.Tasks.TaskCompletionSource`1.TrySetCanceled(CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SetTaskCanceled(HttpRequestMessage request, CancellationTokenSource cancellationTokenSource, TaskCompletionSource`1 tcs)
at System.Net.Http.HttpClient.<>c__DisplayClass59_0.<StartContentBuffering>b__0(Task contentTask)
at System.Threading.Tasks.Task.Execute()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)
at System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(Task task, Boolean taskWasPreviouslyQueued)
at System.Threading.Tasks.TaskScheduler.TryRunInline(Task task, Boolean taskWasPreviouslyQueued)
at System.Threading.Tasks.TaskContinuation.InlineIfPossibleOrElseQueue(Task task, Boolean needsProtection)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task.Finish(Boolean bUserDelegateExecuted)
at System.Threading.Tasks.Task`1.TrySetException(Object exceptionObject)
at System.Threading.Tasks.TaskCompletionSource`1.TrySetException(Exception exception)
at System.Net.Http.HttpContent.<>c__DisplayClass21_0.<LoadIntoBufferAsync>b__0(Task copyTask)
at System.Threading.Tasks.Task.Execute()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)
at System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(Task task, Boolean taskWasPreviouslyQueued)
at System.Threading.Tasks.TaskScheduler.TryRunInline(Task task, Boolean taskWasPreviouslyQueued)
at System.Threading.Tasks.TaskContinuation.InlineIfPossibleOrElseQueue(Task task, Boolean needsProtection)
at System.Threading.Tasks.Task.FinishContinuations()
at System.Threading.Tasks.Task.Finish(Boolean bUserDelegateExecuted)
at System.Threading.Tasks.Task`1.TrySetException(Object exceptionObject)
at System.Threading.Tasks.TaskCompletionSource`1.TrySetException(Exception exception)
at System.Net.Http.StreamToStreamCopy.BufferReadCallback(IAsyncResult ar)
at System.IO.Compression.DeflateStreamAsyncResult.Complete(Object result)
at System.IO.Compression.DeflateStream.ReadCallback(IAsyncResult baseStreamResult)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.ChunkParser.ReadCallback(IAsyncResult ar)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.FixedSizeReader.ReadCallback(IAsyncResult transportResult)
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.ContextAwareResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
Basically, when we use client credential flow to generate access token and use it to call the graph api, let's assume your "App1" had plenty of application api perissions like User.ReadWrite.All,Mail.ReadBasic.All,Files.ReadWrite.All, then the token will contain all these api permissions so that it can be used to search for users/mails/files. As you can see, this token will contain such a big permission.
Then go back to your scenario, you don't need to worry about the token overlapping because token just represent the authorization, since it contained correct api permission, then the token generated by each of your application can be used in other application because they all had the same configuration. In other words they are all valid.
About your exception, I'm not sure which line reported it, you may try to debug you code to find the problematic code. and I'm afraid it may relate to return Task.FromResult(0);. I searched for the error message and here's a question met the same error. Anyway it seems to be an unexpected task issue.
Since you are now using the graph client, I'm afraid you can call graph api directly without obtaining the access token first. Like code below:
var scopes = new[] { "https://graph.microsoft.com/.default" };
string tenantId = "TenantId";
string clientId = "ClientId";
string clientSecret = "ClientSecret";
var clientSecretCredential = new ClientSecretCredential(
tenantId, clientId, clientSecret);
var graphClient = new GraphServiceClient(clientSecretCredential, scopes);
var user = await graphClient.Users.Request().GetAsync();
My Server (System.Net) is working on my local mashine (win10), but not on the ubuntu server, i get this error:
Unhandled exception. System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'size')
at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
at Server.Client.Start() in C:\Users\xxxx\source\repos\Server\Server\Client.cs:line 28
at Server.ClientManager.CreateNewConnection(TcpClient tempClient) in C:\Users\xxxx\source\repos\Server\Server\ClientManager.cs:line 19
at Server.Server.OnClientConnect(IAsyncResult result) in C:\Users\xxxx\source\repos\Server\Server\Server.cs:line 61
at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
at System.Net.ContextAwareResult.CompleteCallback()
at System.Net.ContextAwareResult.<>c.<Complete>b__15_0(Object s)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.ContextAwareResult.Complete(IntPtr userToken)
at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)
at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionCallback(Int32 numBytes, SocketError errorCode)
at System.Net.Sockets.AcceptOverlappedAsyncResult.CompletionCallback(IntPtr acceptedFileDescriptor, Byte[] socketAddress, Int32 socketAddressLen, SocketError errorCode)
at System.Net.Sockets.SocketAsyncContext.AcceptOperation.InvokeCallback(Boolean allowPooling)
at System.Net.Sockets.SocketAsyncContext.OperationQueue`1.ProcessAsyncOperation(TOperation op)
at System.Net.Sockets.SocketAsyncContext.ReadOperation.System.Threading.IThreadPoolWorkItem.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
Aborted (core dumped)
this is the code thats causing the error (its working on my local):
socket.SendBufferSize = 4096;
socket.ReceiveBufferSize = 4096;
stream = socket.GetStream();
recBuffer = new byte[4096];
stream.BeginRead(recBuffer, 0, socket.ReceiveBufferSize, OnReceiveData, null);
Console.WriteLine($"Connection from {socket.Client.RemoteEndPoint.ToString()}");
when i listen with netcat to the port im using (55555) and connect in the game, the connection works, so i think the port/firewall is fine.
i tried exporting it as a linux build, and as a windows build, which i executed via wine, both didn't work.
this is how i published it:
publish
I'm encountering a deadlock when using asynchronous implementation of an EF Core provider.
Say I have the following models:
public class Player
{
public string PlayerId { get; set;}
public string Name { get; set;}
public List<Game> Games { get; set;}
}
public class Game
{
public string GameId { get; set; }
public string PlayerId { get; set; }
public Player Player { get; set;}
}
Now I want to run the following query:
ctx.Players
.Include(p => p.Games)
.Where(p => p.PlayerId == "123")
.Select(p => new {
PlayerId = p.PlayerId,
Games = p.Games.ToList()
}).ToListAsync();
When I run this code via Console Application/XUnit test it works as excepted...
But when I run it via ASP.Net WebApi it get into deadlock and never ends...
I used ConfigureAwait(false) all the way down in order to prevent these kind of situations but it seems that the problematic code is down underneath.
I think that it might be under the System.Interactive.Async library which EFCore use - To be more specific it is under: https://github.com/Reactive-Extensions/Rx.NET/blob/develop/Ix.NET/Source/System.Interactive.Async/ToAsyncEnumerable.cs#L72
there is a call to "Result" which actually blocks the execution thread.
Does anyone encountered this behavior, maybe there is some workaround?
Notice that if I don't load the "Games" entities then everything also works fine...
Edit: Added the StackTrace: (See the call to ToEnumerable)
Not Flagged 21672 5 Worker Thread grpc 0 (cq 0) System.Interactive.Async.dll!System.Linq.AsyncEnumerable.**ToEnumerable_**<NG.Admin.BL.DAL.Entities.Game> Normal
mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout, bool exitContext)
mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout)
mscorlib.dll!System.Threading.ManualResetEventSlim.Wait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)
mscorlib.dll!System.Threading.Tasks.Task.SpinThenBlockingWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)
mscorlib.dll!System.Threading.Tasks.Task.InternalWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)
mscorlib.dll!System.Threading.Tasks.Task.GetResultCore(bool waitCompletionNotification)
mscorlib.dll!System.Threading.Tasks.Task.Result.get()
System.Interactive.Async.dll!System.Linq.AsyncEnumerable.ToEnumerable_<NG.Admin.BL.DAL.Entities.Game>(System.Collections.Generic.IAsyncEnumerable<NG.Admin.BL.DAL.Entities.Game> source)
mscorlib.dll!System.Collections.Generic.List<NG.Admin.BL.DAL.Entities.Game>.List(System.Collections.Generic.IEnumerable<NG.Admin.BL.DAL.Entities.Game> collection)
System.Core.dll!System.Linq.Enumerable.ToList<NG.Admin.BL.DAL.Entities.Game>(System.Collections.Generic.IEnumerable<NG.Admin.BL.DAL.Entities.Game> source)
[Lightweight Function]
Microsoft.EntityFrameworkCore.dll!Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.SelectAsyncEnumerable<Microsoft.EntityFrameworkCore.Storage.ValueBuffer, NG.Admin.BL.Api.Player>.SelectAsyncEnumerator.MoveNext(System.Threading.CancellationToken cancellationToken)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task.TrySetResult(bool result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult(bool result)
NG.Data.Spanner.EF.dll!NG.Data.Spanner.EF.Query.Internal.SpannerAsyncQueryingEnumerable<Microsoft.EntityFrameworkCore.Storage.ValueBuffer>.SpannerAsyncEnumerator.MoveNext(System.Threading.CancellationToken cancellationToken) Line 55
[Resuming Async Method]
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task.TrySetResult(bool result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult(bool result)
Microsoft.EntityFrameworkCore.dll!Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.SelectAsyncEnumerable<Microsoft.EntityFrameworkCore.Storage.ValueBuffer, Microsoft.EntityFrameworkCore.Storage.ValueBuffer>.SelectAsyncEnumerator.MoveNext(System.Threading.CancellationToken cancellationToken)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task.TrySetResult(bool result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult(bool result)
Microsoft.EntityFrameworkCore.Relational.dll!Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable.AsyncEnumerator.MoveNext(System.Threading.CancellationToken cancellationToken)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task.TrySetResult(bool result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult(bool result)
Microsoft.EntityFrameworkCore.Relational.dll!Microsoft.EntityFrameworkCore.Query.Internal.AsyncQueryingEnumerable.AsyncEnumerator.BufferlessMoveNext(bool buffer, System.Threading.CancellationToken cancellationToken)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Threading.Tasks.TaskCompletionSource<System.__Canon>.TrySetResult(System.__Canon result)
Microsoft.EntityFrameworkCore.Relational.dll!System.Threading.Tasks.TaskExtensions.Cast.AnonymousMethod__0(System.Threading.Tasks.Task t)
mscorlib.dll!System.Threading.Tasks.ContinuationTaskFromResultTask.InnerInvoke()
mscorlib.dll!System.Threading.Tasks.Task.Execute()
mscorlib.dll!System.Threading.Tasks.Task.ExecutionContextCallback(object obj)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref System.Threading.Tasks.Task currentTaskSlot)
mscorlib.dll!System.Threading.Tasks.Task.ExecuteEntry(bool bPreventDoubleExecution)
mscorlib.dll!System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(System.Threading.Tasks.Task task, bool taskWasPreviouslyQueued)
mscorlib.dll!System.Threading.Tasks.TaskScheduler.TryRunInline(System.Threading.Tasks.Task task, bool taskWasPreviouslyQueued)
mscorlib.dll!System.Threading.Tasks.TaskContinuation.InlineIfPossibleOrElseQueue(System.Threading.Tasks.Task task, bool needsProtection)
mscorlib.dll!System.Threading.Tasks.StandardTaskContinuation.Run(System.Threading.Tasks.Task completedTask, bool bCanInlineContinuationTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult(object result)
NG.Data.Spanner.EF.dll!NG.Data.Spanner.EF.Storage.Internal.SpannerRelationalCommand.ExecuteAsync(Microsoft.EntityFrameworkCore.Storage.IRelationalConnection connection, string executeMethod, System.Collections.Generic.IReadOnlyDictionary<string, object> parameterValues, bool closeConnection, System.Threading.CancellationToken cancellationToken) Line 41
[Resuming Async Method]
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult(object result)
NG.Data.Spanner.EF.dll!NG.Data.Spanner.EF.Storage.Internal.SpannerRelationalCommand.ExecuteAsync(NG.Data.Spanner.EF.Storage.Internal.IOBehavior ioBehavior, Microsoft.EntityFrameworkCore.Storage.IRelationalConnection connection, string executeMethod, System.Collections.Generic.IReadOnlyDictionary<string, object> parameterValues, bool closeConnection, System.Threading.CancellationToken cancellationToken) Line 128
[Resuming Async Method]
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<System.Data.Common.DbDataReader>.SetResult(System.Data.Common.DbDataReader result)
NG.Data.Spanner.dll!NG.Data.Spanner.SpannerCommand.ExecuteDbDataReaderAsync(System.Data.CommandBehavior behavior, System.Threading.CancellationToken cancellationToken) Line 67
[Resuming Async Method]
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<Google.Cloud.Spanner.V1.ResultSet>.SetResult(Google.Cloud.Spanner.V1.ResultSet result)
NG.Data.Spanner.dll!NG.Data.Spanner.SpannerConnection.RunQuery(string commandText) Line 118
[Resuming Async Method]
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<Google.Cloud.Spanner.V1.ResultSet>.SetResult(Google.Cloud.Spanner.V1.ResultSet result)
Google.Api.Gax.Grpc.dll!Google.Api.Gax.Grpc.ApiCallRetryExtensions.WithRetry.AnonymousMethod__0(Google.Cloud.Spanner.V1.ExecuteSqlRequest request, Google.Api.Gax.Grpc.CallSettings callSettings)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.InvokeMoveNext(object stateMachine)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.MoveNextRunner.Run()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.OutputAsyncCausalityEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__0()
mscorlib.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore.ContinuationWrapper.Invoke()
mscorlib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining, ref System.Threading.Tasks.Task currentTask)
mscorlib.dll!System.Threading.Tasks.Task.FinishContinuations()
mscorlib.dll!System.Threading.Tasks.Task.FinishStageThree()
mscorlib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Threading.Tasks.TaskCompletionSource<System.__Canon>.TrySetResult(System.__Canon result)
mscorlib.dll!System.Threading.Tasks.TaskCompletionSource<System.__Canon>.SetResult(System.__Canon result)
Grpc.Core.dll!Grpc.Core.Internal.AsyncCall<Google.Cloud.Spanner.V1.ExecuteSqlRequest, Google.Cloud.Spanner.V1.ResultSet>.HandleUnaryResponse(bool success, Grpc.Core.Internal.ClientSideStatus receivedStatus, byte[] receivedMessage, Grpc.Core.Metadata responseHeaders)
Grpc.Core.dll!Grpc.Core.Internal.CallSafeHandle.StartUnary.AnonymousMethod__0(bool success, Grpc.Core.Internal.BatchContextSafeHandle context)
Grpc.Core.dll!Grpc.Core.Internal.CompletionRegistry.HandleBatchCompletion(bool success, Grpc.Core.Internal.BatchContextSafeHandle ctx, Grpc.Core.Internal.BatchCompletionDelegate callback)
Grpc.Core.dll!Grpc.Core.Internal.CompletionRegistry.RegisterBatchCompletion.AnonymousMethod__0(bool success)
Grpc.Core.dll!Grpc.Core.Internal.GrpcThreadPool.RunHandlerLoop(Grpc.Core.Internal.CompletionQueueSafeHandle cq, Grpc.Core.Profiling.IProfiler optionalProfiler)
Grpc.Core.dll!Grpc.Core.Internal.GrpcThreadPool.CreateAndStartThread.AnonymousMethod__0()
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state)
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart()
So after a very long debugging I found the problem and did some work around to solve it. I'll try to explain:
I'm building EF Core provider to the Google Spanner db. For this I'm using Google Spanner DotNet API - although it is not yet public and very unmature API (actually it is just auto-generated code).
This API uses the gRPC API and what I saw is that on the first async call with this API, the gRPC thread (The gRPC has its own thread pool implementation) is used and will be used for the next execution calls - since the continuations will run only on this thread from the gRPC thread pool.
EFCore calls the ToEnumerable for the "Games" list which use the System.Interactive.Async lib. On the the ToEnumerable implementation it actually calls "Result" which blocks the execution thread and waits for the result. Because the gRPC thread is used and wait for the execution to be finished we have a deadlock...
My workaround was to use the following code:
GrpcEnvironment.SetCompletionQueueCount(1);
This way the continuations is allowed to run on other threads from the gRPC thread pool.
This is only a workaround and should be used. However it help solving the issue in the meantime...
The real solution should probably be in the EFCore usage of the System.Interactive.Async lib, in regarding to the ToEnumerable... (It should be pure async).
Edit:
If it might be relevant to anyone:
Just released an EF Core provider for the Google Spanner db:
https://github.com/NoGame/NG.Data.Spanner
Your issue is that you are attempting to materialize Games synchronously and the projected object asynchronously. Note the following:
.Select(p => new {
PlayerId = p.PlayerId,
Games = p.Games.ToList() // sync over async is blocking
}).ToListAsync();
What you would expect to work (in regular EF for instance) is:
.Select(p => new {
PlayerId = p.PlayerId,
Games = p.Games // let the provider handle the enumeration materialization
}).ToListAsync();
But according to your comments that doesn't actually return the games on the anonymous projected object. The issue, it seems then, is that EF Core seems to not let you get away with naive assignment (you should update the question to indicate that). I wonder if this isn't related to how .Include() works in EF Core. I am not familiar enough with the core version of EF to offer much help beyond telling you the "cause," but essentially the issue is that you are waiting to get an enumerator from a source that is waiting to materialize based on that enumeration.
I have a WCF service(selfhosted) and client, the client uses client certificate and the service uses a service certificate over HTTPS. This works fine on computer but the other throws this :
System.ServiceModel.Security.MessageSecurityException: The HTTP request was forbidden with client authentication scheme 'Anonymous'. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult result)
--- End of inner exception stack trace ---
at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.<>c__DisplayClass7_0`1.<CreateGenericTask>b__0(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at Myapp.Client.Main.ServiceManagement.ServiceAgents.General.<GetUserConfigurations>d__5.MoveNext() in C:\Project\Myapp\Produkter\MyappUtveckling\Solution\Myapp.Client.Main\Classes\Service Management\Service Agents\General.cs:line 172
In the WCF log I can see this :
Configuration evaluation context not found.
http://msdn.microsoft.com/sv-SE/library/System.ServiceModel.Channels.HttpChannelUnexpectedResponse.aspx
Received bad HTTP response
Myapp.vshost.exe
System.ServiceModel.Security.MessageSecurityException,
System.ServiceModel, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089 The HTTP
request was forbidden with client authentication scheme
'Anonymous'. at
System.ServiceModel.Channels.HttpChannelUtilities.TraceResponseException(Exception
exception) at
System.ServiceModel.Channels.HttpChannelUtilities.ValidateAuthentication(HttpWebRequest
request, HttpWebResponse response, WebException responseException,
HttpChannelFactory1 factory) at
System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest
request, HttpWebResponse response, HttpChannelFactory1 factory,
WebException responseException, ChannelBinding channelBinding) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.ProcessResponse(HttpWebResponse
response, WebException responseException) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.OnGetResponse(IAsyncResult
result) at
System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult
result) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at
System.Net.ContextAwareResult.CompleteCallback(Object state) at
System.Threading.ExecutionContext.RunInternal(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state) at
System.Net.ContextAwareResult.Complete(IntPtr userToken) at
System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result,
IntPtr userToken) at System.Net.HttpWebRequest.ProcessResponse() at
System.Net.HttpWebRequest.SetResponse(CoreResponseData
coreResponseData) at
System.Net.HttpWebRequest.SetAndOrProcessResponse(Object
responseOrException) at
System.Net.ConnectionReturnResult.SetResponses(ConnectionReturnResult
returnResult) at System.Net.Connection.ReadComplete(Int32 bytesRead,
WebExceptionStatus errorStatus) at
System.Net.Connection.ReadCallback(IAsyncResult asyncResult) at
System.Net.Connection.ReadCallbackWrapper(IAsyncResult asyncResult) at
System.Net.LazyAsyncResult.Complete(IntPtr userToken) at
System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result,
IntPtr userToken) at
System.Net.Security._SslStream.ProcessFrameBody(Int32 readBytes,
Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest
asyncRequest) at
System.Net.Security._SslStream.ReadFrameCallback(AsyncProtocolRequest
asyncRequest) at System.Net.AsyncProtocolRequest.CompleteRequest(Int32
result) at
System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32 bytes)
at System.Net.FixedSizeReader.ReadCallback(IAsyncResult
transportResult) at System.Net.LazyAsyncResult.Complete(IntPtr
userToken) at System.Net.ContextAwareResult.CompleteCallback(Object
state) at
System.Threading.ExecutionContext.RunInternal(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state) at
System.Net.ContextAwareResult.Complete(IntPtr userToken) at
System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result,
IntPtr userToken) at
System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32
errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped) at
System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32
errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP) </StackTrace>
<ExceptionString>System.ServiceModel.Security.MessageSecurityException:
The HTTP request was forbidden with client authentication scheme
'Anonymous'. ---> System.Net.WebException: The remote server
returned an error: (403) Forbidden. at
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result) --- End of inner exception stack trace
--- System.Net.WebException, System, Version=4.0.0.0,
Culture=neutral, PublicKeyToken=b77a5c561934e089
The remote server returned an error: (403)
Forbidden. at
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result) </StackTrace> <ExceptionString>System.Net.WebException: The
remote server returned an error: (403) Forbidden. at
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result)
http://msdn.microsoft.com/sv-SE/library/System.ServiceModel.Diagnostics.ThrowingException.aspx
Throwing an exception.
Myapp.vshost.exe
System.ServiceModel.Security.MessageSecurityException,
System.ServiceModel, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089 The HTTP
request was forbidden with client authentication scheme
'Anonymous'. at
System.ServiceModel.Channels.HttpChannelUtilities.ValidateAuthentication(HttpWebRequest
request, HttpWebResponse response, WebException responseException,
HttpChannelFactory1 factory) at
System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest
request, HttpWebResponse response, HttpChannelFactory1 factory,
WebException responseException, ChannelBinding channelBinding) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.ProcessResponse(HttpWebResponse
response, WebException responseException) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.OnGetResponse(IAsyncResult
result) at
System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult
result) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at
System.Net.ContextAwareResult.CompleteCallback(Object state) at
System.Threading.ExecutionContext.RunInternal(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state) at
System.Net.ContextAwareResult.Complete(IntPtr userToken) at
System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result,
IntPtr userToken) at System.Net.HttpWebRequest.ProcessResponse() at
System.Net.HttpWebRequest.SetResponse(CoreResponseData
coreResponseData) at
System.Net.HttpWebRequest.SetAndOrProcessResponse(Object
responseOrException) at
System.Net.ConnectionReturnResult.SetResponses(ConnectionReturnResult
returnResult) at System.Net.Connection.ReadComplete(Int32 bytesRead,
WebExceptionStatus errorStatus) at
System.Net.Connection.ReadCallback(IAsyncResult asyncResult) at
System.Net.Connection.ReadCallbackWrapper(IAsyncResult asyncResult) at
System.Net.LazyAsyncResult.Complete(IntPtr userToken) at
System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result,
IntPtr userToken) at
System.Net.Security._SslStream.ProcessFrameBody(Int32 readBytes,
Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest
asyncRequest) at
System.Net.Security._SslStream.ReadFrameCallback(AsyncProtocolRequest
asyncRequest) at System.Net.AsyncProtocolRequest.CompleteRequest(Int32
result) at
System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32 bytes)
at System.Net.FixedSizeReader.ReadCallback(IAsyncResult
transportResult) at System.Net.LazyAsyncResult.Complete(IntPtr
userToken) at System.Net.ContextAwareResult.CompleteCallback(Object
state) at
System.Threading.ExecutionContext.RunInternal(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state, Boolean
preserveSyncCtx) at
System.Threading.ExecutionContext.Run(ExecutionContext
executionContext, ContextCallback callback, Object state) at
System.Net.ContextAwareResult.Complete(IntPtr userToken) at
System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result,
IntPtr userToken) at
System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32
errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped) at
System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32
errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP) </StackTrace>
<ExceptionString>System.ServiceModel.Security.MessageSecurityException:
The HTTP request was forbidden with client authentication scheme
'Anonymous'. ---> System.Net.WebException: The remote server
returned an error: (403) Forbidden. at
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result) --- End of inner exception stack trace
--- System.Net.WebException, System, Version=4.0.0.0,
Culture=neutral, PublicKeyToken=b77a5c561934e089
The remote server returned an error: (403)
Forbidden. at
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result) </StackTrace> <ExceptionString>System.Net.WebException: The
remote server returned an error: (403) Forbidden. at
System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at
System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult
result)
My first thougt was certificates but both computers matches.
Im creating the channel on the client like this :
private async Task<ChannelFactory<T>> CreateChannelFactory(LoginTypeBase loginType, MyappToken token)
{
var service = await _ConsulService.GetServiceBlocking(loginType.MyappServicesToUse, forceRefresh: true, token: new CancellationTokenSource(TimeSpan.FromSeconds(30)).Token);
if (service == null)
throw new MyappServiceCommunicationException();
var cert = loginType.ClientCertificate;
var uri = loginType.GetMyappClientServiceURL(service.Address, service.Port);
var header = AddressHeader.CreateAddressHeader(nameof(MyappToken), nameof(MyappToken), token);
var endpointAddress = new EndpointAddress(uri, header);
ServiceEndpoint serviceEndpoint = null;
if (loginType.LoginType == LoginType.SmartCard || loginType.LoginType == LoginType.UsernamePasswordSLL)
{
var binding = new NetHttpsBinding("netHttpsBinding");
binding.Security.Mode = BasicHttpsSecurityMode.Transport;
if (loginType.LoginType == LoginType.SmartCard)
binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Certificate;
else
binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.None;
serviceEndpoint = new ServiceEndpoint(ContractDescription.GetContract(typeof(T)), binding, endpointAddress);
}
else
{
var binding = new NetHttpBinding("netHttpBinding");
serviceEndpoint = new ServiceEndpoint(ContractDescription.GetContract(typeof(T)), binding, endpointAddress);
}
serviceEndpoint.EndpointBehaviors.Add(new ProtoEndpointBehavior());
serviceEndpoint.EndpointBehaviors.Add(new CustomMessageInspectorBehavior());
var v = new ChannelFactory<T>(serviceEndpoint);
if (loginType.LoginType == LoginType.SmartCard)
{
v.Credentials.ClientCertificate.Certificate = cert;
//v.Credentials.ClientCertificate.SetCertificate(StoreLocation.CurrentUser, StoreName.My, X509FindType.FindByThumbprint, cert.Thumbprint);
}
return v;
}
The config contains extensions, behaviours and bindings(netHttpsBinding).
The Service is created from code only, nothing from its config file.
Why does it not work on the other computer?
How is it possible to know which piece of code could cause the exception 'SPException was unhandled'?
CODE:
SPList mylist = web.Lists.TryGetList("mylist");
SPListItem item = mylist.Items.Add();
... ... ...
item.Update();
The trace is:
Stack details:
at Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)
at Microsoft.SharePoint.Library.SPRequest.AddOrUpdateItem(String bstrUrl, String bstrListName, Boolean bAdd, Boolean bSystemUpdate, Boolean bPreserveItemVersion, Boolean bPreserveItemUIVersion, Boolean bUpdateNoVersion, Int32& plID, String& pbstrGuid, Guid pbstrNewDocId, Boolean bHasNewDocId, String bstrVersion, Object& pvarAttachmentNames, Object& pvarAttachmentContents, Object& pvarProperties, Boolean bCheckOut, Boolean bCheckin, Boolean bUnRestrictedUpdateInProgress, Boolean bMigration, Boolean bPublish, String bstrFileName, ISP2DSafeArrayWriter pListDataValidationCallback, ISP2DSafeArrayWriter pRestrictInsertCallback, ISP2DSafeArrayWriter pUniqueFieldCallback)
at Microsoft.SharePoint.SPListItem.AddOrUpdateItem(Boolean bAdd, Boolean bSystem, Boolean bPreserveItemVersion, Boolean bNoVersion, Boolean bMigration, Boolean bPublish, Boolean bCheckOut, Boolean bCheckin, Guid newGuidOnAdd, Int32& ulID, Object& objAttachmentNames, Object& objAttachmentContents, Boolean suppressAfterEvents, String filename, Boolean bPreserveItemUIVersion)
at Microsoft.SharePoint.SPListItem.UpdateInternal(Boolean bSystem, Boolean bPreserveItemVersion, Guid newGuidOnAdd, Boolean bMigration, Boolean bPublish, Boolean bNoVersion, Boolean bCheckOut, Boolean bCheckin, Boolean suppressAfterEvents, String filename, Boolean bPreserveItemUIVersion)
at Microsoft.SharePoint.SPListItem.Update()
at Dumper.Program.Main(String[] args) in c:\users\svcspfarm\documents\visual studio 2010\Projects\Dumper\Dumper\Program.cs:line 364
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
Look the sharepoint ULS logs to find out more information about exception.
\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS
For me it looks like you have not setup values for the required list fields.
Have a look at these lines from stack trace:
at Microsoft.SharePoint.SPListItem.Update()
at Dumper.Program.Main(String[] args) in c:\users\svcspfarm\documents\visual studio 2010\Projects\Dumper\Dumper\Program.cs:line 364
The exception is occurring on the call to item.Update();. I had a similar problem when I was attempting to add a subfolder to a list and it can be less than obvious what is causing the problem. I would hazard a guess that somewhere between your call to mylist.Items.Add(); and the Update() that something has not been set correctly. A new list item may have some required fields that need to be initialised prior to the update?