I have a problem running a WCF service that connects to the CRM: It frequently produces CommunicationObjectAbortedExceptions which leave me to wonder if I am doing something wrong. These execptions started occurring after many people started using it, on the test system it worked without problems.
But let's start at the beginning: I wrote two WCF Services that connect to the Microsoft CRM2013 Organization Service using my own library to execute queries on the CRM. These services are regularly called from the CRM which is used by roughly 100-200 people on a daily basis.
This works basically fine, but I frequently get a couple of exceptions which look like the following (see bottom of the Post for the full stacktrace):
System.ServiceModel.CommunicationObjectAbortedException: The HTTP request to 'http://crm/MyOrganization/XRMServices/2011/Organization.svc' was aborted. This may be due to the local channel being closed while the request was still in progress. If this behavior is not desired, then update your code so that it does not close the channel while request operations are still in progress.
By frequently I mean around 100 times a day, most often a couple of those exceptions are thrown every 5-30 minutes in batches of 3-6 exceptions. I have no idea why this is happening. I initialize the connection to the CRM Organization Service using the following class from my library in both services:
public class CrmManager : IDisposable
{
private static CrmConnection s_connection;
public static CrmConnection Connection
{
get
{
if (s_connection == null)
{
s_connection = new CrmConnection("CrmTvTest");
}
return s_connection;
}
}
public static IOrganizationService ServiceProxy
{
get { return s_serviceProxy ?? (s_serviceProxy = new CachedOrganizationService(Connection)); }
}
As can be seen, I connect to the Organization Service once per WCF service, using the CrmConnection to handle the connection details, which is stored in a static variable (acting as a singleton, since establishing the Connection is expensive and should not be done too often to my understanding). It is then passed to the CachedOrganizationService, which is static for the same reasons. The WCF service uses the default instance management (PerSession AFAIK), meaning there is probably 1 connection and organization service per user.
My Connection String looks like this (removed any sensible data, of course):
<connectionStrings>
<add name="CrmTvTest" connectionString="Url=http://crm/MyOrganization; Username=user; Password=pw;"/>
I then use the connection with CrmServiceContext objects to execute queries using this method from my CrmManager class. Which is, of course, always called within a using-statement:
using (CrmServiceContext context = new CrmServiceContext(CrmManager.ServiceProxy))
{
// do some stuff...
}
How can I prevent these exceptions from constantly occuring? I get the feeling this has to do with the Security Tokens used by the CRM connection expiring, but this shouldn't be a problem when I use the CrmConnection class. It should refresh them automatically.
Any advice would be very welcome, since I am pondering this issue for a while now.
UPDATE 1
I switched to using the Developer Extensions and using the CrmConnector class, to no avail (I updated the code above). I also tried passing the CrmConnection class directly to the CrmServiceContext:
using (CrmServiceContext context = new CrmServiceContext(CrmManager.Connection))
which led to the same problems as in this Stackoverflow Question, without using a load-balancer (we initially did, but disabled load-balancing to eliminate the possibility of it causing the problems.
Full Stacktrace:
---> System.Net.WebException: The request was aborted: The request was canceled.
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
at System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.SecurityChannelFactory1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.Xrm.Sdk.IOrganizationService.Execute(OrganizationRequest request)
at Microsoft.Xrm.Sdk.Client.OrganizationServiceProxy.ExecuteCore(OrganizationRequest request)
at Microsoft.Xrm.Sdk.Client.OrganizationServiceContext.Execute(OrganizationRequest request)
at Microsoft.Xrm.Sdk.Linq.QueryProvider.RetrieveEntityCollection(OrganizationRequest request, NavigationSource source)
at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute(QueryExpression qe, Boolean throwIfSequenceIsEmpty, Boolean throwIfSequenceNotSingle, Projection projection, NavigationSource source, List1 linkLookups, String& pagingCookie, Boolean& moreRecords)
at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute[TElement](QueryExpression qe, Boolean throwIfSequenceIsEmpty, Boolean throwIfSequenceNotSingle, Projection projection, NavigationSource source, List1 linkLookups)
at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute[TElement](Expression expression)
at Microsoft.Xrm.Sdk.Linq.QueryProvider.System.Linq.IQueryProvider.Execute[TResult](Expression expression)
at System.Linq.Queryable.SingleOrDefault[TSource](IQueryable`1 source)
at CrmConnector.Entities.Contact.Get(Guid p_id, Boolean p_includeRelatedEntities) in j:\IntDev\Libraries\CrmConnector\Entities\Contact.cs:line 63
at CrmExtensionService.CrmExtension.GetPersonalizedEmailSignature(String p_contactId, String p_systemUserId) in j:\IntDev\Services\CrmExtensionService\CrmExtension.svc.cs:line 460
We had same issue and load balancer was the culprit. Now we resolved this error with load balancer activated by making few configurations in the load balancer. We enabled sticky session with least connection algorithm at the load balancer. So if this is not enabled while load balanced then authenticated connection from one server's request gets routed to different servers even though same session and fails. Once enabling the sticky session (session persistance to be client ip) requests goes to same server (in this case returning connection and not a new connection) it works well.
So, after fiddling around for about two months, we found the issue: The load balancing of the CRM FrontEnd was the culprit. I assumed this was disabled, too, with disabling the load balancing of our CRM Service, but it wasn't. Our CRM Service periodically established a connection with the Organization Service of Server 1, then got switched to Server 2 mid-operation and these exceptions occurred.
We're still trying to figure out how to get this to work with load balancing activated, but for the time being we keep it disabled to prevent these errors from popping up.
There is a similar case here on StackOverflow: Sporadic exceptions calling a web service that is load balanced. We are currently using a webHttpBinding and a quickly attempted to switch over to a basicHttpBinding but didn't get it to work (but as I said, this was just a quick attempt).
Related
I am currently trying with Blazor server to get a list. But when the list is called via the service I just get an error:
Error: System.InvalidOperationException: An exception has been raised that is likely due to a transient failure. Consider enabling transient error resiliency by adding 'EnableRetryOnFailure()' to the 'UseMySql' call.
---> MySqlConnector.MySqlException (0x80004005): Unable to connect to any of the specified MySQL hosts.
at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int32 startTickCount, ILoadBalancer loadBalancer, IOBehavior ioBehavior, CancellationToken cancellationToken) in //src/MySqlConnector/Core/ServerSession.cs:line 433
at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, String logMessage, Int32 startTickCount, IOBehavior ioBehavior, CancellationToken cancellationToken) in //src/MySqlConnector/Core/ConnectionPool.cs:line 363
i am currently calling in the Program.cs:
builder.Services.AddScoped<Organization>().AddDbContext<OrganizationDbContext>().AddEntityFrameworkMySql();
then i try to inject the Organization.
in the Organization there is:
public Organization()
{
this.organizationDBContext = new OrganizationDbContext();
}
so the funny part about all that is in the unit test the external project works just perfect(also if i call new Organization in the OnInitializedAsync sometime it works sometime it doesn't idk why).
My unit test which works perfectly fine:
Organization org = new Organization();
var customers = org.Customers.GetCustomers().ToList();
i get a list with the customers, how should i do this in blazor server?
Thanks!
For all of you who have the same error, this is the right way, keep an eye on your docker environment^^
I have an API with a single endpoint that retrieves documents from a CosmosDB collection. The endpoint works fine on common scenarios. However, when I execute stress tests on the API, to measure how it responds under heavy load, I experience outages on the site (site starts to respond requests with 502 - bad gateway).
Searching on Application Insights, I notice OutOfMemory exceptions raising while executing the sentence to retrieve the documents from the CosmosDB collection. The method that I'm using to read the documents is ReadNextAsync and the logs points this line specifically.
We read and tested the best practices and tips that the Cosmos DB documentation mentions to discard a bad usage of the SDK from our side, but even trying with different configurations (MaxItemCount, MaxBufferedItems, MaxConcurrency), the issue persisted.
After executing several tests, I noticed that if I limit the amount of documents to retrieve from the collection (e.g. using a TOP 40 clause), the exceptions or site outages don’t show. Instead, all requests are processed successfully with 200 status code. As I haven’t had these kind of issues on our Full FWK API, which has the exactly same logic as the .NET Core API described here, I'm wondering whether I could be doing a bad usage of the .NET Core SDK.
In order to share more context, I detailed below the general specifications and also the details on how I configured CosmosDB, along with the implementation to retrieve documents. Additionally, I included logs and a related stack trace found on Application Insights exceptions table.
General Specifications
API .NET CORE 2.2
Microsoft.Azure.Cosmos 3.5.0
Cosmos DB specifications
CosmosDB client connection
Connection mode: Direct
Application Region: West US
Default values for the rest
CosmosDB target collection
~600 documents
~30K size each document
PartitionKey -> id (one logical partition per document)
Write region -> West US
Read regions -> West Us, West Europe, Southeast Asia and Brazil South
Stress scenario details
Execute 400 request per second looking for retrieving up to 200 documents per request.
Document retrieving implementation
var feed = container.GetItemLinqQueryable<T>(false, null, queryRequestOptions).Where(predicate).ToFeedIterator();
var batches = new List<FeedResponse<T>>();
while (feed.HasMoreResults)
{
var batch = await feed.ReadNextAsync();
batches.Add(batch);
}
Application Insights exception stack trace
Response status code does not indicate success: 500 Substatus: 0 Reason: (System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Collections.Generic.List`1.set_Capacity(Int32 value)
at System.Collections.Generic.List`1.EnsureCapacity(Int32 min)
at System.Collections.Generic.List`1.AddWithResize(T item)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseObjectNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParsePropertyNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseObjectNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseArrayNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParsePropertyNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseObjectNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.ParseNode(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator.Parser.Parse(IJsonReader jsonTextReader)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.JsonTextNavigator..ctor(ReadOnlyMemory`1 buffer, Boolean skipValidation)
at Microsoft.Azure.Cosmos.Json.JsonNavigator.Create(ReadOnlyMemory`1 buffer, JsonStringDictionary jsonStringDictionary, Boolean skipValidation)
at Microsoft.Azure.Cosmos.CosmosElements.CosmosElementSerializer.ToCosmosElements(MemoryStream memoryStream, ResourceType resourceType, CosmosSerializationFormatOptions cosmosSerializationOptions)
at Microsoft.Azure.Cosmos.CosmosQueryClientCore.GetCosmosElementResponse(QueryRequestOptions requestOptions, ResourceType resourceType, ResponseMessage cosmosResponseMessage, PartitionKeyRangeIdentity partitionKeyRangeIdentity, SchedulingStopwatch schedulingStopwatch)
at Microsoft.Azure.Cosmos.CosmosQueryClientCore.ExecuteItemQueryAsync[RequestOptionType](Uri resourceUri, ResourceType resourceType, OperationType operationType, RequestOptionType requestOptions, SqlQuerySpec sqlQuerySpec, String continuationToken, PartitionKeyRangeIdentity partitionKeyRange, Boolean isContinuationExpected, Int32 pageSize, SchedulingStopwatch schedulingStopwatch, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.ItemProducer.BufferMoreDocumentsAsync(CancellationToken token)
at Microsoft.Azure.Cosmos.Query.ItemProducer.BufferMoreIfEmptyAsync(CancellationToken token)
at Microsoft.Azure.Cosmos.Query.ItemProducer.TryMoveNextPageAsync(CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.ItemProducerTree.TryMoveNextPageImplementationAsync(CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.ItemProducerTree.ExecuteWithSplitProofingAsync(Func`2 function, Boolean functionNeedsBeReexecuted, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.ItemProducerTree.TryMoveNextPageAsync(CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.CosmosParallelItemQueryExecutionContext.DrainAsync(Int32 maxElements, CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.PipelinedDocumentQueryExecutionContext.ExecuteNextAsync(CancellationToken token)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.LazyCosmosQueryExecutionContext.ExecuteNextAsync(CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextWithNameCacheStaleRetry.ExecuteNextAsync(CancellationToken cancellationToken)
at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CatchAllCosmosQueryExecutionContext.ExecuteNextAsync(CancellationToken cancellationToken)).
{"assembly":"Microsoft.Azure.Cosmos.Client, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode","level":0,"line":0}
{"assembly":"Microsoft.Azure.Cosmos.Client, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.Azure.Cosmos.CosmosResponseFactory.CreateQueryFeedResponseHelper","level":1,"line":0}
{"assembly":"Microsoft.Azure.Cosmos.Client, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.Azure.Cosmos.CosmosResponseFactory.CreateQueryFeedResponse","level":2,"line":0}
{"assembly":"Microsoft.Azure.Cosmos.Client, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35","method":"Microsoft.Azure.Cosmos.FeedIteratorCore`1+<ReadNextAsync>d__5.MoveNext","level":3,"line":0}
I've not used cosmosdb so not sure if this really relevant but accroding to the azure documentation each request is limited 4MB per request.
Am I correct in thinking in the example code you have given above there is no filtering? Meaning all 600 documents (~30k each) are returned?
You might have more success trying to split this into multipl requests
I tryed to connect to OPC UA server using the client provided on this page: https://support.industry.siemens.com/cs/document/42014088/programming-an-opc-ua-net-client-with-c%23-for-the-simatic-net-opc-ua-server?dti=0&lc=en-US . Connection to OPC UA server using the Siemens OPC Scount v10 works fine. When connecting to the OPC UA server using client provided in the article, I get this message:
Could not open UA TCP request channel.
Stack trace of the exception is this:
Server stack trace:
at Opc.Ua.Bindings.UaTcpRequestChannel.OnEndOpen(IAsyncResult result)
at Opc.Ua.Bindings.UaTcpRequestChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.CallOpenOnce.System.ServiceModel.Channels.ServiceChannel.ICallOnce.Call(ServiceChannel channel, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Opc.Ua.ISessionChannel.CreateSession(CreateSessionMessage request)
at Opc.Ua.SessionChannel.CreateSession(CreateSessionMessage request)
at Opc.Ua.SessionClient.CreateSession(RequestHeader requestHeader, ApplicationDescription clientDescription, String serverUri, String endpointUrl, String sessionName, Byte[] clientNonce, Byte[] clientCertificate, Double requestedSessionTimeout, UInt32 maxResponseMessageSize, NodeId& sessionId, NodeId& authenticationToken, Double& revisedSessionTimeout, Byte[]& serverNonce, Byte[]& serverCertificate, EndpointDescriptionCollection& serverEndpoints, SignedSoftwareCertificateCollection& serverSoftwareCertificates, SignatureData& serverSignature, UInt32& maxRequestMessageSize)
at Opc.Ua.Client.Session.Open(String sessionName, UInt32 sessionTimeout, IUserIdentity identity, IList`1 preferredLocales)
at Opc.Ua.Client.Session.Open(String sessionName, IUserIdentity identity)
Any help would be appreciated. OPC UA server runs on Siemens Simatic HMI TP700 Comfort. Configuration on the OPC UA server is default.
After andrewcullen tip, we are getting the below log in the tracelog.txt file and error when catching the exception that says
An unexpected error occurred while connecting to the server.
PID:4196 ************************* Logging started at 02/03/2016 07:41:34 *************************
4196 - 07:41:38.742 GetEndpoints Called. RequestHandle=1, PendingRequestCount=1
4196 - 07:41:38.992 SECURE CHANNEL CREATED [TcpClientChannel UA-TCP 1.00.238.1] [ID=12752] Connected To: opc.tcp://xxx.xxx.xxx.xxx:4870/
4196 - 07:41:39.008 TCPCLIENTCHANNEL SOCKET CONNECTED: 00000698, ChannelId=12752
4196 - 07:41:39.008 SECURE CHANNEL CREATED [Opc.Ua.ChannelBase WCF Client 1.00.238.1] [ID=] Connected To: opc.tcp://xxx.xxx.xxx.xxx:4870/
4196 - 07:41:39.101 GetEndpoints Completed. RequestHandle=1, PendingRequestCount=0
4196 - 07:41:39.132 TCPCLIENTCHANNEL SOCKET CLOSED: 00000698, ChannelId=12752
4196 - 07:41:44.230 Writing rejected certificate to directory:
4196 - 07:41:59.694 CreateSession Called. RequestHandle=1, PendingRequestCount=1
4196 - 07:42:13.672 TCPCLIENTCHANNEL SOCKET CLOSED: 000007C0, ChannelId=0
4196 - 07:42:13.750 CreateSession Completed. RequestHandle=1, PendingRequestCount=0
I got the answer from the Siemens official support:
The application was not tested with Comfort Panel. The code e.g. contains Block Read and Block Write which is not supported from the Panel Server.
So this application will not work.
This Siemens UaClient uses a library 'ClientAPI' which extends the OPC Foundation's Opc.Ua.Core and Opc.Ua.Client. ClientAPI has a lot of nice Helper functions to simplify connecting and subscribing. However, I see in the code for Connect(string Url) that it is using the original WCF-style channel. And your stack trace is showing the WCF types are throwing an exception that is hard to diagnose. I would change two things:
First configure tracing to write to a file. In ClientAPI, find Helpers.CreateClientConfiguration() and add
// add trace config before calling validate
configuration.TraceConfiguration = new TraceConfiguration {
OutputFilePath="tracelog.txt",
DeleteOnLoad = true,
TraceMasks = Utils.TraceMasks.All };
configuration.Validate(ApplicationType.Client);
Second, upgrade the channel type used to connect. In ClientAPI, find Server.Connect(string url) and modify the middle as shown:
// Initialize the channel which will be created with the server.
// SessionChannel channel = SessionChannel.Create(
// configuration,
// endpointDescription,
// endpointConfiguration,
// bindingFactory,
// clientCertificate,
// null);
ITransportChannel channel = WcfChannelBase.CreateUaBinaryChannel(
configuration,
endpointDescription,
endpointConfiguration,
clientCertificate,
configuration.CreateMessageContext());
// Wrap the channel with the session object.
// This call will fail if the server does not trust the client certificate.
// m_Session = new Session(channel, configuration, endpoint);
m_Session = new Session(channel, configuration, endpoint, clientCertificate);
Edit 2/4:
From the tracelog you might find certificate errors. When creating a new session, the client and server both provide and validate each others certificate. By default, UaClient is retrieving it's cert from the windows store LocalMachine\My (aka Personal). The api generates this cert during it's first run, (which requires the first run as administrator) ( to see this cert, run 'certlm.msc').
On the server machine, the server will validate the client's cert, by checking if it matches the certs in its 'TrustedPeerList'. Servers usually use a directory to store the trusted certs. If the client cert is not trusted, the server will copy the client's cert to a 'RejectedCertificates' directory. You are required to copy the cert you find in 'RejectedCertificates' to the trusted cert directory.
Back on the client machine, the client will validate the server's cert. This client uses a windows store for validation 'LocalMachine\My' (aka Personal).
Instead of using a 'Rejected' directory, the client registers an event handler that opens a message box, asking if you wish to accept the server's cert. If you choose to accept, the client sets the eventArg e.Accept = true; To suppress the message box, the server's cert should be imported into the client's 'LocalMachine\My' (aka Personal) using the tool 'certlm.msc'.
Try to ping the server with DNS name. If the server is not accessible the Hosts file in C:\Windows\System32\drivers\etc... must be edited. Open the notepad as administrator, then open the Hosts file and enter the mapping of IP Address to the host name as follows:
xxx.xxx.xxx.xxx host name
I am working on developing a new binding for a Wcf service that is hosted in IIS, I thought I got everything working, but it turns out that the client only works when it is targetting .Net framework 4.5, if I change it to target 4.6 then I get the following error when I try to open a connection:
System.ServiceModel.Security.MessageSecurityException occurred
HResult=-2146233087
Message=The Identity check failed for the outgoing message. The remote endpoint did not provide a domain name system (DNS) claim and therefore did not satisfied DNS identity 'xxx.domain.local'. This may be caused by lack of DNS or CN name in the remote endpoint X.509 certificate's distinguished name.
Source=System.ServiceModel
StackTrace:
at System.ServiceModel.Security.IdentityVerifier.EnsureIdentity(EndpointAddress serviceReference, AuthorizationContext authorizationContext, String errorString)
If I do nothing other than change the target framework in my test code back to 4.5, then it works fine. This makes me think that it could be a bug in .Net 4.6, I know there were Wcf ssl changes made in 4.6
With first chance exceptions turned on I see the following exception that is raised internally in System.ServiceModel
System.ArgumentNullException occurred
HResult=-2147467261
Message=Value cannot be null.
Parameter name: value
ParamName=value
Source=mscorlib
StackTrace:
at System.Enum.TryParseEnum(Type enumType, String value, Boolean ignoreCase, EnumResult& parseResult)
InnerException:
System.ServiceModel.dll!System.ServiceModel.Security.IssuanceTokenProviderBase<System.ServiceModel.Security.Tokens.IssuedSecurityTokenProvider.FederatedTokenProviderState>.DoNegotiation(System.TimeSpan timeout) Unknown System.ServiceModel.dll!System.ServiceModel.Security.IssuanceTokenProviderBase<System.ServiceModel.Security.Tokens.IssuedSecurityTokenProvider.FederatedTokenProviderState>.GetTokenCore(System.TimeSpan timeout) Unknown
System.IdentityModel.dll!System.IdentityModel.Selectors.SecurityTokenProvider.GetToken(System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Security.Tokens.IssuedSecurityTokenProvider.GetTokenCore(System.TimeSpan timeout) Unknown
System.IdentityModel.dll!System.IdentityModel.Selectors.SecurityTokenProvider.GetToken(System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Security.SecurityProtocol.TryGetSupportingTokens(System.ServiceModel.Security.SecurityProtocolFactory factory, System.ServiceModel.EndpointAddress target, System.Uri via, System.ServiceModel.Channels.Message message, System.TimeSpan timeout, bool isBlockingCall, out System.Collections.Generic.IList<System.ServiceModel.Security.SupportingTokenSpecification> supportingTokens) Unknown
System.ServiceModel.dll!System.ServiceModel.Security.TransportSecurityProtocol.SecureOutgoingMessageAtInitiator(ref System.ServiceModel.Channels.Message message, string actor, System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Security.TransportSecurityProtocol.SecureOutgoingMessage(ref System.ServiceModel.Channels.Message message, System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Security.SecurityProtocol.SecureOutgoingMessage(ref System.ServiceModel.Channels.Message message, System.TimeSpan timeout, System.ServiceModel.Security.SecurityProtocolCorrelationState correlationState) Unknown
System.ServiceModel.dll!System.ServiceModel.Channels.SecurityChannelFactory<System.ServiceModel.Channels.IRequestChannel>.SecurityRequestChannel.Request(System.ServiceModel.Channels.Message message, System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Channels.TransactionRequestChannelGeneric<System.ServiceModel.Channels.IRequestChannel>.Request(System.ServiceModel.Channels.Message message, System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Dispatcher.RequestChannelBinder.Request(System.ServiceModel.Channels.Message message, System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Channels.ServiceChannel.Call(string action, bool oneway, System.ServiceModel.Dispatcher.ProxyOperationRuntime operation, object[] ins, object[] outs, System.TimeSpan timeout) Unknown
System.ServiceModel.dll!System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(System.Runtime.Remoting.Messaging.IMethodCallMessage methodCall, System.ServiceModel.Dispatcher.ProxyOperationRuntime operation) Unknown
System.ServiceModel.dll!System.ServiceModel.Channels.ServiceChannelProxy.Invoke(System.Runtime.Remoting.Messaging.IMessage message) Unknown
mscorlib.dll!System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(ref System.Runtime.Remoting.Proxies.MessageData msgData, int type) Unknown
The wcf service being communicated to is targeting 4.6, and as far as I can tell I am specifying the dns identity, which does exist as a CN= in the cert subject. The binding is a custom binding so that I can do federated net.tcp, the client creates everything in code and I don't use the Add Service Reference feature in visual studio, the client code that is creating the binding:
var binding = new CustomBinding(new BindingElement[] {
new TransactionFlowBindingElement(),
security,
new SslStreamSecurityBindingElement(),
new BinaryMessageEncodingBindingElement() {
ReaderQuotas = { MaxDepth = maxReceivedSizeBytes, MaxStringContentLength = maxReceivedSizeBytes, MaxArrayLength = maxReceivedSizeBytes, MaxBytesPerRead = maxReceivedSizeBytes, MaxNameTableCharCount = maxReceivedSizeBytes },
},
new TcpTransportBindingElement {
TransferMode = TransferMode.StreamedResponse,
MaxReceivedMessageSize = maxReceivedSizeBytes,
},
}) {
SendTimeout = sendTimeout,
};
var channelFactory = new ChannelFactory<T>(binding, new EndpointAddress(new Uri(url), EndpointIdentity.CreateDnsIdentity("xxx.domain.local"), new AddressHeader[0]));
Could this be a bug in the 4.6 framework causing different behavior? Would the next steps only be trying to step through and debug framework code to try and find why 4.6 is behaving differently?
EDIT -
I created a small sample project that demonstrates the error, the repro steps are:
(Using VS 2015) Open the WcfSelfHostedServer solution
Add the IdentityFail.pfx cert to your Local Computer, Personal store using mmc
Run the WcfSelfHostedServer project (likely clicking firewall yes allow port 30000)
Open the WcfClient solution
Right click on project > properties, note that it is targetting 4.6.1
Run the project, it will throw the exception described above
Now switch the client to target 4.5.2, it will run fine with no errors
Update -
I found the following that appear related:
https://support.microsoft.com/en-us/kb/3069494
https://msdn.microsoft.com/en-us/library/mt298998(v=vs.110).aspx
But specifying Tls12 at the server and client didn't fix the issue, and even adding the DontEnableSchUseStrongCrypto=true flag didn't affect the DNS Identity Check error even though it got around the Enum.Parse internal error that was being thrown from this line
I needed to look at Retargetting Changes in the .NET Framework 4.6.1, as certificate validation logic changed in that release. (change in behavior for X509CertificateClaimSet.FindClaims that was causing my issue)
The fix is editing my app.config to add:
<runtime>
<AppContextSwitchOverrides value="Switch.System.IdentityModel.DisableMultipleDNSEntriesInSANCertificate=true" />
</runtime>
You can see the changed code on referencesource, and naturally makecert.exe doesn't appear to support generating certificates with "Subject Alternative Name" fields
You can fix in code adding one line.
like this.
AppContext.SetSwitch("Switch.System.IdentityModel.DisableMultipleDNSEntriesInSANCertificate",true);
Installing .net 4.7 on the server solved the problem for me.
Brandon.
It appears that if the flag is 'false' AND a cert does NOT contain SAN entries, we don't add the dns entry.
When hooking up an ASP.NET website to Azure Redis Cache (but this error also occurs when using a local redis instance) using the Microsoft ASP.NET Redis Session State provider, I get a Null Reference Exception. Why? Google tells me nothing. I have tried using the Russian Redis Session State provider, but that randomly corrupts the session state, so I can't use that either.
This is the stack trace:
[NullReferenceException: Object reference not set to an instance of an object.]
Microsoft.Web.Redis.StackExchangeClientConnection.Eval(String script, String[] keyArgs, Object[] valueArgs) +381
Microsoft.Web.Redis.RedisConnectionWrapper.TryUpdateIfLockIdMatch(Object lockId, ISessionStateItemCollection data, Int32 sessionTimeout) +108
Microsoft.Web.Redis.RedisSessionStateProvider.SetAndReleaseItemExclusive(HttpContext context, String id, SessionStateStoreData item, Object lockId, Boolean newItem) +1280
System.Web.SessionState.SessionStateModule.OnReleaseState(Object source, EventArgs eventArgs) +565
System.Web.SessionState.SessionStateModule.OnEndRequest(Object source, EventArgs eventArgs) +139
System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +136
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +69
I have no idea where this problem occurs as the stack trace doesn't touch my code, but it occurs on any url hitting my site after I am logged in, yes using forms authentication.
My redis config looks like this:
<sessionState mode="Custom" customProvider="RedisSessionStateStoreProvider">
<providers>
<clear />
<add name="RedisSessionStateStoreProvider" type="Microsoft.Web.Redis.RedisSessionStateProvider, Microsoft.Web.RedisSessionStateProvider, Version=0.4.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"
host="*******.redis.cache.windows.net"
throwOnError = "true"
connectionTimeoutInMilliseconds = "5000"
retryTimeoutInMilliseconds = "5000"
operationTimeoutInMilliseconds = "5000"
ssl="true"
accessKey="*******************************"
/>
</providers>
... where I have starred out the access key and service uri.
EDIT: I was happy, and thought I had found the solution when I found a post on MSDN Forums (booh!) that dealt with intermittent null reference exceptions where they warned that a nonexistent key being assigned with with a null value will break everything. I found an instance where I was doing this and fixed it,but that didn't actually do the trick and the same problem remains.
I am using Microsoft.Web.RedisSessionStateProvider 0.4.0.0-Pre-121 that allegedly fixes a similar bug
The problem is that I managed to use Null as a key(!!!). What I thought was a properly named constant (like all the other key names) was in fact a variable that was null.
The effects of this mistake differed considerably, so I will post them here if it helps anybody:
Inserting Null as a key caused both the ASP.NET Universal Providers session state provider and RedisAspNetProviders SessionStateProvider to work until the first deserialization after the fatal value with a null key is inserted, after which they blow up with a HttpException (which is correct, probably, although a slightly more helpful error message could be supplied) or an Out Of Memory exception, when I tried storing a List, which is unexpected.
The Microsoft Redis Session State provider blows up with an NRE directly as the idiotic key is inserted and again, us idiots could probably use a better message to save us time.
I tried to use Azure AppFabric as an alternative before I knew what the error was but it was not responding enough to be tested.