I have around 10-20 tasks that each perform a separate job. At some point during this job each task will eventually call the same WCF endpoint asynchronously to get data.
The WCF endpoint does exist on a server farm cluster of 8 servers. Within this endpoint there are multiple things happening (Stored Procedure calls, Data Calculation, SQL inserts, delete, updates)
It sounds like a lot but One single call takes less than 1 second to return.
The problem is running full throttle, the service starts queuing up the calls and the response times grow larger and larger and eventually timeout. Each has to be processed sequentially as the ProcessJob Webservice needs to insert the data into the database as soon as it can.
How can I prevent this queuing and timeout from happening?
private void StartAllJobs()
{
List<Task> task = new List<Task>();
foreach(job in JobList)
{
//Start the job on its own thread
task.Add(Task.Run(await () => RunMyJob(job)));
}
Task.WaitAll(task.ToArray());
}
private async Task<string> RunMyJob(Job job)
{
//Other processing below
var infoList = ProcessOtherStuff(job);
using(var proxy=new serviceclient())
{
Stopwatch sw = new Stopwatch();
sw.Start();
for(var i=0;i<infoList.Count;i++)
//The count can reach up to 20000 records to process
{
try
{
var value = await proxy.ProcessJob(info);
sw.Stop();
Console.Writeline("Process took this long:{0}", sw.Elapsed);
}
catch(Exception ex)
{
Console.Writeline("Exception {0}: Time:{1}, ex.Message, DateTime.Now.ToString());
}
}
return "Complete";
}
}
The error message is something like this
System.TimeoutException: The request channel timed out while waiting for a reply after 00:01:00. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The HTTP request to 'http://localhost/ReportServer/ReportExecution2005.asmx' has exceeded the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.WebException: The operation has timed out
here is the client WCF config (only part that is related to the reporting services, not the entire WCF config)
<bindings>
<wsHttpBinding>
<binding name="ReportExecutionServiceSoap" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
useDefaultWebProxy="true">
<readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647"
maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="TransportCredentialOnly">
<transport clientCredentialType="Non" proxyCredentialType="None"
realm="" />
</security>
</binding>
</wsHttpBinding>
</bindings>
<client>
<endpoint address="https://serverfarm.com/MyServer/MyWebService.svc"
binding="wsHttpBinding" bindingConfiguration="ReportExecutionServiceSoap"
contract="ReportExecutionServiceReference.ReportExecutionServiceSoap"
name="ReportExecutionServiceSoap" />
</client>
Then my service has
<endpoint address="" binding="wsHttpBinding" bindingConfiguration="MyBinding" contract="MyWebService.IMyWebService"/>
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/>
<behaviors>
<serviceBehaviors>
<behavior name="MyBehavior">
<serviceMetadata httpGetEnabled="true"/>
<serviceDebug includeExceptionDetailInFaults="true"/>
<serviceThrottling maxConcurrentInstances="70424" maxConcurrentCalls="10024" maxConcurrentSessions="60400" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<wsHttpeBinding>
<binding name="MyBinding">
<readerQuotas
maxArrayLength="16384" maxBytesPerRead="4096" maxDepth="32" maxNameTableCharCount="16384" maxStringContentLength="102400"/>
<security mode="Transport">
<transportclientCredentialType="None"/>
<messageestablishSecurityContext="false"/>
</security>
</binding>
Related
I'm getting this error in my WCF Service: Additional information: The remote server returned an unexpected response: (413) Request Entity Too Large.
I tried to increase maxReceivedMessageSize to 2147483647 but it still doesn't work. Then, I measured the size of my file and I realized that it has 10 Kb.
Do you guys have any idea of what it might be?
I'm sending you a copy of my code:
Here, I'm basically sending a list of 1o Kb for the business logic layer (WCF service)
else if (comboBoxReason.Text == "Antecipação indevida - automática")
{
AdjustmentAmount = 0;
using (Stream s = new MemoryStream())
{
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(s, AnticipationAdjustment);
long size = s.Length;
}
AdjustmentAmount = Decimal.Round(FinancialAdjustmentsCalculations.AnticipationAutomaticRate(AnticipationAdjustment), 2); //THE ERROR IS HERE
textBoxAdjustment.Text = ValorAjuste.ToString();
AnticipationAdjustment.Clear();
}
That is my Web.Config from the Business Logic Layer (Ext)
<bindings>
<basicHttpBinding>
<binding name="BasicHttpBinding_IInt" closeTimeout="01:00:01" openTimeout="01:00:01"
receiveTimeout="02:00:01" sendTimeout="09:00:00" maxBufferPoolSize="2147483647"
maxBufferSize="2147483647" maxReceivedMessageSize="2147483647"/>
</basicHttpBinding>
And that is my App.Config from the interface:
<binding name="BasicHttpBinding_IExt"
closeTimeout="01:00:01"
openTimeout="01:00:01" receiveTimeout="02:00:01" sendTimeout="09:00:00"
maxBufferPoolSize="2147483647" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647" />
</basicHttpBinding>
I tried to add the readerQuotas and maxItemsInObjectGraph="2147483647", but it still doesn't work:
<binding name="BasicHttpBinding_IExt" closeTimeout="01:00:01"
openTimeout="02:00:01" receiveTimeout="01:00:01" sendTimeout="09:00:00"
maxBufferPoolSize="524288000" maxBufferSize="65536000" maxReceivedMessageSize="65536000">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
</binding>
</basicHttpBinding>
</bindings>
<client>
<endpoint address="http://localhost:56296/ExtService.svc" binding="basicHttpBinding"
bindingConfiguration="BasicHttpBinding_IExt" contract="ExtService.IExt"
name="BasicHttpBinding_IExt" behaviorConfiguration="Behaviors.EndpointBehavior" />
</client>
<behaviors>
<endpointBehaviors>
<behavior name="Behaviors.EndpointBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</endpointBehaviors>
</behaviors>
</system.serviceModel>
I have a WCF service, Server, which accepts requests from a console application Client. When sending the data (which is relatively large), I get the following exception:
System.ServiceModel.ProtocolException: The remote server returned an unexpected response: (413) Request Entity Too Large. ---> System.Net.WebException: The remote server returned an error: (413) Request Entity Too Large.
I thought I had the limits correctly set in the config files, but apparently I'm mistaken?
Here's the relevant part of Server's app.config:
<system.serviceModel>
<services>
<service name="Services.Server">
<endpoint bindingConfiguration="LargeWeb" binding="webHttpBinding" contract="Interfaces.IServer" />
</service>
</services>
<bindings>
<webHttpBinding>
<binding name="LargeWeb" maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647">
<readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
</binding>
</webHttpBinding>
</bindings>
</system.serviceModel>
And here's Client's app.config:
<system.serviceModel>
<client>
<endpoint name="serverEndpoint" binding="webHttpBinding" bindingConfiguration="LargeWeb" contract="Interfaces.IServer" />
</client>
<bindings>
<webHttpBinding>
<binding name="LargeWeb" maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647">
<readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
</binding>
</webHttpBinding>
</bindings>
</system.serviceModel>
And finally, this is how Client calls the Server's method:
using (var cf = new WebChannelFactory<IServer>("serverEndpoint", serverAddress))
{
var channel = cf.CreateChannel();
channel.SomeMethod(some, arguments);
}
What am I doing wrong?
Thank you for any help.
The issue was that Client was residing in a UnitTest project, and the issue arised while running a unit test. When running a unit test, only the UnitTest project's config file is taken into account, so the <service> parts of the Server config need to be copied into it.
This way, only one config file is ever used: both Server and Client read it for their configuration.
After that, things work fine.
I have webHttpBinding for wcf service. Can not post large xml file (1MB) to service getting following error RequestEntityTooLarge (413) is not one of the following: OK (200), Created (201), Accepted (202), NonAuthoritativeInformation (203), NoContent (204), ResetContent (205), PartialContent (206)"} System.Exception {System.ArgumentOutOfRangeException}
Here is configuration for wcf service..
<service name="abc" >
<endpoint address="http://localhost:8741/LTLInboundService/" binding="webHttpBinding" behaviorConfiguration="WebHttpBehaviour" contract="abc" name="webHttpBinding_LTLInboundService" >
</endpoint>
</service>
<webHttpBinding>
<binding name="webHttpBinding_LTLInboundService" closeTimeout="00:10:00"
openTimeout="00:10:00" receiveTimeout="00:10:00" sendTimeout="00:10:00"
allowCookies="false" bypassProxyOnLocal="false"
hostNameComparisonMode="StrongWildcard" maxBufferSize="2147483647"
maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
transferMode="Streamed"
useDefaultWebProxy="true">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647" />
<security mode="Transport">
<transport clientCredentialType="None"/>
</security>
</binding>
</webHttpBinding>
Here is the code that is failing..
using (HttpClient client = new HttpClient("http://localhost:8741/LTLInboundService" + "/SendCompletionReports"))
{
// Initalise a response object
HttpResponseMessage response = null;
string responseFilePath = Path.Combine(#"c:\Samir\abc.xml");
string xmlData = System.IO.File.ReadAllText(responseFilePath);
// Create a content object for the request
HttpContent content = HttpContent.Create(xmlData, Encoding.UTF8, "text/xml");
// Make the request and retrieve the response
response = client.Post("http://localhost:8741/LTLInboundService" + "/SendCompletionReports", content);
// Throw an exception if the response is not a 200 level response
response.EnsureStatusIsSuccessful();
// Retrieve the content of the response for processing
response.Content.LoadIntoBuffer();
}
It fails on response.EnsureStatusIsSuccessful(); statement.
Any idea how to fix this in development and production ?
checkout this site and pay attention to how they have this section configured under the security section
Dealing with Larger Files in Asp.NET
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
</system.webServer>
You defined a webHttpBinding in your config file, but it's not being used by your service because you did not specify it in the bindingConfiguration attribute.
<service name="abc" >
<endpoint address="http://localhost:8741/LTLInboundService/"
binding="webHttpBinding"
behaviorConfiguration="WebHttpBehaviour" contract="abc"
name="webHttpBinding_LTLInboundService" >
</endpoint>
</service>
The name attribute is used for the endpoint configuration, but has no bearing on the binding configuration used. In your <endpoint> element add this:
bindingConfiguration="webHttpBinding_LTLInboundService"
so your defined endpoint will use the values for the binding specified, not the default.
Final endpoint would look like this:
<service name="abc" >
<endpoint address="http://localhost:8741/LTLInboundService/"
binding="webHttpBinding"
bindingConfiguration="webHttpBinding_LTLInboundService"
behaviorConfiguration="WebHttpBehaviour" contract="abc"
name="webHttpBinding_LTLInboundService" >
</endpoint>
</service>
I have a small application that uses WCF to communicate with a webserver. This program is used by some 200 clients, and each client is sending in about 5-20 requests/min.
Looking at the error logs I often get:
The request channel timed out while waiting for a reply after
00:00:59.9989999
The requests are made as follows:
ClientService Client = new ClientService();
Client.Open();
Client.updateOnline(userID);
Client.Close();
This is the app.config
<configuration>
<configSections>
</configSections>
<startup><supportedRuntime version="v2.0.50727"/></startup><system.serviceModel>
<bindings>
<wsHttpBinding>
<binding name="WSHttpBinding_IClientService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true"
allowCookies="false">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="None">
<transport clientCredentialType="Windows" proxyCredentialType="None"
realm="" />
<message clientCredentialType="Windows" negotiateServiceCredential="true" />
</security>
</binding>
</wsHttpBinding>
</bindings>
<client>
<endpoint address="PATH TO SERVICE"
binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_IClientService"
contract="IClientService" name="WSHttpBinding_IClientService">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
</client>
</system.serviceModel>
Of the "many calls" each day about 200-800 fail. And about n-1 are ok. I am very confused what the issue can be. Looking at the server stats, it's hardly building up a sweat - each request takes <2 sec to process. The data it's sending consists of "int" or "very small strings" - so, its not due to size, if it were - there should be a consistent failure..
Suggestions?
Try adding the timeout values to both the service AND the client:
<binding name="BasicHttpBinding_SomeName" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00"
sendTimeout="00:10:00" maxBufferPoolSize="2147483647"
maxBufferSize="2147483647" maxReceivedMessageSize="2147483647">
It seems that your requests are queuing up on server before being handled and then starts to time out. You can try couple of things here to see exactly whats going on
1) Try throttling in your WCF services e.g: Try increasing your concurrent sessions. Take a look
WCF Throttling
2) Try using PerCall rather than Using Sessions here to make sure that no session remains there.
Here is what you can do on your interface to remove session
[ServiceContract(Namespace="YOUR NAMESPACE", SessionMode=SessionMode.NotAllowed)]
and your contract classes implementing those interface
ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
AND .... You should enable tracing to see exactly whats going on .
Check this performance counter - ASP.NET Requests Queued.
One web application can host many web pages and services. IIS processes only 12 concurrent requests per CPU core by default.
This means that even if your service is fast but other pages/services are slow your request has to wait in queue before they are executed.
ASP.NET Requests Queued should be zero or close to zero.
See Performance Counters for ASP.NET on MSDN.
I am able to connect to my WCF service with the Win-form application, however i am not able to do so with my windows service. Whenever i fire open() to the proxy it throws the following error
The server has rejected the client credentials
Inner Exception: System.Security.Authentication.InvalidCredentialException: The server
has rejected the client credentials.
---> System.ComponentModel.Win32Exception: The logon attempt failed
--- End of inner exception stack trace ---
at System.Net.Security.NegoState.ProcessAuthentication(LazyAsyncResult
lazyResult)
at System.Net.Security.NegotiateStream.AuthenticateAsClient(NetworkCredential
credential, ChannelBinding binding, String targetName, ProtectionLevel
requiredProtectionLevel, TokenImpersonationLevel
allowedImpersonationLevel)
at System.Net.Security.NegotiateStream.AuthenticateAsClient(NetworkCredential
credential, String targetName, ProtectionLevel
requiredProtectionLevel, TokenImpersonationLevel
allowedImpersonationLevel)
at System.ServiceModel.Channels.WindowsStreamSecurityUpgradeProvider.WindowsStreamSecurityUpgradeInitiator.OnInitiateUpgrade(Stream
stream, SecurityMessageProperty& remoteSecurity)
Tried searching for the solution, but none fitting my requirements, hence posted.
Please help...
Update 1:
#A.R., Tried using
client.ClientCredentials.Windows.AllowedImpersonationLevel =
System.Security.Principal.TokenImpersonationLevel.Impersonation;
but to no avail.
Update 2:
WCF service Configuration
<system.serviceModel>
<diagnostics performanceCounters="All" />
<bindings>
<netTcpBinding>
<binding name="myBindingForLargeData" maxReceivedMessageSize="5242880" maxConnections="10">
<readerQuotas maxDepth="64" maxStringContentLength="5242880" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384"/>
</binding>
</netTcpBinding>
</bindings>
<services>
<service behaviorConfiguration="WCFService.ServiceBehavior"
name="WCFService.CollectorService">
<endpoint address="" binding="netTcpBinding" bindingConfiguration="myBindingForLargeData"
name="netTcpEndPoint" contract="WCFService.ICollectorService" />
<endpoint address="mex" binding="mexTcpBinding" bindingConfiguration=""
name="mexTcpEndPoint" contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="net.tcp://localhost:8010/WCFService.CollectorService/" />
</baseAddresses>
</host>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="WCFService.ServiceBehavior">
<serviceMetadata httpGetEnabled="False"/>
<serviceDebug includeExceptionDetailInFaults="True" />
<serviceThrottling
maxConcurrentCalls="32"
maxConcurrentSessions="32"
maxConcurrentInstances="32"
/>
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>
Thanks for all your help. i got the answer after few days of some research and trial n error method :) well i know i am late to post the answer, but i think its better late than never.
So Here's the solution
i had to make some changes in my configuration files (both client & server)
On the client side i added <security> tag as shown below
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="netTcpEndPoint" closeTimeout="00:01:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:10:00" transactionFlow="false" transferMode="Buffered" transactionProtocol="OleTransactions" hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxBufferPoolSize="5242880" maxBufferSize="5242880" maxConnections="15" maxReceivedMessageSize="5242880">
<readerQuotas maxDepth="32" maxStringContentLength="5242880" maxArrayLength="16384" maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00" enabled="false" />
<security mode="Transport">
<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
<message clientCredentialType="Windows" />
</security>
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint address="net.tcp://xx.xx.xx.xx:8010/WCFService.CollectorService/" binding="netTcpBinding" bindingConfiguration="netTcpEndPoint" contract="CloudAdapter.CloudCollectorService.ICollectorService" name="netTcpEndPoint">
</endpoint>
</client>
</system.serviceModel>
and also added the same tag on the server side (WCF service configuration), as shown below
<bindings>
<netTcpBinding>
<binding name="myBindingForLargeData" maxReceivedMessageSize="5242880" maxConnections="10">
<readerQuotas maxDepth="64" maxStringContentLength="5242880" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384"/>
<security mode="Transport">
<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
<message clientCredentialType="Windows" />
</security>
</binding>
</netTcpBinding>
</bindings>
Hope this help a person in need :)
So the KEY is to make the <security> tag same over the client and the server configuration files.
Basically what is happening is that your calling service doesn't have the appropriate credentials, like you would have when calling from WinForms. What you need is some impersonation. It takes a bit of setting up, and is kind of annoying, but it will work.
Luckily MSDN has a nice little walkthrough.
http://msdn.microsoft.com/en-us/library/ms731090.aspx
There is some more general information on the topic here:
http://msdn.microsoft.com/en-us/library/ms730088.aspx
UPDATE:
Setting impersonation flags is not enough. You have to actually impersonate a credential to make it work. For example:
// Let's assume that this code is run inside of the calling service.
var winIdentity = ServiceSecurityContext.Current.WindowsIdentity;
using (var impContext = winIdentity.Impersonate())
{
// So this would be the service call that is failing otherwise.
return MyService.MyServiceCall();
}
Check out my answer on this post The server has rejected the client credentials.
Note the security node.
<bindings>
<netTcpBinding>
<binding name="customTcpBinding" maxReceivedMessageSize="20480000" transferMode="Streamed" >
<security mode="None"></security>
</binding>
</netTcpBinding>
</bindings>
What is the authentication mode you are using on your WCF Service? Seems like the winform app is running and providing the correct credentials while your windows service is not running with the specified privileges or the credentials being passed are not valid. Try to inspect your request using Fiddler when made from you winforms vs Windwos service and see the difference.
For me it helped to set on both sides (client + server) the security mode to None:
NetTcpBinding binding = new NetTcpBinding();
binding.Security.Mode = SecurityMode.None;
(Same answer as from spinner_den_g but in C# - no need to edit the app.config)