We have a WCF service as a BL service.
The service is in Mixed Transport mode, have more than 10 different endpoint, binded by BasicHttpBinding, with different contracts and the same address for all of them.
The service runs on its on application pool on IIS-7.
The problem is, the service works fine, but after the first call, even the get the WSDL, the memory usage of the w3wp.exe goes straight to 300 mega, the service memory usage keeps to increase constantly, taking over all the physical memory of the server (98 - 100 %). We didn't get out of memory exception, but this situation slows down other applications and the service so we need to manually refresh the application pool once every couples of days.
I already tried to use memory profiling tool and didn't find any leads to the cause of the problem.
Did anyone encounter this issues? and if yes, what did you do?
Additional information:
The BL service is located above a DAL framework based on NHibernate,
we've already ruled out the memory leak is originating from there.
Config file
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<appSettings>
</appSettings>
<system.web>
<compilation debug="true" targetFramework="4.0" />
<httpRuntime maxRequestLength="20000" requestLengthDiskThreshold="20000" />
</system.web>
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="DefaultServiceBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="true" />
</behavior>
</serviceBehaviors>
<endpointBehaviors>
<behavior name="AnonymousBehavior">
</behavior>
</endpointBehaviors>
</behaviors>
<bindings>
<basicHttpBinding>
<binding name="SecureBinding"
closeTimeout="00:10:00"
openTimeout="00:10:00" receiveTimeout="00:10:00"
sendTimeout="00:10:00" allowCookies="true"
hostNameComparisonMode="StrongWildcard" maxBufferSize="65536000"
maxBufferPoolSize="524288000" maxReceivedMessageSize="65536000"
transferMode="Buffered">
<readerQuotas maxDepth="20000000"
maxStringContentLength="8192000"
maxArrayLength="16384000"
maxBytesPerRead="4096000"
maxNameTableCharCount="16384000" />
<security mode="None">
<transport clientCredentialType="None"/>
</security>
</binding>
</basicHttpBinding>
</bindings>
<services>
<service name="BL.Services.MyService"
behaviorConfiguration="DefaultServiceBehavior">
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SecureBinding"
bindingNamespace="Security/Anonymous"
behaviorConfiguration="WithSecurityContextInspector"
contract="BL.Services.Contracts.IAnonymousClaimsService" />
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SecureBinding"
bindingNamespace="Domain/App"
behaviorConfiguration="WithSecurityContextInspector"
contract="BL.Services.Contracts.IAppService" />
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SecureBinding"
bindingNamespace="Domain/App"
behaviorConfiguration="WithSecurityContextInspector"
contract="BL.Services.Contracts.IAttachmentService" />
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SecureBinding"
bindingNamespace="Domain/Site"
behaviorConfiguration="WithSecurityContextInspector"
contract="BL.Services.Contracts.ISecurityService" />
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SecureBinding"
bindingNamespace="Domain/Transaction"
behaviorConfiguration="WithSecurityContextInspector"
contract="BL.Services.Contracts.ITransactionService" />
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SecureBinding"
bindingNamespace="Domain/ActiveDirectory"
behaviorConfiguration="WithSecurityContextInspector"
contract="BL.Services.Contracts.IActiveDirectoryService" />
<endpoint address=""
binding="basicHttpBinding"
bindingConfiguration="SecureBinding"
bindingNamespace="Domain/Report"
behaviorConfiguration="WithSecurityContextInspector"
contract="BL.Services.Contracts.IReportService" />
<host>
<baseAddresses>
<add baseAddress="//MyService.svc" />
</baseAddresses>
</host>
</service>
</services>
<serviceHostingEnvironment multipleSiteBindingsEnabled="true" />
</system.serviceModel>
<system.webServer>
<modules runAllManagedModulesForAllRequests="true" />
<defaultDocument>
<files>
<add value="MyService.svc" />
</files>
</defaultDocument>
</system.webServer>
</configuration>
The 300MB is not unusual as AnkMannen notes. Heavily used service can easily plateau out around 700MB or more. Your second observation of the service consuming most available server memory but not triggering an out of memory exception is likely due to the non-default config values:
binding:
maxBufferSize="65536000"
maxBufferPoolSize="524288000"
maxReceivedMessageSize="65536000"
transferMode="Buffered"
readerQuotas:
maxDepth="20000000"
maxStringContentLength="8192000"
maxArrayLength="16384000"
maxBytesPerRead="4096000"
maxNameTableCharCount="16384000"
You are actually configuring WCF to consume excessive memory with the values you have chosen. Unless you have encountered a specific condition that required changing the default value for any of those attributes, don't change them. The only value I routinely change is maxReceivedMessageSize from a default of 64K to around 1 to 2 MB, if unavoidable. If you're routinely slinging around messages that are bigger than 3 MB, you should reconsider your data contract design. A lot of the performance issues WCF is accused of are actually misconfigurations not performance problems in WCF itself.
After a long search we found the problem.
Our service used a lot of logic units in a unit of work pattern.
Each logic unit inherited from a BaseLogic class. In the the BaseLogic unit there is an Enterprise Library UnityContainer property which created a factory. Each call created many instances of this factory, changing this property to a static property fixed the problem.
The first initial jump to 300MB is consistent with what I've seen in our applications. Haven't really found a way to decrease that number but it stays at that figure over time.
For the increasing part of memory it sounds like a standard memory leak or at least a GC issue. Are you using entity framework and did you profile with a tool like Red Gates Memory Profiler, not the built in VS profiler?
It's hard to give any more specific answer based on the information in the question.
In the mean time, try to use the IIS auto refresh of the application pool. Set it to a threshold of your choice and let it automatically handle the refresh.
Related
I have just a very basic WCF service implemented and I want the clients to have some properties in their config automatically generated, like maxReceivedMessageSize, maxBufferSize, etc (the default maxReceivedMessageSize is not enough for me). I have the below config in service application's web.config file. I have the properly configured binding (BasicHttpBinding_IService1_YY), that binding is referenced in both service endpoint and client endpoint (I am not sure there is need for client endpoint as well). I also have mexHttpBinding typed endpoint under service. The service points to a behaviour where httpGetEnabled is true.
After all, when I generate the client from this the binding parameters won't be there, it will use the default values. Naturally the wsdl doesn't contain those parameters.
So my question would be is this something that is out of WCF's scope? Do I have to manually maintenance the client config from this perspective if something change? Is there any solution for this issue, because I went through many stackoverflow and other topics and found nothing or I was careless.
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="myServiceBehavior">
<!-- To avoid disclosing metadata information, set the values below to false before deployment -->
<serviceMetadata httpGetEnabled="true" policyVersion="Policy15" httpGetBindingConfiguration="BasicHttpBinding_IService1_YY" />
<!-- To receive exception details in faults for debugging purposes, set the value below to true. Set to false before deployment to avoid disclosing exception information -->
<serviceDebug includeExceptionDetailInFaults="true"/>
</behavior>
</serviceBehaviors>
</behaviors>
<services>
<service name="WcfServiceApplication.Service1" behaviorConfiguration="myServiceBehavior">
<host>
<baseAddresses>
<add baseAddress="http://localhost:51483/"/>
</baseAddresses>
</host>
<endpoint address="" binding="basicHttpBinding" contract="WcfServiceApplication.IService1"
bindingConfiguration="BasicHttpBinding_IService1_YY" name="BasicHttpBinding_IService1_YY" >
</endpoint>
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" >
</endpoint>
</service>
</services>
<bindings>
<basicHttpBinding>
<binding name="BasicHttpBinding_IService1_YY"
maxReceivedMessageSize="20000099" maxBufferSize="20000099" maxBufferPoolSize="20000099"/>
</basicHttpBinding>
</bindings>
<client>
<endpoint address="" binding="basicHttpBinding"
bindingConfiguration="BasicHttpBinding_IService1_YY" contract="WcfServiceApplication.IService1"
name="BasicHttpBinding_IService1_YY" />
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
</client>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true"/>
</system.serviceModel>
And the generated client config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="BasicHttpBinding_IService1_YY" sendTimeout="00:05:00" />
</basicHttpBinding>
</bindings>
<client>
<endpoint address="http://localhost:51483/Service1.svc" binding="basicHttpBinding"
bindingConfiguration="BasicHttpBinding_IService1_YY" contract="IService1"
name="BasicHttpBinding_IService1_YY" />
</client>
</system.serviceModel>
</configuration>
Ok, the name of the binding really appears correctly, but nothing more. When I manually adjust it, then everything works ok, but I think there should be a more suitable solution.
Can you please help me out at this one? Thanks in advance.
The answer is probably: Yes.
The feature of WCF is that the client and server can evolve separately. The server settings are not automatically updated to the client, you need to manually set the parameters you need on the client.
You may try using the "update service reference" feature to update instead of creating a new service reference.
I have a WCF service deployed to IIS 7 and the client. I thought the service is configured to handle concurrency. When I tested with two client, it took double the amount of time. I have spent hours and not yet able to figure out where I missed it. Any help/suggestions would be appreciated. Service1.svc.cs looks like this:
[ServiceBehavior( ConcurrencyMode = ConcurrencyMode.Multiple, InstanceContextMode = InstanceContextMode.PerCall)]
[CallbackBehavior(UseSynchronizationContext=false)]
public class Service1 : IService1
{
public byte[] DownloadFile()
{
byte[] byt = File.ReadAllBytes(#"C:\Temp\TestFile.pdf");
System.Threading.Thread.Sleep(1000);
return byt;
}
}
Configuration:
<system.net>
<connectionManagement>
<add address="*" maxconnection="100"/>
</connectionManagement>
</system.net>
<system.web>
<compilation debug="true" targetFramework="4.0" />
<processModel autoConfig="false" maxWorkerThreads="1400" maxIoThreads="1400" minWorkerThreads="2"/>
<httpRuntime minFreeThreads="1000" minLocalRequestFreeThreads="1000"/>
</system.web>
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="Service1Binding" maxBufferSize="2147483647" maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647" transferMode="Streamed">
<readerQuotas maxDepth="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" />
</binding>
</basicHttpBinding>
</bindings>
<services>
<service name="DocService.Service1">
<endpoint address="http://localhost/DocService/Service1.svc" binding="basicHttpBinding"
bindingConfiguration="" name="Service1Binding" bindingName="Service1Binding"
contract="DocService.IService1" />
<endpoint address="http://localhost/DocService/mex" binding="mexHttpBinding"
name="mexpoint" contract="IMetadataExchange" />
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior>
<serviceMetadata httpGetEnabled="true"/>
<serviceDebug includeExceptionDetailInFaults="false"/>
<serviceThrottling maxConcurrentCalls="100" maxConcurrentInstances="100" maxConcurrentSessions="100"/>
</behavior>
</serviceBehaviors>
</behaviors>
The requests are taking much longer than 1s sleep time and the read time of, say, 10mb file. They should complete in maybe 1.1-2s There is something fishy here.
Comment out the read and return a new byte[0]. Does it take 1.0s now? Do you have concurrency now?
Edit: What client are you using?
Edit 2: Responding to your comment, the file transfer is not limiting concurrency in some way. WCF does not care what you do inside your service method. It knows nothing about files. The problem is that your disk is too slow and that serializing 36mb is probably slow too! You server is just too slow! You can find out, if the disk or the serializing is the problem by replacing the IO with new byte[1024*1024*36].
I get this:
An error occurred while receiving the HTTP response to http://localhost:8732/Design_Time_Addresses/PersistencyService/Service1/. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.
Why do I get this? I assume it is because the method takes about 1 min to complete. How can disable any time limit?
I get this when running in VS a Windows form project that uses a WCF service in the same solution
My WCF configuration:
edit:
<system.serviceModel>
<bindings>
<wsHttpBinding>
<binding name="LongRunning" sendTimeout="00:10:00" />
</wsHttpBinding>
</bindings>
<client>
<endpoint name="Default"
address="http://localhost:8732/Design_Time_Addresses/PersistencyService/Service1/"
binding="wsHttpBinding"
bindingConfiguration="LongRunning"
contract="PersistencyService.IService1" />
</client>
<services>
<service name="PersistencyService.Service1">
<endpoint
address=""
binding="wsHttpBinding" bindingConfiguration=""
contract="PersistencyService.IService1" >
<identity>
<dns value="localhost" />
</identity>
</endpoint>
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="http://localhost:8732/Design_Time_Addresses/PersistencyService/Service1/" />
</baseAddresses>
</host>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior>
<serviceMetadata httpGetEnabled="True"/>
<serviceDebug includeExceptionDetailInFaults="False"/>
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>
The exception message is An existing connection was forcibly closed by the remote host.
I must also add that I get about 70MB of data from the service
On the client side, you need to add some settings to your app.config:
<system.serviceModel>
<bindings>
<wsHttpBinding>
<binding name="LongRunning" sendTimeout="00:10:00" />
</wsHttpBinding>
</bindings>
<client>
<endpoint name="Default"
address="....."
binding="wsHttpBinding"
bindingConfiguration="LongRunning"
contract="IYourServiceContract" />
</client>
</system.serviceModel>
You didn't give us much to go on - no config, nothing.... so I'm left just guessing what settings you might have.
Basically, you need to define a binding configuration for the type of binding you're using, and you need to increase the sendTimeout attribute on that binding configuration (here in my sample: 10 minutes). You cannot completely turn off the timeout - you can increase it, but not turn it off.
Then, your client side config must make a reference to that binding configuration you've defined, by specifying a bindingConfiguration="...." attribute on the <endpoint> configuration, and using the same name for the binding configuration as when you defined it.
Maybe this is enough - but maybe, you'll also need to increase some of the timeouts on the server side. Try this first - if it doesn't work, come back and ask again - and please, next time: provide us with some more useful info, like your code and config!
I am attempting to transfer around 7000-8000 objects that are not large (only 9 properties per object instance). Does anyone know why when I begin to retrieve more than 5000 or so objects that I get connection errors? It works perfectly until I hit some threshold for data size.
I am exposing the retrieval of these objects via WCF's TCP service binding. I have the following sample configuration:
<bindings>
<netTcpBinding>
<binding name="NetTcpBindingConfig"
openTimeout="00:01:00"
sendTimeout="00:05:00"
closeTimeout="00:01:00"
maxBufferPoolSize="2147483647"
maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647">
<readerQuotas maxDepth="2147483647"
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647" />
<security>
<transport/>
</security>
</binding>
</netTcpBinding>
</bindings>
<services>
<service behaviorConfiguration="ServiceBehavior"
name="TestService">
<endpoint address=""
binding="netTcpBinding"
bindingConfiguration="NetTcpBindingConfig"
contract="ServiceInterfaces.ITestService">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
<endpoint address="mex"
binding="mexTcpBinding"
contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="net.tcp://localhost:8526/TestService" />
</baseAddresses>
</host>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="Services.ServiceBehavior">
<serviceMetadata httpGetEnabled="false" />
<serviceDebug includeExceptionDetailInFaults="true" />
</behavior>
</serviceBehaviors>
</behaviors>
From my .NET code I am calling the service using a ChannelFactory with the following sample code:
using (ChannelFactory<ITestervice> channel = new ChannelFactory<ITestService>(BindingConfig, "net.tcp://localhost:8526/TestService"))
{
ITestService testService = channel.CreateChannel();
toReturn = testService.LoadAll();
channel.Close();
}
BindingConfig object is a NetTcpBinding property in my code that is populated as 'new NetTcpBinding("NetTcpBindingConfig")'. My client binding is the exact same as my WCF TCP service binding.
Can anyone offer any insight as to how I can retrieve all of the data (it seems my maximum limit is ~5000 objects with my current setup)? Any help is much appreciated. Thanks.
EDIT:
In case anyone runs into this, see the accepted solution about the MaxItemsInObjectGraph. If, however, you are using ChannelFactory from the Client to consume your services, see the following code to make it work:
foreach (OperationDescription operation in channel.Endpoint.Contract.Operations)
{
DataContractSerializerOperationBehavior dataContractBehavior = operation.Behaviors[typeof(DataContractSerializerOperationBehavior)] as DataContractSerializerOperationBehavior;
if (dataContractBehavior != null)
dataContractBehavior.MaxItemsInObjectGraph = int.MaxValue;
}
Consider increasing MaxItemsInObjectGraph quota as well (its default value is 64k). It should be on both the server and the client side. See the sample configuration:
When I pass a list of objects out of my silverlight app using WCF everything works fine until the List grows too big. It seems that when I exceed 80 items I get the error:
The remote server returned an unexpected response: (404) Not Found
I'm presuming that it's because the List has grown too big as when the List had 70 items everyhing works fine. Strange error message though, right?
In the config file I change the maxBufferSize to the highest value that it will accept but still I can't have more then 80 items in my List.
How can I pass out large objects without having to split the object up?
Thanks Shawn, so where exactly do I do it?
This is my ServiceReferences.ClientConfig
<configuration>
<system.serviceModel>
<client>
<!--"http://sy01911.fw.gsjbw.com/WcfService1/Service1.svc"-->
<endpoint address="http://localhost/WcfService1/Service1.svc"
binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IService11"
contract="SilverlightApplication1.ServiceReference1.IService1"
name="BasicHttpBinding_IService1" />
</client>
<bindings>
<basicHttpBinding>
<binding name="BasicHttpBinding_IService1" maxBufferSize="655360000"
maxReceivedMessageSize="655360000">
<security mode="None" />
</binding>
<binding name="BasicHttpBinding_IService11" maxBufferSize="655360000"
maxReceivedMessageSize="655360000">
<security mode="None" />
</binding>
</basicHttpBinding>
</bindings>
</system.serviceModel>
and this is the server config that you mentioned
<services>
<service name="WcfService1.Service1" behaviorConfiguration="WcfService1.Service1Behavior" >
<!-- Service Endpoints -->
<endpoint address="" binding="basicHttpBinding" contract="WcfService1.IService1" >
<!--
Upon deployment, the following identity element should be removed or replaced to reflect the
identity under which the deployed service runs. If removed, WCF will infer an appropriate identity
automatically.
-->
<identity>
<dns value="localhost"/>
</identity>
</endpoint>
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="WcfService1.Service1Behavior">
<!-- To avoid disclosing metadata information, set the value below to false and remove the metadata endpoint above before deployment -->
<serviceMetadata httpGetEnabled="true"/>
<!-- To receive exception details in faults for debugging purposes, set the value below to true. Set to false before deployment to avoid disclosing exception information -->
<serviceDebug includeExceptionDetailInFaults="true"/>
</behavior>
</serviceBehaviors>
</behaviors>
If you are sending out a large number of items from WCF, also make sure that maxItemsInObjectGraph is a relatively high number
<behaviors>
<serviceBehaviors>
<behavior name="YourBahvior">
<dataContractSerializer maxItemsInObjectGraph="6553600"/>
</behavior>
</serviceBehaviors>
</behaviors>
There are two config files. The silverlight clientconfig will let you send the larger message, but if you'r eusing WCF, there is a server web.config that limits the size of the received message (to prevent DDOS attacks).
In server side, change the config file to make the service can accept large message.
Add a basicHttpBinding configuration in <system.serviceModel> section:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="MyBasicHttpBinding" maxReceivedMessageSize="300000000">
<security mode="None"/>
<readerQuotas maxStringContentLength="300000000"/>
</binding>
</basicHttpBinding>
</bindings>
.......
Add the configuration to the service binding.
<endpoint address="" binding="basicHttpBinding" bindingConfiguration="MyBasicHttpBinding" contract="WcfService1.IService1">