We Keep Coding

WCF Memory Increases dramatically

I am using WCF service. The problem I have is its starts using double memory.
I am using HTTPS binding
<wsHttpBinding>
<binding name="secureHttpBinding" closeTimeout="04:01:00" openTimeout="04:01:00" receiveTimeout="04:10:00" sendTimeout="04:01:00" allowCookies="false"
bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true">
<readerQuotas maxDepth="128" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
<security mode="Transport" >
<transport clientCredentialType="None"/>
</security>
</binding>
</wsHttpBinding>
<endpoint address="https://localhost/test.svc"
binding="wsHttpBinding"
bindingConfiguration="secureHttpBinding"
contract="IWcfContract"
name="SecureBasicHttpBinding_WcfContract">
Here is the code I am using to upload
using (Stream fileStream = File.OpenRead(logsZipFullPath))
{
// Call web server
UploadFileResponse response = _webServiceHelper.UploadFile(fileStream, currentDate, ".zip", string.Empty);
fileStream.Close();
}
Here is my model
[MessageContract]
public class UploadFileRequest : IDisposable
{
[MessageBodyMember(Order = 1)]
public Stream FileByteStream;
[MessageHeader(MustUnderstand = true)]
public string FileDescription;
}
My zip file is of 80MB.
The problem I have is at the start of the service its using 26mb which is quite fine. At the first call it uses 136MB after call completes it goes down to 26mb. which is also fine. after the second call to upload it starts using 346MB
which again gets down to 26mb after service call. My question is why it is using 346MB when the file is of only 80MB? My GC and disponse has been called correctly. But, is this normal behaviour or I am missing anything?

finally found a work around for this. According to this post
wsHttpBinding is the full-blown binding, which supports a ton of WS-*
features and standards - it has lots more security features, you can
use sessionful connections, you can use reliable messaging, you can
use transactional control - just a lot more stuff, but wsHttpBinding
is also a lot *heavier" and adds a lot of overhead to your messages as
they travel across the network
I think this is the main reason why the memory usage is high. Our requirement was to use the HTTPS certificate and we are able to use this with basicHttpBinding. SO, I changed my settings like this
<basicHttpBinding>
<binding name="BasicHttpBinding_WcfContract" closeTimeout="04:01:00" openTimeout="04:01:00" receiveTimeout="04:10:00" sendTimeout="04:01:00" allowCookies="false"
bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard" maxBufferSize="2147483647" maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647" messageEncoding="Text" textEncoding="utf-8" transferMode="Streamed" useDefaultWebProxy="true">
<readerQuotas maxDepth="128" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
<security mode="Transport"> <-- this is the main change
<transport clientCredentialType="None" proxyCredentialType="None" realm=""/>
<message clientCredentialType="UserName" algorithmSuite="Default"/>
</security>
</binding>
</basicHttpBinding>
Note: I changed security mode none to transport to support https.
Now, everything works without the issue.
Two possible things which causes memory issues.
wsHttpBinding overheads
Stream mode is not supported in wsHttpBinding

Error 413 Request Entity Too Large

I am running into the error (413) Request Entity Too Large. I have done some searching around and all I can see is that the maxRecievedMessageSize needs to be added to the binding and that binding needs to be added in the bindingConfiguration. I have taken those steps on both my client web.config and service web.config.
Client
<bindings>
<basicHttpBinding>
<binding name="secureHttpBinding" maxReceivedMessageSize="2147483647" maxBufferPoolSize="2147483647" maxBufferSize="2147483647">
<readerQuotas maxDepth="2000000" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="Transport">
<transport clientCredentialType="None"/>
</security>
</binding>
</basicHttpBinding>
</bindings>
...
<endpoint address="https://mysite.com/WcfServices/MyService.svc"
binding="basicHttpBinding" bindingConfiguration="secureHttpBinding"
contract="MyService.IMyService" name="BasicHttpBinding_IMyService" />
WCF
<basicHttpBinding>
<binding name="secureHttpBinding" maxBufferPoolSize="2147483647" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647">
<readerQuotas maxDepth="2000000" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
<security mode="Transport">
<transport clientCredentialType="None"/>
</security>
</binding>
</basicHttpBinding>
I do not have any <service> tags specified explicitly in my WCF web.config, but I have had success before not explicitly defining the tag. Are there any other issues that could be in play here?

When using SSL/TLS, IIS can, under certain circumstances, buffer the entire HTTP request before passing it off to WCF. The default size of this buffer is only 48k, and if the buffer is exceeded IIS will return a 413 error.
To work around this, the size of the buffer has to be increased via the uploadReadAheadSize setting. In IIS 7.5, you can get to this setting via IIS Manager:
Select the web site in question under "Sites".
Launch "Configuration Editor".
Select Section: system.webServer/serverRuntime, From: ApplicationHost.config.
Set uploadReadAheadSize to the desired buffer size in bytes. Make sure it's larger than the largest request you might send.
Click "Apply".
For other IIS versions this setting may be in a different place.
There can be other reasons for getting a 413 as well, but this is the one I've run into. Of course, if you're not hosting the service in IIS this obviously wouldn't apply.

The basicHttpBinding you have defined in the service config file is not being used because it is not assigned to an endpoint. In WCF 4.0+, you will get default endpoints with default bindings if you don't explicitly specify an endpoint - and the default binding for http is basicHttpBinding. But this also means you will get the default values for the binding.
You can either add a service endpoint explicitly and assign your binding to it, or you can make your definition the default definition for basicHttpBinding by omitting the name attribute in the definition, like this:
<basicHttpBinding>
<binding maxBufferPoolSize="2147483647" maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647">
<readerQuotas maxDepth="2000000" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"/>
<security mode="Transport">
<transport clientCredentialType="None"/>
</security>
</binding>
</basicHttpBinding>
The above config snippet will then set your definition for basicHttpBinding as the default for any services uses basicHttpBinding in that application.

The request channel timed out while waiting for a reply

I have a small application that uses WCF to communicate with a webserver. This program is used by some 200 clients, and each client is sending in about 5-20 requests/min.
Looking at the error logs I often get:
The request channel timed out while waiting for a reply after
00:00:59.9989999
The requests are made as follows:
ClientService Client = new ClientService();
Client.Open();
Client.updateOnline(userID);
Client.Close();
This is the app.config
<configuration>
<configSections>
</configSections>
<startup><supportedRuntime version="v2.0.50727"/></startup><system.serviceModel>
<bindings>
<wsHttpBinding>
<binding name="WSHttpBinding_IClientService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true"
allowCookies="false">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="None">
<transport clientCredentialType="Windows" proxyCredentialType="None"
realm="" />
<message clientCredentialType="Windows" negotiateServiceCredential="true" />
</security>
</binding>
</wsHttpBinding>
</bindings>
<client>
<endpoint address="PATH TO SERVICE"
binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_IClientService"
contract="IClientService" name="WSHttpBinding_IClientService">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
</client>
</system.serviceModel>
Of the "many calls" each day about 200-800 fail. And about n-1 are ok. I am very confused what the issue can be. Looking at the server stats, it's hardly building up a sweat - each request takes <2 sec to process. The data it's sending consists of "int" or "very small strings" - so, its not due to size, if it were - there should be a consistent failure..
Suggestions?

Try adding the timeout values to both the service AND the client:
<binding name="BasicHttpBinding_SomeName" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00"
sendTimeout="00:10:00" maxBufferPoolSize="2147483647"
maxBufferSize="2147483647" maxReceivedMessageSize="2147483647">

It seems that your requests are queuing up on server before being handled and then starts to time out. You can try couple of things here to see exactly whats going on
1) Try throttling in your WCF services e.g: Try increasing your concurrent sessions. Take a look
WCF Throttling
2) Try using PerCall rather than Using Sessions here to make sure that no session remains there.
Here is what you can do on your interface to remove session
[ServiceContract(Namespace="YOUR NAMESPACE", SessionMode=SessionMode.NotAllowed)]
and your contract classes implementing those interface
ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
AND .... You should enable tracing to see exactly whats going on .

Check this performance counter - ASP.NET Requests Queued.
One web application can host many web pages and services. IIS processes only 12 concurrent requests per CPU core by default.
This means that even if your service is fast but other pages/services are slow your request has to wait in queue before they are executed.
ASP.NET Requests Queued should be zero or close to zero.
See Performance Counters for ASP.NET on MSDN.

Send Large Data with WCF without Streaming

We are trying to build a web service that can send/receive large data files from kb~gb sizes.
I know that we should use streaming to preform that so is not going to crash the memory.
But problem is we don't want to change our design, or more like, now we are going to try to see what will happen when we transfers large file say just 150mb file in a byte[] buffered transfers mode with WCF. (I know is goign to buffer the whole file into memory... and crash/exception if we transfer gb size files...)
But even so, they want to see what happen, so I have my wsHttpBinding in WCF config:
<system.web>
<compilation debug="true"/>
<httpRuntime maxRequestLength="524288"/>
</system.web>
....
<wsHttpBinding>
<binding name="wsHttpBinding1" closeTimeout="00:10:00" openTimeout="00:10:00"
receiveTimeout="00:30:00" sendTimeout="00:30:00" bypassProxyOnLocal="false"
hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="52428800" maxReceivedMessageSize="65536000"
messageEncoding="Mtom" useDefaultWebProxy="true">
<security mode="None" />
</binding>
</wsHttpBinding>
Then my client app config:
<wsHttpBinding>
<binding name="WSHttpBinding_IPrintService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:30:00" sendTimeout="00:30:00"
bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="52428800" maxReceivedMessageSize="65536000"
messageEncoding="Mtom" textEncoding="utf-8" useDefaultWebProxy="true"
allowCookies="false">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="None"/>
</binding>
</wsHttpBinding>
When I transfers small data it works fine, but when I try to transfers 150mb file, it gives me exceptions:
InnerException = {"Unable to write data to the transport
connection: An operation on a socket could not be performed because
the system lacked sufficient buffer space or because a queue was
full."}
I tried to increase maxReceivedMessage and such, but it still having the same problem....
======
Edit: I tried to increase all possible value above to... 2147483647.... I can sent a 30mb file successfully..but still not 150mb.... And I also tried to use this CustomBinding:
<customBinding>
<binding name="LargeCustomBinding"
receiveTimeout="01:00:00"
openTimeout="01:00:00"
closeTimeout="01:00:00"
sendTimeout="01:00:00">
<binaryMessageEncoding
maxReadPoolSize="2147483647"
maxSessionSize="2147483647"
maxWritePoolSize="2147483647">
<readerQuotas
maxArrayLength="2147483647"
maxDepth="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"
maxStringContentLength="2147483647" />
</binaryMessageEncoding>
<httpTransport
maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647"
maxBufferPoolSize="2147483647"/>
</binding>
</customBinding>
But I am still getting the same problem....
=========
Edit 2: I guess I should put concern on how I sending the data, maybe I was wrong, I have an Object MyData:
byte[] LargeData = File.ReadAllBytes(this.LargeDataPath.Text);
MyData.LargeData = new List<KeyValuePair<string, byte[]>>()
{
new KeyValuePair<string, byte[]>("DATA", LargeData)
};
myServiceClient.SendDataAsync( new SendDataRequest(MyData));
There should be no problem? All I did is just sending one big byte[] inside a list.....

You should be sending a smaller chunks of data with a separate send method calls.
Put the data you want to send into a byte array.
Then realize the logic on both ends, that will know how to handle sizes and just start sending the data in a smaller pieces, one by one. When you have sent an entire file, it will get concatenated into an array on a client and you will be able to do further processing.

Fix this first:
maxStringContentLength="8192"
maxArrayLength="16384"
And ensure you have
maxStringContentLength="2147483646"
maxArrayLength="2147483646"
On both client and server.
I was sending 30Mb over WCF without problems.
And check out this

WCF listenBacklog and maxConnections can't be set higher than 10. Why not?

My service works great under low load. But under high load I start to get connection errors.
I know about other settings but I am trying to change the listenBacklog parameter in particular for my TCP Buffered binding.
If I set listenBacklog="10" I am able to telnet into the port where my WCF service is running.
If I change listenBacklog to anything higher than 10 it will not let me telnet into my service when it is running.
No errors seem to be thrown.
What can I do?
I get the same problem when I change my maxConnections away from 10. All other properties of the binding I can set higher without a problem.
Here is what my binding looks like:
<bindings>
<netTcpBinding>
<binding name="NetTcpBinding_IMyService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
transactionFlow="false" transferMode="Buffered" transactionProtocol="OleTransactions"
hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxBufferPoolSize="524288"
maxBufferSize="1048576" maxConnections="10" maxReceivedMessageSize="1048576">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="Transport">
<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign">
</transport>
<message clientCredentialType="Windows" />
</security>
</binding>
...
I really need to increase the values of maxConnections and listenBacklog

If you're running on a Win2000, XP, Vista or Win7 machine, then the OS is limiting you to 10 concurrent TCP connections. Try running on a Windows Server machine to confirm.

This is almost certainly a throttling issue. Try specifying non-default values for the service throttling behavior values. See this page for some guidance.

Try to remove all references to HTTP bindings and MEX.