I'm trying to set up some tests to see how WCF services work. I have code similar to this for basicHttp, netTcp, netPipe bindings calling a WCF service with those bindings.
using (_log = new StreamWriter(#"C:\TestRunnerLog.txt", true))
{
var methodCallInformation = new MethodCallInformation { Method = NetPipeServiceCall, TotalTimeTakenForAllMethodCalls = new TimeSpan(), TotalNumberOfTimesMethodCalled = 0, TimesErrored = 0 };
for (var i = 0; i < 25000; i++)
{
try
{
var stopwatch = new Stopwatch();
stopwatch.Start();
using(var channelFactory = new ChannelFactory<IChatHub>(new NetTcpBinding { Security = new NetTcpSecurity { Mode = SecurityMode.None } }, "net.tcp://localhost/test/test.svc"))
{
var channel = channelFactory.CreateChannel();
await channel.GetPeople();
}
stopwatch.Stop();
return stopwatch.Elapsed;
methodCallInformation.TotalTimeTakenForAllMethodCalls += stopwatch.Elapsed;
}
catch (Exception e)
{
methodCallInformation.TimesErrored++;
}
methodCallInformation.TotalNumberOfTimesMethodCalled++;
}
await _log.WriteLineAsync(string.Format("net.tcp:\tTimesCalled:{0}\tTotalTime:{1}\tAverageTime:{2}\tErrors:{3}", methodCallInformation.TotalNumberOfTimesMethodCalled, methodCallInformation.TotalTimeTakenForAllMethodCalls, new TimeSpan(methodCallInformation.TotalTimeTakenForAllMethodCalls.Ticks / methodCallInformation.TotalNumberOfTimesMethodCalled), methodCallInformation.TimesErrored));
}
I'm confused by the results I'm getting though. I'm running all iterations locally and have the service and client all running locally on my machine. Based on my research, netPipe should be the fastest, followed by netTcp and then basicHttp. The code I have randomizes these calls instead of the each loop only calling one binding for all iterations as in the sample above.
----------------------------------------------------------------------------------------------
http: TimesCalled:24981 TotalTime:00:00:46.0452454 AverageTime:00:00:00.0018432 Errors:0
net.tcp: TimesCalled:25168 TotalTime:00:00:16.7776697 AverageTime:00:00:00.0006666 Errors:17593
net.pipe: TimesCalled:24786 TotalTime:00:00:50.5698790 AverageTime:00:00:00.0020402 Errors:0
I'm trying to figure out what I'm doing incorrectly to get all of those errors with net.tcp and why the timing seems to be so slow for net.pipe.
Here are the bindings I've set up on the service:
<bindings>
<basicHttpBinding>
<binding name="basicHttpDefaultBinding" maxReceivedMessageSize="2147483647" maxBufferPoolSize="2147483647" maxBufferSize="2147483647">
<readerQuotas maxArrayLength="2147483647" maxStringContentLength="2147483647" maxDepth="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="None">
<transport clientCredentialType="None" />
</security>
</binding>
</basicHttpBinding>
<netTcpBinding>
<binding name="netTcpDefaultBinding" maxBufferPoolSize="2147483647" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647">
<readerQuotas maxArrayLength="2147483647" maxStringContentLength="2147483647" maxDepth="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="None">
<transport clientCredentialType="None" />
<message clientCredentialType="None" />
</security>
</binding>
</netTcpBinding>
<netNamedPipeBinding>
<binding name="netNamedPipeDefaultBinding" maxBufferPoolSize="2147483647" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647">
<readerQuotas maxArrayLength="2147483647" maxStringContentLength="2147483647" maxDepth="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="None">
</security>
</binding>
</netNamedPipeBinding>
</bindings>
Any help would be greatly appreciated.
Your NetTcp test is likely completing so much faster due to the fact that it is throwing an exception ~17,000 out of ~25,000 times. You are more or less just testing how fast your application can throw exceptions.
Running locally, the differences between the bindings are going to be almost imperceptible and environmental factors will likely be enough to skew your results. Over a network the results may be different, primarily due to message encoding (binary vs. text with binary being more efficient). That hopefully answers your question of why Named Pipes are "slow" in comparison to NetTcp.
Secondly, to determine why NetTcp is failing, you need to stop hiding the exception.
catch (Exception e)
{
// you should also be reporting your exception somewhere here...
methodCallInformation.TimesErrored++;
}
Or just set Visual Studio to break on all exceptions and run in debug mode. Speculating why something might be producing an error, in my experience, is generally not a good way to troubleshoot a problem. It is a method that should be reserved only when there are no other options available. If you provide the exception details, we can likely provide more insight.
Related
I am using WCF service. The problem I have is its starts using double memory.
I am using HTTPS binding
<wsHttpBinding>
<binding name="secureHttpBinding" closeTimeout="04:01:00" openTimeout="04:01:00" receiveTimeout="04:10:00" sendTimeout="04:01:00" allowCookies="false"
bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true">
<readerQuotas maxDepth="128" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
<security mode="Transport" >
<transport clientCredentialType="None"/>
</security>
</binding>
</wsHttpBinding>
<endpoint address="https://localhost/test.svc"
binding="wsHttpBinding"
bindingConfiguration="secureHttpBinding"
contract="IWcfContract"
name="SecureBasicHttpBinding_WcfContract">
Here is the code I am using to upload
using (Stream fileStream = File.OpenRead(logsZipFullPath))
{
// Call web server
UploadFileResponse response = _webServiceHelper.UploadFile(fileStream, currentDate, ".zip", string.Empty);
fileStream.Close();
}
Here is my model
[MessageContract]
public class UploadFileRequest : IDisposable
{
[MessageBodyMember(Order = 1)]
public Stream FileByteStream;
[MessageHeader(MustUnderstand = true)]
public string FileDescription;
}
My zip file is of 80MB.
The problem I have is at the start of the service its using 26mb which is quite fine. At the first call it uses 136MB after call completes it goes down to 26mb. which is also fine. after the second call to upload it starts using 346MB
which again gets down to 26mb after service call. My question is why it is using 346MB when the file is of only 80MB? My GC and disponse has been called correctly. But, is this normal behaviour or I am missing anything?
finally found a work around for this. According to this post
wsHttpBinding is the full-blown binding, which supports a ton of WS-*
features and standards - it has lots more security features, you can
use sessionful connections, you can use reliable messaging, you can
use transactional control - just a lot more stuff, but wsHttpBinding
is also a lot *heavier" and adds a lot of overhead to your messages as
they travel across the network
I think this is the main reason why the memory usage is high. Our requirement was to use the HTTPS certificate and we are able to use this with basicHttpBinding. SO, I changed my settings like this
<basicHttpBinding>
<binding name="BasicHttpBinding_WcfContract" closeTimeout="04:01:00" openTimeout="04:01:00" receiveTimeout="04:10:00" sendTimeout="04:01:00" allowCookies="false"
bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard" maxBufferSize="2147483647" maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647" messageEncoding="Text" textEncoding="utf-8" transferMode="Streamed" useDefaultWebProxy="true">
<readerQuotas maxDepth="128" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
<security mode="Transport"> <-- this is the main change
<transport clientCredentialType="None" proxyCredentialType="None" realm=""/>
<message clientCredentialType="UserName" algorithmSuite="Default"/>
</security>
</binding>
</basicHttpBinding>
Note: I changed security mode none to transport to support https.
Now, everything works without the issue.
Two possible things which causes memory issues.
wsHttpBinding overheads
Stream mode is not supported in wsHttpBinding
i'm currently working on a client/server program which is using wshttpdualbinding for communication in vs with c#. So i'm calling webservices over this binding from the server.
Everything worked fine, but in the last days i'm encountering the following problem: if i call a service from the server, the client just stops working and stops reacting. Nothing gets send from the client.
I can't post much, because it is a big project, but i'll try to post the best snippets:
binding:
<bindings>
<basicHttpBinding>
<binding name="BasicHttpBinding_AuthenticationService" closeTimeout="03:10:00"
openTimeout="03:10:00" receiveTimeout="03:10:00" sendTimeout="03:10:00"
bypassProxyOnLocal="false" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
useDefaultWebProxy="true" allowCookies="false" messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered" >
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="None">
<transport clientCredentialType="None" proxyCredentialType="None"
realm="" />
<message clientCredentialType="UserName" algorithmSuite="Default" />
</security>
</binding>
</basicHttpBinding>
<wsDualHttpBinding>
<binding name="WSDualHttpBinding_Service" closeTimeout="03:10:00"
openTimeout="03:10:00" receiveTimeout="03:10:00" sendTimeout="03:10:00"
bypassProxyOnLocal="false" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
useDefaultWebProxy="true" messageEncoding="Text" textEncoding="utf-8" >
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="None">
<message clientCredentialType="UserName" algorithmSuite="Default"/>
</security>
</binding>
</wsDualHttpBinding>
</bindings>
problem causing code:
Clusterrationalemapping cr = new Clusterrationalemapping
{
Textbricks = rationale,
Cluster = cluster,
Project = CurrentProject
};
var clusterrationaleId = 0;
if (cluster.AddClusterRationaleMapping(cr))
{
clusterrationaleId = _service.SaveItem(cr);
cr.Id = clusterrationaleId;
if(rationale.Clusterrationalemappings == null) rationale.Clusterrationalemappings = new List<Clusterrationalemapping>();
rationale.Clusterrationalemappings.Add(cr);
}
A function gets called which creates a new clusterrationalemapping for a specific cluster. The rational and CurrentProject variables are always the same. just cluster is different.
Most of the time everything works and service.SaveItem returns the new Id as expected. But for one special cluster it is not working. service.SaveItem just stops and freezes the tool until i get some exception (server did not respond in time/gave no answer).
I'm sure that the wsDualHttpBinding is set correctly. If i comment out the line Project = CurrentProject and replace it with Project = new Project(); everything works fine for this cluster again.
So it looks like clusterrationalmapping might get too big for sending. But the exception is not saying that. And i got it from the server - from that direction it worked.
I used Microsoft Service Trace Viewer for debugging purpose and i can see there, that it instantly throws an exception that the endpoint could not be reached. So i'm pretty sure that the service dies when serializing the object.
Is there any way to debug into the service to see when exactly, or better why, it fails?
Thanks in advance for your help.
Since I have not been able to find a solution - I still think that there has been a problem with the serializer and a too big/nested class, i changed the binding to nettcp and it is working now.
I am running into the error (413) Request Entity Too Large. I have done some searching around and all I can see is that the maxRecievedMessageSize needs to be added to the binding and that binding needs to be added in the bindingConfiguration. I have taken those steps on both my client web.config and service web.config.
Client
<bindings>
<basicHttpBinding>
<binding name="secureHttpBinding" maxReceivedMessageSize="2147483647" maxBufferPoolSize="2147483647" maxBufferSize="2147483647">
<readerQuotas maxDepth="2000000" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<security mode="Transport">
<transport clientCredentialType="None"/>
</security>
</binding>
</basicHttpBinding>
</bindings>
...
<endpoint address="https://mysite.com/WcfServices/MyService.svc"
binding="basicHttpBinding" bindingConfiguration="secureHttpBinding"
contract="MyService.IMyService" name="BasicHttpBinding_IMyService" />
WCF
<basicHttpBinding>
<binding name="secureHttpBinding" maxBufferPoolSize="2147483647" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647">
<readerQuotas maxDepth="2000000" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
<security mode="Transport">
<transport clientCredentialType="None"/>
</security>
</binding>
</basicHttpBinding>
I do not have any <service> tags specified explicitly in my WCF web.config, but I have had success before not explicitly defining the tag. Are there any other issues that could be in play here?
When using SSL/TLS, IIS can, under certain circumstances, buffer the entire HTTP request before passing it off to WCF. The default size of this buffer is only 48k, and if the buffer is exceeded IIS will return a 413 error.
To work around this, the size of the buffer has to be increased via the uploadReadAheadSize setting. In IIS 7.5, you can get to this setting via IIS Manager:
Select the web site in question under "Sites".
Launch "Configuration Editor".
Select Section: system.webServer/serverRuntime, From: ApplicationHost.config.
Set uploadReadAheadSize to the desired buffer size in bytes. Make sure it's larger than the largest request you might send.
Click "Apply".
For other IIS versions this setting may be in a different place.
There can be other reasons for getting a 413 as well, but this is the one I've run into. Of course, if you're not hosting the service in IIS this obviously wouldn't apply.
The basicHttpBinding you have defined in the service config file is not being used because it is not assigned to an endpoint. In WCF 4.0+, you will get default endpoints with default bindings if you don't explicitly specify an endpoint - and the default binding for http is basicHttpBinding. But this also means you will get the default values for the binding.
You can either add a service endpoint explicitly and assign your binding to it, or you can make your definition the default definition for basicHttpBinding by omitting the name attribute in the definition, like this:
<basicHttpBinding>
<binding maxBufferPoolSize="2147483647" maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647">
<readerQuotas maxDepth="2000000" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"/>
<security mode="Transport">
<transport clientCredentialType="None"/>
</security>
</binding>
</basicHttpBinding>
The above config snippet will then set your definition for basicHttpBinding as the default for any services uses basicHttpBinding in that application.
We are trying to build a web service that can send/receive large data files from kb~gb sizes.
I know that we should use streaming to preform that so is not going to crash the memory.
But problem is we don't want to change our design, or more like, now we are going to try to see what will happen when we transfers large file say just 150mb file in a byte[] buffered transfers mode with WCF. (I know is goign to buffer the whole file into memory... and crash/exception if we transfer gb size files...)
But even so, they want to see what happen, so I have my wsHttpBinding in WCF config:
<system.web>
<compilation debug="true"/>
<httpRuntime maxRequestLength="524288"/>
</system.web>
....
<wsHttpBinding>
<binding name="wsHttpBinding1" closeTimeout="00:10:00" openTimeout="00:10:00"
receiveTimeout="00:30:00" sendTimeout="00:30:00" bypassProxyOnLocal="false"
hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="52428800" maxReceivedMessageSize="65536000"
messageEncoding="Mtom" useDefaultWebProxy="true">
<security mode="None" />
</binding>
</wsHttpBinding>
Then my client app config:
<wsHttpBinding>
<binding name="WSHttpBinding_IPrintService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:30:00" sendTimeout="00:30:00"
bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="52428800" maxReceivedMessageSize="65536000"
messageEncoding="Mtom" textEncoding="utf-8" useDefaultWebProxy="true"
allowCookies="false">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="None"/>
</binding>
</wsHttpBinding>
When I transfers small data it works fine, but when I try to transfers 150mb file, it gives me exceptions:
InnerException = {"Unable to write data to the transport
connection: An operation on a socket could not be performed because
the system lacked sufficient buffer space or because a queue was
full."}
I tried to increase maxReceivedMessage and such, but it still having the same problem....
======
Edit: I tried to increase all possible value above to... 2147483647.... I can sent a 30mb file successfully..but still not 150mb.... And I also tried to use this CustomBinding:
<customBinding>
<binding name="LargeCustomBinding"
receiveTimeout="01:00:00"
openTimeout="01:00:00"
closeTimeout="01:00:00"
sendTimeout="01:00:00">
<binaryMessageEncoding
maxReadPoolSize="2147483647"
maxSessionSize="2147483647"
maxWritePoolSize="2147483647">
<readerQuotas
maxArrayLength="2147483647"
maxDepth="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"
maxStringContentLength="2147483647" />
</binaryMessageEncoding>
<httpTransport
maxBufferSize="2147483647"
maxReceivedMessageSize="2147483647"
maxBufferPoolSize="2147483647"/>
</binding>
</customBinding>
But I am still getting the same problem....
=========
Edit 2: I guess I should put concern on how I sending the data, maybe I was wrong, I have an Object MyData:
byte[] LargeData = File.ReadAllBytes(this.LargeDataPath.Text);
MyData.LargeData = new List<KeyValuePair<string, byte[]>>()
{
new KeyValuePair<string, byte[]>("DATA", LargeData)
};
myServiceClient.SendDataAsync( new SendDataRequest(MyData));
There should be no problem? All I did is just sending one big byte[] inside a list.....
You should be sending a smaller chunks of data with a separate send method calls.
Put the data you want to send into a byte array.
Then realize the logic on both ends, that will know how to handle sizes and just start sending the data in a smaller pieces, one by one. When you have sent an entire file, it will get concatenated into an array on a client and you will be able to do further processing.
Fix this first:
maxStringContentLength="8192"
maxArrayLength="16384"
And ensure you have
maxStringContentLength="2147483646"
maxArrayLength="2147483646"
On both client and server.
I was sending 30Mb over WCF without problems.
And check out this
My service works great under low load. But under high load I start to get connection errors.
I know about other settings but I am trying to change the listenBacklog parameter in particular for my TCP Buffered binding.
If I set listenBacklog="10" I am able to telnet into the port where my WCF service is running.
If I change listenBacklog to anything higher than 10 it will not let me telnet into my service when it is running.
No errors seem to be thrown.
What can I do?
I get the same problem when I change my maxConnections away from 10. All other properties of the binding I can set higher without a problem.
Here is what my binding looks like:
<bindings>
<netTcpBinding>
<binding name="NetTcpBinding_IMyService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
transactionFlow="false" transferMode="Buffered" transactionProtocol="OleTransactions"
hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxBufferPoolSize="524288"
maxBufferSize="1048576" maxConnections="10" maxReceivedMessageSize="1048576">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="Transport">
<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign">
</transport>
<message clientCredentialType="Windows" />
</security>
</binding>
...
I really need to increase the values of maxConnections and listenBacklog
If you're running on a Win2000, XP, Vista or Win7 machine, then the OS is limiting you to 10 concurrent TCP connections. Try running on a Windows Server machine to confirm.
This is almost certainly a throttling issue. Try specifying non-default values for the service throttling behavior values. See this page for some guidance.
Try to remove all references to HTTP bindings and MEX.