I have a wcf server using NetNamedPipesBinding.
I can see when the server is loaded with requests the reply is very slow (1-7 seconds).
The application code runs very fast but the time between sending the reply and receiving the reply takes long.
Is this because there are lots of messages at the pipe and they are processed sequentially ? is there a way to improve that ?
there are only 2 processes involves (caller and service) and the calls are 2 way, the caller process uses different threads to call.
Thanks.
If you are creating a separate Thread for each request, you could be starving your system. Since both client and server are on the same machine, it may be the client's fault the server is slow.
There are lots of ways to do multithreading in .NET and a new Thread may be the worst. At worst you should move your calls to the thread pool (http://msdn.microsoft.com/en-us/library/3dasc8as.aspx)
or you may want to use the async methods of the proxy (http://msdn.microsoft.com/en-us/library/ms730059.aspx).
Related
I am using WCF and NamedPipes for InterProcess communication.
Each process listens on a unique named pipe easily discoverable by other processes.
The problem i am getting has to do with 2 processes trying to talk to each other at almost the same time. I assume on different pipes but who knows what goes on with WCF...
Essentially the flow is this : process 1 calls a process 2 method on pipe 2 at almost the same time that process 2 calls a process 1 method on pipe 1. When this happens i keep getting TimeoutExceptions on my ICommunicationObject.Close() call.
I have tried NamedPipes and NetTcp and both have the same problem. BasicHttp on the other hand does not...Works fine but i should not have to use BasicHttp for IPC on the same machine.
Another important info. If i enqueue my reply call using Dispatcher.BeginInvoke() then it works.
It looks like enqueuing the next call gives WCF time to complete the first call before starting the next call back. But why does that even matter when the processes are communicating ON DIFFERENT PIPES??
Any input is greatly appreciated
This a classic deadlock situation. Both sides post work on each other's UIThread and both are waiting for the other blocked side to reply.
I switched the callers to instead use Tasks and Dispatch the remote calls on a background thread and all is fine.
I have a client .NET application and a server .NET application, connected through sockets.
The client sends a string of 20 or so characters every 500 milliseconds.
On my local development machine, this works perfectly, but once the client and the server are on two different servers, the server is not receiving the string immediately when it's sent. The client still sends perfectly, I've confirmed this with Wireshark. I have also confirmed that the the server does receive the strings every 500 milliseconds.
The problem is that my server application that is waiting for the message only actually receives the message every 20 seconds or so - and then it receives all the content from those 20 seconds.
I use asynchronous sockets and for some reason the callback is just not invoked more than once every 20 seconds.
In AcceptCallback it establishes the connection and call BeginReceive
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0, new AsyncCallback(ReadCallback), state);
This works on my local machine, but on my production server the ReadCallback doesn't happen immediately.
The BufferSize is set to 1024. I also tried setting it to 10. It makes a difference in how much data it will read from the socket at one time once the ReadCallback is invoked, but that's not really the problem here. Once it invokes ReadCallback, the rest works fine.
I'm using Microsofts Asynchronous Server Socket Example so you can see there what my ReadCallback method looks like.
How can I get the BeginReceive callback immediately when data arrives at the server?
--
UPDATE
This has been solved. It was because the server had a a single processor and single core. After adding another core, the problem was instantly solved. ReadCallback is now called immediately when the call goes through to the server.
Thankyou all for your suggestions!!
One approach might be to adjust the SO_SNDBUF option for the send side. SInce you are not running into this problem when both server/client are on the same box, it is possible that having a small buffer is throttling the send side since due to (a possible) slower sending rate between the servers. If the sender cannot send fast enough, then the send-side buffer might be filling up sooner.
Update: we did some debugging and turns out that the issue is with the application being slower.
It might be that the Nagle algorithm is waiting on the sender side for more packets. If you are sending small chunks of data, they will be merged in one so you don't pay a huge TCP header overhead for small data.
You can disable it using: StreamSocketControl.NoDelay
See: http://msdn.microsoft.com/en-us/library/windows/apps/windows.networking.sockets.streamsocketcontrol.nodelay
The Nagle algorithm might be disabled for loopback and this is a possible explanation of why it works when you have both the sender and the receiver on the same machine.
By the request of OP, duplicating my "comment/answer" here.
My guess was, the problem appeared because of thread scheduling on a single-core machine. This is an old problem, almost extinct in the modern age of hyper-threading/multi-core processors. When a thread is spawned in the course of execution of the program, it needs scheduled time to run.
On a single-core machine, if one thread continues to execute without explicitly passing control to OS scheduler (by waiting for mutex/signal or by calling Sleep), the execution of any other thread (in the same process and with lower priority) may be postponed indefinitely by the scheduler. Hence, in the case described, the asynchronous network thread was (most likely) just starved for execution time - getting only pieces from time to time.
Adding second CPU/core, obviously, fixed that by providing a parallel scheduling environment.
I am developing a Windows RT application that needs to get data from a MVC WebApi server.
The problem is that the response can take from few seconds to 3 minutes.
Which is the best approach to solve it?
For now, I call async to the web api and put a long timeout value to avoid exceptions. Is it a good way? I do not like too much because the server have a open connection opened all time. Can it affect significantly to the server performance?
Is there some thing like "callback" but for web services? I mean that the server calls to the client to send the data.
Yes, there are ways to get server to callback client, for example WCF duplex communication. However, such techniques will usually keep the connection open (in most cases this is TCP session). Most web servers do not support numerous concurrent requests and thus each prolonged call to the server will increment the number of concurrently connected clients. This will lead to heavy resource utilisation at the point where it shouldn't be. If you have many clients, such architecture is bound to fail.
REST requests shall be lightweight, small and fast. Consider using a database to store temporary results and worker servers, to process the load. This is a server-side problem, not client-side.
Finally I solved it using WebSockets (thanks oleksii). It keeps the connection open but I avoid to poll for the result repeatedly. Now, when the server finishes the process, sends the data directly to the client. WebSockets is a protocol that relays over TCP and has been standardized.
http://en.wikipedia.org/wiki/WebSocket
Edit (again): Let me simplify my problem. I have a Windows Service that exposes some WCF endpoints with methods like:
int ExecuteQuery(string query) {
// asynchronously execute query that may take 1 second to 20 minutes
return queryId;
}
string GetStatus(int queryId) {
// return the status of the query (# of results so far, etc)
}
What is the best way to implement the ExecuteQuery method? Should I just call ThreadPool.QueueUserWorkItem to get my query going?
Note that the actual work behind executing a query is done by load-balanced black box. I want to be able to have several queries going at the same time.
The analogy is a web browser that is downloading multiple files simultaneously and you have a download manager that can track the status of each file.
Take a look at Microsoft Message Queuing (MSMQ):
Microsoft Message Queuing (MSMQ) technology enables applications running at different times to communicate across heterogeneous networks and systems that may be temporarily offline. MSMQ provides guaranteed message delivery, efficient routing, security, and priority-based messaging. It can be used to implement solutions for both asynchronous and synchronous messaging scenarios.
It's good to know that Windows Communication Foundation (WCF) can leverage queuing services offered by MSMQ.
Either this is a trick question or a no-brainer... ThreadPool.QueueUserWorkItem is about the easiest way to go when you want to execute a piece of code concurrently. I'm sure you already knew that, so technically you have already answered your own question.
So if this is not a trick question, then are you asking exactly how to pass the query in the ThreadPool.QueueUserWorkItem?
I use a Windows service for a very similar task and it works very well. I use database tables to queue requests and responses, as it gives me a persistent queue that can be accessed over the network from remote ASP.Net applications, and concurrency control through transactions.
A supervisor thread on a timer spawns workers whenever incoming requests need servicing. I use a separate database tables for configuration and control so that I can administer the service and pause the supervisor from an application without while leaving the service core running. Logging to a separate table is a convenient way to see what's happening from web apps and a local admin app.
I wouldn't use the ThreadPool for long-running threads, but instead create a worker class that runs in its own thread and uses callback methods to update the supervisor with progress and completion status.
Adding to the MSMQ answer, you could think about looking at using an Enterprise Service Bus (ESB) to handle these sorts of things, if future scalability is a concern. Check out NServiceBus for one .NET example.
I would use WWF (4.0):
You can start long running transactions that can be handle in a few machines, execute task in parallel, failure support, friendly coding, you can manage it with appfabric, it is free...
We have a remoting singleton server running in a separate windows service (let's call her RemotingService). The clients of the RemotingService are ASP.NET instances (many many).
Currently, the clients remoting call RemotingService and blocks while the RemotingService call is serviced. However, the remoting service is getting complicated enough (with more RPC calls and complex algorithms) that the asp.net worker threads are blocked for a significantly long time (4-5 seconds).
According to this msdn article, doing this will not scale well because an asp.net worker thread is blocked for each remoting RPC. It advises switching to async handlers to free up asp.net worker threads.
The purpose of an asynchronous handler
is to free up an ASP.NET thread pool
thread to service additional requests
while the handler is processing the
original request.
This seems fine, except the remoting call still takes up a thread from the thread pool.
Is this the same thread pool as the asp.net worker threads?
How should I go about turning my remoting singleton server into an async system such that I free up my asp.net worker threads?
I've probably missed out some important information, please let me know if there is anything else you need to know to answer the question.
The idea behind using the ThreadPool is that through it you can control the amount of synchronous threads, and if those get too many, then the thread pool automatically manages the waiting of newer threads.
The Asp.Net worked thread (AFAIK) doesn't come from the Thread Pool and shouldn't get affected by your call to the remoting service (unless this is a very slow processor, and your remoting function is very CPU intensive - in which case, everything on your computer will be affected).
You could always host the remoting service on a different physical server. In that case, your asp.net worker thread will be totally independent of your remoting call (if the remoting call is called on a separate thread that is).