I'm not exactly sure how a WCF proxy class will handle sending requests through plinq. Does the following code snippit look ok, or does it look like it could cause problems with shared state across multiple threads? (also, I already understand that a using block is not ideal, this is just an example)
using (var proxy = new ServiceProxyOfSomeSort())
{
_aBunchOfMessagesToSend.AsParallel()
.WithDegreeOfParallelism(SomeDegree).ForAll(m =>
{
proxy.SomeOperation(m);
}
}
Should I be creating the proxy once per thread? Is it ok to share the proxy across threads? I don't want to create more proxies than I need to because that is a somewhat expensive operation.
Edit:
I don't really have any of the implementation details of the service on the server side. From a requirements standpoint they should have developed it so that multiple clients could call it at anytime (async). Assuming that they can handle async calls (which may be a big assumption) I'm just trying to figure out if this an acceptable approach from the client side. It is working, I just don't know if there are any gotchas with this approach.
In response to your question about the proxy: you should create a new one per thread. Reusing the proxy tends to work for a while and then throw a fault after tens of requests. As they love to say, its behavior is undefined. The performance overhead of creating new proxies is not huge (after the first one is created), so it shouldn't be a big deal.
As for the discussion of multiple concurrent requests swamping the server: a few, or even a dozen, should be fine - the server can probably handle as many requests as your processor can create threads. Then again, the sudden influx of requests from a single source might be interpreted as a Denial Of Service attack, so you should be wary, particularly if your control over the service is limited.
Related
I have to add N (independent) items frequently in redis cache using stackexhange.redis C# each with different expiration time so that there is minimum time at client side and min blocking & cost at server side. Redis server will receive hundreds of get requests per second so I don't want to mess with the get time at all.
I have read the documentation here and answer here. I could not find a single method that does this operation. Considering different options:
Using transaction- This will block any operation at server side. So this should not be right solution
Using batching- This will block any operation at client side till all the batch operation is complete. This should not be right solution.
Using pipelining - This will not block any operation at client side and server side. But it can send multiple requests (packets less than N) may consume more network but may increase memory consumption at client side which may induce latency.
Using fire and forget - This also will not block any operation at client side and server side. But it will send multiple requests (more packets than pipelining) which may consume more network bandwidth but no memory consumption at client side.
Which should be the best approach?
I assumed competing operations means 2 inserts and one get and insert cannot go together though they may be accessing different keys. Am I correct in this otherwise what does it mean?
Redis is single-threaded when it comes to either read or write on a database.
What's the best solution in your case? Who knows, it might depend on a lot of variables and each use case should be analyzed separately to implement the right solution.
Redis MULTI can't be avoided unless you want to corrupt your data if something goes wrong in your application layer. Actually, if you want to avoid many requests to Redis, you should use Lua scripts instead.
In the other hand, the point of Redis is trying to make many operations but be sure that those are as small as possible because of the Redis single-threaded nature. Right, it's blazing fast, unless you execute an operation that takes too much time.
In summary, I wouldn't be too concerned about sending many requests as its an in-memory database and works at the lightspeed. Also, consider the wonders of Redis Cluster (i.e. sharding) to being able to optimize your scenario.
Finally, I would take a look at this Redis tutorial: Redis latency problems troubleshooting
You should add to your list of options, Lua scripting. See EVAL.
Also, consider the data structure that you will use. For example you can use MSET to send multiple values in Redis with one hop.
This question already has answers here:
Best Practice for WCF Service Proxy lifetime?
(4 answers)
Reuse of WCF service clients
(2 answers)
Closed 9 years ago.
I have a UI application in which I consume a WCF service like this
public MainWindow()
{
....
mServiceClient = new ServiceClient("EndPointTCP");
}
Should I create the client as a member of class and close the client along with exit of my application or Should I create a new client whenever its required in a method and close there itself.
It depends solely onwhat you want to achieve. There is no "best way to do it" since both ways are possible, good, and have different tradeoffs.
Holding the clientobject simply wastes resources. It also may leak context data between calls. You might have a bug that will cause mClient.Buy(100) and mClient.Sell(100) to work properly when used alone, but fail when used together mClient.Buy(100); mClient.Sell(100). Dropping and re-creating fresh instance each time could save you from that one bug, but obviously that's not a good argument for it.
Recreating the client each time a call is to be made has however the vague benefit of .. having a fresh client every time. If your app has a possibility to dynamically change the endpoint during the app's runtime, then automatically your client will be always using the newest addresses/logins/passwords/etc.
However, not recreating the client object at every time is simply faster. Mind that it's WCF layer, so the actual underlying connection can be any. Were it some protocol with heavy setup with some keyexchange, encryption, etc, you may find that creating a new client every time might create a new connection at every time and it will slow down eveyrthing, while keeping the instance will work blazingly fast, since connection might be kept open and reused. Often you try to keep the connection when you have to perform many and often calls to the service, like 24h/day twice/sec monitoring some remote value for breaching safe limits.
On yet the other hand, you might not want the connection to linger. Your remote service may have thousands of clients and limited resources, so you might want to close the connection ASAP so others may connect. Often you'd do it like that when the calls to the service are really rarely done, only once in a time, ie. when user clicks after returning from coffe break.
Please don't get me wrong: all above is just conjuring some vague "facts" from a void. I do not know your app, nor your service, nor your bindings (sans "endpoint TCP"). The most important factors are all on your side and they sit in the actual way how your app and that remote service work and interoperate. If you care about what you ask, you must first simply research the topic on your side. Best - simply try both ways and check if it works and how does it perform. The difference would be something like 2..6 lines of code, so that's, well, rather quick.
There are already some similar questions:
Reuse of WCF service clients
Reusing a WCF service client or creating one each time?
In my opinion it depends on your application type (scalability, performance requirements, ...) but usually I think that it's more safe to recreate the ServiceClient each time. In this way you don't need special code if there are connections problems between requests and with the latest version of WCF seems that there isn't a big performance impact.
See http://msdn.microsoft.com/en-us/library/aa738757.aspx.
Consider also that ServiceClient is not thread safe (at least reading MSDN).
I'm tasked to create a web application. I'm currently using c# & asp.net (mvc - but i doubt its relevant to the question) - am a rookie developer and somewhat new to .net.
Part of the logic in the application im building is to make requests to an external smsgateway by means of hitting a particular url with a request - either as part of a user-initiated action in the webapp (could be a couple of messages send) or as part of a scheduledtask run daily (could and will be several thousand message send).
In relation to a daily task, i am afraid that looping - say - 10.000 times in one thread (especially if im also to take action depending on the response of the request - like write to a db) is not the best strategy and that i could gain some performance/timesavings from some parallelization.
Ultimately i'm more afraid that thousands of users at the same time (very likely) will perform the action that triggers a request. With a naive implementation that spawns some kind of background thread (whatever its called) for each request i fear a scenario with hundreds/thousands of requests at once.
So if my assumptions are correct - how do i deal with this? do i have to manually spawn some appropriate number of new Thread()s and coordinate their work from a producer/consumer-like queue or is there some easy way?
Cheers
If you have to make 10,000 requests to a service then it means that the service's API is anemic - probably CRUD-based, designed as a thin wrapper over a database instead of an actual service.
A single "request" to a well-designed service should convey all of the information required to perform a single "unit of work" - in other words, those 10,000 requests could very likely be consolidated into one request, or at least a small handful of requests. This is especially important if requests are going to a remote server or may take a long time to complete (and 2-3 seconds is an extremely long time in computing).
If you do not have control over the service, if you do not have the ability to change the specification or the API - then I think you're going to find this very difficult. A single machine simply can't handle 10,000 outgoing connections at once; it will struggle with even a few hundred. You can try to parallelize this, but even if you achieve a tenfold increase in throughput, it's still going to take half an hour to complete, which is the kind of task you probably don't want running on a public-facing web site (but then, maybe you do, I don't know the specifics).
Perhaps you could be more specific about the environment, the architecture, and what it is you're trying to do?
In response to your update (possibly having thousands of users all performing an action at the same time that requires you to send one or two SMS messages for each):
This sounds like exactly the kind of scenario where you should be using Message Queuing. It's actually not too difficult to set up a solution using WCF. Some of the main reasons why one uses a message queue are:
There are a large number of messages to send;
The sending application cannot afford to send them synchronously or wait for any kind of response;
The messages must eventually be delivered.
And your requirements fit this like a glove. Since you're already on the Microsoft stack, I'd definitely recommend an asynchronous WCF service backed by MSMQ.
If you are working with SOAP, or some other type XML request, you may not have an issue dealing with the level of requests in a loop.
I set up something similar using a SOAP server with 4-5K requests with no problem...
A SOAP request to a web service (assuming .NET 2.0 and superior) looks something like this:
WebServiceProxyClient myclient = new WebServiceProxyClient();
myclient.SomeOperation(parameter1, parameter2);
myclient.Close();
I'm assuming that this code will will be embedded into your business logic that you will be trigger as part of the user initiated action, or as part of the scheduled task.
You don't need to do anything especial in your code to cope with a high volume of users. This will actually be a matter of scalling on your platform.
When you say 10.000 request, what do you mean? 10.000 request per second/minute/hour, this is your page hit per day, etc?
I'd also look into using an AsyncController, so that your site doesn't quickly become completely unusable.
I'm working on an application that may generate thousands of messages in a fairly tight loop on a client, to be processed on a server. The chain of events is something like:
Client processes item, places in local queue.
Local queue processing picks up messages and calls web service.
Web service creates message in service bus on server.
Service bus processes message to database.
The idea being that all communications are asynchronous, as there will be many clients for the web service. I know that MSMQ can do this directly, but we don't always have that kind of admin capability on the clients to set things up like security etc.
My question is about the granularity of the messages at each stage. The simplest method would mean that each item processed on the client generates one client message/web service call/service bus message. That's fine, but I know it's better for the web service calls to be batched up if possible, except there's a tradeoff between large granularity web service DTOs, versus short-running transactions on the database. This particular scenario does not require a "business transaction", where all or none items are processed, I'm just looking to achieve the best balance of message size vs. number of web service calls vs. database transactions.
Any advice?
Chatty interfaces (i.e. lots and lots of messages) will tend to have a high overhead from dispatching the incoming message (and, on the client, the reply) to the correct code to process the message (this will be a fixed cost per message). While big messages tend to use the resources in processing the message.
Additionally a lot of web service calls in progress will mean a lot of TCP/IP connections to manage, and concurrency issues (including locking in a database) might become an issue.
But without some details of the processing of the message it is hard to be specific, other than the general advice against chatty interfaces because of the fixed overheads.
Measure first, optimize later. Unless you can make a back-of-the-envelope estimate that shows that the simplest solution yields unacceptably high loads, try it, establish good supervisory measurements, see how it performs and scales. Then start thinking about how much to batch and where.
This approach, of course, requires you to be able to change the web service interface after deployment, so you need a versioning approach to deal with clients which may not have been redesigned, supporting several WS versions in parallel. But not thinking about versioning almost always traps you in suboptimal interfaces, anyway.
Abstract the message queue
and have a swappable message queue backend. This way you can test many backends and give yourself an easy bail-out should you pick the wrong one or grow to like a new one that appears. The overhead of messaging is usually packing and handling the request. Different systems are designed for different levels traffic and different symmetries over time.
If you abstract out the basic features you can swap the mechanics in and out as your needs change, or are more accurately assessed.
You can also translate messages from differing queue types at various portions of the application or message route as the recipient's stresses change because they are handling, for example 1000:1/s vs 10:1/s on a higher level.
Good Luck
I am working on a class library that retrieves information from a third-party web site. The web site being accessed will stop responding if too many requests are made within a set time period (~0.5 seconds).
The public methods of my library directly relate to a resource an file on the web server. In other words, each time a method is called, an HttpWebRequest is created and sent to the server. If all goes well, an XML file is returned to the caller. However, if this is the second web request in less than 0.5s, the request will timeout.
My dilemma lies in how I should handle request throttling (if at all). Obviously, I don't want the caller sit around waiting for a response -- especially if I'm completely certain that their request will timeout.
Would it make more sense for my library to queue and throttle the webrequests I create, or should my library simply throw an exception if the a client does not wait long enough between API calls?
The concept of a library is to give its client code as little to worry about as possible. Therefore I would make it the libraries job to queue requests and return results in a timely manner. In an ideal world you would use a callback or delegate model so that the client code can operate in asynchronously, not blocking the UI. You could also offer the option for skipping the queue, (and failing if it operates too soon) and possibly even offer priorities within the queue model.
I also believe it is the responsibility of the library author to default to being a good citizen, and for the library's default operation to be to comply to the conditions of the data provider.
I'd say both - you're dealing with two independent systems and both should take measures to defend themselves from excessive load. The web server should refuse incoming connections, and the client library should take steps to reduce the requests it makes to a slow or unresponsive external service. A common pattern for dealing with this on the client is 'circuit breaker' which wraps calls to an external service, and fails fast for a certain period after failure.
That's the Web server's responsibility, imo. Because the critical load depends on hardware, network bandwidth, etc a lot of things that are outside of your application's control, it should not concern itself with trying the deal with it. IIS can throttle traffic based on various configuration options.
What kind of client is it? Is this an interactive client, for eg: GUI based app?
In that case, you can equate that to a webbrowser scenario, and let the timeout surface to the caller. Also, if you know for sure that this webserver is throttling requests, you can tell the client that he has to wait for a given time period before retrying. In that way, the client will not keep on re-issuing requests, and will know when the first timeout occurs that it is futile to issue requests too fast.