Context
The recommended way to use HttpClient is to create one instance and share it (according to the Microsoft documentation). There are many examples out there where using an HttpClient per request (on a server) yielded issues.
Problem
In my situation I have to run up to 20 simultaneous "sessions" which cannot interfere with one another. By "session" I mean isolated set of operations on a service or multiple services. No data should be shared between these sessions and specifically cookies.
These sessions are long lasting sessions (can last for days). But there may be only a maximum of 20 concurrent sessions at the same time. A slow start while instantiating these sessions is acceptable (up to 5 seconds).
Question
Should I use a pool? Should I reuse the same HttpClient instance? Should I spawn up to 20 HttpClients? Moreover, considering that they run concurrently, is it correct to assume that concurrent calls to a single HttpClient will be blocking?
The recommendation for creating only a single instance of HttpClient is because it creates a TCP connection that persists for a period of time. Eventually, it will get garbarge collected or closed naturally, but in the meantime, you could potentially exhaust the maximum amount of open connections on the server. There's also a bit of resource utilization from each connection that could prove problematic when stacked.
However, this is one of those things where there's no hard and fast rule. Certain things, like creating a new HttpClient per operation, are obviously problematic, but generally one instance per request to your server (not per request sent through HttpClient) shouldn't be problematic unless you're servicing a high number of concurrent requests.
You can also just make it a singleton across your application. HttpClient is not technically a user or session-specific thing. It would only be problematic if your changing default headers that should not be leaked across sessions. For example, you'd want to attach something like an Authorization header as part of creating a request, not as a default header when setting up the client. As long as you're smart about how you use it, there's no problem having it shared. That being "smart about how you use it" thing can sometimes be a point of failure, though. Even if you are, will the next developer be. More to the point, will they even know they should be? You might even forget, yourself.
Personally, I'd start making it request-scoped, where each request to your web application gets a separate instance, but that same instance will be used for all operations with HttpClient during that request. You can then do some profiling to see how the server is handling this, and if it does become a problem, then you can look for alternative solutions.
Related
I am calling a webservice in multi threaded environment. Lot of my calls fail due to operation time out or bad request but none of the calls fail if I do it in linear fashion which means there is problem with invoking webservice with multiple threads. After lot of analysis, I found out that there is limit of concurrent connection which is causing these exception so I fixed it by adding below code.
ServicePointManager.DefaultConnectionLimit = 2 * _ThreadCount;
What I dont know is the possible disadvantage of increasing this limit. By default, Connection limit is 2. If anyone knows any disadvantages, please do let me know.
The MSDN says:
Changing the DefaultConnectionLimit property has no effect on existing
ServicePoint objects; it affects only ServicePoint objects that are
initialized after the change. If the value of this property has not
been set either directly or through configuration, the value defaults
to the constant DefaultPersistentConnectionLimit.
and
Note
Any changes to the DefaultConnectionLimit property affect both HTTP
1.0 and HTTP 1.1 connections. It is not possible to separately alter the connection limit for HTTP 1.0 and HTTP 1.1 protocols. When used in
the server environment (ASP.NET) DefaultConnectionLimit defaults to
higher number of connections, which is 10.
No, there should not be any disadvantages other than that your AppDomain will consume more resources. But in your case it's trivial.
In fact, it can actually help you use less resources (memory) since pending requests are queued up internally in the ServicePoint. Read here for more information: Big size of ServicePoint object after several hours sending HTTP request in parallel
let me give you the picture....I have around 46K tasks,these task are ran in batch of 100 (each task will call webservice) so I have 100 threads calling webserivce simultaneously. is it still trivial? or will it have some impact in my case?
It will of course have an impact. But the impact depends on many factors. A service point is per host.
If your tasks are mostly against the same host, increase DefaultConnectionLimit to a larger value (expected number of tasks in the current execution batch).
If you are mostly doing requests against different hosts, the limit in your question works fine.
Regarding the usage of resources, it's probably fine as long as your server is not very busy by other applications.
You should also realize, that it may hurt you at the other end.
You may get throttled or black listed, if you put a big load on the webservice you are calling. See for example this blog.
This question already has answers here:
Best Practice for WCF Service Proxy lifetime?
(4 answers)
Reuse of WCF service clients
(2 answers)
Closed 9 years ago.
I have a UI application in which I consume a WCF service like this
public MainWindow()
{
....
mServiceClient = new ServiceClient("EndPointTCP");
}
Should I create the client as a member of class and close the client along with exit of my application or Should I create a new client whenever its required in a method and close there itself.
It depends solely onwhat you want to achieve. There is no "best way to do it" since both ways are possible, good, and have different tradeoffs.
Holding the clientobject simply wastes resources. It also may leak context data between calls. You might have a bug that will cause mClient.Buy(100) and mClient.Sell(100) to work properly when used alone, but fail when used together mClient.Buy(100); mClient.Sell(100). Dropping and re-creating fresh instance each time could save you from that one bug, but obviously that's not a good argument for it.
Recreating the client each time a call is to be made has however the vague benefit of .. having a fresh client every time. If your app has a possibility to dynamically change the endpoint during the app's runtime, then automatically your client will be always using the newest addresses/logins/passwords/etc.
However, not recreating the client object at every time is simply faster. Mind that it's WCF layer, so the actual underlying connection can be any. Were it some protocol with heavy setup with some keyexchange, encryption, etc, you may find that creating a new client every time might create a new connection at every time and it will slow down eveyrthing, while keeping the instance will work blazingly fast, since connection might be kept open and reused. Often you try to keep the connection when you have to perform many and often calls to the service, like 24h/day twice/sec monitoring some remote value for breaching safe limits.
On yet the other hand, you might not want the connection to linger. Your remote service may have thousands of clients and limited resources, so you might want to close the connection ASAP so others may connect. Often you'd do it like that when the calls to the service are really rarely done, only once in a time, ie. when user clicks after returning from coffe break.
Please don't get me wrong: all above is just conjuring some vague "facts" from a void. I do not know your app, nor your service, nor your bindings (sans "endpoint TCP"). The most important factors are all on your side and they sit in the actual way how your app and that remote service work and interoperate. If you care about what you ask, you must first simply research the topic on your side. Best - simply try both ways and check if it works and how does it perform. The difference would be something like 2..6 lines of code, so that's, well, rather quick.
There are already some similar questions:
Reuse of WCF service clients
Reusing a WCF service client or creating one each time?
In my opinion it depends on your application type (scalability, performance requirements, ...) but usually I think that it's more safe to recreate the ServiceClient each time. In this way you don't need special code if there are connections problems between requests and with the latest version of WCF seems that there isn't a big performance impact.
See http://msdn.microsoft.com/en-us/library/aa738757.aspx.
Consider also that ServiceClient is not thread safe (at least reading MSDN).
I'm not exactly sure how a WCF proxy class will handle sending requests through plinq. Does the following code snippit look ok, or does it look like it could cause problems with shared state across multiple threads? (also, I already understand that a using block is not ideal, this is just an example)
using (var proxy = new ServiceProxyOfSomeSort())
{
_aBunchOfMessagesToSend.AsParallel()
.WithDegreeOfParallelism(SomeDegree).ForAll(m =>
{
proxy.SomeOperation(m);
}
}
Should I be creating the proxy once per thread? Is it ok to share the proxy across threads? I don't want to create more proxies than I need to because that is a somewhat expensive operation.
Edit:
I don't really have any of the implementation details of the service on the server side. From a requirements standpoint they should have developed it so that multiple clients could call it at anytime (async). Assuming that they can handle async calls (which may be a big assumption) I'm just trying to figure out if this an acceptable approach from the client side. It is working, I just don't know if there are any gotchas with this approach.
In response to your question about the proxy: you should create a new one per thread. Reusing the proxy tends to work for a while and then throw a fault after tens of requests. As they love to say, its behavior is undefined. The performance overhead of creating new proxies is not huge (after the first one is created), so it shouldn't be a big deal.
As for the discussion of multiple concurrent requests swamping the server: a few, or even a dozen, should be fine - the server can probably handle as many requests as your processor can create threads. Then again, the sudden influx of requests from a single source might be interpreted as a Denial Of Service attack, so you should be wary, particularly if your control over the service is limited.
I'm tasked to create a web application. I'm currently using c# & asp.net (mvc - but i doubt its relevant to the question) - am a rookie developer and somewhat new to .net.
Part of the logic in the application im building is to make requests to an external smsgateway by means of hitting a particular url with a request - either as part of a user-initiated action in the webapp (could be a couple of messages send) or as part of a scheduledtask run daily (could and will be several thousand message send).
In relation to a daily task, i am afraid that looping - say - 10.000 times in one thread (especially if im also to take action depending on the response of the request - like write to a db) is not the best strategy and that i could gain some performance/timesavings from some parallelization.
Ultimately i'm more afraid that thousands of users at the same time (very likely) will perform the action that triggers a request. With a naive implementation that spawns some kind of background thread (whatever its called) for each request i fear a scenario with hundreds/thousands of requests at once.
So if my assumptions are correct - how do i deal with this? do i have to manually spawn some appropriate number of new Thread()s and coordinate their work from a producer/consumer-like queue or is there some easy way?
Cheers
If you have to make 10,000 requests to a service then it means that the service's API is anemic - probably CRUD-based, designed as a thin wrapper over a database instead of an actual service.
A single "request" to a well-designed service should convey all of the information required to perform a single "unit of work" - in other words, those 10,000 requests could very likely be consolidated into one request, or at least a small handful of requests. This is especially important if requests are going to a remote server or may take a long time to complete (and 2-3 seconds is an extremely long time in computing).
If you do not have control over the service, if you do not have the ability to change the specification or the API - then I think you're going to find this very difficult. A single machine simply can't handle 10,000 outgoing connections at once; it will struggle with even a few hundred. You can try to parallelize this, but even if you achieve a tenfold increase in throughput, it's still going to take half an hour to complete, which is the kind of task you probably don't want running on a public-facing web site (but then, maybe you do, I don't know the specifics).
Perhaps you could be more specific about the environment, the architecture, and what it is you're trying to do?
In response to your update (possibly having thousands of users all performing an action at the same time that requires you to send one or two SMS messages for each):
This sounds like exactly the kind of scenario where you should be using Message Queuing. It's actually not too difficult to set up a solution using WCF. Some of the main reasons why one uses a message queue are:
There are a large number of messages to send;
The sending application cannot afford to send them synchronously or wait for any kind of response;
The messages must eventually be delivered.
And your requirements fit this like a glove. Since you're already on the Microsoft stack, I'd definitely recommend an asynchronous WCF service backed by MSMQ.
If you are working with SOAP, or some other type XML request, you may not have an issue dealing with the level of requests in a loop.
I set up something similar using a SOAP server with 4-5K requests with no problem...
A SOAP request to a web service (assuming .NET 2.0 and superior) looks something like this:
WebServiceProxyClient myclient = new WebServiceProxyClient();
myclient.SomeOperation(parameter1, parameter2);
myclient.Close();
I'm assuming that this code will will be embedded into your business logic that you will be trigger as part of the user initiated action, or as part of the scheduled task.
You don't need to do anything especial in your code to cope with a high volume of users. This will actually be a matter of scalling on your platform.
When you say 10.000 request, what do you mean? 10.000 request per second/minute/hour, this is your page hit per day, etc?
I'd also look into using an AsyncController, so that your site doesn't quickly become completely unusable.
I am working on a class library that retrieves information from a third-party web site. The web site being accessed will stop responding if too many requests are made within a set time period (~0.5 seconds).
The public methods of my library directly relate to a resource an file on the web server. In other words, each time a method is called, an HttpWebRequest is created and sent to the server. If all goes well, an XML file is returned to the caller. However, if this is the second web request in less than 0.5s, the request will timeout.
My dilemma lies in how I should handle request throttling (if at all). Obviously, I don't want the caller sit around waiting for a response -- especially if I'm completely certain that their request will timeout.
Would it make more sense for my library to queue and throttle the webrequests I create, or should my library simply throw an exception if the a client does not wait long enough between API calls?
The concept of a library is to give its client code as little to worry about as possible. Therefore I would make it the libraries job to queue requests and return results in a timely manner. In an ideal world you would use a callback or delegate model so that the client code can operate in asynchronously, not blocking the UI. You could also offer the option for skipping the queue, (and failing if it operates too soon) and possibly even offer priorities within the queue model.
I also believe it is the responsibility of the library author to default to being a good citizen, and for the library's default operation to be to comply to the conditions of the data provider.
I'd say both - you're dealing with two independent systems and both should take measures to defend themselves from excessive load. The web server should refuse incoming connections, and the client library should take steps to reduce the requests it makes to a slow or unresponsive external service. A common pattern for dealing with this on the client is 'circuit breaker' which wraps calls to an external service, and fails fast for a certain period after failure.
That's the Web server's responsibility, imo. Because the critical load depends on hardware, network bandwidth, etc a lot of things that are outside of your application's control, it should not concern itself with trying the deal with it. IIS can throttle traffic based on various configuration options.
What kind of client is it? Is this an interactive client, for eg: GUI based app?
In that case, you can equate that to a webbrowser scenario, and let the timeout surface to the caller. Also, if you know for sure that this webserver is throttling requests, you can tell the client that he has to wait for a given time period before retrying. In that way, the client will not keep on re-issuing requests, and will know when the first timeout occurs that it is futile to issue requests too fast.