We Keep Coding

How to Improve ASP.Net Web Service performance

We have one old ASP.Net asmx webservice in our application which receives bulk requests at sometime. Service is taking less than 5 seconds for a single request. But It is taking more than a minute when it receives 20 or more concurrent requests. Following is the way it is implemented,
1)receives a request with input data from external clients
2)Will get 20 possibilities from database for one request based on input data after validation
3)Then It will iterate all 20 possibilities using foreach and gets solutions either from other external service or data base based on possibility data.  Here in old implementation we have used Parallel.Foreach to perform all 20 calls (service calls or DB calls) parallely to improve the performance.
4)After that Service will send back the all 20 solutions to the client.
This old approach is working fine for few (1or 2 ) requests and resonse time of asmx service is very fast(less than 5 seconds)  considering external service calls which are taking 2-3 seconds .But This approach is taking more than 60 seconds when the number of concurrent requests are more  than 20.Concurrent requests are pushing CPU utilization to 100% and thread pool starvation as per experts analysis and there by causing requests to queue for threads allocation.
So we got a recommendation to replace parallel extensions and complete service with async/await implementation from end to end.I have implemented async/await end to end and also replaced Parallel.foreach with Task.WhenAll in TPL. But response time has increased a lot after this implementation.for a single request 20 secconds and it its taking more than 2 minutes for bulk requests.
I also tried async foreach in place of parallel.foreach as mentioned in below article  but still performance is really bad.
https://stackoverflow.com/questions/14673728/run-async-method-8-times-in-parallel/14674239#14674239 
As per logs basic issue is with external service calls/DB calls inside foreach in both old parallel or new async/await implementations.But these service responses are very fast for a single request. Async implementation is taking more time in completing service calls than parallel extensions implementation.
I think service should not take more than 20 seconds for bulk request if it is lessa than  5 seconds for single request.
Can anyone please me what should be the way forward here to improve the performance ?
Thanks in advance.
Regards,
Raghu.

Looks like a lot of things happening here at the same time. I believe you have on nderlying issue that causes many side effects.
I will make the assumption that your server is sufficient in terms of CPU and memory to handle the concurrent connections (though the CPU 100% makes me wonder).
It seems to me that your problem, is that the parallel tasks (or threads), compete for the same resources. That would explain why multiple requests take much more time and why the async paradigm takes even more.
Let me explain:
The problem in practice
Parallel implementation: 1 or 2 request need minimum synchronization, so even if they compete for the same resources, it should be fine.
When 20 threads, try to access the same resources, a lot is happening and you come to a situation known as livelock.
When you switch to async, no requests await for a thread (they are waiting on the IO threads), so you make the problem even worse.
(I suspect that the problem is on your database. If your database server is the same machine, it would also explain the utilization).
The solution
Instead of trying to up the parallelism, find the contested resources and identify the problem.
If it's in your database (most probable scenario), then you need to identify the queries causing the trouble and fix them (indexes, statistics, query plans and whatnot). DB profilers showing locks and query execution plans are your friends for this.
If the problem is in your code, try to minimize the race conditions and imporve your algorithms.
To get a hint of where to look for, use the Visual Studio profiling tools: https://learn.microsoft.com/en-us/visualstudio/profiling/profiling-feature-tour?view=vs-2019 or any external .net profiling software.

Using TPL in .NET

I have to refactor a fairly time-consuming process in one of my applications and after doing some research I think it's a perfect match for using TPL. I wanted to clarify my understanding of it and ask if there are any more issues which I should take into account.
In few words, I have a windows service, which runs overnight and sends out emails with data updates to around 10000 users. At presence, the whole process takes around 8 hrs to complete. I would like to reduce it to 2 hrs max.
Application workflow follows steps below:
1. Iterate through all users list
2. Check if this user has to be notified
3. If so, create an email body by calling external service
4. Send an email
Analysis of the code has shown that step 3 is the most time-consuming one and takes around 3,5 sec to complete. It means, that when processing 10000 users, my application waits well over 6 hrs in total for a response from the external service! I think this is a reason good enough to try to introduce some asynchronous and parallel processing.
So, my plan is to use Parallel class and ForEach method to iterate through users in step 1. As I can understand this should distribute processing each user into a separate thread, making them run in parallel? Processes are completely independent of each other and each doesn't return any value. In the case of any exception being thrown it will be persisted in logs db. As with regards to step 3, I would like to convert a call to external service into an async call. As I can understand this would release the resources on the thread so it could be reused by the Parallel class to start processing next user from the list?
I had a read through MS documentation regarding TPL, especially Potential Pitfalls in Data and Task Parallelism document and the only point I'm not sure about is "Avoid Writing to Shared Memory Locations". I am using a local integer to count a total number of emails processed. As with regards to all of the rest, I'm quite positive they're not applicable to my scenario.
My question is, without any implementation as yet. Is what I'm trying to achieve possible (especially the async await part for external service call)? Should I be aware of any other obstacles that might affect my implementation? Is there any better way of improving the workflow?
Just to clarify I'm using .Net v4.0

Yes, you can use the TPL for your problem. If you cannot influence your external problem, then this might be the best way.
However, you can make the best gains if you can get your external source to accept batches. Because this source could actually optimize the performance. Right now you have a message overhead of 10000 messages to serialize, send, work on, receive and deserialize. This is stuff that could be done once. In addition, your external source might be able to optimize the work they do if they know they will get multiple records.
So the bottom line is: if you need to optimize locally, the TPL is fine. If you want to optimize your whole process for actual gains, try to find out if your external source can help you, because that is where you can make some real progress.

You didn't show any code, and I'm assuming that step 4 (send an e-mail) is not that fast either.
With the presented case, unless your external service from step 3 (create an email body by calling external service) processes requests in parallel and supports a good load of simultaneous requests, you will not gain much with this refactor.
In other words, test the external service and the e-mail server first for:
Parallel request execution
The way to test this is to send at least 2 simultaneous requests and observe how long it takes to process them.
If it takes about double the time of a single, the requests have some serial processing, either they're queued or some broad lock is being taken.
Load test
Go up to 4, 8, 12, 16, 20, etc, and see where it starts to degrade.
You should set a limit on the amount of simultaneous requests to something that keeps execution time above e.g. 80% of the time it takes to process a single request, assuming you're the sole consumer
Or a few requests before it starts degrading (e.g. divide by the number of consumers) to leave the external service available for other consumers.
Only then can you decide if the refactor is worth. If you can't change the external service or the e-mail server, you must weight it they offer enough parallel capability without degrading.
Even so, be realistic. Don't let your service push the external service and the e-mail server to their limits in production.

What are the downsides to Request throttling using delay (C# .Net 4 Web Server)

We are running a Http Api and want to be able to set a limit to the number of requests a user can do per time unit. When this limit has been reached, we don't want the users to receive errors, such as Http 429. Instead we want to increase the response times. This has the result that the users can continue to work, but slower, and can then choose to upgrade or not upgrade its paying plan. This solution can quite easily be implemented using Thread.sleep (or something similar) for x number of seconds, on all requests of a user that has passed its limit.
We think that in worst case there might be a problem with the number of possible connections for a single server, since as long as we keep delaying the response, we keep a connection open, and therefore limiting the number of possible other connections.
All requests to the Api is running asynchronously. The Server itself is built to be scalable and is running behind a load balancer. We can start up additional servers if necessary.
When searching for this type of throttling, we find very few examples of this way of limiting the users, and the examples we found seemed not concerned at all about connections running out. So we wonder is this not a problem?
Are there any downsides to this that we are missing, or is this a feasible solution? How many connections can we have open simultaneously without starting to get problems? Can our vision be solved in another way, that is without giving errors to the user?

Thread.Sleep() is pretty much the worst possible thing you can do on a web server. It doesn't matter that you are running things asynchronously because that only applies to I/O bound operations and then frees the thread to do more work.
By using a Sleep() command, you will effectively be taking that thread out of commission for the time it sleeps.
ASP.Net App Pools have a limited number of threads available to them, and therefore in the worst case scenario, you will max out the total number of connections to your server at 40-50 (whatever the default is), if all of them are sleeping at once.
Secondly
This opens up a major attack vector in terms of DOS. If I am an attacker, I could easily take out your entire server by spinning up 100 or 1000 connections, all using the same API key. Using this approach, the server will dutifully start putting all the threads to sleep and then it's game over.
UPDATE
So you could use Task.Delay() in order to insert an arbitrary amount of latency in the response. Under the hood it uses a Timer which is much lighter weight than using a thread.
await Task.Delay(numberOfMilliseconds);
However...
This only takes care of one side of the equation. You still have an open connection to your server for the duration of the delay. Because this is a limited resource it still leaves you vulnerable to a DOS attack that wouldn't have normally existed.
This may be an acceptable risk for you, but you should at least be aware of the possibility.

Why not simply add a "Please Wait..." on the client to artificially look like it's processing? Adding artificial delays on server costs you, it leaves connections as well as threads tied up unnecessarily.

Prevent overload on remote system when using Parallel.ForEach()

We've built this app that needs to have some calculations done on a remote machine (actually a MatLab server). We're using web services to connect to the MatLab server and perform the calculations.
In order to speed things up, we've used Parallel.ForEach() in order to have multiple service calls going at the same time. If we're very conservative in setting ParallelOptions.MaxDegreeOfParallelism (DOP) to 4 or something, everything works fine and well.
However, if we let the framework decide on the DOP it will spawn so many threads that it forces the remote machine on its knees and timeouts start occurring ( > 10 minutes ).
How can we solve this issue? What I would LOVE to be able to do is use the response time to throttle the calls. If response time is less than 30 sec, keep adding threads, as soon as it's over 30 sec, use less. Any suggestions?
N.B. Related to the response in this question: https://stackoverflow.com/a/20192692/896697

Simplest way would be to tune for the best number of concurrent requests and hardcode that as you have done so far, however there are some nicer options if you are willing to put in some effort.
You could move from a Parallel.ForEach to using a thread pool. That way as things come back from the remote server you can either manually or programatically tune the number of available threads. reducing/increasing the number of available threads as things slow down/speed up, or even kill them if needed.
You could also do a variant of the above using Tasks which are the newer way of doing parallel/async stuff in .net.
Another option would be to use a timers and/or jobs model to schedule jobs every x milliseconds, which could then be throttled/relaxed as results returned from the server. The easiest way to get started would be using Quartz.Net.

c# Webcrawler optimization [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I have Webcrawles writed in C#, it uses multithreading. For now it can download and parse about 1000 links / min but when I run ex. 3 instances at the same time every one instance can reach 1000 links / min so I have 3000 links/min. One instance uses up to 2% CPU, 100MB RAM, and 1% network speed. Now I wonder it is possible to one instance can reach 3000 links/min or more when I have avaialable resources (cpu,ram,network)?
Structure of my code:
ThreadSafeFileBuffer<string> successWriter = new ThreadSafeFileBuffer<string>("ok.txt");
IEnumerable<string> lines = File.ReadLines("urls.txt");
var options = new ParallelOptions
{
CancellationToken = _cts.Token,
MaxDegreeOfParallelism = 500
};
Parallel.ForEach(lines, options, (line, loopState, idx) =>
{
var crawler = new Crawler(line);
var result = crawler.Go(); //download,parse
successWriter.AddResult(result);
}
I have Windows 7,CPU i7,16GB RAM,SSD disk

The problem with using Parallel.ForEach on a list of URLs is that those lists often contain many URLs from the same site and you end up with multiple concurrent requests to the same site. Some sites frown on that and will block you or insert artificial delays.
1,000 requests per minute works out to 16 or 17 requests per second, which is pretty much the limit of what you can do without resorting to extraordinary measures. A large part of the problem is DNS resolution, which can take a surprisingly long time. In addition, the default .NET ServicePointManager limits you to 2 concurrent requests on any given site. If you want to support more than that, you need to change the ServicePointManager.DefaultConnectionLimit property.
You definitely don't want to add hundreds of threads. I did that once. It's painful. What you need is a handful of threads that can make asynchronous requests very quickly. My testing shows that a single thread can't sustain more than 15 requests per second because HttpRequest.BeginGetResponse does a lot of synchronous work before going asynchronous. As the documentation states:
The BeginGetResponse method requires some synchronous setup tasks to complete (DNS resolution, proxy detection, and TCP socket connection, for example) before this method becomes asynchronous.
You can speed that up somewhat by increasing the size of your DNS client cache and by having a local DNS cache on a separate machine, but there's a limit to what you can achieve there.
I don't know how much crawling you're doing. If you're doing a lot, then you need to implement a politeness policy that takes into account the robots.txt file, limits how often it hits a particular site, limits the types of URLs it downloads (no use downloading an MP3 or .doc file if you can't do anything with it, for example), etc. To prevent your crawler from being blocked, your crawler becomes at the core a politeness policy enforcer that just happens to download web pages.
I started writing about this some time back, but then didn't finish (other projects took precedence). See http://blog.mischel.com/2011/12/13/writing-a-web-crawler-introduction/ for the first post and links to the other posts in the thread. Also see http://blog.mischel.com/2011/12/26/writing-a-web-crawler-queue-management-part-1/. It's something I've been wanting to get back to, but after almost two years I still haven't managed it.
You'll also run into proxy problems, URL filtering problems (here and here), weird redirects, and asynchronous calls that aren't completely asynchronous.

You do not need more threads, as those threads all spend their time waiting. You need an asynchronous program, that doesn't block threads waiting for web replies.
The problem with threads is that they are a rather expensive resource, because of the memory required for their stack, and the work they create for the OS thread scheduler. In your program, this scheduler keeps on switching threads to that they can all take turns waiting. But they're not doing anything usefull.

In a webcrawler your going to be spending most of your time waiting for web requests. So if you have blocking I/O your program is not going to be processing at full speed, also async IO wont help if the program sits idle waiting for a callback. Sounds like you just need to add more threads to your main app and be processing in parallel.
But it's hard to tell, since you've not posted any code.,

Yes, it is. Find out where your bottleneck is and improve on the performance.
Edit:
If you are using Parallel.ForEach, you can try an overload using the ParallelOptions parameter. Setting the MaxDegreeOfParallelism property might help.

Actually the number of links/min is directly proportional with the number of crawler threads running at the same time.
In your first case; you have 3 processes with n threads per. (total 3n threads)
Try to run 3n threads in one process.
Actually this depends on your operating system and CPU, too. Because old versions of windows (like XP) doesn't support parallel multithreading over different cpu cores.

Parallelism with the TPL is bad design for a web crawler. A Parallel.ForEach() loop starts only a bunch of requests (5-50), because it is designed to perform time consuming computing in parallel and not to perform thousands of requests in parallel which almost do nothing. To get the data you want to have, you must be able to preform a really large amount (10000+) of requests in parallel. Async Operations are the key to that.
I have developed the Crawler Engine of the Crawler-Lib Framework. It is a workflow enabled crawler which can easily extended to do any kind of requests or even processing you want to have.
It is designed to give you high throughput out of the box.
Here is the engine: http://www.crawler-lib.net/crawler-lib-engine
Here are some Youtube Videos, showing how the Crawler-Lib engine works: http://www.youtube.com/user/CrawlerLib
I know this project is not open source, but there is a free version.