We Keep Coding

Why do .NET threads have inferior performance to separate .NET processes?

Lately I've been observing an interesting phenomenon, and before I reengineer my whole software architecture based on it, I'd like to know why this happens, and if it's perhaps possible to make thread performance on par with process performance.
Generally, the task is to download certain data. If we make one process with 6 threads, based on the Parallel library, the downloads take around 10s.
If we, however, make 6 processes, each being single threaded, and download the same data, the whole thing will only take around 6s.
The numbers are thoroughly verified and statistically significant, so do take them for granted.
The observation holds over a large (100s of trials) dataset and I've observed no deviation from this behavior.
Basically, the question is, why a non-synchronizing multithreaded process is slower than a few separate processes with the exact same working code, and how it can be fixed?
Thanks in advance!
Note: I've read similar questions but the answers haven't been satisfactory and practical.

My guess is the same as svick's: you probably have some kind of bottleneck inserted by the runtime.
In general, you can use a tool like Fiddler or Wireshark to see how the 10 downloads are interleaving. In your case, I would expect that there would only be two active at any one time and that once one finishes, another will start immediately.
Before you go and change the setting, you should understand why it's there. It is written into the HTTP spec as suggested client behavior so as to not overwhelm the server. If your code is going to be distributed out to hundreds/thousands/millions of machines, you should consider the effects of 10 simultaneous downloads per client.

What is non-thread-safety for?

There are a lot of articles and discussions explaining why it is good to build thread-safe classes. It is said that if multiple threads access e.g. a field at the same time, there can only be some bad consequences. So, what is the point of keeping non thread-safe code? I'm focusing mostly on .NET, but I believe the main reasons are not language-dependent.
E.g. .NET static fields are not thread-safe. What would be the result if they were thread-safe by default? (without a need to perform "manual" locking). What are the benefits of using (actually defaulting to) non-thread-safety?
One thing that comes to my mind is performance (more of a guess, though). It's rather intuitive that, when a function or field doesn't need to be thread-safe, it shouldn't be. However, the question is: what for? Is thread-safety just an additional amount of code you always need to implement? In what scenarios can I be 100% sure that e.g. a field won't be used by two threads at once?

Writing thread-safe code:
Requires more skilled developers
Is harder and consumes more coding efforts
Is harder to test and debug
Usually has bigger performance cost
But! Thread-safe code is not always needed. If you can be sure that some piece of code will be accessed by only one thread the list above becomes huge and unnecessary overhead. It is like renting a van when going to neighbor city when there are two of you and not much luggage.

Thread safety comes with costs - you need to lock fields that might cause problems if accessed simultaneously.
In applications that have no use of threads, but need high performance when every cpu cycle counts, there is no reason to have safe-thread classes.

So, what is the point of keeping non thread-safe code?
Cost. Like you assumed, there usually is a penalty in performance.
Also, writing thread-safe code is more difficult and time consuming.

Thread safety is not a "yes" or "no" proposition. The meaning of "thread safety" depends upon context; does it mean "concurrent-read safe, concurrent write unsafe"? Does it mean that the application just might return stale data instead of crashing? There are many things that it can mean.
The main reason not to make a class "thread safe" is the cost. If the type won't be accessed by multiple threads, there's no advantage to putting in the work and increase the maintenance cost.

Writing threadsafe code is painfully difficult at times. For example, simple lazy loading requires two checks for '== null' and a lock. It's really easy to screw up.
[EDIT]
I didn't mean to suggest that threaded lazy loading was particularly difficult, it's the "Oh and I didn't remember to lock that first!" moments that come fast and hard once you think you're done with the locking that are really the challenge.

There are situations where "thread-safe" doesn't make sense. This consideration is in addition to the higher developer skill and increased time (development, testing, and runtime all take hits).
For example, List<T> is a commonly-used non-thread-safe class. If we were to create a thread-safe equivalent, how would we implement GetEnumerator? Hint: there is no good solution.

Turn this question on its head.
In the early days of programming there was no Thread-Safe code because there was no concept of threads. A program started, then proceeded step by step to the end. Events? What's that? Threads? Huh?
As hardware became more powerful, concepts of what types of problems could be solved with software became more imaginative and developers more ambitious, the software infrastructure became more sophisticated. It also became much more top-heavy. And here we are today, with a sophisticated, powerful, and in some cases unnecessarily top-heavy software ecosystem which includes threads and "thread-safety".
I realize the question is aimed more at application developers than, say, firmware developers, but looking at the whole forest does offer insights into how that one tree evolved.

So, what is the point of keeping non thread-safe code?
By allowing for code that isn't thread safe you're leaving it up to the programmer to decide what the correct level of isolation is.
As others have mentioned this allows for complexity reduction and improved performance.
Rico Mariani wrote two articles entitled "Putting your synchronization at the correct level" and
Putting your synchronization at the correct level -- solution that have a nice example of this in action.
In the article he has a method called DoWork(). In it he calls other classes Read twice Write twice and then LogToSteam.
Read, Write, and LogToSteam all shared a lock and were thread safe. This is good except for the fact that because DoWork was also thread safe all the synchronizing work in each Read, Write and LogToSteam was a complete waste of time.
This is all related to the nature Imperative Programming. Its side effects cause the need for this.
However if you had an development platform where applications could be expressed as pure functions where there were no dependencies or side effects then it would be possible to create applications where the threading was managed without developer intervention.

So, what is the point of keeping non thread-safe code?
The rule of thumb is to avoid locking as much as possible. The Ideal code is re-entrant and thread safe with out any locking. But that would be utopia.
Coming back to reality, a good programmer tries his level best to have a sectional locking as opposed to locking the entire context. An example would be to lock few lines of code at a time in various routines than locking everything in a function.
So Also, one has to refactor the code to come up with a design that would minimize the locking if not get rid of it in entirity.
e.g. consider a foobar() function that gets new data on each call and uses switch() case on a type of data to changes a node in a tree. The locking can be mostly avoided (if not completely) As each case statement would touch a different node in a tree. This may be a more specific example but i think it elaborates my point.

Multicore programming: the hard parts

I'm writing a book on multicore programming using .NET 4 and I'm curious to know what parts of multicore programming people have found difficult to grok or anticipate being difficult to grok?

What's a useful unit of work to parallelize, and how do I find/organize one?
All these parallelism primitives aren't helpful if you fork a piece of work that is smaller than the forking overhead; in fact, that buys you a nice slowdown instead of what you are expecting.
So one of the big problems is finding units of work that are obviously more expensive than the parallelism primitives. A key problem here is that nobody knows what anything costs to execute, including the parallelism primitives themselves. Clearly calibrating these costs would be very helpful. (As an aside, we designed, implemented, and daily use a parallel programming langauge, PARLANSE whose objective was to minimize the cost of the parallelism primitives by allowing the compiler to generate and optimize them, with the goal of making smaller bits of work "more parallelizable").
One might also consider discussion big-Oh notation and its applications. We all hope that the parallelism primitives have cost O(1). If that's the case, then if you find work with cost O(x) > O(1) then that work is a good candidate for parallelization. If your proposed work is also O(1), then whether it is effective or not depends on the constant factors and we are back to calibration as above.
There's the problem of collecting work into large enough units, if none of the pieces are large enough. Code motion, algorithm replacement, ... are all useful ideas to achieve this effect.
Lastly, there's the problem of synchnonization: when do my parallel units have to interact, what primitives should I use, and how much do those primitives cost? (More than you expect!).

I guess some of it depends on how basic or advanced the book/audience is. When you go from single-threaded to multi-threaded programming for the first time, you typically fall off a huge cliff (and many never recover, see e.g. all the muddled questions about Control.Invoke).
Anyway, to add some thoughts that are less about the programming itself, and more about the other related tasks in the software process:
Measuring: deciding what metric you are aiming to improve, measuring it correctly (it is so easy to accidentally measure the wrong thing), using the right tools, differentiating signal versus noise, interpreting the results and understanding why they are as they are.
Testing: how to write tests that tolerate unimportant non-determinism/interleavings, but still pin down correct program behavior.
Debugging: tools, strategies, when "hard to debug" implies feedback to improve your code/design and better partition mutable state, etc.
Physical versus logical thread affinity: understanding the GUI thread, understanding how e.g. an F# MailboxProcessor/agent can encapsulate mutable state and run on multiple threads but always with only a single logical thread (one program counter).
Patterns (and when they apply): fork-join, map-reduce, producer-consumer, ...
I expect that there will be a large audience for e.g. "help, I've got a single-threaded app with 12% CPU utilization, and I want to learn just enough to make it go 4x faster without much work" and a smaller audience for e.g. "my app is scaling sub-linearly as we add cores because there seems to be contention here, is there a better approach to use?", and so a bit of the challenge may be serving each of those audiences.

Since you write a whole book for multi-core programming in .Net.
I think you can also go beyond multi-core a little bit.
For example, you can use a chapter talking about parallel computing in a distributed system in .Net. Unlikely, there is no mature frameworks in .Net yet. DryadLinq is the closest. (On the other side, Hadoop and its friends in Java platform are really good.)
You can also use a chapter demonstrating some GPU computing stuff.

One thing that has tripped me up is which approach to use to solve a particular type of problem. There's agents, there's tasks, async computations, MPI for distribution - for many problems you could use multiple methods but I'm having difficulty understanding why I should use one over another.

To understand: low level memory details like the difference between acquire and release semantics of memory.
Most of the rest of the concepts and ideas (anything can interleave, race conditions, ...) are not that difficult with a little usage.
Of course the practice, especially if something is failing sometimes, is very hard as you need to work at multiple levels of abstraction to understand what is going on, so keep your design simple and as far as possible design out the need for locking etc. (e.g. using immutable data and higher level abstractions).

Its not so much theoretical details, but more the practical implementation details which trips people up.
What's the deal with immutable data structures?
All the time, people try to update a data structure from multiple threads, find it too hard, and someone chimes in "use immutable data structures!", and so our persistent coder writes this:
ImmutableSet set;
ThreadLoop1()
foreach(Customer c in dataStore1)
set = set.Add(ProcessCustomer(c));
ThreadLoop2()
foreach(Customer c in dataStore2)
set = set.Add(ProcessCustomer(c));
Coder has heard all their lives that immutable data structures can be updated without locking, but the new code doesn't work for obvious reasons.
Even if your targeting academics and experienced devs, a little primer on the basics of immutable programming idioms can't hurt.
How to partition roughly equal amounts of work between threads?
Getting this step right is hard. Sometimes you break up a single process into 10,000 steps which can be executed in parallel, but not all steps take the same amount of time. If you split the work on 4 threads, and the first 3 threads finish in 1 second, and the last thread takes 60 seconds, your multithreaded program isn't much better than the single-threaded version, right?
So how do you partition problems with roughly equal amounts of work between all threads? Lots of good heuristics on solving bin packing problems should be relevant here..
How many threads?
If your problem is nicely parallelizable, adding more threads should make it faster, right? Well not really, lots of things to consider here:
Even a single core processor, adding more threads can make a program faster because more threads gives more opportunities for the OS to schedule your thread, so it gets more execution time than the single-threaded program. But with the law of diminishing returns, adding more threads increasing context-switching, so at a certain point, even if your program has the most execution time the performance could still be worse than the single-threaded version.
So how do you spin off just enough threads to minimize execution time?
And if there are lots of other apps spinning up threads and competing for resources, how do you detect performance changes and adjust your program automagically?

I find the conceptions of synchronized data moving across worker nodes in complex patterns very hard to visualize and program.
Usually I find debugging to be a bear, also.

Line of business applications: Will F# make my life easy?

I develop mainly line of business applications.No scientific operations. No complex calculations. Just tie User Interface to database. The only reason I use threading is to do some work in background and still keep UI responding.
This may not be the best approach but this is what I follow
1.Create an working application first(without threads) and give it to end user to play for the sake of feedback.
2.Once all requirements are locked, I try to use threads wherever it makes sense to improve performance.
The code for steps 1 & 2 is overwhelmingly different and threading code dominates the actual code.
1.Will F# make my life easier in case of Line of Business applications?
2.Are there any particular UI technologies that will fit best with F#? I mainly work on ASP.NET & Silverlight. WPF now & then.
3.Are there any good examples of Line of business applications/demos with F#?
I am not asking whether it is possible to develop Line of Business application in F#. I am asking whether F# will make my life easier to develop Line of Business applications when compared to C#? My main concern will be threading & UI synchronization.

I'm in the same boat as you, doing lots and lots of line-of-business type apps, nothing "fun" like games or compilers or search engines.
Your mileage will vary, but at least in my own experience a lot of dev teams are reluctant to jump right into F# because no one else on the team knows or has even heard of it. And right off the bat, you have to fight those questions like "what does it do differently from C#?" If you can convince your boss to let you write a few demo apps with it, go for it!
So with that being said, I find that C# isn't very good at business rules, but it handles GUIs like a champ; F#'s immutability makes GUI development awkward, but business rules and workflows feel natural. So the two languages have their own strengths and compliments one another's weaknesses.
In my own LOB apps, F# runs circles around C# in a few areas:
F#'s async workflows and mailbox processors orders of magnitude easier to work with than native threads and even task parallel library. Since using mailbox processors for interthread communication, I don't even remember the last time I've had to lock or thread.join() anything for syncronization at all.
Defining business rules engines and DSLs with unions FTW! Every non-trivial LOB app I've ever worked on has its own half-baked rules language and interpreter, and its almost always based on recursively switching on an enum to drill through the rules and find a match. Current project I have now contains 1300+ public classes, 900 or so are simple container classes to represent a rule. I think representing the rules as an F# union would substantially reduce the code bloat and make for a better engine.
Immutable code just works better -- if you get a new object with an invalid state, you don't have to search far to find the offending line of code, everything you need to know is on the call stack. If you have a mutable object with an invalid state, sometimes you have to spend a lot of time tracking it down. You can write immutable code in C#, but its really hard not to fall back on mutability, especially when you're modifying an object in a loop.
I hate nulls. I can't give an exact estimate, but it feels like half the bugs we get in production are null reference exceptions -- an object is improperly initialized and you don't know about it until you're 30 stack frames deep in code. I think F# would help us write more bug free code the first time around.
C# usually works well when:
You're writing GUI code or working with inherently mutable systems.
Exposing a DLL or web service to many different clients.
Your boss won't let you use the tools you want ;)
So if you can get over the "why would we want to use a new language hurdle", I think F# will indeed make your life easier.

That's a very hard question to answer - I think that F# would make some aspects of your life much easier and some a bit harder.
On the plus side, dealing with threading should be relatively painless - F# async workflows are a huge benefit. Also, F# Interactive makes rapidly iterating and exploring code very easy. To me, this is a huge benefit over C#, since I can test out minor code changes without going through a full build.
On the down side, the tooling for F# isn't where it is for C#, which means that you won't have access to GUI-builders, refactoring tools, etc. It's hard to say how big of a productivity hit this would cause you without knowing more about your scenario. One workaround would be to create a multi-language solution which has a thin C# front-end, but again the complexity may not be worth the benefit.

#Juliet and #kvb have good answers, I just want to reiterate how useful F# is for making threading easy. In my blog post "An RSS Dashboard in F#, part six (putting it all together with a WPF GUI)", I say
...Note the nifty use of ‘async’ here – I
use Async.StartImmediate, which means
all the code runs on the UI thread
(where the Click handler starts), but
I can still do non-blocking sleeps
that don’t jam up the UI. This is one
way that F# async just blows the doors
off everything else.
...
Our “ago” information (“3 minutes
ago”) will quickly get stale if we
don’t refresh it, so we start a loop
that redraws every minute. Once
again, F# async kicks butt, as I can
just write this as though it were a
synchronous loop running on the UI
thread, but the non-blocking sleep
call ensures that the UI stays live.
Awesome. ...
but that blog post is an example of using F# to hand-code the entire UI. That implies trading away all of the great GUI tooling you get with C# or VB. I imagine that a hybrid approach can potentially net almost all of the benefits of both (with the cost of having two projects in the solution where you previous just had one), but I don't (yet) have direct experience of my own to share here.
(Is there a canonical "problem example" of a C# GUI app where you need to add threading to improve perf or keep the app live during some long operation? If so, I should check it out.)

Something you might like to see:
The First Substantial Line of Business Application in F#
A big LOB app in F#.

To address this I posted some thoughts of mine in using F#,
http://fadsworld.wordpress.com/2011/04/13/f-in-the-enterprise-i/ http://fadsworld.wordpress.com/2011/04/17/fin-the-enterprise-ii-2/
I'm also planning to do a video tutorial to finish up the series and show how F# can contribute in UX programming.
I'm only talking in context of F# here.
-Fahad

How to prove that multithreading is working?

How can I prove that multithreading is working in my C# programs? This is for a testing requirement. For example, I'm going to have to add some locking to a logger class (yes I know, I shouldn't have written my own logging class), and I need a test case to prove that the change will work.

If you want to test that your locking code is correctly synchronizing access to your log(s), you need to construct tests that guarantee contention. This may require you to refactor your code so that you can inject a mock log writer class that can hold the log's lock for arbitrary periods of time.
This is a broad topic, and you can find several related questions on StackOverflow, which are all worth reading:
How do I perform a Unit Test using threads?
How to write an automated test for thread safety
What are some strategies to unit test a scheduler?
Unit testing a multithreaded application?
Should I unit test for multithreading problems before writing any lock? (.NET C# TDD)
CHESS is a framework under development for identifying "assertion violations, deadlocks, livelocks, data-races, and memory-model errors." I haven't actually used this, but it looks like it might be extremely helpful.

Well, this may sound wrong but the truth is you can't prove multi-threaded behavior with unit-tests. Having said that, you can gain some confidence in the code with testing and over time it might actually present an issue.
<rant>
Multi-threaded code is the bane of my existence in many a project. Often people/developers do not have the expertise required to do a good job. Bugs often go unnoticed for long periods of time before anyone sees it in the wild, and then you can't reproduce the issue to identify whats going on. Further, attempting to 'fix' broken multi-threaded code via debugging is often not a viable approach.
</rant>
Anyway, go ahead and test it, there is no harm in doing that much and it's easy enough to do. Just fire up N number of threads, have them all wait on a ManualRestEvent, and then call your api in a tight loop a couple of hundred-thousand times :). But first I would recommend everyone on your team do a code review. Walk every line of code thinking about it executing in parallel. Ask yourself:
Do I really need this lock()?
What's the least amount of code that MUST be in the lock()?
Can I make this object/state immutable and avoid the locking?
Is there any way for a caller to have code execute inside the lock?
Review all members accessed and changed inside a lock for 'volatile'?
Are you using System.Threading.Thread.MemoryBarrier correctly?
If multiple locks are involved are they always obtained in the same order?
[wiki add here]

You just can't :) It's all depends on timing and it might blow up at any time. You have to mentally check every possible situation and that is the only way to go. That's why a lot of developers think multithreading is impossible to get right.

I have actually found Thread.Sleep() to be very useful to simulate various different race conditions. However, for obvious reasons you need to ensure that you either remove (or use configuration to disable) the Thread.Sleep before deploying to production.
In Robert C Martin's book "Clean Code", he recommends using "jiggling strategies" in your unit tests to ferret out multi-threading issues. "Jiggling" involves adding random wait times to your code so that threads run in different order at different times. You can then run your unit tests many times and your jiggling may route out some flaws. The important thing is NOT to ignore any unit test failures involving multithreading just because they pass the next time you run the test.

You actually can't. You can, however, write some debug-time code (start of routine, end of routine, special actions routine takes, ...) that writes to a console, so you can see that routines run at the same time.

Thread.Sleep. If you're suffering from race conditions with multithreading a well placed Thread.Sleep can increase the size of the race condition making it easier to reproduce.
WriteA();
// potential race condition as another bit of code might read all the state
// and only get A in their read.
WriteB();
to
WriteA();
Thread.Sleep(60000);
WriteB();
Then you can write code that reproduces the problem. Then you can write code that fixes the problem. Then you can assert that your fix works. Profit!

Another thread posted a related answer, using Microsoft's CHESS program.