We Keep Coding

How to execute a method with parameters as different threads independently in C#? [duplicate]

I have a question concerning parallel for loops. I have the following code:
public static void MultiplicateArray(double[] array, double factor)
{
for (int i = 0; i < array.Length; i++)
{
array[i] = array[i] * factor;
}
}
public static void MultiplicateArray(double[] arrayToChange, double[] multiplication)
{
for (int i = 0; i < arrayToChange.Length; i++)
{
arrayToChange[i] = arrayToChange[i] * multiplication[i];
}
}
public static void MultiplicateArray(double[] arrayToChange, double[,] multiArray, int dimension)
{
for (int i = 0; i < arrayToChange.Length; i++)
{
arrayToChange[i] = arrayToChange[i] * multiArray[i, dimension];
}
}
Now I try to add parallel function:
public static void MultiplicateArray(double[] array, double factor)
{
Parallel.For(0, array.Length, i =>
{
array[i] = array[i] * factor;
});
}
public static void MultiplicateArray(double[] arrayToChange, double[] multiplication)
{
Parallel.For(0, arrayToChange.Length, i =>
{
arrayToChange[i] = arrayToChange[i] * multiplication[i];
});
}
public static void MultiplicateArray(double[] arrayToChange, double[,] multiArray, int dimension)
{
Parallel.For(0, arrayToChange.Length, i =>
{
arrayToChange[i] = arrayToChange[i] * multiArray[i, dimension];
});
}
The issue is, that I want to save time, not to waste it. With the standard for loop it computes about 2 minutes, but with the parallel for loop it takes 3 min. Why?

Parallel.For() can improve performance a lot by parallelizing your code, but it also has overhead (synchronization between threads, invoking the delegate on each iteration). And since in your code, each iteration is very short (basically, just a few CPU instructions), this overhead can become prominent.
Because of this, I thought using Parallel.For() is not the right solution for you. Instead, if you parallelize your code manually (which is very simple in this case), you may see the performance improve.
To verify this, I performed some measurements: I ran different implementations of MultiplicateArray() on an array of 200 000 000 items (the code I used is below). On my machine, the serial version consistently took 0.21 s and Parallel.For() usually took something around 0.45 s, but from time to time, it spiked to 8–9 s!
First, I'll try to improve the common case and I'll come to those spikes later. We want to process the array by N CPUs, so we split it into N equally sized parts and process each part separately. The result? 0.35 s. That's still worse than the serial version. But for loop over each item in an array is one of the most optimized constructs. Can't we do something to help the compiler? Extracting computing the bound of the loop could help. It turns out it does: 0.18 s. That's better than the serial version, but not by much. And, interestingly, changing the degree of parallelism from 4 to 2 on my 4-core machine (no HyperThreading) doesn't change the result: still 0.18 s. This makes me conclude that the CPU is not the bottleneck here, memory bandwidth is.
Now, back to the spikes: my custom parallelization doesn't have them, but Parallel.For() does, why? Parallel.For() does use range partitioning, which means each thread processes its own part of the array. But, if one thread finishes early, it will try to help processing the range of another thread that hasn't finished yet. If that happens, you will get a lot of false sharing, which could slow down the code a lot. And my own test with forcing false sharing seems to indicate this could indeed be the problem. Forcing the degree of parallelism of the Parallel.For() seems to help with the spikes a little.
Of course, all those measurements are specific to the hardware on my computer and will be different for you, so you should make your own measurements.
The code I used:
static void Main()
{
double[] array = new double[200 * 1000 * 1000];
for (int i = 0; i < array.Length; i++)
array[i] = 1;
for (int i = 0; i < 5; i++)
{
Stopwatch sw = Stopwatch.StartNew();
Serial(array, 2);
Console.WriteLine("Serial: {0:f2} s", sw.Elapsed.TotalSeconds);
sw = Stopwatch.StartNew();
ParallelFor(array, 2);
Console.WriteLine("Parallel.For: {0:f2} s", sw.Elapsed.TotalSeconds);
sw = Stopwatch.StartNew();
ParallelForDegreeOfParallelism(array, 2);
Console.WriteLine("Parallel.For (degree of parallelism): {0:f2} s", sw.Elapsed.TotalSeconds);
sw = Stopwatch.StartNew();
CustomParallel(array, 2);
Console.WriteLine("Custom parallel: {0:f2} s", sw.Elapsed.TotalSeconds);
sw = Stopwatch.StartNew();
CustomParallelExtractedMax(array, 2);
Console.WriteLine("Custom parallel (extracted max): {0:f2} s", sw.Elapsed.TotalSeconds);
sw = Stopwatch.StartNew();
CustomParallelExtractedMaxHalfParallelism(array, 2);
Console.WriteLine("Custom parallel (extracted max, half parallelism): {0:f2} s", sw.Elapsed.TotalSeconds);
sw = Stopwatch.StartNew();
CustomParallelFalseSharing(array, 2);
Console.WriteLine("Custom parallel (false sharing): {0:f2} s", sw.Elapsed.TotalSeconds);
}
}
static void Serial(double[] array, double factor)
{
for (int i = 0; i < array.Length; i++)
{
array[i] = array[i] * factor;
}
}
static void ParallelFor(double[] array, double factor)
{
Parallel.For(
0, array.Length, i => { array[i] = array[i] * factor; });
}
static void ParallelForDegreeOfParallelism(double[] array, double factor)
{
Parallel.For(
0, array.Length, new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount },
i => { array[i] = array[i] * factor; });
}
static void CustomParallel(double[] array, double factor)
{
var degreeOfParallelism = Environment.ProcessorCount;
var tasks = new Task[degreeOfParallelism];
for (int taskNumber = 0; taskNumber < degreeOfParallelism; taskNumber++)
{
// capturing taskNumber in lambda wouldn't work correctly
int taskNumberCopy = taskNumber;
tasks[taskNumber] = Task.Factory.StartNew(
() =>
{
for (int i = array.Length * taskNumberCopy / degreeOfParallelism;
i < array.Length * (taskNumberCopy + 1) / degreeOfParallelism;
i++)
{
array[i] = array[i] * factor;
}
});
}
Task.WaitAll(tasks);
}
static void CustomParallelExtractedMax(double[] array, double factor)
{
var degreeOfParallelism = Environment.ProcessorCount;
var tasks = new Task[degreeOfParallelism];
for (int taskNumber = 0; taskNumber < degreeOfParallelism; taskNumber++)
{
// capturing taskNumber in lambda wouldn't work correctly
int taskNumberCopy = taskNumber;
tasks[taskNumber] = Task.Factory.StartNew(
() =>
{
var max = array.Length * (taskNumberCopy + 1) / degreeOfParallelism;
for (int i = array.Length * taskNumberCopy / degreeOfParallelism;
i < max;
i++)
{
array[i] = array[i] * factor;
}
});
}
Task.WaitAll(tasks);
}
static void CustomParallelExtractedMaxHalfParallelism(double[] array, double factor)
{
var degreeOfParallelism = Environment.ProcessorCount / 2;
var tasks = new Task[degreeOfParallelism];
for (int taskNumber = 0; taskNumber < degreeOfParallelism; taskNumber++)
{
// capturing taskNumber in lambda wouldn't work correctly
int taskNumberCopy = taskNumber;
tasks[taskNumber] = Task.Factory.StartNew(
() =>
{
var max = array.Length * (taskNumberCopy + 1) / degreeOfParallelism;
for (int i = array.Length * taskNumberCopy / degreeOfParallelism;
i < max;
i++)
{
array[i] = array[i] * factor;
}
});
}
Task.WaitAll(tasks);
}
static void CustomParallelFalseSharing(double[] array, double factor)
{
var degreeOfParallelism = Environment.ProcessorCount;
var tasks = new Task[degreeOfParallelism];
int i = -1;
for (int taskNumber = 0; taskNumber < degreeOfParallelism; taskNumber++)
{
tasks[taskNumber] = Task.Factory.StartNew(
() =>
{
int j = Interlocked.Increment(ref i);
while (j < array.Length)
{
array[j] = array[j] * factor;
j = Interlocked.Increment(ref i);
}
});
}
Task.WaitAll(tasks);
}
Example output:
Serial: 0,20 s
Parallel.For: 0,50 s
Parallel.For (degree of parallelism): 8,90 s
Custom parallel: 0,33 s
Custom parallel (extracted max): 0,18 s
Custom parallel (extracted max, half parallelism): 0,18 s
Custom parallel (false sharing): 7,53 s
Serial: 0,21 s
Parallel.For: 0,52 s
Parallel.For (degree of parallelism): 0,36 s
Custom parallel: 0,31 s
Custom parallel (extracted max): 0,18 s
Custom parallel (extracted max, half parallelism): 0,19 s
Custom parallel (false sharing): 7,59 s
Serial: 0,21 s
Parallel.For: 11,21 s
Parallel.For (degree of parallelism): 0,36 s
Custom parallel: 0,32 s
Custom parallel (extracted max): 0,18 s
Custom parallel (extracted max, half parallelism): 0,18 s
Custom parallel (false sharing): 7,76 s
Serial: 0,21 s
Parallel.For: 0,46 s
Parallel.For (degree of parallelism): 0,35 s
Custom parallel: 0,31 s
Custom parallel (extracted max): 0,18 s
Custom parallel (extracted max, half parallelism): 0,18 s
Custom parallel (false sharing): 7,58 s
Serial: 0,21 s
Parallel.For: 0,45 s
Parallel.For (degree of parallelism): 0,40 s
Custom parallel: 0,38 s
Custom parallel (extracted max): 0,18 s
Custom parallel (extracted max, half parallelism): 0,18 s
Custom parallel (false sharing): 7,58 s

Svick already provided a great answer but I'd like to emphasize that the key point is not to "parallelize your code manually" instead of using Parallel.For() but that you have to process larger chunks of data.
This can still be done using Parallel.For() like this:
static void My(double[] array, double factor)
{
int degreeOfParallelism = Environment.ProcessorCount;
Parallel.For(0, degreeOfParallelism, workerId =>
{
var max = array.Length * (workerId + 1) / degreeOfParallelism;
for (int i = array.Length * workerId / degreeOfParallelism; i < max; i++)
array[i] = array[i] * factor;
});
}
which does the same thing as svicks CustomParallelExtractedMax() but is shorter, simpler and (on my machine) performs even slightly faster:
Serial: 3,94 s
Parallel.For: 9,28 s
Parallel.For (degree of parallelism): 9,58 s
Custom parallel: 2,05 s
Custom parallel (extracted max): 1,19 s
Custom parallel (extracted max, half parallelism): 1,49 s
Custom parallel (false sharing): 27,88 s
My: 0,95 s
Btw, the keyword for this which is missing from all the other answers is granularity.

See Custom Partitioners for PLINQ and TPL:
In a For loop, the body of the loop is provided to the method as a delegate. The cost of invoking that delegate is about the same as a virtual method call. In some scenarios, the body of a parallel loop might be small enough that the cost of the delegate invocation on each loop iteration becomes significant. In such situations, you can use one of the Create overloads to create an IEnumerable<T> of range partitions over the source elements. Then, you can pass this collection of ranges to a ForEach method whose body consists of a regular for loop. The benefit of this approach is that the delegate invocation cost is incurred only once per range, rather than once per element.
In your loop body, you are performing a single multiplication, and the overhead of the delegate call will be very noticeable.
Try this:
public static void MultiplicateArray(double[] array, double factor)
{
var rangePartitioner = Partitioner.Create(0, array.Length);
Parallel.ForEach(rangePartitioner, range =>
{
for (int i = range.Item1; i < range.Item2; i++)
{
array[i] = array[i] * factor;
}
});
}
See also: Parallel.ForEach documentation and Partitioner.Create documentation.

Parallel.For involves more complex memory management. That result could vary depending on cpu specs, like #cores, L1 & L2 cache...
Please take a look to this interesting article:
http://msdn.microsoft.com/en-us/magazine/cc872851.aspx

from http://msdn.microsoft.com/en-us/library/system.threading.tasks.parallel.aspx and http://msdn.microsoft.com/en-us/library/dd537608.aspx
you are not creating three thread/process that execute your for, but the iteration of the for is tryed to be executet in parallel, so even with only one for you are using multiple thread/process.
this mean that interation with index = 0 and index = 1 may be executed at the same time.
Probabily you are forcing to use too much thread/process, and the overhead for the creation/execution of them is bigger that the speed gain.
Try to use three normal for but in three different thread/process, if your sistem is multicore (3x at least) it should take less than one minute

Why thread creation is so fast?

I read in books that thread creation is expensive (not so expensive like process creation but nevertheless it is) and we should avoid it. I write the test code and I was shocked how fast thread creation is.
using System;
using System.Diagnostics;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
namespace ConsoleApplication1
{
class Program
{
static int testVal = 0;
static void Main(string[] args)
{
const int ThreadsCount = 10000;
var watch = Stopwatch.StartNew();
for (int i = 0; i < ThreadsCount; i++)
{
var myThread = new Thread(MainVoid);
myThread.Start();
}
watch.Stop();
Console.WriteLine("Test value ={0}", testVal);
Console.WriteLine("Ended in {0} miliseconds", watch.ElapsedMilliseconds);
Console.WriteLine("{0} miliseconds per thread ", (double)watch.ElapsedMilliseconds / ThreadsCount);
}
static void MainVoid()
{
Interlocked.Increment(ref testVal);
}
}
}
Output:
Test value =10000
Ended in 702 miliseconds
0,0702 miliseconds per thread.
Is my code wrong or thread creation is so fast and advices in books are wrong? (I see only some extra memory consumption per thread but no creation time.)

Thread creation is pretty slow. Consider this bit of code that contrasts the speed of doing things inline, and doing things with multiple threads:
private void DoStuff()
{
const int ThreadsCount = 10000;
var sw = Stopwatch.StartNew();
int testVal = 0;
for (int i = 0; i < ThreadsCount; ++i)
{
Interlocked.Increment(ref testVal);
}
sw.Stop();
Console.WriteLine(sw.ElapsedTicks);
sw = Stopwatch.StartNew();
testVal = 0;
for (int i = 0; i < ThreadsCount; ++i)
{
var myThread = new Thread(() =>
{
Interlocked.Increment(ref testVal);
});
myThread.Start();
}
sw.Stop();
Console.WriteLine(sw.ElapsedTicks);
}
On my system, doing it inline requires 200 ticks. With threads it's almost 2 million ticks. So using threads here takes approximately 10,000 times as long. I used ElapsedTicks here rather than ElapsedMilliseconds because with ElapsedMilliseconds the output for the inline code was 0. The threads version takes around 700 milliseconds. Context switches are expensive.
In addition, your test is fundamentally flawed because you don't explicitly wait for all of the threads to finish before harvesting the result. It's quite possible that you could output the value of testVal before the last thread finishes incrementing it.
When timing code, by the way, you should be sure to run it in release mode without the debugger attached. In Visual Studio, use Ctrl+F5 (start without debugging).

StopWatch and 0 milliseconds via c#

I try to calculate time between operations.
So,write two methods with equal code ,but use different ways.
At first way i do like that:
private static void calcAverageTimeUid(ISomeObject someObj, int N,ISnapshot _Snapshot)
{
Stopwatch stopWatch = new Stopwatch();
int averageTime = 0;
var uid = someObj.Uid;
for (int i = 0; i < N; i++)
{
stopWatch.Start();
var coll = _Snapshot.GetObject(uid);
stopWatch.Stop();
TimeSpan ts = stopWatch.Elapsed;
averageTime = averageTime + ts.Milliseconds;
}
averageTime = averageTime / N;
}
And i have result averageTime such as 500 milliseconds. N=1000000 and more.
But , i rewrite this method to two methods: mainCalc ,wich should contains come other methods,f.e.
to get average time of uid,id,name and so on.
mainCalc:
private static void mainCalc(ISomeObject someObj,int N,ISnapshot _Snapshot)
{
int averageTimeUID = 0;
for (int i = 0; i < N; i++)
{
var tmp=calcAverageTimeUid2(someObj,N,_Snapshot);
averageTimeUID+=tmp;
}
averageTimeUID = averageTimeUID / N;
}
And other method:
private static int calcAverageTimeUid2(ISomeObject someObj,int N,ISnapshot _Snapshot)
{
Stopwatch stopWatch = new Stopwatch();
var prop = someObj.Uid;
stopWatch.Start();
var obj = _Snapshot.GetObject(prop);
stopWatch.Stop();
TimeSpan ts = stopWatch.Elapsed;
return ts.Milliseconds;
}
So, i run mainCalc and run calcAcerageTimeUid2 in this method.And result of stopWatch=0 milliseconds!
It is wrong result or not? I dont understand-what way of use stopWatch right?
P.S. delete one of excess StopWatch.
P.P.S. Thank all of you!

Your first routine should be
for (int i = 0; i < N; i++)
{
stopWatch.Start();
var coll = _Snapshot.GetObject(uid);
stopWatch.Stop();
}
averageTime = stopWatch.Elapsed / N;
Note, stopWatch.Start() does not reset the stopwatch back to zero.

Milliseconds is not TotalMilliseconds.
Milliseconds is the entire number of milliseconds of the TimeSpan. Not the total milliseconds which is a double, so you are losing the precision under 1ms.
And why do you return an int, instead of the TimeSpan?
Try this code :
private static void mainCalc(ISomeObject someObj, int N, ISnapshot _Snapshot)
{
var averageTimeUID = TimeSpan.Zero;
for (int i = 0; i < N; i++)
{
averageTimeUID += calcAverageTimeUid2(someObj,N,_Snapshot);
}
averageTimeUID = new TimeSpan( averageTimeUID.Ticks / N );
}
The other method:
private static TimeSpan calcAverageTimeUid2(ISomeObject someObj, int N, ISnapshot _Snapshot)
{
var stopWatch = new Stopwatch();
var prop = someObj.Uid;
stopWatch.Start();
var obj = _Snapshot.GetObject(prop);
stopWatch.Stop();
return stopWatch.Elapsed;
}

The reason you get different results is because you are rounding the number of milliseconds in different places. In your first method, you use one stopwatch and continuously Start() and Stop() it. Your operation must take less than 1 ms, but when you repeatedly start and stop the same stopwatch, the total number of ticks will still increase. That is why with N=1000000 you got only 500 ms.
In the second method, you start and stop a new stopwatch each time, and return the milliseconds. Since each operation is averaging 500/1000000 = 0.00005 ms, the ticks of the stopwatch will accumulate some small value, but the ElapsedMilliseconds (or Milliseconds of the TimeSpan) will still be 0.
EDIT: To solve your problems, the first loop should use the final Elapsed value of the stopwatch once the loop is complete (like the 2nd example in sgmoore's answer). The second method should return the ticks from the method rather than milliseconds and then calculate the milliseconds from the tick frequency of the stopwatch.
In summary, the first operation you are summing a bunch of values like 0.00005, in the second you are summing a bunch of 0s.

Exit the loop after specific time in C#

I have a requirement in my project (C#, VS2010, .NET 4.0) that a particular for loop must finish within 200 milliseconds. If it doesn't then it has to terminate after this duration without executing the remaining iterations. The loop generally goes for i = 0 to about 500,000 to 700,000 so the total loop time varies.
I have read following questions which are similar but they didn't help in my case:
What is the best way to exit out of a loop after an elapsed time of 30ms in C++
How to execute the loop for specific time
So far I have tried using a Stopwatch object to track the elapsed time but it's not working for me. Here are 2 different methods I have tried so far:
Method 1. Comparing the elapsed time within for loop:
Stopwatch sw = new Stopwatch();
sw.Start();
for (i = 0; i < nEntries; i++) // nEntries is typically more than 500,000
{
// Do some stuff
...
...
...
if (sw.Elapsed > TimeSpan.FromMilliseconds(200))
break;
}
sw.Stop();
This doesn't work because if (sw.Elapsed > TimeSpan.FromMilliseconds(200)) takes more than 200 milliseconds to complete. Hence useless in my case. I am not sure whether TimeSpan.FromMilliseconds() generally takes this long or it's just in my case for some reason.
Method 2. Creating a separate thread to compare time:
Stopwatch sw = new Stopwatch();
sw.Start();
bool bDoExit = false;
int msLimit = 200;
System.Threading.ThreadPool.QueueUserWorkItem((x) =>
{
while (bDoExit == false)
{
if (sw.Elapsed.Milliseconds > msLimit)
{
bDoExit = true;
sw.Stop();
}
System.Threading.Thread.Sleep(10);
}
});
for (i = 0; i < nEntries; i++) // nEntries is typically more than 500,000
{
// Do some stuff
...
...
...
if (bDoExit == true)
break;
}
sw.Stop();
I have some other code in the for loop that prints some statistics. It tells me that in case of Method 2, the for loop definitely breaks before completing all the iterations but the loop timing is still 280-300 milliseconds.
Any suggestions to break a for loop strictly with-in 200 milliseconds or less?
Thanks.

For a faster comparison try comparing
if(sw.ElapsedMilliseconds > 200)
break;
You should do that check in the beggining of your loop and also during the processing, ("// Do some stuff" part of the code) because it is possible, for example, that processing starts at 190 (beginning of the loop), lasts 20 and ends at 210.
You could also measure average execution time of your processing (this is approximate because it relies on average time), this way loop should last 200 milliseconds or less, here is a demo that you can put in a Main method of a Console application and easily modify it for your application:
Stopwatch sw = new Stopwatch();
sw.Start();
string a = String.Empty;
int i;
decimal sum = 0, avg = 0, beginning = 0, end = 0;
for (i = 0; i < 700000; i++) // nEntries is typically more than 500,000
{
beginning = sw.ElapsedMilliseconds;
if (sw.ElapsedMilliseconds + avg > 200)
break;
// Some processing
a += "x";
int s = a.Length * 100;
Thread.Sleep(19);
/////////////
end = sw.ElapsedMilliseconds;
sum += end - beginning;
avg = sum / (i + 1);
}
sw.Stop();
Console.WriteLine(
"avg:{0}, count:{1}, milliseconds elapsed:{2}", avg, i + 1,
sw.ElapsedMilliseconds);
Console.ReadKey();

Another option would be to use CancellationTokenSource:
CancellationTokenSource source = new CancellationTokenSource(100);
while(!source.IsCancellationRequested)
{
// Do stuff
}

Use the first one - simple and have better chances to be precise than second one.
Both cases have the same kind of termination condition, so both should behave are more-or-less the same. Second is much more complicated due to usage of threads and Sleep, so I'd use first one. Also second one is much less precise due to sleeps.
There are abolutely no reasons for TimeSpan.FromMilliseconds(200) to take any significant amount of time (as well as calling it in every iteration).

Using cancellation token:
var cancellationToken = new CancellationTokenSource(TimeSpan.FromSeconds(15)).Token;
while (!cancellationToken.IsCancellationRequested)
{
//Do stuff...
}

I don't know if this is that exactly, but I think it's worth a try using a System.Timers.Timer:
int msLimit = 200;
int nEntries = 500000;
bool cancel = false;
System.Timers.Timer t = new System.Timers.Timer();
t.Interval = msLimit;
t.Elapsed += (s, e) => cancel = true;
t.Start();
for (int i = 0; i < nEntries; i++)
{
// do sth
if (cancel) {
break;
}
}

How do you convert Stopwatch ticks to nanoseconds, milliseconds and seconds?

This is a very basic question...quite embarassing, but here goes:
I have a Stopwatch block in C# code. I determine the elapsed time in ticks and then want to convert to ns, ms, s. I know that the Frequency is provided by the Stopwatch class and that it is required for conversion.
Thanks

Stopwatch.Elapsed is a TimeSpan, so you can do myStopwatch.Elapsed.TotalMilliseconds, myStopwatch.Elapsed.TotalSeconds, etc.
// Create new stopwatch
Stopwatch stopwatch = new Stopwatch();
// Begin timing
stopwatch.Start();
// Do something
for (int i = 0; i < 1000; i++)
{
Thread.Sleep(1);
}
// Stop timing
stopwatch.Stop();
// Write result
Console.WriteLine("Time elapsed (s): {0}", stopwatch.Elapsed.TotalSeconds);
Console.WriteLine("Time elapsed (ms): {0}", stopwatch.Elapsed.TotalMilliseconds);
Console.WriteLine("Time elapsed (ns): {0}", stopwatch.Elapsed.TotalMilliseconds * 1000000);
Output:
Time elapsed (s): 2.4976622
Time elapsed (ms): 2497.6622
Time elapsed (ns): 2497662200
Note that stopwatch.ElapsedMilliseconds returns a long and is thus only precise up to the millisecond, while stopwatch.Elapsed.TotalMilliseconds returns a double and has the same precision as stopwatch.ElapsedTicks, except it's easier to use. ILSpy shows that TimeSpan.TotalMilliseconds is computed using ticks anyway.

According to MSDN, Frequency tells you the number of ticks per second. Therefore:
Stopwatch sw = new Stopwatch();
// ...
double ticks = sw.ElapsedTicks;
double seconds = ticks / Stopwatch.Frequency;
double milliseconds = (ticks / Stopwatch.Frequency) * 1000;
double nanoseconds = (ticks / Stopwatch.Frequency) * 1000000000;

Stopwatch.Frequency gives you ticks/second.
So, if you have ticks, you can just divide by frequency to get seconds:
long ticks = sw.ElapsedTicks;
double ns = 1000000000.0 * (double)ticks / Stopwatch.Frequency;
double ms = ns / 1000000.0;
double s = ms / 1000;
For example, you can do:
static void Main()
{
Stopwatch sw = new Stopwatch();
sw.Start();
System.Threading.Thread.Sleep(3456);
sw.Stop();
long ticks = sw.ElapsedTicks;
double ns = 1000000000.0 * (double)ticks / Stopwatch.Frequency;
double ms = ns / 1000000.0;
double s = ms / 1000;
Console.WriteLine("{0}, {1}, {2}", ns, ms, s);
Console.ReadKey();
}
Which, on my system, prints:
3455650175.58075, 3455.65017558075, 3.45565017558075

Use this class:
public static class Utility
{
public static long ElapsedNanoSeconds(this Stopwatch watch)
{
return watch.ElapsedTicks * 1000000000 / Stopwatch.Frequency;
}
public static long ElapsedMicroSeconds(this Stopwatch watch)
{
return watch.ElapsedTicks * 1000000 / Stopwatch.Frequency;
}
}
Then you can get the elapsed nanoseconds/microseconds just like this:
var stopwatch = Stopwatch.StartNew();
//... measured code part
Console.WriteLine(stopwatch.ElapsedNanoSeconds());
// OR
Console.WriteLine(stopwatch.ElapsedMicroSeconds());
For milliseconds you can use the ElapsedMilliseconds() method of Stopwatch.

From the MSDN docs:
Use the Frequency and IsHighResolution fields to determine the precision and resolution of the Stopwatch timing implementation.
long frequency = Stopwatch.Frequency;
Console.WriteLine(" Timer frequency in ticks per second = {0}",
frequency);
long nanosecPerTick = (1000L*1000L*1000L) / frequency;
Console.WriteLine(" Timer is accurate within {0} nanoseconds",
nanosecPerTick);

Use the Elapsed property:
stopwatch.Elapsed.TotalMilliseconds