We Keep Coding

How to call async method in a thread but wait for it in c#

I have a thread which is responsible for calling a webapi from 4 websites exactly every 2 seconds. The Webapi call method should not be awaited because if a website is not available it will wait 5 second to get timeout and then the next website call will be delayed.
As HttpClient in .NET 4.7.2 has only async methods , it should be used with await, and if not , compiler gives warning and we may get unexpected behavior (as Microsoft says) .
So should I use Task.Run or call Threadpool.QueueUserWorkItem to make a webapi call in parallel.
Here is sudocode :
public class Test1
{
private AutoResetEvent waitEvent = new AutoResetEvent(false);
private volatile bool _terminated = false;
public void Start()
{
Thread T = new Thread(ProcThread);
T.Start();
}
private async void ProcThread()
{
while (!_terminated)
{
await CallWebApi(); <=========== this line
waitEvent.WaitOne(2000);
}
}
private async Task CallWebApi()
{
HttpClient client = new HttpClient();
.....
.....
}
}

So you have an async procedure that uses a HttpClient to fetch some information and process the fetched data:
async Task CallWebApiAsync() {...}
Improvement 1: it is good practice to suffix async methods with async. This is done to make it possible to let an async version exist next to a non-async version that does something similarly.
Inside this method you are using one of the HttpClient methods to fetch the information. As CallWebApiAsync is awaitable, I assume the async methods are used (GetAsync, GetStreamAsync, etc), and that the method only awaits when it needs the result of the async method.
The nice thing about this is, that as a user of CallWebApiAsync, as long as you don't await the call, you are free to do other things, even if the website isn't reacting. The problem is: after 2 seconds, you want to call the method again. But what to do if the method hasn't finished yet.
Improvement 2 Because you want to be able to start a new Task, while the previous one has not finished: remember the started tasks, and throw them away when finished.
HashSet<Task> activeTasks = new HashSet<Task>(); // efficient add, lookup, and removal
void TaskStarted(Task startedTask)
{
// remember the startedTask
activeTasks.Add(startedTask);
}
void TaskCompleted(Task completedTask)
{
// If desired: log or process the results
LogFinishedTask(completedTask);
// Remove the completedTask from the set of ActiveTasks:
activeTasks.Remove(completedTask);
}
It might be handy to remove all completed tasks at once:
void RemoveCompletedTasks()
{
var completedTasks = activeTasks.Where(task => task.IsCompleted).ToList();
foreach (var task in completedTasks)
{
TaskCompleted(completedTask);
}
}
Now we can adjust your ProcThread.
Improvement 3: in async-await always return Task instead of void and Task<TResult> instead of TResult. Only exception: eventhandlers return void.
async Task ProcThread()
{
// Repeatedly: start a task; remember it, and wait 2 seconds
TimeSpan waitTime = TimeSpan.FromSeconds(2);
while (!terminationRequested)
{
Task taskWebApi = CallWebApiAsync();
// You didn't await, so you are free to do other things
// Remember the task that you started.
this.TaskStarted(taskWebApi);
// wait a while before you start new task:
await Task.Delay(waitTime);
// before starting a new task, remove all completed tasks
this.RemoveCompletedTasks();
}
}
Improvement 4: Use TimeSpan.
TimeSpan.FromSeconds(2) is much easier to understand what it represents than a value 2000.
How to stop?
The problem is of course, after you request termination there might still be some tasks running. You'll have to wait for them to finish. But even then: some tasks might not finish at all within reasonable time.
Improvement 5: use CancellationToken to request cancellation.
To cancel tasks in a neat way, class CancellationToken is invented. Users who start a task create a CancellationTokenSource object, and ask this object for a CancellationToken. This token is passed to all async methods. As soon as the user wants to cancel all tasks that were started using this CancellationTokenSource, he requests the CancellationTokenSource to cancel.
All tasks that have a token from this source have promised to regularly check the token to see if cancellation is requested. If so, the task does some cleanup (if needed) and returns.
Everything summarized in one class:
class Test1
{
private HttpClient httpClient = new HttpClient(...);
private HashSet<TTask> activeTasks = new HashSet<TTask>();
public async Task StartAsync(CancellationToken cancellationToken)
{
// repeated CallWebApiAsync until cancellation is requested
TimeSpan waitTime = TimeSpan.FromSeconds(2);
// repeat the following until OperationCancelled
try
{
while (true))
{
// stop if cancellation requested
cancellationToken.ThrowIfCancellationRequested();
var taskWebApi = this.CallWebApiAsync(cancellationToken);
this.activeTasks.Add(taskWebApi);
await Task.Delay(waitTime, cancellationToken);
// remove all completed tasks:
activeTasks.RemoveWhere(task => task.IsCompleted);
}
}
catch (OperationCanceledException exception)
{
// caller requested to cancel. Wait until all tasks are finished.
await Task.WhenAll(this.activeTasks);
// if desired do some logging for all tasks that were not completed.
}
}
And the adjusted CallWebApiAsync:
private async Task CallWebApiAsync(CancellationToken cancellationToken)
{
const string requestUri = ...
var httpResponseMessage = await this.httpClient.GetAsync(requestUri, cancellationToken);
// if here: cancellation not requested
this.ProcessHttpResponse(httpResponseMessage);
}
private void ProcessHttpRespons(HttpResponseMessage httpResponseMessage)
{
...
}
}
Usage:
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
Test1 test = new Test1();
Task taskCallWebApiRepeatedly = test.StartAsync(cancellationTokenSource.Token);
// because you didn't await, you are free to do other things, while WebApi is called
// every 2 seconds
DoSomethingElse();
// you get bored. Request cancellation:
cancellationTokenSource.Cancel();
// of course you need to await until all tasks are finished:
await Task.Wait(taskCallWebApiRepeatedly);
Because everyone promises to check regularly if cancellation is requested, you are certain that within reasonable time all tasks are finished, and have cleaned up their mess. The definition or "reasonable time" is arbitrary, but let's say, less than 100 msec?

If all you want is to execute a method every two seconds, then a System.Timers.Timer is probably the most suitable tool to use:
public class Test1
{
private readonly HttpClient _client;
private readonly System.Timers.Timer _timer;
public Test1()
{
_client = new HttpClient();
_timer = new System.Timers.Timer();
_timer.Interval = 2000;
_timer.Elapsed += Timer_Elapsed;
}
private void Timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
var fireAndForgetTask = CallWebApiAsync();
}
private async Task CallWebApiAsync()
{
var html = await _client.GetStringAsync("http://example.com");
//...
}
public void Start() => _timer.Start();
public void Stop() => _timer.Stop();
}

something like this. BTW take this as pseudo code as I am typing sitting on my bed:)
List<Task> tasks = new List<Task>();
tasks.Add(CallWebApi());
while (! await Task.WhenAny(tasks))
{
tasks.Add(CallWebApi()); <=========== this line
await Task.Delay(2000);
}

Queuing asynchronous task in C#

I have few methods that report some data to Data base. We want to invoke all calls to Data service asynchronously. These calls to data service are all over and so we want to make sure that these DS calls are executed one after another in order at any given time. Initially, i was using async await on each of these methods and each of the calls were executed asynchronously but we found out if they are out of sequence then there are room for errors.
So, i thought we should queue all these asynchronous tasks and send them in a separate thread but i want to know what options we have? I came across 'SemaphoreSlim' . Will this be appropriate in my use case?
Or what other options will suit my use case? Please, guide me.
So, what i have in my code currently
public static SemaphoreSlim mutex = new SemaphoreSlim(1);
//first DS call
public async Task SendModuleDataToDSAsync(Module parameters)
{
var tasks1 = new List<Task>();
var tasks2 = new List<Task>();
//await mutex.WaitAsync(); **//is this correct way to use SemaphoreSlim ?**
foreach (var setting in Module.param)
{
Task job1 = SaveModule(setting);
tasks1.Add(job1);
Task job2= SaveModule(GetAdvancedData(setting));
tasks2.Add(job2);
}
await Task.WhenAll(tasks1);
await Task.WhenAll(tasks2);
//mutex.Release(); // **is this correct?**
}
private async Task SaveModule(Module setting)
{
await Task.Run(() =>
{
// Invokes Calls to DS
...
});
}
//somewhere down the main thread, invoking second call to DS
//Second DS Call
private async Task SendInstrumentSettingsToDS(<param1>, <param2>)
{
//await mutex.WaitAsync();// **is this correct?**
await Task.Run(() =>
{
//TrackInstrumentInfoToDS
//mutex.Release();// **is this correct?**
});
if(param2)
{
await Task.Run(() =>
{
//TrackParam2InstrumentInfoToDS
});
}
}

Initially, i was using async await on each of these methods and each of the calls were executed asynchronously but we found out if they are out of sequence then there are room for errors.
So, i thought we should queue all these asynchronous tasks and send them in a separate thread but i want to know what options we have? I came across 'SemaphoreSlim' .
SemaphoreSlim does restrict asynchronous code to running one at a time, and is a valid form of mutual exclusion. However, since "out of sequence" calls can cause errors, then SemaphoreSlim is not an appropriate solution since it does not guarantee FIFO.
In a more general sense, no synchronization primitive guarantees FIFO because that can cause problems due to side effects like lock convoys. On the other hand, it is natural for data structures to be strictly FIFO.
So, you'll need to use your own FIFO queue, rather than having an implicit execution queue. Channels is a nice, performant, async-compatible queue, but since you're on an older version of C#/.NET, BlockingCollection<T> would work:
public sealed class ExecutionQueue
{
private readonly BlockingCollection<Func<Task>> _queue = new BlockingCollection<Func<Task>>();
public ExecutionQueue() => Completion = Task.Run(() => ProcessQueueAsync());
public Task Completion { get; }
public void Complete() => _queue.CompleteAdding();
private async Task ProcessQueueAsync()
{
foreach (var value in _queue.GetConsumingEnumerable())
await value();
}
}
The only tricky part with this setup is how to queue work. From the perspective of the code queueing the work, they want to know when the lambda is executed, not when the lambda is queued. From the perspective of the queue method (which I'm calling Run), the method needs to complete its returned task only after the lambda is executed. So, you can write the queue method something like this:
public Task Run(Func<Task> lambda)
{
var tcs = new TaskCompletionSource<object>();
_queue.Add(async () =>
{
// Execute the lambda and propagate the results to the Task returned from Run
try
{
await lambda();
tcs.TrySetResult(null);
}
catch (OperationCanceledException ex)
{
tcs.TrySetCanceled(ex.CancellationToken);
}
catch (Exception ex)
{
tcs.TrySetException(ex);
}
});
return tcs.Task;
}
This queueing method isn't as perfect as it could be. If a task completes with more than one exception (this is normal for parallel code), only the first one is retained (this is normal for async code). There's also an edge case around OperationCanceledException handling. But this code is good enough for most cases.
Now you can use it like this:
public static ExecutionQueue _queue = new ExecutionQueue();
public async Task SendModuleDataToDSAsync(Module parameters)
{
var tasks1 = new List<Task>();
var tasks2 = new List<Task>();
foreach (var setting in Module.param)
{
Task job1 = _queue.Run(() => SaveModule(setting));
tasks1.Add(job1);
Task job2 = _queue.Run(() => SaveModule(GetAdvancedData(setting)));
tasks2.Add(job2);
}
await Task.WhenAll(tasks1);
await Task.WhenAll(tasks2);
}

Here's a compact solution that has the least amount of moving parts but still guarantees FIFO ordering (unlike some of the suggested SemaphoreSlim solutions). There are two overloads for Enqueue so you can enqueue tasks with and without return values.
using System;
using System.Threading;
using System.Threading.Tasks;
public class TaskQueue
{
private Task _previousTask = Task.CompletedTask;
public Task Enqueue(Func<Task> asyncAction)
{
return Enqueue(async () => {
await asyncAction().ConfigureAwait(false);
return true;
});
}
public async Task<T> Enqueue<T>(Func<Task<T>> asyncFunction)
{
var tcs = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
// get predecessor and wait until it's done. Also atomically swap in our own completion task.
await Interlocked.Exchange(ref _previousTask, tcs.Task).ConfigureAwait(false);
try
{
return await asyncFunction().ConfigureAwait(false);
}
finally
{
tcs.SetResult();
}
}
}

Please keep in mind that your first solution queueing all tasks to lists doesn't ensure that the tasks are executed one after another. They're all running in parallel because they're not awaited until the next tasks is startet.
So yes you've to use a SemapohoreSlim to use async locking and await. A simple implementation might be:
private readonly SemaphoreSlim _syncRoot = new SemaphoreSlim(1);
public async Task SendModuleDataToDSAsync(Module parameters)
{
await this._syncRoot.WaitAsync();
try
{
foreach (var setting in Module.param)
{
await SaveModule(setting);
await SaveModule(GetAdvancedData(setting));
}
}
finally
{
this._syncRoot.Release();
}
}
If you can use Nito.AsyncEx the code can be simplified to:
public async Task SendModuleDataToDSAsync(Module parameters)
{
using var lockHandle = await this._syncRoot.LockAsync();
foreach (var setting in Module.param)
{
await SaveModule(setting);
await SaveModule(GetAdvancedData(setting));
}
}

One option is to queue operations that will create tasks instead of queuing already running tasks as the code in the question does.
PseudoCode without locking:
Queue<Func<Task>> tasksQueue = new Queue<Func<Task>>();
async Task RunAllTasks()
{
while (tasksQueue.Count > 0)
{
var taskCreator = tasksQueue.Dequeu(); // get creator
var task = taskCreator(); // staring one task at a time here
await task; // wait till task completes
}
}
// note that declaring createSaveModuleTask does not
// start SaveModule task - it will only happen after this func is invoked
// inside RunAllTasks
Func<Task> createSaveModuleTask = () => SaveModule(setting);
tasksQueue.Add(createSaveModuleTask);
tasksQueue.Add(() => SaveModule(GetAdvancedData(setting)));
// no DB operations started at this point
// this will start tasks from the queue one by one.
await RunAllTasks();
Using ConcurrentQueue would be likely be right thing in actual code. You also would need to know total number of expected operations to stop when all are started and awaited one after another.

Building on your comment under Alexeis answer, your approch with the SemaphoreSlim is correct.
Assumeing that the methods SendInstrumentSettingsToDS and SendModuleDataToDSAsync are members of the same class. You simplay need a instance variable for a SemaphoreSlim and then at the start of each methode that needs synchornization call await lock.WaitAsync() and call lock.Release() in the finally block.
public async Task SendModuleDataToDSAsync(Module parameters)
{
await lock.WaitAsync();
try
{
...
}
finally
{
lock.Release();
}
}
private async Task SendInstrumentSettingsToDS(<param1>, <param2>)
{
await lock.WaitAsync();
try
{
...
}
finally
{
lock.Release();
}
}
and it is importend that the call to lock.Release() is in the finally-block, so that if an exception is thrown somewhere in the code of the try-block the semaphore is released.

What task is being returned and why is this task status RanToCompletion

I am rather new to task based programming and trying to determine how to return a task and verify that it has been started. The code that I got to work was not what I was expecting. The console application is as follows:
public static void Main(string[] args)
{
var mySimple = new Simple();
var cts = new CancellationTokenSource();
var task = mySimple.RunSomethingAsync(cts.Token);
while (task.Status != TaskStatus.RanToCompletion)
{
Console.WriteLine("Starting...");
Thread.Sleep(100);
}
Console.WriteLine("It is started");
Console.ReadKey();
cts.Cancel();
}
public class Simple
{
public async void RunSomething(CancellationToken token)
{
var count = 0;
while (true)
{
if (token.IsCancellationRequested)
{
break;
}
Console.WriteLine(count++);
await Task.Delay(TimeSpan.FromMilliseconds(1000), token).ContinueWith(task => { });
}
}
public Task RunSomethingAsync(CancellationToken token)
{
return Task.Run(() => this.RunSomething(token));
}
}
The output is:
Starting...
0
It is started
1
2
3
4
Why is the task that is being returned have a status as TaskStatus.RanToCompletion compared to TaskStatus.Running as we see that the while loop is still executing? Am I checking the status of the task of putting the RunSomething task on the threadpool rather than the RunSomething task itself?

RunSomething is an async void method, meaning it exposes no means of the caller ever determining when it finishes, they can only ever start the operation and then have no idea what happens next. You then wrap a call to it inside of Task.Run, this is schedluing a thread pool thread to start RunSomething. It will then complete as soon as it has finished starting that Task.
If RunSomething actually returned a Task, then the caller would be able to determine when it actually finished, and if you waited on it it wouldn't actually indicate that it was done until that asynchronous operation was actually finished (there would be no reason to use Task.Run to start it in another thead, you'd be better off just calling it directly and not wasting the effort of moving that to a thread pool thread).

Never use async void (https://msdn.microsoft.com/en-us/magazine/jj991977.aspx)
instead you should use async Task
If you need to call an async method from a non-async (such as from a static void main) you should do something like this:
mySimple.RunSomethingAsync(cts.Token).GetAwaiter().GetResult();
That will effectively make the method a synchronous call.

You can use async void, but only for events.

Task being marked as RanToCompletion at await, when still Running

I'm still getting up to speed with async & multi threading. I'm trying to monitor when the Task I Start is still running (to show in a UI). However it's indicating that it is RanToCompletion earlier than I want, when it hits an await, even when I consider its Status as still Running.
Here is the sample I'm doing. It all seems to be centred around the await's. When it hits an await, it is then marked as RanToCompletion.
I want to keep track of the main Task which starts it all, in a way which indicates to me that it is still running all the way to the end and only RanToCompletion when it is all done, including the repo call and the WhenAll.
How can I change this to get the feedback I want about the tskProdSeeding task status?
My Console application Main method calls this:
Task tskProdSeeding;
tskProdSeeding = Task.Factory.StartNew(SeedingProd, _cts.Token);
Which the runs this:
private async void SeedingProd(object state)
{
var token = (CancellationToken)state;
while (!token.IsCancellationRequested)
{
int totalSeeded = 0;
var codesToSeed = await _myRepository.All().ToListAsync(token);
await Task.WhenAll(Task.Run(async () =>
{
foreach (var code in codesToSeed)
{
if (!token.IsCancellationRequested)
{
try
{
int seedCountByCode = await _myManager.SeedDataFromLive(code);
totalSeeded += seedCountByCode;
}
catch (Exception ex)
{
_logger.InfoFormat(ex.ToString());
}
}
}
}, token));
Thread.Sleep(30000);
}
}

If you use async void the outer task can't tell when the task is finished, you need to use async Task instead.
Second, once you do switch to async Task, Task.Factory.StartNew can't handle functions that return a Task, you need to switch to Task.Run(
tskProdSeeding = Task.Run(() => SeedingProd(_cts.Token), _cts.Token);
Once you do both of those changes you will be able to await or do a .Wait() on tskProdSeeding and it will properly wait till all the work is done before continuing.
Please read "Async/Await - Best Practices in Asynchronous Programming" to learn more about not doing async void.
Please read "StartNew is Dangerous" to learn more about why you should not be using StartNew the way you are using it.
P.S. In SeedingProd you should switch it to use await Task.Delay(30000); insetad of Thread.Sleep(30000);, you will then not tie up a thread while it waits. If you do this you likely could drop the
tskProdSeeding = Task.Run(() => SeedingProd(_cts.Token), _cts.Token);
and just make it
tskProdSeeding = SeedingProd(_cts.Token);
because the function no-longer has a blocking call inside of it.

I'm not convinced that you need a second thread (Task.Run or StartNew) at all. It looks like the bulk of the work is I/O-bound and if you're doing it asynchronously and using Task.Delay instead of Thread.Sleep, then there is no thread consumed by those operations and your UI shouldn't freeze. The first thing anyone new to async needs to understand is that it's not the same thing as multithreading. The latter is all about consuming more threads, the former is all about consuming fewer. Focus on eliminating the blocking and you shouldn't need a second thread.
As others have noted, SeedingProd needs to return a Task, not void, so you can observe its completion. I believe your method can be reduced to this:
private async Task SeedingProd(CancellationToken token)
{
while (!token.IsCancellationRequested)
{
int totalSeeded = 0;
var codesToSeed = await _myRepository.All().ToListAsync(token);
foreach (var code in codesToSeed)
{
if (token.IsCancellationRequested)
return;
try
{
int seedCountByCode = await _myManager.SeedDataFromLive(code);
totalSeeded += seedCountByCode;
}
catch (Exception ex)
{
_logger.InfoFormat(ex.ToString());
}
}
await Task.Dealy(30000);
}
}
Then simply call the method, without awaiting it, and you'll have your task.
Task mainTask = SeedingProd(token);

When you specify async on a method, it compiles into a state machine with a Task, so SeedingProd does not run synchronously, but acts as a Task even if returns void. So when you call Task.Factory.StartNew(SeedingProd) you start a task that kick off another task - that's why the first one finishes immediately before the second one. All you have to do is add the Task return parameter instead of void:
private async Task SeedingProdAsync(CancellationToken ct)
{
...
}
and call it as simply as this:
Task tskProdSeeding = SeedingProdAsync(_cts.Token);

The lack of non-capturing Task.Yield forces me to use Task.Run, why follow that?

Apologies in advance if this question is opinion-based. The lack of Task.Yield version which wouldn't capture the execution context was already discussed here. Apparently, this feature was present in some form in early versions of Async CTP but was removed because it could easily be misused.
IMO, such feature could be as easily misused as Task.Run itself. Here's what I mean. Imagine there's an awaitable SwitchContext.Yield API which schedules the continuation on ThreadPool, so the execution will always continues on a thread different from the calling thread. I could have used it in the following code, which starts some CPU-bound work from a UI thread. I would consider it a convenient way of continuing the CPU-bound work on a pool thread:
class Worker
{
static void Log(string format, params object[] args)
{
Debug.WriteLine("{0}: {1}", Thread.CurrentThread.ManagedThreadId, String.Format(format, args));
}
public async Task UIAction()
{
// UI Thread
Log("UIAction");
// start the CPU-bound work
var cts = new CancellationTokenSource(5000);
var workTask = DoWorkAsync(cts.Token);
// possibly await for some IO-bound work
await Task.Delay(1000);
Log("after Task.Delay");
// finally, get the result of the CPU-bound work
int c = await workTask;
Log("Result: {0}", c);
}
async Task<int> DoWorkAsync(CancellationToken ct)
{
// start on the UI thread
Log("DoWorkAsync");
// switch to a pool thread and yield back to the UI thread
await SwitchContext.Yield();
Log("after SwitchContext.Yield");
// continue on a pool thread
int c = 0;
while (!ct.IsCancellationRequested)
{
// do some CPU-bound work on a pool thread: counting cycles :)
c++;
// and use async/await too
await Task.Delay(50);
}
return c;
}
}
Now, without SwitchContext.Yield, DoWorkAsync would look like below. It adds some extra level of complexity in form of async delegate and task nesting:
async Task<int> DoWorkAsync(CancellationToken ct)
{
// start on the UI thread
Log("DoWorkAsync");
// Have to use async delegate
// Task.Run uwraps the inner Task<int> task
return await Task.Run(async () =>
{
// continue on a pool thread
Log("after Task.Yield");
int c = 0;
while (!ct.IsCancellationRequested)
{
// do some CPU-bound work on a pool thread: counting cycles :)
c++;
// and use async/await too
await Task.Delay(50);
}
return c;
});
}
That said, implementing SwitchContext.Yield may actually be quite simple and (I dare to say) efficient:
public static class SwitchContext
{
public static Awaiter Yield() { return new Awaiter(); }
public struct Awaiter : System.Runtime.CompilerServices.INotifyCompletion
{
public Awaiter GetAwaiter() { return this; }
public bool IsCompleted { get { return false; } }
public void OnCompleted(Action continuation)
{
ThreadPool.QueueUserWorkItem((state) => ((Action)state)(), continuation);
}
public void GetResult() { }
}
}
So, my question is, why should I prefer the second version of DoWorkAsync over the first one, and why would using SwitchContext.Yield be considered a bad practice?

You don't have to put the Task.Run in DoWorkAsync. Consider this option:
public async Task UIAction()
{
// UI Thread
Log("UIAction");
// start the CPU-bound work
var cts = new CancellationTokenSource(5000);
var workTask = Task.Run(() => DoWorkAsync(cts.Token));
// possibly await for some IO-bound work
await Task.Delay(1000);
Log("after Task.Delay");
// finally, get the result of the CPU-bound work
int c = await workTask;
Log("Result: {0}", c);
}
This results in code with much clearer intent. DoWorkAsync is a naturally synchronous method, so it has a synchronous signature. DoWorkAsync neither knows nor cares about the UI. The UIAction, which does care about the UI thread, pushes off the work onto a background thread using Task.Run.
As a general rule, try to "push" any Task.Run calls up out of your library methods as much as possible.