We Keep Coding

Best approach to a class that would run in background polling other API and saving data

So I have a website written in .NET Core C# and I would like to run a process in the background that would make API calls to other website and save the data in database.
I have created ApiAccessor class and would like to invoke the method from the controller (which uses dependency injections for it's database connection), but if I pass them to the ApiAccessor (it would be async) the connection is already disposed of. I've tried injecting it from the get go, but it will still say that the interfaces are disposed, by the time it finishes. I can only do await on it, but this would cause user to wait for too long. What approach should I take with this one? I am a newbie at DI. Maybe some Singleton class? I would still don't know how to pass dependency injections to singleton
ApiAccessor:
IUserAccount _userAccounts;
public ApiAccessor(IConfiguration configuration, IUserAccount userAccounts)
{
_configuration = configuration;
_userAccounts = userAccounts;
}
//...
MethodToPollApi(){
var newUserIdToAdd = // just some kind of new data from api
_userAccounts.Add(newUserIdToAdd) // accessing DB, that causes errors
}
Controller:
void Index(){
MethodToPollApi();
return View();
}

I would consider an idea of using as called background jobs. There are a few popular frameworks for this type of solutions. Within them: custom implementation based on IHostedService, Quartz.NET, Hangfire, and many more available.
I used to play with many of them, personally prefer Hangfire as it self bootstrapped, provide nice UI for jobs dashboard, and really easy to use - for instance, that is how triggering jobs light look like with it:
Run once immediately:
var jobId = BackgroundJob.Enqueue(() => Console.WriteLine("Fire-and-forget!"));
Run delayed:
BackgroundJob.Schedule(() => Console.WriteLine("Delayed!"), TimeSpan.FromDays(7));
Run repeating:
var jobId = RecurringJob.AddOrUpdate(() => Console.WriteLine("Recurring!"), Cron.Daily);
Pick up completed job and continue:
BackgroundJob.ContinueWith(jobId, () => Console.WriteLine("Continuation!"));

Continuing the answer from #Dmitry. With Hangfire you can do something like this.
services.AddHangfire(x => x.UseSqlServerStorage("<Your connection string>"));
Hope this helps.

How to cache data from repository call

I am working in a multi-layered web application that has ASP.NET MVC as its front-end client. A particular page of this web application is taking a very long time to load. Around 30 seconds.
I downloaded dotTrace and ran it on my application (following this tutorial). I found out that the reason my application is slow.
It turns out it is because one particular method that I have does a load of work (takes time), and that same method gets called a total of 4 times.
Here is a screenshot from dotTrace showing the above:
The method in question is GetTasks(). So in order to improve the speed of the web application I want to cache the data returned from GetTasks() for each request.
If my thinking is correct, this would really improve on the speed issues I am having.
My question is, how can I achieve this? I have never done such a thing before. For each new request, how can I cache the data returned from GetTasks(), and use that for all subsequent calls to GetTasks().

Have you considered the Cache Aside pattern?
You can implement it easily using LazyCache
//probably in my constructor (or use dependency injection)
this.cache = new CachingService()
public List<MyTasks> GetTasks()
{
return cache.GetOrAdd<List<MyTasks>>("get-tasks", () = > {
//go and get the tasks here.
});
}
For more information see https://alastaircrabtree.com/the-easy-way-to-add-caching-to-net-application-and-make-it-faster-is-called-lazycache/

One of the most popsular solution is to cache results. I can shouw you my solution.
First of all install Nuget package: LazyCache
Then you can use wrapper that I've created wrapper: code. You can extract and interface or whatever.
Then you can use it like this:
private readonly CacheManager cacheManager = new CacheManager();
// or injected via ctor
public IEnumerable<Task> GetTasks()
{
return this.cacheManager.Get("Tasks", ctx => this.taskRepository.GetAll());
}
public void AddTask(Task task)
{
this.taskRepository.Create(task);
/// other code
// we need to tell the cache that it should get fresh collectiion
this.cacheManager.Signal("Tasks");
}

WCF Channel and ChannelFactory Caching

So I've decided to up the performance a bit in my WCF application, and attempt to cache Channels and the ChannelFactory. There's two questions I have about all of this that I need to clear up before I get started.
1) Should the ChannelFactory be implemented as a singleton?
2) I'm kind of unsure about how to cache/reuse individual channels. Do you have any examples of how to do this you can share?
It's probably important to note that my WCF service is being deployed as a stand alone application, with only one endpoint.
EDIT:
Thank you for the responses. I still have a few questions though...
1)I guess I'm confused as to where the caching should occur. I'm delivering a client API that uses this code to another department in our company. Does this caching occur on the client?
2)The client API will be used as part of a Silverlight application, does this change anything? In particular, what caching mechanisms are available in such a scenario?
3)I'm still not clear about the design of the GetChannelFactory method. If I have only one service, should only one ChannelFactory ever be created and cached?
I still haven't implemented any caching feature (because I'm utterly confused about how it should be done!), but here's what I have for the client proxy so far:
namespace MyCompany.MyProject.Proxies
{
static readonly ChannelFactory<IMyService> channelFactory =
new ChannelFactory<IMyService>("IMyService");
public Response DoSomething(Request request)
{
var channel = channelFactory.CreateChannel();
try
{
Response response = channel.DoSomethingWithService(request);
((ICommunicationObject)channel).Close();
return response;
}
catch(Exception exception)
{
((ICommenicationObject)channel).Abort();
}
}
}

Use the ChannelFactory to create an instance of the factory, then cache that instance. You can then create communicatino channels as needed/desired from the cached istance.
Do you have a need for multiple channel factories (i.e.., are there multiple services)? In my experience, that's where you'll see the biggest benefit in performance. Creating a channel is a fairly inexpensive task; it's setting everything up at the start that takes time.
I would not cache individual channels - I'd create them, use them for an operation, and then close them. If you cache them, they may time out and the channel will fault, then you'll have to abort it and create a new one anyway.
Not sure why you'd want to usea singleton to implement ChannelFactory, especially if you're going to create it and cache it, and there's only one endpoint.
I'll post some example code later when I have a bit more time.
UPDATE: Code Examples
Here is an example of how I implemented this for a project at work. I used ChannelFactory<T>, as the application I was developing is an n-tier app with several services, and more will be added. The goal was to have a simple way to create a client once per life of the application, and then create communication channels as needed. The basics of the idea are not mine (I got it from an article on the web), though I modified the implementation for my needs.
I have a static helper class in my application, and within that class I have a dictionary and a method to create communication channels from the channelf factory.
The dictionary is as follows (object is the value as it will contain different channel factories, one for each service). I put "Cache" in the example as sort of a placeholder - replace the syntax with whatever caching mechanism you're using.
public static Dictionary<string, object> OpenChannels
{
get
{
if (Cache["OpenChannels"] == null)
{
Cache["OpenChannels"] = new Dictionary<string, object>();
}
return (Dictionary<string, object>)Cache["OpenChannels"];
}
set
{
Cache["OpenChannels"] = value;
}
}
Next is a method to create a communication channel from the factory instance. The method checks to see if the factory exists first - if it does not, it creates it, puts it in the dictionary and then generates the channel. Otherwise it simply generates a channel from the cached instance of the factory.
public static T GetFactoryChannel<T>(string address)
{
string key = typeof(T.Name);
if (!OpenChannels.ContainsKey(key))
{
ChannelFactory<T> factory = new ChannelFactory<T>();
factory.Endpoint.Address = new EndpointAddress(new System.Uri(address));
factory.Endpoint.Binding = new BasicHttpBinding();
OpenChannels.Add(key, factory);
}
T channel = ((ChannelFactory<T>)OpenChannels[key]).CreateChannel();
((IClientChannel)channel).Open();
return channel;
}
I've stripped this example down some from what I use at work. There's a lot you can do in this method - you can handle multiple bindings, assign credentials for authentication, etc. Its pretty much your one stop shopping center for generating a client.
Finally, when I use it in the application, I generally create a channel, do my business, and close it (or abort it if need be). For example:
IMyServiceContract client;
try
{
client = Helper.GetFactoryChannel<IMyServiceContract>("http://myserviceaddress");
client.DoSomething();
// This is another helper method that will safely close the channel,
// handling any exceptions that may occurr trying to close.
// Shouldn't be any, but it doesn't hurt.
Helper.CloseChannel(client);
}
catch (Exception ex)
{
// Something went wrong; need to abort the channel
// I also do logging of some sort here
Helper.AbortChannel(client);
}
Hopefully the above examples will give you something to go on. I've been using something similar to this for about a year now in a production environment and it's worked very well. 99% of any problems we've encountered have usually been related to something outside the application (either external clients or data sources not under our direct control).
Let me know if anything isn't clear or you have further questions.

You could always just make your ChannelFactory static for each WCF Contract...
You should be aware that from .Net 3.5 the proxy objects are pooled for performance reasons by the channel factory. Calling the ICommunicationObject.Close() method actually returns the object to the pool in the hope it can be reused.
I would look at the profiler if you want to do some optimisation, if you can prevent just one IO call being made in your code it could far outweigh any optimisation you will make with the channel factory. Don't pick an area to optimise, use the profiler to find where you can target an optimisation. If you have an SQL database for instance, you will probably find some low hanging fruit in your queries that will get you orders of magnitude performance increases if these haven't already been optimised.

Creating the Channel costs the performance so much. actually , WCF already has the cache mechanism for the ChannelFactory if you use the ClientBase in the client instead of the pure ChannelFactory. But the cache will be expired if you make some anditional operations(Please google it for details if you want).
For the ErOx's issue i got another solution i think it is better. see below:
namespace ChannelFactoryCacheDemo
{
public static class ChannelFactoryInitiator
{
private static Hashtable channelFactories = new Hashtable();
public static ChannelFactory Initiate(string endpointName)
{
ChannelFactory channelFactory = null;
if (channelFactories.ContainsKey(endpointName))//already cached, get from the table
{
channelFactory = channelFactories[endpointName] as ChannelFactory;
}
else // not cached, create and cache then
{
channelFactory = new ChannelFactory(endpointName);
lock (channelFactories.SyncRoot)
{
channelFactories[endpointName] = channelFactory;
}
}
return channelFactory;
}
}
class AppWhereUseTheChannel
{
static void Main(string[] args)
{
ChannelFactory channelFactory = ChannelFactoryInitiator.Initiate("MyEndpoint");
}
}
interface IMyContract { }
}
you can customize the logic and the parameters of the Initiate method yourself if you got another requirement. but this initiator class is not limited only one endpoint. it is powerful for all of the endpoint in your application. hopefully. it works well for you. BTW. this solution is not from me. i got this from a book.

Webservice/WCF (timer update engine)

I am trying to build (csharp) one webservice /WCF engine that make two actions:
Have one timer (thread), that will run in each 10-10 minutes, requesting some information (connecting with other server to grab some info - status) to update in one database. (This must be automatic and no human action will be available). The idea is the webservice automaticaly (10x10 minutes) update the database with the recent information status.
One service method that get some information from one database. (This is one simple method that gives the information when someone request it). This method will responsible to select the status info from database.
The problem is the step 1, because step 2 is very easy.
Can anyone help me, with ideas or some code, how to the step 1.
Any pattern should be used here?

Since it's a webapp (for instance, a "WCF Service Application" project type in VS2010), you can hook into the application events.
By default that project template type doesn't create a Global.asax, so you'll need to "add new item" and choose "Global Application Class" (it won't be available if you already have a Global.asax, FWIW).
Then you can just use the start and end events on the application to start and stop your timer, so something like:
public class Global : System.Web.HttpApplication
{
private static readonly TimeSpan UpdateEngineTimerFrequency = TimeSpan.FromMinutes(10);
private Timer UpdateEngineTimer { get; set; }
private void MyTimerAction(object state)
{
// do engine work here - call other servers, bake cookies, etc.
}
protected void Application_Start(object sender, EventArgs e)
{
this.UpdateEngineTimer = new Timer(MyTimerAction,
null, /* or whatever state object you need to pass */
UpdateEngineTimerFrequency,
UpdateEngineTimerFrequency);
}
protected void Application_End(object sender, EventArgs e)
{
this.UpdateEngineTimer.Dispose();
}
}

The Single Responsibility Principle suggests that you should split these two responsibilities into two services. One (a Windows Service) would handle the Timer. The second, the WCF Service, would have the single operation to query the database and return the data.
These are independent functions, and should be implemented independently.
Additionally, I would recommend against depending on IIS or Application_Start and similar methods. That will prevent your WCF service from being hosted in WAS or some other environment. Keep in mind that WCF is much more flexible than ASMX web services. It doesn't restrict where you host your service. You should think carefully before you place such restrictions on your own service.

Fire and Forget (Asynch) ASP.NET Method Call

We have a service to update customer information to server. One service call takes around few seconds, which is normal.
Now we have a new page where at one instance around 35-50 Costumers information can be updated. Changing service interface to accept all customers together is out of question at this point.
I need to call a method (say "ProcessCustomerInfo"), which will loop through customers information and call web service 35-50 times. Calling service asynchronously is not of much use.
I need to call the method "ProcessCustomerInfo" asynchronously. I am trying to use RegisterAsyncTask for this. There are various examples available on web, but the problem is after initiating this call if I move away from this page, the processing stops.
Is it possible to implement Fire and Forget method call so that user can move away (Redirect to another page) from the page without stopping method processing?

Details on: http://www.codeproject.com/KB/cs/AsyncMethodInvocation.aspx
Basically you can create a delegate which points to the method you want to run asynchronously and then kick it off with BeginInvoke.
// Declare the delegate - name it whatever you would like
public delegate void ProcessCustomerInfoDelegate();
// Instantiate the delegate and kick it off with BeginInvoke
ProcessCustomerInfoDelegate d = new ProcessCustomerInfoDelegate(ProcessCustomerInfo);
simpleDelegate.BeginInvoke(null, null);
// The method which will run Asynchronously
void ProcessCustomerInfo()
{
// this is where you can call your webservice 50 times
}

This was something I whipped just to do that...
public class DoAsAsync
{
private Action action;
private bool ended;
public DoAsAsync(Action action)
{
this.action = action;
}
public void Execute()
{
action.BeginInvoke(new AsyncCallback(End), null);
}
private void End(IAsyncResult result)
{
if (ended)
return;
try
{
((Action)((AsyncResult)result).AsyncDelegate).EndInvoke(result);
}
catch
{
/* do something */
}
finally
{
ended = true;
}
}
}
And then
new DoAsAsync(ProcessCustomerInfo).Execute();
Also need to set the Async property in the Page directive <%# Page Async="true" %>
I'm not sure exactly how reliable this is, however it did work for what I needed it for. Wrote this maybe a year ago.

I believe the issue is the fact is your web service is expecting a client to return the response to, that the service call itself is not a one way communication.
If you're using WCF for your webservices look at http://moustafa-arafa.blogspot.com/2007/08/oneway-operation-in-wcf.html for making a one way service call.
My two cents: IMO whoever put the construct on you that you're not able to alter the service interface to add a new service method is the one making unreasonable demands. Even if your service is a publicly consumed API adding a new service method shouldn't impact any existing consumers.

Sure you can.

I think what you are wanting is a true background thread:
Safely running background threads in ASP.NET 2.0
Creating a Background Thread to Log IP Information