I'm pushing a lot of data into CRM at times, like bulk updates. It is done in a workflow (CodeActivity) class, with its Execute() method. I can use the argument (the context) passed to that method to get the ServiceFactory, and then in turn create the IOrganizationService.
I then iterate a list of objects to update, change properties, and call service.Update(myObj). So far so good.
Now, to speed things up, I thought I'd run 2-4 threads that do the update in parallel.
Question 1
Should I reuse the service I just created, or create a new one, if I run two or more threads? I.e. create one service per thread, or share it?
I'm basically doing Parallel.Invoke(action1, action2, action3); where each action needs a service instance to call service.Update(myObj);
Question 2
If calling serviceFactory.CreateOrganizationService(context.UserId), will it actually create a new service, or return an existing one?
I have everything else nailed down, and it is working just fine, but wanted to get a recommendation/best practice for when multithreading within a workflow's Execute() method.
I would be a little surprised if mulit-threading your application would actually improves performance (But if you can post data that says otherwise, I would love to see it)
Question 1:
The CrmOrganizationService that is used on the client side is not thread safe, I doubt the server version that implements the IOrganizationService is thread safe as well.
Question 2:
I am not aware of Microsoft specifying one way or the other, so that would lead me to believe that whatever the answer is, you can't count on it. The interface doesn't define the answer, so it could potentially be changed with the next update. With that being said, I would just test it yourself:
if(Object.ReferenceEquals(serviceFactory.CreateOrganizationService(context.UserId), serviceFactory.CreateOrganizationService(context.UserId)){
throw new Exception("Returns the same IOrganizationService");
}
throw new Exception("Returns a new IOrganizationService");
So, to summarize, it works just fine to run multiple threads in CRM, from within a workflow. Do this:
Use the executionContext parameter passed to the Execute() method.
Create an IOrganizationService like this:
IOrganizationServiceFactory serviceFactory = executionContext.GetExtension<IOrganizationServiceFactory>();
IOrganizationService service = serviceFactory.CreateOrganizationService(context.UserId);
Now use that service for one thread/task. If you run many threads, create one service per thread.
The other issues I had was related to a specific CRM Addon.
Related
I'm using Entity Framework 6 (DbContext) in a WPF application, and I'd like to find a way to properly cancel the async data load methods (ToListAsync & FirstOrDefaultAsync) so I can immediately start another request.
I've been trying to stick with the single context per form (tab in my case) standard, and so far, I've been dealing with the non-thread safe nature of these calls by ensuring that the UI is disabled during requests, so the user can't start any new requests while one's in progress. However, I've run into a use case now where this is just not possible. I need to keep the UI responsive during some long-running requests, and in order to do that, I need a way to cancel the current request and start another one right away.
I've tried leveraging the CancellationToken support that was added to the Async methods, but I've found that when I cancel the request, it doesn't actually cancel anything. It will properly throw the OperationCanceledException, but the request is still in progress, and when I try to make another request after that, I still get NotSupportedException (A second operation started on this context...)
I'm using change-tracking, so changing the app to use a new Context for every request is not realistic.
Also, I've temporarily gotten around the issue by disposing the current context and creating a new one every time this particular view model makes a request while one's already in progress. This technically solves my issue, but I'm wondering if there's a way to do it while staying with the same context.
So, does anyone have any experience with this? I find it hard to believe I'm the first one who's run into this issue, but all other answers I've found on here for similar questions either recommend I use the CancellationToken (which doesn't work properly) or are a bit older and don't apply to the Async methods.
EDIT 1:
Since no one's answered this yet, I'm really starting to wonder what my options are here. A bit of background. I'm converting a Silverlight application to WPF.The Silverlight application was using WCF RIA services with EF 4.1, but with the WPF application, we decided to just use EF6.1.
With Silverlight & WCF, there is no limit to the number of async calls you can make at time, and we actually have a single context for the entire application (bad, I know, but simple and we never had any issues). We just bind directly to the entities, and use change-tracking to save the changes made by the user.
Is there just no way to do this in WPF, using EF 6.1 and the Async methods, in a real world application, where sometimes, you just need to cancel what the app is in the progress of doing and do what the user wants, without crashing and burning?
Posting my solution for now. I'm not a huge fan, but it's all I've been able to get to work that wouldn't require completely rewriting this application from scratch.
I'm now using the AsyncLock class from AsyncEx in repository methods that are accessing or saving tracked entities. Each of the DbContext objects use their own lock, so I'm not blocking other calls from other contexts or for untracked entities.
As an example, my GetListAsync method in my repository:
public async virtual Task<IList<T>> GetListAsync(IDbContext context,
Expression<Func<T, bool>> where,
IOrderByClause<T>[] orderBy = null,
params Expression<Func<T, object>>[] navigationProperties)
{
using (await context.ContextLock.LockAsync())
{
IList<T> list = await GetListQuery(context, where, orderBy, navigationProperties).ToListAsync();
return list;
}
}
GetListQuery creates the query using the nav properties and where and order by clauses.
I'll probably also add a timeout as well, using a CancellationToken.
I have a web app that uses the iBATIS.NET library.
The DAO is configured by using an XmlDocument, so the code is very similar to what they show in the documentation:
XmlDocument anXmlDoc = someSupportClass.GetDynamicXmlDocument();
DomDaoManagerBuilder builder = new DomDaoManagerBuilder();
builder.Configure(anXmlDoc);
The documentation says right below that:
The DaoManager instance that is built from a dao.config file is aware
of all of the contexts contained within the configuration file. The
context basically bundles DAO implementations together with a session
handler. The DaoManager knows which DAOs and session handler belong to
which context. When you request a DAO instance from the DaoManager,
the proper session handler will be provided with it. Therefore, there
is no need to access the context or session handler directly. Your DAO
knows how it works.
This does not appear to be true. In a threaded process, the following exception is thrown randomly:
WebSessionStore: Could not obtain reference to HttpContext at
IBatisNet.DataAccess.SessionStore.WebSessionStore.ObtainSessionContext()
at
IBatisNet.DataAccess.SessionStore.WebSessionStore.get_LocalSession()
at IBatisNet.DataAccess.DaoManager.IsDaoSessionStarted() at
IBatisNet.DataAccess.Configuration.DaoProxy.Intercept(IInvocation
invocation, Object[] arguments)
The application may run fine for days or even months at a time, but every once in awhile this exception is thrown. The only solution I can find mentioned anywhere is to swap out the SessionStore for a HybridWebThreadSessionStore (mentioned here).
As the documentation says above, this shouldn't be necessary. If I wanted to force it though, how would I go about overriding the SessionStore created by the DomDaoManagerBuilder?
The issue seems to be caused by a race condition in the iBatis.NET framework when the application starts up. If a call to the database is made on the main thread before a call to the database is made on a separate thread, then the separate threads will raise the error since they don't have an HttpContext stored in the SessionStore.
Basically it went Startup -> db call on main thread -> db call on separate thread
When it should have been Startup -> db call on separate thread -> db call on main thread
Basically I just changed the order in which events fire to make sure the db call on the thread is called at least once before anything on the main thread happens.
This post describes the same problem
THE SETUP
In my Global.asax, I kick-start a background process using the Timer
class. This Timer class uses a delgate DoTimerBasedStuff() which is
called periodically. The DoTimerBasedStuff() method accesses two
different databases, MySQL and SQL Server. Both are setup properly
using their MySQLMapper and SQLServerMapper classes.
Aside from a couple of these background threads, the application also
has a web front which accesses both databases using typical .aspx
pages and classes inherited from Page. Nothing fancy.
THE PROBLEM
If the application is started by going to index.aspx using a browser
(this page doesn't do any db stuff but triggers Global.asax) and no
other actions are taken until the first call of DoTimerBasedStuff()
all is well and both the background process and the web users are
fine. IBatis does everything right.
BUT if I start the application by going to index.aspx and then
immediately do some work via the web which accesses a database (before
DoTimerBasedStuff() is executed), the DoTimerBasedStuff() method has
trouble accessing the HttpContext and gives the following error:
Exception: WebSessionStore: Could not obtain reference to HttpContext
at IBatisNet.DataMapper.SessionStore.WebSessionStore.ObtainSessionContext()
at IBatisNet.DataMapper.SessionStore.WebSessionStore.get_LocalSession()
at IBatisNet.DataMapper.SqlMapper.QueryForObject(String
...
The error occurred because you run ibatis in different threads.
You can make use of a new instance of HybridWebThreadSessionStore just before making a query to the database.
var map = new Hashtable
{
{ "FilterA", "MyFilter" }
};
SqlMap.SessionStore = new HybridWebThreadSessionStore(SqlMap.Id);
var listadoJobs = SqlMap.QueryForList<EventoJob>(SbsIbatisConstantes.ListarJobs, map).ToList();
You can check this reference Here
That worked for me.
I need to insert some data and I'm doing it by calling WCF from my UI and passing a list of objects. Then the service calls a lower business layer which inserts sequentialy the items (calling several other managers and making a lots of call/insert throught ObjectContext)
Now the problem I can't understand is this:
If I call the service more times and passing items one by one everything works fine, inserts goes in parallel and I get some performance benefit concerning time.
If I try to call a parallel foreach in the service class I got an exeception, because ObjectContext is it not thread-safe, but I can't lock code every time I use it because it happens too many time
Why if I call WCF does it work? Is there a way to do the same in my manager class?
Thank you
Depending on your service configuration a new instance is created for every parallel service-call. But using a parallel loop within the service will cause the same ObjectContext to be used multiple times. So basically, calling parallel via WCF creates multiple ObjectContexts where executing within the WCF service only uses one (which as you know, is not thread-safe). Depending on the nature of your inserts this might be okay. You could also spin up multiple ObjectContexts within the service.
Some of this is wild guessing, because you can actually influence the behavior of the WCF service to not run multiple instances, but judging from the behavior your experience, this should be the reason, why you can perform parallel inserts using the WCF service and not within the WCF service.
I have a RESTful C# web service (using Open Rasta) that I want to run IronPython scripts that talk to a CouchDB.
One thing I could use some clarification on is: How often do I need a new instance of the python engine and the scope? one each per application? per session? per request?
I currently have a static engine at the application level along with a dictionary of compiled scripts; then, per request, I create a new scope and execute the code within that scope...
Is that correct? thread safe? and as performant as it could be?
EDIT: regarding the bounty Please also answer the question I posed in reply to Jeff: Will a static instance of the engine cause sequential requests from different clients to wait in line to execute? if so I will probably need everything on a per-request basis.
A ScriptRuntime/ScriptEngine per application and a Scope per request is exactly how it should be done. Runtimes/Engine are thread-safe and Scopes are not.
Per request is the way to go unless all of your code is thread safe. You may get better performance using per application (per session implies you have th notion of "sesions" between you client and server), however the implication there is that all of your code in the "application" is thread safe.
So per-request is what you should use unless you know your code to be thread safe.
Note also that per application will be faster only if:
In order to make things thread safe
you've not blocking threads in any
way.
To a certain extent if the
business layer/data layer are
extremely "heavy" (take a lot of
time to instantiate) then some
performance benefit may be gained.
I have a web app that currently uses the current HttpContext to store a LINQ Data Context. The context is persisted for the current request, on a per user basis, per Rick Strahl's blog:
string ocKey = "ocm_" + HttpContext.Current.GetHashCode().ToString("x")
Thread.CurrentContext.ContextID.ToString();
if (!HttpContext.Current.Items.Contains(ocKey))
{
// Get new Data Context and store it in the HTTP Context
}
However, I have some scripts that execute from the global.asax file, that don't have an HttpContext. The HttpContext.Current is NULL, because the server is the one making the "request".
Is there an equivalent object that I can use to store the Data Context? So I don't have to worry about re-creating it, and attaching/detaching objects? I only want to persist the context for the lifetime of my processes.
UPDATED:
I am currently trying to use a static variable in my DAL helper class. on the first call to one of the methods in the class the DataContext is instantiated, and stored in the static variable. At the end of my process, I call another method that calls Dispose on the DataContext, and sets the static variable to NULL.
Can you not just use a static variable specifically for those scripts? That will have the same life-time as the AppDomain. You should probably think carefully about any concurrency concerns, but it sounds like the simplest way to keep a value around.
(I've just checked, and although one instance of HttpApplication can be used to service multiple requests, each one only serves one request at a time - which suggests that multiple instances are created for concurrent request processing. I haven't validated this, but it does sound like it wouldn't be safe to keep it in an instance variable.)
EDIT: Josh's answer suggests that you want this to be per-thread. That sounds slightly odd to me, as unless you've got a lot of these events occurring, you're quite likely to only ever see them execute on different threads, making the whole sharing business pointless. If you really do want that sort of thing, I'd suggest just using an instance variable in the HttpApplication-derived class - for exactly the reason described in the paragraph above :)
Why not use the current HttpContext? The scripts in your global.asax file are all the result of a request coming into the server, so there should be a context associated with that request which you can grab.
I don't understand the need for generating the key based on the hashcode or the thread. There is going to be a separate instance of HttpContext for each request that comes in, and that instance is going to be specific to the thread that is processing the request. Because of that, the key is pretty much worthless when it's based on the instance of HttpContext and the thread.
Also, how do you dispose of the DataContext when you are done? It implements IDisposable for a reason, so I would recommend against a shared instance like this.
UPDATE
In the comments, it indicates that there is a timer that is running that is executing the scripts. Instead of the timer, I would recommend setting up a Scheduled Task which will call a webservice or predetermined page on the site which will perform the task. Then you will always have an HttpContext to work with.
HttpContext.Current is a static method and should be available from anywhere as long as the code is executing within the context of a request.
In your case your not executing within the context of a request, You could look at using Application.Cache but I would caution against holding a DataContext open. I am not very famillar with linq to entities, so I could be wrong, but generally caching data base related items such as connections is bad.
I would also recommend that you consider moving the logic out of your global.asax and to a windows service. This would let you have more control over these tasks, for example you can shut them down seperatley of the web site.
Edit
As JS points out you could use a static variable. You could also define an instance variable marked with ThreadLocal attribute. This will give each thread its own copy of the variable, and can eliminate contention. Since you want each thread to have its own copy anyways.
Is there a reason why these need to be handled the same way as the other DataContexts? It seems to me that if the context is only needed inside the event handling routine, you shouldn't need to keep it around. Especially if it is in Application_Start (as per your comment), I wouldn't bother caching it anywhere -- just use it locally and pass it to the other methods as needed.
Set the DataContext as the state parameter when creating the timer. Based on the info you posted on the comments, it seems to me that your DataContext is more related to the timers than anything else.
Also avoid using the same DataContext for different timers, because you would end up with mixed modifications from the different timers. Also make sure your same timer logic isn't run twice, since it would cause the same i.e. too short period with no control.