I am calling the SQL Server Reporting Services Webservice from inside an asp.net application. I need to fire off all the subscriptions for a given report. But I don't want the user to have to wait around for it to all happen, so I want to have all the webservice calls inside a separate thread, and return to the user straight away.
I am using code that looks something like this:
public static void FireAllAsync(string ReportPath)
{
Hashtable paramValues = new Hashtable();
ThreadPool.QueueUserWorkItem(new WaitCallback(FireAll), ReportPath);
}
public static void FireAll(object ReportPath)
{
ReportingService rs = new ReportingService();
rs.Credentials = System.Net.CredentialCache.DefaultCredentials;
Subscription[] SubList = rs.ListSubscriptions((string)ReportPath, null);
foreach (Subscription CurSub in SubList) {
rs.FireEvent(CurSub.EventType, CurSub.SubscriptionID);
}
}
Calling FireAll works fine, but trying to call FireAllAsync to use the Thread fails with a 401 error. I believe the problem with the credentials not getting passed through properly. ie this line of code here:
rs.Credentials = System.Net.CredentialCache.DefaultCredentials;
I do not have a good understanding of how the credentials cache works, so I can't figure out why it doesn't like being in a separate thread.
I have tried grabbing the credentials in the outer function, and passing them through as an argument, but the same error occurs.
Does anyone have any ideas what may be happening?
When your primary thread is executing, it has an impersonation context prepared for it. In an authenticated scenario, IIS has authenticated the user, and then set the thread token - not the process token - for that user.
Because the thread token is set for the user, that thread can then act on behalf of the user (on the local box; acting on behalf of the user on another box requires delegation to be set up if Integrated Windows Authentication (NTLM/Kerberos) is used).
But, if you spin off a worker thread without any identity information, that worker thread will be spawned without its own token, so it'll use the process token.
If you can do what you need to do on the initial thread, you don't have a delegation problem, just a lack of the user token on the worker thread.
Threadpool threads will run as the process identity (NetworkService by default, so the computer account of the hosting machine, unless your App Pool is set to run as SomeUser, in which case all worker threads will run as SomeUser) - and if that user has access to whatever it is the initial user wants, all is good.
If not, 401sville.
There are web references that can help work around this, with various possible answers:
Link
http://aspalliance.com/articleViewer.aspx?aId=650&pId=2
Related
Problem: If the DB is offline when this service is started, this service will not start as it fails inside this line: var container = new BootStrapper().Container; on start.
private static void Main(string[] args)
{
Logger.Info("Engine Service is bootstrapping...");
AppDomain.CurrentDomain.UnhandledException += UncaughtExceptions.DomainException;
Directory.SetCurrentDirectory(AppDomain.CurrentDomain.BaseDirectory);
var container = new BootStrapper().Container;
var controller = container.Resolve<EngineController>();
ServiceBase.Run(controller.MainView as ServiceBase);
container.Dispose();
}
The reason it fails there is that it runs this code where it adds the nhibernate facility container.AddFacility<NHibernateFacility>(); and fails with a connection timeout.
public void Install(IWindsorContainer container, IConfigurationStore store)
{
var isAutoTxFacilityRegistered = container.Kernel.GetFacilities().Any(f => f is AutoTxFacility);
if (!isAutoTxFacilityRegistered) container.AddFacility<AutoTxFacility>();
container.Register(
Component.For<INHibernateInstaller>().ImplementedBy<CieFluentInstaller>().IsDefault().LifestyleTransient(),
Classes.FromThisAssembly().Pick().WithService.DefaultInterfaces().LifestyleTransient()
);
var isNHibernateFacilityRegistered = container.Kernel.GetFacilities().Any(f => f is NHibernateFacility);
if (!isNHibernateFacilityRegistered) container.AddFacility<NHibernateFacility>();
}
If the windows service start up takes longer than 30 seconds (which it may if updates or backups are being conducted on the DB) the app service fails to start.
I'm using FluentNhibernate, NHibernate, Castle Windsor with NHibernateFacility.
Things I've tried:
Can't do it from the service start event because it fails before it
gets to the view or controller. The view and controller have no
direct access to the IoC container, only via an injected IoCFactory
as per Castle Windsor recommendations.
I've tried to spawn a thread in the main and start it off there with
a retry loop but because the service "waits" inside the
ServiceBase.Run method, I can't seem to get the correct returns to
make it "fake start" while in a retry loop.
Investigated lengthening the service start timeout, but can't access
the servicebase/view since it fails before then and a system wide
change at hundreds of production sites is not an option.
Question: How can I make it so that the windows service "starts" when DB is offline given the design?
You need to divide your startup actions into two categories:
Actions that must happen fairly immediately and/or won't fix themselves
in case of failure. Things such as a mandatory configuration file
missing, for which administrator intervention would be required.
Actions that we're OK to delay, or - more importantly - actions that can
fail due to transient errors. Such errors can be network failure or that
we happened to start somewhat faster than the database server after a
reboot.
You service OnStart code should follow this basic structure:
OnStart:
Perform the immediate category 1 tasks and exit if any of these fail.
Launch the main application thread.
One approach to the "main application thread" is to follow this basic
structure:
ManualResetEvent shutdownRequestedEvent = new ManualResetEvent()
RealMain:
while (!shutdownRequestedEvent.WaitOne(0) && !bootstrapPerformed)
{
try
{
PerformBootstrap()
bootstrapPerformed = true
}
catch (Exception ex)
{
LogError(ex)
}
if (!bootstrapPerformed)
shutdownRequestedEvent.WaitOne(some timeout)
}
Second bootstrap action similar to above, etc.
Third bootstrap action similar to above, etc.
Eventually, start performing real work, while listening to
the shutdownRequestedEvent.
The services OnShutdown would signal the shutdownRequestedEvent and then
wait for the RealMain thread to exit.
If the RealMain thread serves no purpose other then setup, it should perhaps
be allowed to exit when it's done with all bootstrap tasks.
Another thing to be careful about is to make sure your service, during normal operation, can withstand the temporary loss of access to a resource due to transient errors. For example, your service shouldn't crash just because someone reboots the database server. It should just wait patiently and retry forever.
An alternative approach that can work in some cases is to handle the bootstrapping as a dependency of whatever the real task is. For instance, launch the real task, the real task will request a database session, to get that we must have the session factory, if we don't yet have the session factory, launch the session factory initialization. If the session factory
cannot be created, exception bubbles up and the whole task fails. The remaining
work is now to wait a little while and then retry the task. Repeat forever.
Turned out to be a bug in NHibernate that prevented doing any of the above. Between Nibernate 2.0 and 3.0 you have to add the following to the NHibernate v3.0+ config (or in this case the FluentNHibernate):
cfg.SetProperty("hbm2ddl.keywords", "none");
This allows NHibernate to properly bootstrap itself and get to the controller now without error.
As I understand Azure Worker roles run by the help of Host application called WaWorkerHost.exe and there is another application called WaHostBootstrapper.exe which checks if WaWorkerHost.exe is running and if not it will run the WaWorkerHost.exe.
How often does this 'worker role status check' occurs?
How can I quickly restart the Worker role myself? I can either reboot the machine worker role is running and wait for few minutes or chose the following traditional method:
Taskkill /im /f WaWorkerHost.exe
and wait for few minutes for the WaHostBootstrapper.exe to kick in but this very inefficient and slow.
Is there any (instant)method of restarting the worker role?
Can I run something like the following and expect similar results to the WaHostBootstapper.exe or there are other consideration?
WaWorkerHost.exe {MyAzureWorkerRole.dll}
The bootstrapper checks the WaWorkerHost status every 1 second.You can see it in the bootsrapper logs (c:\resources\WaHostBootstrapper.txt), by looking at interval of the trace:
"Getting status from client WaWorkerHost.exe"
You can use AzureTools which is a utility used by Azure support team.
One of the its features is gracefully recycle the role instance:
Alternatively, you can restart the instance programmatically:
Upload management certificate to your subscription.
Use the following code to programmatically restart the instance:
Using Microsoft Azure Compute Management library:
X509Certificate2 cert = new X509Certificate2("");
var credentials = new CertificateCloudCredentials("your_subscription_id", cert);
using (var managementClient = new ComputeManagementClient(credentials))
{
OperationStatusResponse response =
await managementClient.Deployments.RebootRoleInstanceByDeploymentSlotAsync(
"cloud_service_name",
DeploymentSlot.Production, // or staging
"instance_name");
}
This is not recommended, for three reasons:
The bootsrapper checks every second, which should be enough for most cases.
It could lead to weird issues. For example, you kill the worker, bootstrapper identifies that the worker is down, you manually start the worker, bootstrapper also tries to start the worker and fail (will crash? will enter zombie state?). It can lead to unhealthy bootstrapper, means that nothing takes care of the worker process.
It depends, of course, on what's the bootstrapper does other than starting the worker. But even if it is currently does nothing other than starting the role, you cannot know for sure if tomorrow Azure team will decide to add it more responsibilities/actions.
If the role itself is aware that it needs to restart, it can call RoleEnvironment.RequestRecycle to cause the role instance to be restarted.
I created a page to send thousands of emails to our clients, almost 8K emails.
The sending process is taking hours, but after a while I couldn't access any page (get waiting...) in the site that is hosting the page except for static files (images etc...).
Using: IIS 6 and .Net 4.0
Code:
public static bool Send(MailSettings settings, Action<string, string[], bool> Sent = null)
{
System.Net.Mail.SmtpClient client;
...
foreach(){
try{ client.Send(message);}catch{...client.Dispose();...}
Sent.BeginInvoke(stringValue, stringArray, boolValue, null, null);
if(count++>N){
count=1;
System.Threading.Thread.Sleep(1000);
}
}
...
}
public void SentComplete(string email, string[] value, bool isSent)
{
....//DB logging
}
Note: Other sites using the same Application pool were fine!
Questions:
Is there a IIS 6.0 limitation to the number of threads for the same website?
Any Ideas if my code was causing any performance issues? Am I using Action right?
There are many things wrong with this code:
Your try catch block is very weird. You are disposing an object in
one iteration but it can be used by others. Try using block instead.
There is a maximum time for asp.net request to execute.
Don't put thread sleep in asp.net!
Yes, there is a maximum thread count in asp.net pool
If you end up blocking for one reason, you can also be blocked by the session (if you have one)... Does it work if you open a different browser?
You're firing off a whole bunch of actions to be executed in the thread pool. There is a max number of threads that the thread pool will create, after that the work items sent to the pool simply get queued up. Since you're flooding the thread pool with so many operations you're preventing the thread pool from ever having an opportunity to get a chance to work on the items added by ASP.NET to handle pages. Since static items don't need to push work to the thread pool, those items can be serviced.
You shouldn't be firing off so many items in parallel. You should be limiting the degree of parallelism to a reasonably small fixed amount. Let those handful of items that you start each process a large number of operations that you have so that the threads in the thread pool have the possibility of working on other things as well.
We regularly send 10000 client emails. I store all the details of the email in a database and then call a web service to send them. This just chuggs through them and affects nothing else. I do put the thread to sleep (in the web service) for 100ms between each call to Send ... if I don't do this firing off so many emails seems to overwhelm our mail server and we get some odd things happening.
I use advapi32.dll's logonuser method to access data over our network.
I know it change the thread's user to the information i give it, but i was wondering if there's a way to reverse it.
I want to access the data and then return to the local user credentials.
Some time ago I created a small impersonator class.
Basically you wrap your code to execute under another user simply inside a using block:
using ( new Impersonator( "myUsername", "myDomainname", "myPassword" ) )
{
...
<code that executes under the new context>
...
}
Worked very well for my projects.
You can call RevertToSelf.
That said, there is something to be said for spinning up a dedicated thread for the impersonation task, and terminating it when the impersonation work is complete. This will segregate the impersonation work so that if any callbacks or messages are processed on the main thread they will be performed in the context of the principal user rather than the impersonated user. In fact, the more I think about this the stronger I feel that a dedicated thread is the solution.
Some Details
I am working with VisualWebGUI, so this app is like ASP.NET, and it is deployed on IIS 7 (for testing)
For my 'Web Site', Anonymous Authentication is set to a specific user (DomainName\DomainUser). In my web.config, I have impersonation on. This is how I got my app to access the share in the first place.
The Problem
There is a point in the the app where we use the Thread class, something similar to:
Thread myThread = new Thread(new ThreadStart(objInstance.PublicMethod));
myThread.Start();
What I have noticed is that I can write to my logs (text file on the share), everywhere throughout my code, except in the thread that I kicked off. I added some debugging output and what I see for users is:
The thread that's kicked off: NT AUTHORITY\NETWORK SERVICE
Everywhere else in my code: DomainName\DomainUser (described in my IIS setup)
OK, for some reason the thread gets a different user (NETWORK SERVICE). Fine. But, my share (and the actual log file) was given 'Full Control' to the NETWORK SERVICE user (this share resides on a different server than the one that my app is running).
If NETWORK SERVICE has rights to this folder, why do I get access denied? Or is there a way to have the thread I kick off have the same user as the process?
You can also get the new thread to impersonate the user that issued the request. For example, if you are starting your request in the Page.Load event it might look like this.
public partial class MyPage : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
Thread myThread = new Thread(new ParameterizedThreadStart(ThreadMethod));
myThread.Start(HttpContext.Current.User);
}
private void ThreadMethod(object state)
{
WindowsPrincipal principal = state as WindowsPrincipal;
WindowsImpersonationContext impersonationContext = null;
try
{
if (principal != null)
{
Thread.CurrentPrincipal = principal;
impersonationContext = WindowsIdentity.Impersonate(((WindowsIdentity)principal.Identity).Token);
}
// Do your user specific stuff here...
}
finally
{
if (impersonationContext != null)
{
impersonationContext.Undo();
}
}
}
}
You will notice I passed the Principal from the ASP.NET thread through to the new thread, I obviously assume that you are using Windows Integrated Authentication so I did not do much in the way of error checking, this is just a quick sample.
Note: For VWG you would get the user from the VWG context and run the impersonation code in the appropriate function, this example is ASP.NET jsut because the environment is the same, just I do not know the VWG objects off the top of my head.
NT AUTHORITY\NETWORK SERVICE is a local computer account. To the remote server it looks like your IIS computer's Active Directory account (COMPUTERNAME$). You need to grant access to the log directory to the AD Computer Account of your IIS box.
HTH