We Keep Coding

Two identical Azure deployments; one always throws 404 - how to find out what the difference is?

I am deploying a C# ASP.NET Core web service to Azure using Pulumi. I can deploy it in 3 ways:
Run it locally from Visual Studio, i.e., not using Azure at all.
Deploy it to Azure from my local developer computer.
Deploy it to Azure from Jenkins (whicn runs on a different computer).
I have this problem:
When I run it locally, I can call the service fine, e.g. from Postman or from a C# application. The web service returns what I expect.
When I deploy it to Azure from my local machine, I can also call it fine. The web service returns what I expect.
When I deploy it to Azure from Jenkins and then try to call the webservice, it returns "NotFound" to all calls no matter what I do. (This presumably means HTTP 404.)
The deployments in 2 and 3 should be exactly the same. My question is: How can I find out what the difference is between these two deployments in Azure?
The Jenkins-deployed webservice exhibits the following curious behaviour:
It does not log any exceptions (even when I wait several minutes for them to show up).
If I go to my resource group -> Application Insights -> Logs and search for "requests", it does list requests. Curiously, it says that it returned HTTP 200 to all the requests, even though what I get when calling them is 404.
The above is true even for web service calls that should never return 200 (they should return 201).
The above is true even for web service calls to methods that shouldn't even exist (i.e., when I deliberately corrupt the method URI before calling the service).
During deployment I authenticate with Azure using a service principal. My Jenkinsfile looks like this:
withVaultSecrets([
"path/to/secret/in/vault": [
"sp_name", "application_id", "object_id", "sp_secret"
]
]){
script {
env.PULUMI_CONFIG_PASSPHRASE = 'jenkinspassphrase'
env.ARM_CLIENT_ID = "${application_id}"
env.ARM_CLIENT_SECRET = "${sp_secret}"
env.ARM_TENANT_ID = "${azure_dev_tenant_id}"
env.ARM_SUBSCRIPTION_ID = "${azure_dev_subscription_id}"
env.AZURE_CLIENT_ID = "${application_id}"
env.AZURE_CLIENT_SECRET = "${sp_secret}"
env.AZURE_TENANT_ID = "${azure_dev_tenant_id}"
}//script
dir("./src/deploy/KmsStack"){
powershell "pulumi login --local";
powershell "pulumi stack init jenkinsfunctionaltest --secrets-provider=passphrase"
powershell "pulumi up --yes"
}//dir
}//withVaultSecrets
The script which I use to deploy locally looks like this, with the same service principal credentials:
cd $PSScriptRoot
cd webapi
dotnet publish /p:DisableGitVersionTask=true
cd ../deploy/KmsStack
$env:PULUMI_CONFIG_PASSPHRASE = 'jenkinspassphrase'
$env:ARM_CLIENT_ID = ...
$env:ARM_CLIENT_SECRET = ...
$env:ARM_TENANT_ID = ...
$env:ARM_SUBSCRIPTION_ID = ...
$env:AZURE_CLIENT_ID = ...
$env:AZURE_CLIENT_SECRET = ...
$env:AZURE_TENANT_ID = ...
pulumi logout
pulumi login --local
pulumi stack rm jenkinsfunctionaltest -y
pulumi stack init jenkinsfunctionaltest --secrets-provider=passphrase
pulumi stack select jenkinsfunctionaltest
pulumi up --yes
How can I find out why these two deployed services behave differently? The Azure portal GUI is rich and has lots of sections. Can you recommend me where to look? Might there be some security settings that differ? How can I find them?
Thanks in advance!

We found out what was wrong. It was not an Azure issue. The problem was that we were deploying a bad ZIP file. The ZIP file was missing web.config, which meant that the web application could not start up.
We were zipping our published web application by having this in the CSPROJ file:
<Target Name="ZipOutputPath" AfterTargets="Publish">
<ZipDirectory SourceDirectory="$(OutputPath)\publish" DestinationFile="$(MSBuildProjectDirectory)\kmswebapp.zip" Overwrite="true" />
</Target>
This turned out not to work because the compiler does things in a different order than we expected. At the time when it generated the ZIP file, web.config was not generated yet, so web.config never got packed into the ZIP file. Hence Azure could not start the application.
When we deployed from our local machines, it worked because we didn't clean the publish directory before each run, so there would be a web.config left over from the previous run, and this old (but unchanged) web.config would get packed into the ZIP file and deployed to Azure, so Azure would know how to start the application.
We solved it by removing the above from our CSPROJ file and doing (roughly) this in our Jenkinsfile:
powershell "dotnet publish ./src/webapi/WebAPI.csproj"
powershell "if (!(Test-Path('${publishDirectoryPath}/web.config'))){throw 'We need web.config to exist in the publish directory'}"
powershell "Compress-Archive -Path '${publishDirectoryPath}/*' -DestinationPath './src/webapi/kmswebapp.zip' -Force"
This generates a proper ZIP file including web.config, and Azure can now start our application so it can respond properly to requests.

ASP.NET Web Application - on deploy, System.Speech.dll object not getting set to an instance of an object

I have an ASP.NET web application that uses System.Speech to transform text to a WAV file. It works fine locally but when I deploy it to the server, I get the below error message. This is using Windows Server 2012, ASP.NET 4.5, and IIS 8.5:
Object reference not set to an instance of an object.
System.Speech
at System.Speech.Internal.ObjectTokens.RegistryDataKey..ctor(String fullPath, RegistryDataKey copyKey)
at System.Speech.Internal.ObjectTokens.SAPICategories.DefaultDeviceOut()
at System.Speech.Internal.Synthesis.VoiceSynthesis..ctor(WeakReference speechSynthesizer)
at System.Speech.Synthesis.SpeechSynthesizer.get_VoiceSynthesizer()
at QuinnSDS.handlerTransform.<>c__DisplayClass6.<ProcessRequest>b__1()
The code which is generating this error message runs on the server:
if (context.Request.ContentLength > 0)
{
string line = new StreamReader(context.Request.InputStream).ReadToEnd();
// ********* generate wav file voicing the response *****************
// Using Microsoft voices
// initiate new instance of speech synthesizer
Thread t = new Thread(() =>
{
try
{
// The object creation works fine
System.Speech.Synthesis.SpeechSynthesizer synth = new System.Speech.Synthesis.SpeechSynthesizer();
if (synth != null)
{
// The code breaks at synth.GetInstalledVoices() below. It will break any time I try to do anything with the synth object
foreach (System.Speech.Synthesis.InstalledVoice voice in synth.GetInstalledVoices())
{
System.Speech.Synthesis.VoiceInfo info = voice.VoiceInfo;
string voiceName = info.Name;
ws.WriteLine(voiceName);
}
}
}
catch (Exception e)
{
ws.WriteLine(e.Message);
ws.WriteLine(e.Source);
ws.WriteLine(e.StackTrace);
}
//... code continues...
It does not break when the Speech Synthesis object is created; it breaks whenever I try to use that object in any way.
I'm not sure if it's an access issue but I'm pretty new to ASP.NET and IIS and I can't figure out how to give the web app access to the GAC or if that's even what the problem is. I tried changing the property Local Copy for the System.Speech reference to True in Visual Studio, before I deploy the app, but that hasn't worked. I searched online and while the "object reference not set to an instance of an object" seems fairly common, I cannot find any similar issues where it is because of a .NET framework class library...I have run the text-to-speech code locally on the server and it ran fine. I have not run the entire app locally on the server because the web app requires speech input and there is not a microphone on the server.
Any ideas of anything to try would be most welcome!

What user account is the code running under when executed from ASP.NET? If the Speech API is touching the registry like the call stack suggests, it possibly has different permissions than the account you used to run the code manually.
If you can't just make the application pool for your site run with the same account you log into the machine with, I've had some success using Process Monitor to track down this kind of problem before. Basically, execute the code that fails while Process Monitor is running and look for 'ACCESS DENIED' in the 'Result' column (or anything else that looks suspicious). Quickly switching the application pool to use your standard user account will be the fastest way to rule out security or permission related problems though.

Keep C# application running

I'm building a Windows Service that uses FileSystemWatcher, and runs in the background.
I don't want to keep on uninstalling and installing the service every time I want to debug, so would like to do most of my development in a normal program before moving it into a service. But I'm quite new to this, and when I run it, it just runs through the block and exits.
What would be a good way to keep the program running?

http://einaregilsson.com/run-windows-service-as-a-console-program/
I've used this before to debug my service as a Console application based on whether its running in an interactive user environment.
public partial class DemoService : ServiceBase
{
static void Main(string[] args)
{
DemoService service = new DemoService();
if (Environment.UserInteractive)
{
service.OnStart(args);
Console.WriteLine("Press any key to stop program");
Console.Read();
service.OnStop();
}
else
{
ServiceBase.Run(service);
}
}

while (true)
{
// Execute your program's functionality here.
}

I wrote a 7 part series a while ago titled: Building a Windows Service. It covers all the intricacies of building services, making them friendly to debug, and self-installing.
The basic feature set I was looking for was as follows:
Building a service that can also be used from the console
Proper event logging of service startup/shutdown and other activities
Allowing multiple instances by using command-line arguments
Self installation of service and event log
Proper event logging of service exceptions and errors
Controlling of start-up, shutdown and restart options
Handling custom service commands, power, and session events
Customizing service security and access control
The final result was a Visual Studio project template that creates a working service, complete with all of the above, in a single step. It's been a great time saver for me.
see Building a Windows Service – Part 7: Finishing touches for a link to the project template and install instructions.

Here’s documentation from MSDN # http://msdn.microsoft.com/en-us/library/7a50syb3(v=vs.80).aspx?ppud=4 . I have tried it before and it works under .NET Framework 3.x. I could not find my descriptive notes on it, at the moment.
Use the pragma #If DEBUG for debugging purposes like console outputs. Another is using the Debug object.
If you have any trouble with this, say so. I may be able to find my notes or make a Windows Service app myself, just to see if the steps on MSDN still work.

How can I programmatically stop or start a website in IIS (6.0 and 7.0) using MsBuild?

I have Windows Server 2003 (IIS 6.0) and Windows Server 2008 (IIS 7.0) servers, and I use MSBuild for deploying web applications.
I need to do a safe deploy, and do this:
Stop a website in IIS 6 (or an Application in IIS 7), not stop AppPool.
Check if the website is stopped; not running.
If the website is stopped, do another task for deploy.
Start the website IIS 6 (or Application in IIS 7),
How can I achieve this?
Update: Key for me: IIS6WebSite and IIS6AppPool (and for IIS7), do wait for stopped status when try Stop Website or AppPool?
When I execute Stop Action for Website (or Stop Action for AppPool), I need be sure 100% that Website is stopped, and then, and only if Website is Stopped, I can execute other targets.

By adding a reference to Microsoft.Web.Administration (which can be found inX:\Windows\System32\inetsrv, or your systems equivalent) you can achieve nice managed control of the situation with IIS7, as sampled below:
namespace StackOverflow
{
using System;
using System.Linq;
using Microsoft.Web.Administration;
class Program
{
static void Main(string[] args)
{
var server = new ServerManager();
var site = server.Sites.FirstOrDefault(s => s.Name == "Default Web Site");
if (site != null)
{
//stop the site...
site.Stop();
if (site.State == ObjectState.Stopped)
{
//do deployment tasks...
}
else
{
throw new InvalidOperationException("Could not stop website!");
}
//restart the site...
site.Start();
}
else
{
throw new InvalidOperationException("Could not find website!");
}
}
}
}
Obviously tailor this to your own requirements and through your deployment build script execute the resulting application.
Enjoy. :)

Write a script, e.g. PowerShell, which will stop/start IIS web site programmatically relying on command-line argument, e.g. start-stop.ps1 /stop 1
Put it into MsBuild script as a custom step
Check this to find out how to restart IIS AppPool
IIS WMI objects reference

So you have your answer above for IIS7. What you're missing is IIS6. So here you go. This is using a COM interop object as that's all that is available for IIS 6. Also, because it's in vb, you'll have to figure out how to convert it. http://www.codeproject.com/Articles/16686/A-C-alternative-for-the-Visual-Basic-GetObject-fun should get you on the right track. you could also create a vb project just for this code but that's kind of silly.
Dim WebServiceObj As Object
dim IisSiteId as Integer = 0
WebServiceObj = GetObject("IIS://localhost/W3SVC/" & IisSiteId)
WebServiceObj.Stop()
WebServiceObj.Start()

How to create a publisher within a console application

I'm just starting out with NServiceBus and have updated the PubSub sample to work with .NET 4.0 Framework. That's working perfectly ok. This runs one publisher and two subscribers within the "NServiceBus.Host.exe" environment - so it is that which takes responsibility for setting up an instance of the Bus and doing any relevant subscriptions. That's all working fine (as you'd expect) but I'm now trying to move the publisher out of being run within "NServiceBus.Host.exe" into it's own console application (eventually I would like to publish messages from a website so this seems a good small step in that direction).
If I start the 3 projects (my console app, Sub1 and Sub2) it creates 5 msmq on my local machine, but instead of the endpoint.config...subscriptions Q, it creates a generic "nservicebus_subscriptions" Q. If I enable journals, I see that the MyPublisherInputQueue has three (processed) completion messages with an errorcode 0, whilst the subscriber1inputqueue and subscriber2inputqueue each have one. This all seems good, but if I then publish messages, the publisher doesn't appear to throw any errors, but no messages make it as far as the subscribers (they just sit waiting for a message). Also no messages appear in either the new message or journal for any of the MQs.
I'm obviously missing some step(s) - Is the console application not opening itself up for subscriptions? If so, what steps are needed to do that? What steps am I missing that run when you host the publisher in the nservicebus.host.exe?
To create the console application I've done the following:
Within the pubsub solution, created a console application.
Added references in the console application to mymessages, nservicebus.dll and nservicebus.core.dll
From the existing pub code and example on the NServiceBus website, added following code to "Main()"
Code:
IBus Bus = Configure.With()
.Log4Net()
.DefaultBuilder()
.MsmqSubscriptionStorage()
.XmlSerializer()
.MsmqTransport()
.UnicastBus()
.LoadMessageHandlers()
.CreateBus()
.Start();
bool publishIEvent = true;
do
{
Console.ReadLine();
var eventMessage = publishIEvent ? Bus.CreateInstance<IEvent>() : new EventMessage();
eventMessage.EventId = Guid.NewGuid();
eventMessage.Time = DateTime.Now.Second > 30 ? (DateTime?)DateTime.Now : null;
eventMessage.Duration = TimeSpan.FromSeconds(99999D);
Bus.Send(eventMessage);
Console.WriteLine("Published event with Id {0}.", eventMessage.EventId);
} while (true);
Created an app.config for the new console application using the contents of the existing "publisher" app.config.
Added "NServiceBus.Integration" to command line arguments for console application project.

If you look at your logs, it is likely that NServiceBus is telling you that you need to have your publisher be configured to be transactional. The way to do that is to include .IsTransactional(true) after .MsmqTransport().