I wanted to add logging to an existing azure cloud service using NLog and Azure Diagnostics, i went through all the steps described here: http://msdn.microsoft.com/en-us/library/azure/dn482131.aspx
But, unfortunately when testing the cloud service the WADLogsTable table storage does not created, and no information gets stored here. Not on local development storage, nor on azure.
diagnostics.wadcfg:
<?xml version="1.0" encoding="utf-8"?>
<DiagnosticMonitorConfiguration configurationChangePollInterval="PT1M" overallQuotaInMB="4096" xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">
<DiagnosticInfrastructureLogs />
<Directories>
<IISLogs container="wad-iis-logfiles" directoryQuotaInMB="1024" />
<CrashDumps container="wad-crash-dumps" />
</Directories>
<Logs bufferQuotaInMB="1024" scheduledTransferPeriod="PT1M" scheduledTransferLogLevelFilter="Verbose" />
<PerformanceCounters bufferQuotaInMB="512">
<PerformanceCounterConfiguration counterSpecifier="\Memory\Available MBytes" sampleRate="PT3M" />
</PerformanceCounters>
<WindowsEventLog bufferQuotaInMB="1024" scheduledTransferPeriod="PT1M" scheduledTransferLogLevelFilter="Verbose">
<DataSource name="Application!*" />
</WindowsEventLog>
</DiagnosticMonitorConfiguration>
ServiceConfiguration.Local.csfg (the same at Cloud, just with the correct table storage account)
<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="AAService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="4" osVersion="*" schemaVersion="2014-01.2.3">
<Role name="Web.Services.AAService">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=true" />
</ConfigurationSettings>
</Role>
</ServiceConfiguration>
ServiceDefinition.csdef
<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="AAService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2014-01.2.3">
<WebRole name="Web.Services.AAService" vmsize="ExtraSmall">
<Sites>
<Site name="Web">
<Bindings>
<Binding name="Endpoint1" endpointName="Endpoint1" />
</Bindings>
</Site>
</Sites>
<Endpoints>
<InputEndpoint name="Endpoint1" protocol="http" port="80" />
</Endpoints>
<Imports>
<Import moduleName="Diagnostics" />
</Imports>
<LocalResources>
<LocalStorage name="Web.Services.AAService.svclog" sizeInMB="1000" cleanOnRoleRecycle="false" />
</LocalResources>
</WebRole>
</ServiceDefinition>
WebRole entrypoint
public override bool OnStart()
{
// To enable the AzureLocalStorageTraceListner, uncomment relevent section in the web.config
DiagnosticMonitorConfiguration diagnosticConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();
diagnosticConfig.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
diagnosticConfig.Directories.DataSources.Add(AzureLocalStorageTraceListener.GetLogDirectory());
DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", diagnosticConfig);
// For information on handling configuration changes
// see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
return base.OnStart();
}
cloud service web.config also has:
<system.diagnostics>
<trace>
<listeners>
<add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=2.3.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="AzureDiagnostics">
<filter type="" />
</add>
</listeners>
</trace>
</system.diagnostics>
Does anyone have any ide why is this doesnt make any trace logs? I called Trace.WriteLine() many places where the application goes through for sure, but still no WADLogsTable.
Thanks for the help
Get rid of all the Diagnostic* code in OnStart. The <Import moduleName="Diagnostics" /> in your csdef will automatically start the diagnostics agent, and the diagnostics agent will read the settings from the .wadcfg. Your code in OnStart is unnecessary and is overriding the wadcfg settings due to the order of precedence of loading WAD configuration.
Also note that if you are trying to update an existing cloud service then you will need to first delete the XML file in blob storage. If a file already exists in blob storage then the diagnostics agent will ignore the .wadcfg settings. The file you are looking for is in wad-control-container and will be named with the deployment ID and role name.
Related
I am developing APIs with Service Fabric into a big solution. After I created the services I needed (an Actor, a stateful and a stateless with .NET Framework) and I made a walking skeleton of the APIs.
I started to test and service fabric always threw the error "Service does not exist" (I'm 100% sure that the uri was right) when was the time to call the Actor (same error with both ActorProxyFactory and ActorProxy). I wathced on Cluster Explorer and under my SF App there was only the other two service. But the ActorServiceType was registered.
So I decide to create two simple SF app with an actor and I got the same error and also I don't see them on Cluster explorer.
In no projects I touched the ServiceManifest, ApplicationManifest or whatelse.
Here the versions of the tools I use:
Windows 11 Enterprise
Visual Studio Enterprise v16.11.9 with .Net Framework 4.7.1
Service Fabric Runtime 8.1.321.9590
Serive Fabric SDK 5.1.321.9590
Nuget Microsoft.SerivceFabric 8.1.321
Nuget Microsoft.ServiceFabric.Actors 5.1.321 (For the nuget packages I tried everything)
Microsoft.VisualStudio.Azure.Fabric.MSBuild 1.7.6 (also this I tried
every version)
My auto-generated setting.xml:
<?xml version="1.0" encoding="utf-8"?>
<Settings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Section Name="MyActorServiceReplicatorConfig">
<Parameter Name="ReplicatorEndpoint" Value="MyActorServiceReplicatorEndpoint" />
<Parameter Name="BatchAcknowledgementInterval" Value="0.005" />
</Section>
<Section Name="MyActorServiceReplicatorSecurityConfig">
<Parameter Name="CredentialType" Value="None" />
</Section>
<!-- The content will be generated during build -->
</Settings>
My auto-generated service-manifest.xml
<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="MyActorPkg" Version="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<ServiceTypes>
<StatefulServiceType ServiceTypeName="MyActorServiceType" HasPersistedState="true">
<Extensions>
<Extension Name="__GeneratedServiceType__" GeneratedId="5f4d2e71-68d5-43f4-b8a3-60990017b54d|Persisted">
<GeneratedNames xmlns="http://schemas.microsoft.com/2015/03/fabact-no-schema">
<DefaultService Name="MyActorService" />
<ReplicatorEndpoint Name="MyActorServiceReplicatorEndpoint" />
<ReplicatorConfigSection Name="MyActorServiceReplicatorConfig" />
<ReplicatorSecurityConfigSection Name="MyActorServiceReplicatorSecurityConfig" />
<StoreConfigSection Name="MyActorServiceLocalStoreConfig" />
<ServiceEndpointV2_1 Name="MyActorServiceEndpointV2_1" />
</GeneratedNames>
</Extension>
</Extensions>
</StatefulServiceType>
</ServiceTypes>
<CodePackage Name="Code" Version="1.0.0">
<EntryPoint>
<ExeHost>
<Program>MyActor.exe</Program>
</ExeHost>
</EntryPoint>
</CodePackage>
<ConfigPackage Name="Config" Version="1.0.0" />
<Resources>
<Endpoints>
<Endpoint Name="MyActorServiceEndpointV2_1" />
<Endpoint Name="MyActorServiceReplicatorEndpoint" />
</Endpoints>
</Resources>
<!-- The content will be generated during build -->
</ServiceManifest>
my auto-generated application-manifest:
<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="MyActorPkg" Version="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<ServiceTypes>
<StatefulServiceType ServiceTypeName="MyActorServiceType" HasPersistedState="true">
<Extensions>
<Extension Name="__GeneratedServiceType__" GeneratedId="5f4d2e71-68d5-43f4-b8a3-60990017b54d|Persisted">
<GeneratedNames xmlns="http://schemas.microsoft.com/2015/03/fabact-no-schema">
<DefaultService Name="MyActorService" />
<ReplicatorEndpoint Name="MyActorServiceReplicatorEndpoint" />
<ReplicatorConfigSection Name="MyActorServiceReplicatorConfig" />
<ReplicatorSecurityConfigSection Name="MyActorServiceReplicatorSecurityConfig" />
<StoreConfigSection Name="MyActorServiceLocalStoreConfig" />
<ServiceEndpointV2_1 Name="MyActorServiceEndpointV2_1" />
</GeneratedNames>
</Extension>
</Extensions>
</StatefulServiceType>
</ServiceTypes>
<CodePackage Name="Code" Version="1.0.0">
<EntryPoint>
<ExeHost>
<Program>MyActor.exe</Program>
</ExeHost>
</EntryPoint>
</CodePackage>
<ConfigPackage Name="Config" Version="1.0.0" />
<Resources>
<Endpoints>
<Endpoint Name="MyActorServiceEndpointV2_1" />
<Endpoint Name="MyActorServiceReplicatorEndpoint" />
</Endpoints>
</Resources>
<!-- The content will be generated during build -->
</ServiceManifest>
my local1node.xml file:
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/TEST2.MYACTOR.SF" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters />
</Application>
screens cluster explorer:
screen cluster explorer
screen cluster explorer service type
The solution of this problem found on this github issue with a step-by-step guide.
In few words if you have more than installation of Visual Studio (2019 and 2022) a service fabric package cache go in conflict.
I have a test-assembly (MyTestProject) where I want to write some logging using log4net. Thus I created a config-file with the same name as the assembly where I set up the logging as suggested here:
<?xml version="1.0" encoding="utf-8" ?>
<!-- .NET application configuration file
This file must have the exact same name as your application with
.config appended to it. For example if your application is testApp.exe
then the config file must be testApp.exe.config it must also be in the
same directory as the application. -->
<configuration>
<configSections>
<!-- Register the section handler for the log4net section -->
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net" />
<sectionGroup name="NUnit">
<!-- For .NET 2.0 Beta 2 replace the lines with the following -->
<section name="TestCaseBuilder" type="System.Configuration.NameValueSectionHandler, System, Version=2.0.50215.44, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
<section name="TestRunner" type="System.Configuration.NameValueSectionHandler, System, Version=2.0.50215.44, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
</sectionGroup>
</configSections>
<NUnit>
<TestCaseBuilder>
<add key="OldStyleTestCases" value="false" />
</TestCaseBuilder>
<TestRunner>
<!-- Valid values are STA,MTA. Others ignored. -->
<add key="ApartmentState" value="STA" />
<!-- See ThreadPriority enum for other valid values -->
<add key="ThreadPriority" value="Normal" />
</TestRunner>
</NUnit>
<appSettings>
<add key="ApartmentState" value="STA" />
<add key="apartment" value="STA" />
</appSettings>
<!-- This section contains the log4net configuration settings -->
<log4net debug="true">
<!-- Define some output appenders -->
<appender name="LogFileAppender" type="log4net.Appender.FileAppender,log4net" >
<param name="File" value="D:/data.log" />
<param name="AppendToFile" value="false" />
<layout type="log4net.Layout.PatternLayout,log4net">
<param name="ConversionPattern" value="%d [%t] %-5p %c [%x] <%X{auth}> - %m%n" />
</layout>
</appender>
<!-- Setup the root category, add the appenders and set the default priority -->
<root>
<level value="DEBUG" />
<appender-ref ref="LogFileAppender" />
</root>
</log4net>
</configuration>
Within my code I set up the logging as follows:
[TestFixture]
public class MyTest
{
private readonly ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
[TestFixtureSetUp]
public void Init()
{
log.Info("Something to log");
...
}
}
However when I run my tests no such file D:/data.log is created. Even more suspicious when I debug the code and add a watch to the appenders from log I get an empty collection.
I post this together as it stuck me for hours.
I looked into the directory where the file should stay, there was one (apperently from an earlier test-run as its timestamp was not recent). Trying to delete the file got me a message that it is currently in use from nunit-agent-x86.exe. So I killed the process in TaskManager and re-run my tests. Now everything works correct and I get my logging into the file.
This Could have been solved if you give value for key, AppendToFile as true..
As logger appends logging to the existing file and wont try to delete it.
I have created a wpp.targets file which does all the deployment in azure. I want to change the appsetting property of webconfig during deployment to azure. I found a sample in http://sedodream.com/PermaLink,guid,25ddd39e-59de-4e35-becc-de19dcc5e4ea.aspx , which uses the pubxml and parameters.xml. I want to use wpp.targets instead of pubxml.
<appSettings>
<!-- TODO: set this in the azure config -->
<add key="customer" value="xyz" />
</appSettings>
Need to update the customer value to "client" during deployment and web.config file should reflect the changes in the deployed folder like below
<appSettings>
<add key="customer" value="client" />
</appSettings>
ANSWER
I have achieved using the ProjectParametersXMLFile in msDeploy
msbuild Api.csproj /p:ProjectParametersXMLFile="c:\parameter.xml"/p:PublishSettingsFile=%publishFileLocation%
You can use parameter xml file
for example
<parameters >
<parameter name="customer" description="web Job queue name" defaultValue="sitetest1">
<parameterEntry kind="XmlFile"
scope="\\web.config$"
match="/configuration/appSettings/add[#key='customer']/#value" />
</parameter>
<parameter name="customer" description="web Job queue name" defaultValue="sitetest">
<parameterEntry kind="XmlFile"
scope="\\app.config$"
match="/configuration/appSettings/add[#key='customer']/#value" />
</parameter>
</parameters>
this file will change the appsettings when you deploy in azure based on default value you provide
I have an Azure cloud service app that is comprised of a worker role (1 instance only) and a web role (multiple instances). The web role is also where the Microsoft.WindowsAzure.Caching's (version 2.2) in-role co-located cache cluster resides. I have Azure diagnostics set up and in my WadLogsTable I see this pair of warnings every few minutes in production:
WARNING: DeadServerCallback Called, Server URI: [net.tcp://100.74.158.31:20005], Underlying exception - ; TraceSource 'w3wp.exe' event
WARNING: DeadServerCallback: Matches My Server, Cleaning Pending Requests; TraceSource 'w3wp.exe' event
The net.tcp://100.74.158.31:20005 corresponds to one of the web roles. The warnings sporatically alternate between the web roles (i.e. sometimes it says net.tcp://100.74.158.51:20005).
Notice that there is nothing after "Underlying exception - ". There are also no related exceptions in the WadWindowsEventLogsTable (I know that Azure exception logging is working because I will occasionally see other exceptions in that table). I believe I've also turned on all the Caching diagnostics that I could.
So my issue is that I am seeing all these warnings but I don't know why and I don't know how to remedy the situation. It doesn't appear that these warnings are causing the app to crash, but still they are worrisome. Any help would be appreciated.
Here is my ServiceConfiguration.cscfg:
<?xml version="1.0"?>
<ServiceConfiguration serviceName="CloudService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="3" osVersion="*" schemaVersion="2013-10.2.2">
<Role name="WebRole">
<Instances count="2" />
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=XXXXX;AccountKey=XXXXX" />
<Setting name="Microsoft.WindowsAzure.Plugins.Caching.NamedCaches" value="{"caches":[{"name":"default","policy":{"eviction":{"type":0},"expiration":{"defaultTTL":10,"isExpirable":true,"type":2},"serverNotification":{"isEnabled":false}},"secondaries":0}]}" />
<Setting name="Microsoft.WindowsAzure.Plugins.Caching.DiagnosticLevel" value="3" />
<Setting name="Microsoft.WindowsAzure.Plugins.Caching.CacheSizePercentage" value="30" />
<Setting name="Microsoft.WindowsAzure.Plugins.Caching.ConfigStoreConnectionString" value="DefaultEndpointsProtocol=https;AccountName=XXXXX;AccountKey=XXXXX" />
<Setting name="Microsoft.WindowsAzure.Plugins.Caching.ClientDiagnosticLevel" value="3" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="Me" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="XXXXX" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2014-08-05T23:59:59.0000000-07:00" />
</ConfigurationSettings>
<Certificates>
<Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="XXXXX" thumbprintAlgorithm="sha1" />
<Certificate name="www.myapp.com" thumbprint="XXXXX" thumbprintAlgorithm="sha1" />
</Certificates>
</Role>
<Role name="WorkerRole">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Caching.ClientDiagnosticLevel" value="3" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="Me" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="XXXXX" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2014-08-05T23:59:59.0000000-07:00" />
<Setting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" />
<!-- For Windows Azure ServiceManagement. -->
<Setting name="SubscriptionId" value="XXXXX" />
<Setting name="ServiceManagementCertificateThumbprint" value="value="XXXXX" " />
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=XXXXX;AccountKey=XXXXX" />
</ConfigurationSettings>
<Certificates>
<Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="XXXXX" thumbprintAlgorithm="sha1" />
<!-- For Windows Azure ServiceManagement. -->
<Certificate name="ServiceManagementCertificate" thumbprint="thumbprint="XXXXX" " thumbprintAlgorithm="sha1" />
</Certificates>
</Role>
</ServiceConfiguration>
Here are pertinent parts of the Web.config of my web role:
<dataCacheClients>
<dataCacheClient name="default" connectionPool="true" useLegacyProtocol="false" isCompressionEnabled="false" maxConnectionsToServer="2">
<autoDiscover isEnabled="true" identifier="WebRole" />
</dataCacheClient>
</dataCacheClients>
<cacheDiagnostics>
<crashDump dumpLevel="Full" dumpStorageQuotaInMB="100" scheduledTransferPeriodInMinutes="5" />
</cacheDiagnostics>
Here are pertinent parts of the App.config of my worker role:
<dataCacheClients>
<dataCacheClient name="default" connectionPool="true" useLegacyProtocol="false" isCompressionEnabled="false" maxConnectionsToServer="2">
<autoDiscover isEnabled="true" identifier="WebRole" />
</dataCacheClient>
</dataCacheClients>
<cacheDiagnostics>
<crashDump dumpLevel="Full" dumpStorageQuotaInMB="100" scheduledTransferPeriodInMinutes="5" />
</cacheDiagnostics>
Additional notes:
I've probably tried every possible combination of attribute settings for both the <dataCacheClient> entries in the web role's
Web.config and worker role's App.config.
The cloud service has been upgraded to use Windows Azure Tools version 2.2.
I've Googled and found a few posts related to these warnings but it seems that most people only experienced this in the compute emulator, not on production like I am. Nevertheless, I've tried most of their suggested solutions and nothing has helped.
As long as you dont see any failures or timeouts or drop in quality of the service, you should be good. The deadserver callback should be just cleaning up old unused connections. To validate this hypothesis :
Do you have any other connection errors happening in verbose logging ? Can you see if the last request to this server was a long time back ? Do you have bursty traffic ?
Can we read certificate section withi ServiceConfiguration.cscfg file using c#? There is method inside RoleEnvironment class to read ConfigurationSettings, but not certificate section.
<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="WindowsAzureProject7" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1" osVersion="*">
<Role name="MvcWebRole1" >
<Instances count="1" />
<Certificates>
<Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="625FBBB3B7A25C4B9D1C49D1CB1E3AE196C1A083" thumbprintAlgorithm="sha1" />
</Certificates>
</Role>
</ServiceConfiguration>
Yes, there is no API to read certificate details as far as i know, however what you can do is create a configuration setting and add your certificate specific details and read it directly from the same API. Here is the trick, I used in past:
<ServiceConfiguration serviceName="RW" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
<Role name="RR">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="AppFolder" value="RailsApp" />
<Setting name="CertificateThumb" value="*************" />
</ConfigurationSettings>
<Certificates>
<Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="*****************************" thumbprintAlgorithm="sha1" />
</Certificates>
</Role>
</ServiceConfiguration>
Now, in my role specific code, I can call RoleEnvironment.GetConfigurationSettingValue to get the certificate thumb as below:
string certThumb = RoleEnvironment.GetConfigurationSettingValue("CertificateThumb");