We Keep Coding

How to send log to Cloud Watch in C# application

I want to be able to send application logs to Cloud Watch Log. and I got to know that there is a Cloud Watch Agent service that runs in the background and reads logs from log file and send only the delta (extra logs) to Cloud Watch Log. All this makes sense to me. Then I got to know about NLog a C# logging framework, and wrote below POC to send logs.
static void Main(string[] args)
{
ConfigureNLog();
var logger = NLog.LogManager.GetCurrentClassLogger();
logger.Info("Hello World");
logger.Log(LogLevel.Info, "Sample informational message");
}
static void ConfigureNLog()
{
var accessKey = ConfigurationManager.AppSettings.Get("AWSAccessKey");
var secretKey = ConfigurationManager.AppSettings.Get("AWSSecretKey");
var config = new LoggingConfiguration();
var awsTarget = new AWSTarget()
{
LogGroup = "NLog.ProgrammaticConfigurationExample",
Region = "us-east-1",
Credentials = new BasicAWSCredentials(accessKey, secretKey)
};
config.AddTarget("aws", awsTarget);
config.LoggingRules.Add(new LoggingRule("*", LogLevel.Debug, awsTarget));
LogManager.Configuration = config;
}
Now when I run above code, I am able to send log to Cloud Watch. But I am confused now, where is the significance of Cloud Watch Agent?
Since I am directly sending log data, does that mean I don't need Cloud Watch Agent in my scenario?
In case I want to use Cloud Watch Agent then I need to use FILE as a target for logs by NLog and then tell Cloud Watch Agent to send that log file to Cloud Watch Log??
Is my understanding correct? Please help me in understanding the flow.
Is below flow correct?
NLog write log to File -> Cloud Agent read log from there -> Send log
to Cloud Watch
Question: How to use Cloud Watch Agent in above POC to send data via NLog?

Cloud Watch Agent runs on your server and can watch logs files that are produced. These log files can be anything, IIS Logs, Time Logs, Event Log, Etc. When the log file is updated, CWA will grab the updates and send to Cloud Watch. This is the generic behavior of the CWA and is great for Event Logs and OS logging.
By modifying the AWS.EC2.Windows.CloudWatch.json CWA json file, you can configure it to watch log files for certain formats and send changes to CW outside the standard/example ones it does by default. You can update the json to your NLog entry layout format and have it watch for that specific format in the file. CW Does have a delay sending.
Now you have Nlog which writes log files. You can have NLog send the log entries to a file and have the Cloud Watch Agent watch that file, pick up the change and send it or you can have NLog send the entries directly to CW. Since you are writing directly to CW through a NLog target, you don't need the Cloud Agent for your NLog files. I suggest keeping CWA for other log files like IIS or event logs.
I guess it is a matter a preference on how you do it. I think NLog Targets with layouts is easier than dealing with the CloudWatch json file to try and match the log format. I only use CWA to send log files I have no control over and use a NLog Target to send my NLog entries.
I can post an example CWA json snippet for a 3rd party log file I monitor with CWA if you need an example.

When an application just have to write to a file, then it lives a very simple life with few problems.
When an application suddenly have to handle network traffic (with timeouts, disconnects, retries, connectivity, latency, etc.) then it suddenly will have issues with things queuing up, taking memory, using sockets, causing garbage collections, stalls, etc. (And loosing all pending logevents on crash)
Depending on the lifetime of your application and the critically of your applications, then it can be useful to give it a simple life. And let a friend like Cloud Watch Agent worry about the network-stuff.
See also https://github.com/NLog/NLog.Extensions.Logging/wiki/NLog-cloud-logging-with-Azure-function-or-AWS-lambda

Event logger in Windows 10 Universal Apps

I am trying to create an event log for a Windows Universal Application.
Earlier we had System.Diagnostics EventLog to log events, but I could not find anything similar on the Windows 10 Universal Apps platform.
Is it possible to create logs for Windows 10 and can these logs be written to a file for accessing it later?
I searched a lot, but could not find anything.

FileLoggingSession
Since Windows 8.1 there are FileLoggingSession and LoggingChannel classes in the Windows.Foundation.Diagnostics namespace, which can perform logging to files when configured to do so. You can read more in the official documentation.
Initialization, usage and retrieving the log file can be done like in the following snippet, of course you need to create interfaces, singletons etc. to make it usable:
// Initialization
FileLoggingSession fileLoggingSession = new FileLoggingSession("session");
var loggingChannel = new LoggingChannel("channel");
fileLoggingSession.AddLoggingChannel(loggingChannel);
// Log messages
loggingChannel.LogMessage("error message", LoggingLevel.Error);
// When file is needed
var file = await fileLoggingSession.CloseAndSaveToFileAsync();
// Do anything with file
LoggingSession
Just as FileLoggingSession writes logs to a file but the main difference is that FileLoggingSession writes logs immediately to the file, and LoggingSession does not, and you need to manually request writing the logs to a file with the SaveToFileAsync method. From the documentation:
The FileLoggingSession class sends logged messages to disk files as they are logged. The FileLoggingSession class uses sequential logging, which means that all messages are sent to a disk file, and a sequential history of messages is retained. This is distinct from the LoggingSession class, which sends logged messages to disk on-demand, and this happens when there's a problem and the immediate history of in-memory messages is needed for analysis.
MetroLog
You have another alternatives if you do not wan't to use FileLoggingSession or LoggingSession classes. One good solution is MetroLog which has a FileStreamingTarget target that makes it very simple to log in a Windows/Phone app.
You create the logger when you need it, for example in a page:
public sealed partial class LogSamplePage : Win8Sample.Common.LayoutAwarePage
{
private ILogger Log = LogManagerFactory.DefaultLogManager.GetLogger<LogSamplePage>();
}
Then you can use it in the page like this:
// flat strings...
if (this.Log.IsInfoEnabled)
this.Log.Info("I've been navigated to.");
// formatting...
if (this.Log.IsDebugEnabled)
this.Log.Debug("I can also format {0}.", "strings");
// errors...
try
{
this.DoMagic();
}
catch(Exception ex)
{
if (this.Log.IsWarnEnabled)
this.Log.Warn("You can also pass in exceptions.", ex);
}
MetroEventSource
The second solution is this logging sample on MSDN sample gallery by Can Bilgin where you have the MetroEventSource class. You can log messages for example an error like this:
MetroEventSource.Log.Error("Here is the error message");
If you use this logger don't forget to initialize it on application run, as described in the sample project.

Append to an online text log file from my offline C# app

I'm looking for customer feedback regarding one or more of my apps (optional, and they would be notified), so I'd like to send some text to be appended to an online log file from my offline C# Winforms app.
If it makes a difference, the server is Linux based, and I don't mind if the public can access it too, so no need to worry about encryption or anything (no personal details or anything like that would be stored in them).
What would be the C# code required to do such a thing? (Pretend the website is: http://www.website.com/logfile.txt). Would I have to read the file wholesale, and write it back wholesale, or is there a more efficient 'append' operation I could use?
EDIT: Looks to be harder than I imagined. If I have to make a simple PHP script to help with this task, so be it, though code for that would be appreciated as well if that's the case.

You have a couple options.
First, you could check out a service like loggly which is an online log file. You would have a personal API key to post data to from your application.
If you don't want to do that, you could write your own API that has a simple Post with a string parameter. It would then be responsible for opening the file, adding the text and saving it. The Winform could just fire and forget, knowing that the API can handle it.
In C#, you could use either RestSharp or the HttpClient to send the data to the API.
I think the problem you're going to run into, is having the Winform app save the file. If it was a local file, you can just append to a file (without reading the whole thin into memory.) But on a remote machine, you'd first have to download the entire file. The next problem would be making sure that the Winform app can save the file. Without something like an API call, you could run into a lot of issues.

This may not be robust enough for your needs, but this my solution.
The easiest way I can think of doing this is to have your C# application send the text to a web script. Since you said you didn't care if the data was encrypted I thought why not just pass the text as a get parameter to a PHP script.
This example is very simplistic; you may want to add other checks to meet your needs:
The C# code would look like:
string loggerUrl = "http://www.YourDomainExample.com/Logger.php?text=";
string textToLog = WebUtility.UrlEncode("This text came from my C# desktop application");
HttpWebRequest myWebRequest = (HttpWebRequest)HttpWebRequest.Create(loggerUrl + textToLog);
HttpWebResponse myWebResponse = (HttpWebResponse)myWebRequest.GetResponse();
myWebResponse.Close();
The PHP script residing on your web server would look like:
<?php
$text = htmlspecialchars($_GET["text"]);
$log = "log.txt";
$fh = fopen($log, 'a') or die("can't open file"); // Open log in append mode
$textToWrite = "$text\n"; //Write each comment on a line
fwrite($fh, $textToWrite);
fclose($fh);
?>
By doing it this way, basically anything that can call the url can append text to your log. So your logger could be part of a desktop application, run on a mobile phone or a web application etc.
To test that your PHP script is working correctly, you can use your Web Browser as a client and just go to http://www.YourDomainExample.com/Logger.php?text=Test from webbrowser and check for log.txt on your web server

If you want to log into a remote destination, I see two solutions. Both are using log4net:
Solution 1:
You can set up log4net to log into a database. You can see here for the configuration.
Solution 2:
You can derive your logging class from AppenderSkeleton and configure the behaviour to log into anything you want.
internal class MyAppender : AppenderSkeleton
{
/// <summary>
/// Subclasses of <see cref="T:log4net.Appender.AppenderSkeleton"/> should implement this method
/// to perform actual logging.
/// </summary>
/// <param name="loggingEvent">The event to append.</param>
protected override void Append(LoggingEvent loggingEvent)
{
/* Here you can do whatever you want with your loggingEvent */
}
}

C# Windows Service Textwriter

I've created a standard windows service that uses the LocalSystem account. For the log files I use textwriter to write to a specified file within C:\Users\useraccount directory. The problem is, when running as a service under LocalSystem, it doesn't want to create or write to the file at all.
string dir = #"C:\Users\useraccount\log.txt";
StreamWriter sw = File.AppendText(dir);
As you can see, the directory is hardcoded in so there isn't any base directory conflicts seeing as LocalSystem would start in System32 or something of that nature. The permissions on the folder lets the System account access it fully (windows 7) so why am I not able to create/write to that file?
Thanks for any input!
Edit:
Apparently the logging program thread is running as LocalSystem as well, when I really need it to be running as a standard user. So how do I execute a threaded process from the service to run under the local user account instead of under LocalSystem.
I use Thread.new(process) where process is an additional program. The process program needs to receive input before it writes anything, and it isn't receiving input because it's on the wrong account. How would I fix this?

Should do the same trick as using(...) but did you try:
sw.Close();
Without it there's just an empty file but it throws no exceptions..

Try this code and check what exception reports:
try
{
using (StreamWriter sw = File.AppendText(dir))
{
sw.WriteLine("test");
}
}
catch (Exception ex)
{
Debug.Print(ex.Message);
}
Just a note: it's a good idea to use using(...) with all classes implementing IDisposable interface, so you can be sure they are freed when exiting block!

First of all I would use Log4Net instead of rolling your own logging mechanism, unless there is a good reason not to. This way you can create the log file you want but also (via a config or code) add log appenders such as windows event log. Then I would make sure I log any and all exceptions. You say no exception is being thrown, I expect there is an exception being thrown on the logging thread which is not 'bubbling' up, so catch it and log it (Log.Error(ex)). You can then see these exceptions in your event log if you have configured an event log appender.
Lastly, I'm not sure how you would assume the security context of the logged on user. Sounds like that would be a big securtity hole, unless you write something that prompts the user to enter their credentials or grant the process the right to impersonate/assume the user security context.. Perhaps that is the answer, the service responds to a user logon by prompting the user to grant this right. I'll have a look around for a better answer to this point.

Write a log file

I have a recent problem . I can upload file in my intetpub/wwwrooot/folder
But I can't write a log file in this same folder ...
I have all the permissions for the network service. Everything is on my server.
DirectoryInfo di = new DirectoryInfo(~/);
// Get a reference to each file in that directory.
FileInfo[] fiArr = di.GetFiles();
string strLogText = di;
// Create a writer and open the file:
StreamWriter log;
if (!System.IO.File.Exists("C:\\inetpub\\wwwroot\\logfile.txt"))
{
log = new StreamWriter("C:\\inetpub\\wwwroot\\logfile.txt");
}
else
{
log = System.IO.File.AppendText("C:\\inetpub\\wwwroot\\logfile.txt");
}
// Write to the file:
log.WriteLine(DateTime.Now);
log.WriteLine(strLogText);
log.WriteLine();
// Close the stream:
log.Close();
The error is the access is denied !
It works locally , but on my server it doesnt. On the folder Inetpub , I just need to allow writting for Network service ? That is strange because I can upload file and writting is already enable

Emged in case of exceptions your code does not close the streams on the log file and this is surely not good.
You should use a using statement around the streams so in any case streams are closed and disposed also in case of exceptions.
As Chris has suggested I would absolutely opt for a logging Framework and I would also avoid writing in that wwwroot folder.
ELMAH or NLog or Log4Net are good and easy alternatives far better than any custom logging lie you are doing right now and the big advantage of these technologies/libraries is that you can change the behaviour at runtime simply by editing the configuration file, no need to rebuild or redeploy anything...
my favourite is actually Log4Net, check these ones for a simple example on how to use it:
http://logging.apache.org/log4net/release/manual/configuration.html
Log4Net in App object?

Depending on the version of your server (windows 2008 and above), that directory has additional protection against writes.
I'd highly recommend you look into ELMAH to do your logging. It gives you a number of options including in memory or database backed and collects a LOT of additional data you might want.
Further, opening up various physical directory locations for write access is a HUGE security no-no.

On the server, is the web app running under an Application Pool that has alternate credentials, other than the normal network service account? If you haven't done so already, try turning on Auditing to see what user is trying to access the file.