I have a queue that processes objects in a while loop. They are added asynchronously somewhere.. like this:
myqueue.pushback(String value);
And they are processed like this:
while(true)
{
String path = queue.pop();
if(process(path))
{
Console.WriteLine("Good!");
}
else
{
queue.pushback(path);
}
}
Now, the thing is that I'd like to modify this to support a TTL-like (time to live) flag, so the file path would be added o more than n times.
How could I do this, while keeping the bool process(String path) function signature? I don't want to modify that.
I thought about holding a map, or a list that counts how many times the process function returned false for a path and drop the path from the list at the n-th return of false. I wonder how can this be done more dynamically, and preferably I'd like the TTL to automatically decrement itself at each new addition to the process. I hope I am not talking trash.
Maybe using something like this
class JobData
{
public string path;
public short ttl;
public static implicit operator String(JobData jobData) {jobData.ttl--; return jobData.path;}
}
I like the idea of a JobData class, but there's already an answer demonstrating that, and the fact that you're working with file paths give you another possible advantage. Certain characters are not valid in file paths, and so you could choose one to use as a delimiter. The advantage here is that the queue type remains a string, and so you would not have to modify any of your existing asynchronous code. You can see a list of reserved path characters here:
http://en.wikipedia.org/wiki/Filename#Reserved_characters_and_words
For our purposes, I'll use the percent (%) character. Then you can modify your code as follows, and nothing else needs to change:
const int startingTTL = 100;
const string delimiter = "%";
while(true)
{
String[] path = queue.pop().Split(delimiter.ToCharArray());
int ttl = path.Length > 1?--int.Parse(path[1]):startingTTL;
if(process(path[0]))
{
Console.WriteLine("Good!");
}
else if (ttl > 0)
{
queue.pushback(string.Format("{0}{1}{2}", path[0], delimiter,ttl));
}
else
{
Console.WriteLine("TTL expired for path: {0}" path[0]);
}
}
Again, from a pure architecture standpoint, a class with two properties is a better design... but from a practical standpoint, YAGNI: this option means you can avoid going back and changing other asynchronous code that pushes into the queue. That code still only needs to know about the strings, and will work with this unmodified.
One more thing. I want to point out that this is a fairly tight loop, prone to running away with a cpu core. Additionally, if this is the .Net queue type and your tight loop gets ahead of your asynchronous produces to empty the queue, you'll throw an exception, which would break out of the while(true) block. You can solve both issues with code like this:
while(true)
{
try
{
String[] path = queue.pop().Split(delimiter.ToCharArray());
int ttl = path.Length > 1?--int.Parse(path[1]):startingTTL;
if(process(path[0]))
{
Console.WriteLine("Good!");
}
else if (ttl > 0)
{
queue.pushback(string.Format("{0}{1}{2}", path[0], delimiter,ttl));
}
else
{
Console.WriteLine("TTL expired for path: {0}" path[0]);
}
}
catch(InvalidOperationException ex)
{
//Queue.Dequeue throws InvalidOperation if the queue is empty... sleep for a bit before trying again
Thread.Sleep(100);
}
}
If the constraint is that bool process(String path) cannot be touched/changed then put the functionality into myqueue. You can keep its public signatures of void pushback(string path) and string pop(), but internally you can track your TTL. You can either wrap the string paths in a JobData-like class that gets added to the internal queue, or you can have a secondary Dictionary keyed by path. Perhaps even something as simple as saving the last poped path and if the subsequent push is the same path you can assume it was a rejected/failed item. Also, in your pop method you can even discard a path that has been rejected too many time and internally fetch the next path so the calling code is blissfully unaware of the issue.
You could abstract/encapsulate the functionality of the "job manager". Hide the queue and implementation from the caller so you can do whatever you want without the callers caring. Something like this:
public static class JobManager
{
private static Queue<JobData> _queue;
static JobManager() { Task.Factory.StartNew(() => { StartProcessing(); }); }
public static void AddJob(string value)
{
//TODO: validate
_queue.Enqueue(new JobData(value));
}
private static StartProcessing()
{
while (true)
{
if (_queue.Count > 0)
{
JobData data = _queue.Dequeue();
if (!process(data.Path))
{
data.TTL--;
if (data.TTL > 0)
_queue.Enqueue(data);
}
}
else
{
Thread.Sleep(1000);
}
}
}
private class JobData
{
public string Path { get; set; }
public short TTL { get; set; }
public JobData(string value)
{
this.Path = value;
this.TTL = DEFAULT_TTL;
}
}
}
Then your processing loop can handle the TTL value.
Edit - Added a simple processing loop. This code isn't thread safe, but should hopefully give you an idea.
Related
what is proper way to save all lines from text file to objects. I have .txt file something like this
0001Marcus Aurelius 20021122160 21311
0002William Shakespeare 19940822332 11092
0003Albert Camus 20010715180 01232
From this file I know position of each data that is written in file, and all data are formatted.
Line number is from 0 to 3
Book author is from 4 to 30
Publish date is from 31 to 37
Page num. is from 38 to 43
Book code is from 44 to 49
I made class Data which holds information about start, end position, value, error.
Then I made class Line that holds list of type Data, and list that holds all error founded from some line. After load data from line to object Data I loop through lineError and add errors from all line to list, because I need to save errors from each line to database.
My question is this proper way to save data from file to object and after processing same data saving to database, advice for some better approach?
public class Data
{
public int startPosition = 0;
public int endPosition = 0;
public object value = null;
public string fieldName = "";
public Error error = null;
public Data(int start, int end, string name)
{
this.startPosition = start;
this.endPosition = end;
this.fieldName = name;
}
public void SetValueFromLine(string line)
{
string valueFromLine = line.Substring(this.startPosition, this.endPosition - this.startPosition);
// if else statment that checks validity of data (lenght, empty value)
this.value = valueFromLine;
}
}
public class Line
{
public List<Data> lineData = new List<Data>();
public List<Error> lineError = new List<Error>();
public Line()
{
AddObjectDataToList();
}
public void AddObjectDataToList()
{
lineData.Add(new Data(0, 3, "lineNumber"));
lineData.Add(new Data(4, 30, "bookAuthor"));
lineData.Add(new Data(31, 37, "publishData"));
lineData.Add(new Data(38, 43, "pageNumber"));
lineData.Add(new Data(44, 49, "bookCode"));
}
public void LoadLineDataToObjects(string line)
{
foreach(Data s in lineData)
{
s.SetValueFromLine(line);
}
}
public void GetAllErrorFromData()
{
foreach (Data s in lineData)
{
if(s.error != null)
{
lineError.Add(s.error);
}
}
}
}
public class File
{
public string fileName;
public List<Line> lines = new List<Line>();
}
I assume that the focus is on using OOP. I also assume that parsing is a secondary task and I will not consider options for its implementation.
First of all, it is necessary to determine the main acting object. Strange as it may seem, this is not a Book, but the string itself (e.g. DataLine). Initially, I wanted to create a Book from a string (through a separate constructor), but that would be a mistake.
What actions should be able to perform DataLine? - In fact, only one - process. I see two acceptable options for this method:
process returns Book or throws exceptions. (Book process())
process returns nothing, but interacts with another object. (void process(IResults result))
The first option has the following drawbacks:
It is difficult to test (although this applies to the second option). All validation is hidden inside DataLine.
It is impossible/difficult to return a few errors.
The program is aimed at working with incorrect data, so expected exceptions are often generated. This violates the ideology of exceptions. Also, there are small fears of slowing performance.
The second option is devoid of the last two drawbacks. IResults can contain methodserror(...), to return several errors, and success(Book book).
The testability of the process method can be significantly improved by adding IValidator. This object can be passed as a parameter to the DataLine constructor, but this is not entirely correct. First, this unnecessary expense of memory because it will not give us tangible benefits. Secondly, this does not correspond to the essence of the DataLine class. DataLine represents only a line that can be processed in one particular way. Thus, a good solution is the void process (IValidator validator, IResults result).
Summarize the above (may contain syntax errors):
interface IResults {
void error (string message);
void success (Book book);
}
interface IValidator {
// just example
bool checkBookCode (string bookCode);
}
class DataLine {
private readonly string _rawData;
// constructor
/////////////////
public void process (IValidator validator, IResults result) {
// parse _rawData
bool isValid = true; // just example! maybe better to add IResults.hasErrors ()
if (! validator.checkBookCode (bookCode)) {
result.error("Bad book code");
isValid = false;
}
if (isValid) {
result.success(new Book (...));
// or even result.success (...); to avoid cohesion (coupling?) with the Book
}
}
}
The next step is to create a model of the file with the lines. Here again there are many options and nuances, but I would like to pay attention to IEnumerable<DataLine>. Ideally, we need to create a DataLines class that will support IEnumerable<DataLine> and load from a file or from IEnumerable<string>. However, this approach is relatively complex and redundant, it makes sense only in large projects. A much simpler version:
interface DataLinesProvider {
IEnumerable <DataLine> Lines ();
}
class DataLinesFile implements DataLinesProvider {
private readonly string _fileName;
// constructor
////////////////////
IEnumerable <DataLine> Lines () {
// not sure that it's right
return File
. ReadAllLines (_fileName)
.Select (x => new DataLine (x));
}
}
You can infinitely improve the code, introduce new and new abstractions, but here you must start from common sense and a specific problem.
P. S. sorry for "strange" English. Google not always correctly translate such complex topics.
I have an application where I need to create files with a unique and sequential number as part of the file name. My first thought was to use (since this application does not have any other data storage) a text file that would contain a number and I would increment this number so then my application would always create a file with a unique id.
Then I thought that maybe at a time when there are more than one user submitting to this application at the same time, one process might be reading the txt file before it has been written by the previous process. So then I am looking for a way to read and write to a file (with try catch so then I can know when it's being used by another process and then wait and try to read from it a few other times) in the same 'process' without unlocking the file in between.
If what I am saying above sounds like a bad option, could you please give me an alternative to this? How would you then keep track of unique identification numbers for an application like my case?
Thanks.
If it's a single application then you can store the current number in your application settings. Load that number at startup. Then with each request you can safely increment it and use the result. Save the sequential number when the program shuts down. For example:
private int _fileNumber;
// at application startup
_fileNumber = LoadFileNumberFromSettings();
// to increment
public int GetNextFile()
{
return Interlocked.Increment(ref _fileNumber);
}
// at application shutdown
SaveFileNumberToSettings(_fileNumber);
Or, you might want to make sure that the file number is saved whenever it's incremented. If so, change your GetNextFile method:
private readonly object _fileLock = new object();
public int GetNextFile()
{
lock (_fileLock)
{
int result = ++_fileNumber;
SaveFileNumbertoSettings(_fileNumber);
return result;
}
}
Note also that it might be reasonable to use the registry for this, rather than a file.
Edit: As Alireza pointed in the comments, it is not a valid way to lock between multiple applications.
You can always lock the access to the file (so you won't need to rely on exceptions).
e.g:
// Create a lock in your class
private static object LockObject = new object();
// and then lock on this object when you access the file like this:
lock(LockObject)
{
... access to the file
}
Edit2: It seems that you can use Mutex to perform inter-application signalling.
private static System.Threading.Mutex m = new System.Threading.Mutex(false, "LockMutex");
void AccessMethod()
{
try
{
m.WaitOne();
// Access the file
}
finally
{
m.ReleaseMutex();
}
}
But it's not the best pattern to generate unique ids. Maybe a sequence in a database would be better ? If you don't have a database, you can use Guids or a local database (even Access would be better I think)
I would prefer a complex and universal solution with the global mutex. It uses a mutex with name prefixed with "Global\" which makes it system-wide i.e. one mutex instance is shared across all processes. if your program runs in friendly environment or you can specify strict permissions limited to a user account you can trust then it works well.
Keep in mind that this solution is not transactional and is not protected against thread-abortion/process-termination.
Not transactional means that if your process/thread is caught in the middle of storage file modification and is terminated/aborted then the storage file will be left in unknown state. For instance it can be left empty. You can protect yourself against loss of data (loss of last used index) by writing the new value first, saving the file and only then removing the previous value. Reading procedure should expect a file with multiple numbers and should take the greatest.
Not protected against thread-abortion means that if a thread which obtained the mutex is aborted unexpectedly and/or you do not have proper exception handling then the mutex could stay locked for the life of the process that created that thread. In order to make solution abort-protected you will have to implement timeouts on obtaining the lock i.e. replace the following line which waits forever
blnResult = iLock.Mutex.WaitOne();
with something with timeout.
Summing this up I try to say that if you are looking for a really robust solution you will come to utilizing some kind of a transactional database or write a kind of such a database yourself :)
Here is the working code without timeout handling (I do not need it in my solution). It is robust enough to begin with.
using System;
using System.IO;
using System.Security.AccessControl;
using System.Security.Principal;
using System.Threading;
namespace ConsoleApplication31
{
class Program
{
//You only need one instance of that Mutex for each application domain (commonly each process).
private static SMutex mclsIOLock;
static void Main(string[] args)
{
//Initialize the mutex. Here you need to know the path to the file you use to store application data.
string strEnumStorageFilePath = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData),
"MyAppEnumStorage.txt");
mclsIOLock = IOMutexGet(strEnumStorageFilePath);
}
//Template for the main processing routine.
public static void RequestProcess()
{
//This flag is used to protect against unwanted lock releases in case of recursive routines.
bool blnLockIsSet = false;
try
{
//Obtain the lock.
blnLockIsSet = IOLockSet(mclsIOLock);
//Read file data, update file data. Do not put much of long-running code here.
//Other processes may be waiting for the lock release.
}
finally
{
//Release the lock if it was obtained in this particular call stack frame.
IOLockRelease(mclsIOLock, blnLockIsSet);
}
//Put your long-running code here.
}
private static SMutex IOMutexGet(string iMutexNameBase)
{
SMutex clsResult = null;
clsResult = new SMutex();
string strSystemObjectName = #"Global\" + iMutexNameBase.Replace('\\', '_');
//Give permissions to all authenticated users.
SecurityIdentifier clsAuthenticatedUsers = new SecurityIdentifier(WellKnownSidType.AuthenticatedUserSid, null);
MutexSecurity clsMutexSecurity = new MutexSecurity();
MutexAccessRule clsMutexAccessRule = new MutexAccessRule(
clsAuthenticatedUsers,
MutexRights.FullControl,
AccessControlType.Allow);
clsMutexSecurity.AddAccessRule(clsMutexAccessRule);
//Create the mutex or open an existing one.
bool blnCreatedNew;
clsResult.Mutex = new Mutex(
false,
strSystemObjectName,
out blnCreatedNew,
clsMutexSecurity);
clsResult.IsMutexHeldByCurrentAppDomain = false;
return clsResult;
}
//Release IO lock.
private static void IOLockRelease(
SMutex iLock,
bool? iLockIsSetInCurrentStackFrame = null)
{
if (iLock != null)
{
lock (iLock)
{
if (iLock.IsMutexHeldByCurrentAppDomain &&
(!iLockIsSetInCurrentStackFrame.HasValue ||
iLockIsSetInCurrentStackFrame.Value))
{
iLock.MutexOwnerThread = null;
iLock.IsMutexHeldByCurrentAppDomain = false;
iLock.Mutex.ReleaseMutex();
}
}
}
}
//Set the IO lock.
private static bool IOLockSet(SMutex iLock)
{
bool blnResult = false;
try
{
if (iLock != null)
{
if (iLock.MutexOwnerThread != Thread.CurrentThread)
{
blnResult = iLock.Mutex.WaitOne();
iLock.IsMutexHeldByCurrentAppDomain = blnResult;
if (blnResult)
{
iLock.MutexOwnerThread = Thread.CurrentThread;
}
else
{
throw new ApplicationException("Failed to obtain the IO lock.");
}
}
}
}
catch (AbandonedMutexException iMutexAbandonedException)
{
blnResult = true;
iLock.IsMutexHeldByCurrentAppDomain = true;
iLock.MutexOwnerThread = Thread.CurrentThread;
}
return blnResult;
}
}
internal class SMutex
{
public Mutex Mutex;
public bool IsMutexHeldByCurrentAppDomain;
public Thread MutexOwnerThread;
}
}
Evening guys,
This may seem like a stupid question but im having some issues figuring out where i should be placing my user input verification checks. I have checks, for the following:
file exists
Correct extension
Access to the file
Input in 2 NumericUpDown controls
One numericUpDown is always greater than the other.
Assignment to static properties.
Im assuming each one of these should at least be a single method but i then have a single method which has a large number of lines which solely check the result of the method calls to the list above. ie
public void VerifyData()
{
if(VerifyNumber1OnTheList != true)
{
LogError("The file specified is incorrect")
return;
}
if(VerifyNumber2OnTheList != ......
Any suggestions on how i should actually be laying this out would be appreciated.
Thanks
Putting a series of validation methods is ok.
This is user input, so you're doing defensive programming here, not looking for exceptional errors. It seems like you'd want to know why the user input is bad. If so, a simple approach is to use a collecting object to visit each validation method
public void VerifyData(ValidationErrors errors)
{
ValidateFileExists(errors);
ValidateExtension(errors);
ValidateFileAccess(errors);
...
}
private void ValidateFileExists(ValidationErrors errors)
{
if(!File.Exists...)
{
errors.Add("File does not exists.");
}
}
public void CallingMethod(UserInput input)
{
_dataToVerify = input;
var errors = new ValidationErrors();
VerifyData(errors);
if(errors.Count > 0)
ShowErrors(errors);
else
ShowSuccess();
}
I just put the data in an instance variable for purposes of simplicity, you could pass that into your validation method. This approach is simple but allows you to tell the user something about the input and how to correct it.
All right here is a quick sample I wrote up for you. Architecture obviously is your choice here. Modify as needed. I believe this should be a great jumping off point for you.
Cheers
Matt
using System.Collections.Generic;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
//This would simulate the event handler that calls your validation event
List<string> errorList = Validation.VerifyData();
if (errorList.Count != 0)
{
ErrorHandler.HandleError(errorList);
return;
}
//Do stuff if validation actually passed here.
}
}
public static class Validation
{
public static List<string> VerifyData()
{
List<string> errorList = new List<string>();
//File exists
if (true)
errorList.Add("File doesn't exist.");
//File has correct extension
if (true)
errorList.Add("File doesn't exist.");
//Has access to the file
if (true)
errorList.Add("File doesn't exist.");
//INput in 2 NumericUpDownControls
if (true)
errorList.Add("File doesn't exist.");
//One NumericUpDown is always greater than the other
if (true)
errorList.Add("File doesn't exist.");
//Assignment to static properties
if (true)
errorList.Add("File doesn't exist.");
return errorList;
}
}
public static class ErrorHandler
{
public static void HandleError(List<string> errorMessageList )
{
//Display your message here. This could return a dialog result as well for further processing.
}
}
}
Your method above seems like a perfectly fine way of laying out your code.
My only change would be in how you handle your checking. Use well-named methods rather than booleans during your check:
public void VerifyData()
{
if(FileExists())
{
LogError("The file specified is incorrect")
return;
}
if(CorrectExtension())
I'm building a T4 template that will help people construct Azure queues in a consistent and simple manner. I'd like to make this self-documenting, and somewhat consistent.
First I made the queue name at the top of the file, the queue names have to be in lowercase so I added ToLower()
The public constructor uses the built-in StorageClient API's to access the connection strings. I've seen many different approaches to this, and would like to get something that works in almost all situations. (ideas? do share)
I dislike the unneeded HTTP requests to check if the queues have been created so I made is a static bool . I didn't implement a Lock(monitorObject) since I don't think one is needed.
Instead of using a string and parsing it with commas (like most MSDN documentation) I'm serializing the object when passing it into the queue.
For further optimization I'm using a JSON serializer extension method to get the most out of the 8k limit. Not sure if an encoding will help optimize this any more
Added retry logic to handle certain scenarios that occur with the queue (see html link)
Q: Is "DataContext" appropriate name for this class?
Q: Is it a poor practice to name the Queue Action Name in the manner I have done?
What additional changes do you think I should make?
public class AgentQueueDataContext
{
// Queue names must always be in lowercase
// Is named like a const, but isn't one because .ToLower won't compile...
static string AGENT_QUEUE_ACTION_NAME = "AgentQueueActions".ToLower();
static bool QueuesWereCreated { get; set; }
DataModel.SecretDataSource secDataSource = null;
CloudStorageAccount cloudStorageAccount = null;
CloudQueueClient cloudQueueClient = null;
CloudQueue queueAgentQueueActions = null;
static AgentQueueDataContext()
{
QueuesWereCreated = false;
}
public AgentQueueDataContext() : this(false)
{
}
public AgentQueueDataContext(bool CreateQueues)
{
// This pattern of setting up queues is from:
// ttp://convective.wordpress.com/2009/11/15/queues-azure-storage-client-v1-0/
//
this.cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
this.cloudQueueClient = cloudStorageAccount.CreateCloudQueueClient();
this.secDataSource = new DataModel.SecretDataSource();
queueAgentQueueActions = cloudQueueClient.GetQueueReference(AGENT_QUEUE_ACTION_NAME);
if (QueuesWereCreated == false || CreateQueues)
{
queueAgentQueueActions.CreateIfNotExist();
QueuesWereCreated = true;
}
}
// This is the method that will be spawned using ThreadStart
public void CheckQueue()
{
while (true)
{
try
{
CloudQueueMessage msg = queueAgentQueueActions.GetMessage();
bool DoRetryDelayLogic = false;
if (msg != null)
{
// Deserialize using JSON (allows more data to be stored)
AgentQueueEntry actionableMessage = msg.AsString.FromJSONString<AgentQueueEntry>();
switch (actionableMessage.ActionType)
{
case AgentQueueActionEnum.EnrollNew:
{
// Add to
break;
}
case AgentQueueActionEnum.LinkToSite:
{
// Link within Agent itself
// Link within Site
break;
}
case AgentQueueActionEnum.DisableKey:
{
// Disable key in site
// Disable key in AgentTable (update modification time)
break;
}
default:
{
break;
}
}
//
// Only delete the message if the requested agent has been missing for
// at least 10 minutes
//
if (DoRetryDelayLogic)
{
if (msg.InsertionTime != null)
if (msg.InsertionTime < DateTime.UtcNow + new TimeSpan(0, 10, 10))
continue;
// ToDo: Log error: AgentID xxx has not been found in table for xxx minutes.
// It is likely the result of a the registratoin host crashing.
// Data is still consistent. Deleting queued message.
}
//
// If execution made it to this point, then we are either fully processed, or
// there is sufficent reason to discard the message.
//
try
{
queueAgentQueueActions.DeleteMessage(msg);
}
catch (StorageClientException ex)
{
// As of July 2010, this is the best way to detect this class of exception
// Description: ttp://blog.smarx.com/posts/deleting-windows-azure-queue-messages-handling-exceptions
if (ex.ExtendedErrorInformation.ErrorCode == "MessageNotFound")
{
// pop receipt must be invalid
// ignore or log (so we can tune the visibility timeout)
}
else
{
// not the error we were expecting
throw;
}
}
}
else
{
// allow control to fall to the bottom, where the sleep timer is...
}
}
catch (Exception e)
{
// Justification: Thread must not fail.
//Todo: Log this exception
// allow control to fall to the bottom, where the sleep timer is...
// Rationale: not doing so may cause queue thrashing on a specific corrupt entry
}
// todo: Thread.Sleep() is bad
// Replace with something better...
Thread.Sleep(9000);
}
Q: Is "DataContext" appropriate name for this class?
In .NET we have a lot of DataContext classes, so in the sense that you want names to appropriately communicate what the class does, I think XyzQueueDataContext properly communicates what the class does - although you can't query from it.
If you want to stay more aligned to accepted pattern languages, Patterns of Enterprise Application Architecture calls any class that encapsulates access to an external system for a Gateway, while more specifically you may want to use the term Channel in the language of Enterprise Integration Patterns - that's what I would do.
Q: Is it a poor practice to name the Queue Action Name in the manner I have done?
Well, it certainly tightly couples the queue name to the class. This means that if you later decide that you want to decouple those, you can't.
As a general comment I think this class might benefit from trying to do less. Using the queue is not the same thing as managing it, so instead of having all of that queue management code there, I'd suggest injecting a CloudQueue into the instance. Here's how I implement my AzureChannel constructor:
private readonly CloudQueue queue;
public AzureChannel(CloudQueue queue)
{
if (queue == null)
{
throw new ArgumentNullException("queue");
}
this.queue = queue;
}
This better fits the Single Responsibility Principle and you can now implement queue management in its own (reusable) class.
I'm using the Windows Event Log to record some events. Events within the Windows Event Log can be assigned a handful of properties. One of which, is an EventID.
Now I want to use the EventId to try and group related errors. I could just pick a number for each call to the logging method I do, but that seems a little tedious.
I want the system to do this automatically. It would choose an eventId that is "unique" to the position in the code where the logging event occurred. Now, there's only 65536 unique event IDs, so there are likely to be collisions but they should be rare enough to make the EventId a useful way to group errors.
One strategy would be to take the hashcode of the stacktrace but that would mean that the first and second calls in the following code would have generate the same event ID.
public void TestLog()
{
LogSomething("Moo");
// Do some stuff and then a 100 lines later..
LogSomething("Moo");
}
I thought of walking up the call stack using the StackFrame class which has a GetFileLineNumber method. The problem with this strategy is that it will only work when built with debug symbols on. I need it to work in production code too.
Does anyone have any ideas?
Here is some code you can use to generate an EventID with the properties I describe in my question:
public static int GenerateEventId()
{
StackTrace trace = new StackTrace();
StringBuilder builder = new StringBuilder();
builder.Append(Environment.StackTrace);
foreach (StackFrame frame in trace.GetFrames())
{
builder.Append(frame.GetILOffset());
builder.Append(",");
}
return builder.ToString().GetHashCode() & 0xFFFF;
}
The frame.GetILOffset() method call gives the position within that particular frame at the time of execution.
I concatenate these offsets with the entire stacktrace to give a unique string for the current position within the program.
Finally, since there are only 65536 unique event IDs I logical AND the hashcode against 0xFFFF to extract least significant 16-bits. This value then becomes the EventId.
The IL offset number is available without debug symbols. Combined with the stack information and hashed, I think that would do the trick.
Here's an article that, in part, covers retrieving the IL offset (for the purpose of logging it for an offline match to PDB files--different problem but I think it'll show you what you need):
http://timstall.dotnetdevelopersjournal.com/getting_file_and_line_numbers_without_deploying_the_pdb_file.htm
Create a hash using the ILOffset of the last but one stack frame instead of the line number (i.e. the stack frame of your TestLog method above).
*Important: This post focuses at solving the root cause of what it appears your problem is instead of providing a solution you specifically asked for. I realize this post is old, but felt it important to contribute. *
My team had a similar issue, and we changed the way we managed our logging which has reduced production support and bug patching times significantly. Pragmatically this works in most enterprise apps my team works on:
Prefix log messages with the "class name"."function name".
For true errors, output the captured Exception to the event logger.
Focus on having clear messages as part of the peer code review as opposed to event id's.
Use a unique event id for each function, just go top to bottom and key them.
when it becomes impractical to code each function a different event ID, each class should just just have a unique one (collisions be damned).
Utilize Event categories to reduce event id reliance when filtering the log
Of course it matters how big your apps are and how sensitive the data is. Most of ours are around 10k to 500k lines of code with minimally sensitive information. It may feel oversimplified, but from a KISS standpoint it pragmatically works.
That being said, using an abstract Event Log class to simplify the process makes it easy to utilize, although cleanup my be unpleasant. For Example:
MyClass.cs (using the wrapper)
class MyClass
{
// hardcoded, but should be from configuration vars
private string AppName = "MyApp";
private string AppVersion = "1.0.0.0";
private string ClassName = "MyClass";
private string LogName = "MyApp Log";
EventLogAdapter oEventLogAdapter;
EventLogEntryType oEventLogEntryType;
public MyClass(){
this.oEventLogAdapter = new EventLogAdapter(
this.AppName
, this.LogName
, this.AppName
, this.AppVersion
, this.ClassName
);
}
private bool MyFunction() {
bool result = false;
this.oEventLogAdapter.SetMethodInformation("MyFunction", 100);
try {
// do stuff
this.oEventLogAdapter.WriteEntry("Something important found out...", EventLogEntryType.Information);
} catch (Exception oException) {
this.oEventLogAdapter.WriteEntry("Error: " + oException.ToString(), EventLogEntryType.Error);
}
return result;
}
}
EventLogAdapter.cs
class EventLogAdapter
{
//vars
private string _EventProgram = "";
private string _EventSource = "";
private string _ProgramName = "";
private string _ProgramVersion = "";
private string _EventClass = "";
private string _EventMethod = "";
private int _EventCode = 1;
private bool _Initialized = false;
private System.Diagnostics.EventLog oEventLog = new EventLog();
// methods
public EventLogAdapter() { }
public EventLogAdapter(
string EventProgram
, string EventSource
, string ProgramName
, string ProgramVersion
, string EventClass
) {
this.SetEventProgram(EventProgram);
this.SetEventSource(EventSource);
this.SetProgramName(ProgramName);
this.SetProgramVersion(ProgramVersion);
this.SetEventClass(EventClass);
this.InitializeEventLog();
}
public void InitializeEventLog() {
try {
if(
!String.IsNullOrEmpty(this._EventSource)
&& !String.IsNullOrEmpty(this._EventProgram)
){
if (!System.Diagnostics.EventLog.SourceExists(this._EventSource)) {
System.Diagnostics.EventLog.CreateEventSource(
this._EventSource
, this._EventProgram
);
}
this.oEventLog.Source = this._EventSource;
this.oEventLog.Log = this._EventProgram;
this._Initialized = true;
}
} catch { }
}
public void WriteEntry(string Message, System.Diagnostics.EventLogEntryType EventEntryType) {
try {
string _message =
"[" + this._ProgramName + " " + this._ProgramVersion + "]"
+ "." + this._EventClass + "." + this._EventMethod + "():\n"
+ Message;
this.oEventLog.WriteEntry(
Message
, EventEntryType
, this._EventCode
);
} catch { }
}
public void SetMethodInformation(
string EventMethod
,int EventCode
) {
this.SetEventMethod(EventMethod);
this.SetEventCode(EventCode);
}
public string GetEventProgram() { return this._EventProgram; }
public string GetEventSource() { return this._EventSource; }
public string GetProgramName() { return this._ProgramName; }
public string GetProgramVersion() { return this._ProgramVersion; }
public string GetEventClass() { return this._EventClass; }
public string GetEventMethod() { return this._EventMethod; }
public int GetEventCode() { return this._EventCode; }
public void SetEventProgram(string EventProgram) { this._EventProgram = EventProgram; }
public void SetEventSource(string EventSource) { this._EventSource = EventSource; }
public void SetProgramName(string ProgramName) { this._ProgramName = ProgramName; }
public void SetProgramVersion(string ProgramVersion) { this._ProgramVersion = ProgramVersion; }
public void SetEventClass(string EventClass) { this._EventClass = EventClass; }
public void SetEventMethod(string EventMethod) { this._EventMethod = EventMethod; }
public void SetEventCode(int EventCode) { this._EventCode = EventCode; }
}
Thanks for the idea of hashing the call stack, I was going to ask that very same question of how to pick an eventId.
I recommend putting a static variable in LogSomething that increments each time it is called.
Now I want to use the EventId to try
and group related errors.
You have filters in event viewer so why (Go to find ? You have 65536 unique event IDs too.
Or rather use log4net or something ??
just my ideas....