This situation might seem strange but this is what i have to do:
Situation, i have a sharepoint portal and there was such an issue that there might be a problem while retrieving user profiles that there might be too slow when a lot of people online and perfrom that kind of action, so there was made a descision to make a console application to test it out.
The console application needs to simulate behavior for retrieving the user profiles with as if many different users are doing that.
And there must be a log written.
The first question is this kind of testing a good way to really know where exactly the problme is?
And the other question is about my application, i have a strange behavior:
public class Program
{
static void Main(string[] args)
{
string filePath = #"C:\Users\User\Desktop\logfile.txt";
string siteUrl = #"http://siteurl";
int threads = 1;
//Multiplicator multiplicator = new Multiplicator(filePath, siteUrl, threads);
//Console.ReadLine();
for (int i = 0; i < 100; i++)
{
Thread t = new Thread(Execute);
t.Start();
}
Console.WriteLine("Main thread: " + Thread.CurrentThread.Name);
// Simultaneously, do something on the main thread.
}
static void Execute()
{
for (int i = 0; i < 100; i++)
{
using (SPSite ospSite = new SPSite(#"http://siteurl"))
{
SPServiceContext serviceContext = SPServiceContext.GetContext(ospSite);
UserProfileManager profileManager = new UserProfileManager(serviceContext);
UserProfile userProfile = profileManager.GetUserProfile("User Name");
string message = "Retrieved: " + userProfile.DisplayName + " on " +DateTime.Now + "by " Thread.CurrentThread.Name;
Console.WriteLine(message);
}
}
}
}
So the problem is i never get the name of the thread written why?
Thread.CurrentThread.Name is empty, is it normal, maybe i initialize the threading wrong? Altho many sources said that it is done like this?
You have not set the name of the thread. You should do so before you start it, and you can incorporate the iteration number in the name, if you like:
Thread t = new Thread(Execute);
t.Name = "My Thread" + i.ToString();
t.Start();
They are not given names automatically. The name can only be set once, after which you would get an InvalidOperationException
MSDN Reference: Thread.Name
Incidentally, creating 100 threads is probably not a good idea under normal circumstances.
You need to give it a name. Just when you create the thread, just name it based on i
Thread t = new Thread(Execute) { Name = i.ToString() };
Ok, I will go for your first question. No, there is a better way to do this.
Put the site under load. Maybe a friend can hit F5 all the time or you run a batch file with 1000 lines of a curl get
Attach the Visual Studio debugger to the webserver process
Hit break 10 times and see where it stops most of the time. That is your hotspot/problem.
This is called the poor mans profiler. It is built into every Visual Studio ;-)
In general, it is easy to find such problems by doing profiling. There are even sophisticated tools for this.
Related
I am executing/processing very big files in multi threaded mode in a console app.
When I don't update/write to the console from threads, for testing the whole process take about 1 minute.
But when I try to update/write to console from threads to show the progress, the process stuck and it never finishes (waited several minutes even hours). And also console text/window does not updated as it should.
Update-1: As requested by few kind responder, i added minimal code that can reproduce the same error/problem
Here is the code from the thread function/method:
using System;
using System.Collections;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
namespace Large_Text_To_Small_Text
{
class Program
{
static string sAppPath;
static ArrayList objThreadList;
private struct ThreadFileInfo
{
public string sBaseDir, sRFile;
public int iCurFile, iTFile;
public bool bIncludesExtension;
}
static void Main(string[] args)
{
string sFileDir;
DateTime dtStart;
Console.Clear();
sAppPath = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
sFileDir = #"d:\Test";
dtStart = DateTime.Now;
///process in multi threaded mode
List<string> lFiles;
lFiles = new List<string>();
lFiles.AddRange(Directory.GetFiles(sFileDir, "*.*", SearchOption.AllDirectories));
if (Directory.Exists(sFileDir + "-Processed") == true)
{
Directory.Delete(sFileDir + "-Processed", true);
}
Directory.CreateDirectory(sFileDir + "-Processed");
sPrepareThreading();
for (int iFLoop = 0; iFLoop < lFiles.Count; iFLoop++)
{
//Console.WriteLine(string.Format("{0}/{1}", (iFLoop + 1), lFiles.Count));
sThreadProcessFile(sFileDir + "-Processed", lFiles[iFLoop], (iFLoop + 1), lFiles.Count, Convert.ToBoolean(args[3]));
}
sFinishThreading();
Console.WriteLine(DateTime.Now.Subtract(dtStart).ToString());
Console.ReadKey();
return;
}
private static void sProcSO(object oThreadInfo)
{
var inputLines = new BlockingCollection<string>();
long lACounter, lCCounter;
ThreadFileInfo oProcInfo;
lACounter = 0;
lCCounter = 0;
oProcInfo = (ThreadFileInfo)oThreadInfo;
var readLines = Task.Factory.StartNew(() =>
{
foreach (var line in File.ReadLines(oProcInfo.sRFile))
{
inputLines.Add(line);
lACounter++;
}
inputLines.CompleteAdding();
});
var processLines = Task.Factory.StartNew(() =>
{
Parallel.ForEach(inputLines.GetConsumingEnumerable(), line =>
{
lCCounter++;
/*
some process goes here
*/
/*If i Comment out these lines program get stuck!*/
//Console.SetCursorPosition(0, oProcInfo.iCurFile);
//Console.Write(oProcInfo.iCurFile + " = " + lCCounter.ToString());
});
});
Task.WaitAll(readLines, processLines);
}
private static void sPrepareThreading()
{
objThreadList = new ArrayList();
for (var iTLoop = 0; iTLoop < 5; iTLoop++)
{
objThreadList.Add(null);
}
}
private static void sThreadProcessFile(string sBaseDir, string sRFile, int iCurFile, int iTFile, bool bIncludesExtension)
{
Boolean bMatched;
Thread oCurThread;
ThreadFileInfo oProcInfo;
Salma_RecheckThread:
bMatched = false;
for (int iTLoop = 0; iTLoop < 5; iTLoop++)
{
if (objThreadList[iTLoop] == null || ((System.Threading.Thread)(objThreadList[iTLoop])).IsAlive == false)
{
oProcInfo = new ThreadFileInfo()
{
sBaseDir = sBaseDir,
sRFile = sRFile,
iCurFile = iCurFile,
iTFile = iTFile,
bIncludesExtension = bIncludesExtension
};
oCurThread = new Thread(sProcSO);
oCurThread.IsBackground = true;
oCurThread.Start(oProcInfo);
objThreadList[iTLoop] = oCurThread;
bMatched = true;
break;
}
}
if (bMatched == false)
{
System.Threading.Thread.Sleep(250);
goto Salma_RecheckThread;
}
}
private static void sFinishThreading()
{
Boolean bRunning;
Salma_RecheckThread:
bRunning = false;
for (int iTLoop = 0; iTLoop < 5; iTLoop++)
{
if (objThreadList[iTLoop] != null && ((System.Threading.Thread)(objThreadList[iTLoop])).IsAlive == true)
{
bRunning = true;
}
}
if (bRunning == true)
{
System.Threading.Thread.Sleep(250);
goto Salma_RecheckThread;
}
}
}
}
And here is the screenshot, if I try to update console window:
You see? Nor the line number (oProcInfo.iCurFile) or the whole line is correct!
It should be like this:
1 = xxxxx
2 = xxxxx
3 = xxxxx
4 = xxxxx
5 = xxxxx
Update-1: To test just change the sFileDir to any folder that has some big text file or if you like you can download some big text files from following link:
https://wetransfer.com/downloads/8aecfe05bb44e35582fc338f623ad43b20210602005845/bcdbb5
Am I missing any function/method to update console text from threads?
I can't reproduce it. In my tests the process always runs to completion, without getting stuck. The output is all over the place though, because the two lines below are not synchronized:
Console.SetCursorPosition(0, oProcInfo.iCurFile);
Console.Write(oProcInfo.iCurFile + " = " + lCCounter.ToString());
Each thread of the many threads involved in the computation invokes these two statements concurrently with the other threads. This makes it possible for one thread to preempt another, and move the cursor before the first thread has the chance to write in the console. To solve this problem you must add proper synchronization, and the easiest way to do it is to use the lock statement:
class Program
{
static object _locker = new object();
And in the sProcSO method:
lock (_locker)
{
Console.SetCursorPosition(0, oProcInfo.iCurFile);
Console.Write(oProcInfo.iCurFile + " = " + lCCounter.ToString());
}
If you want to know more about thread synchronization, I recommend this online resource: Threading in C# - Part 2: Basic Synchronization
If you would like to hear my opinion about the code in the question, and you don't mind receiving criticism, my opinion is that honestly the code is so much riddled with problems that the best course of action would be to throw it away and restart from scratch. Use of archaic data structures (ArrayList???), liberal use of casting from object to specific types, liberal use of the goto statement, use of hungarian notation in public type members, all make the code difficult to follow, and easy for bugs to creep in. I found particularly problematic that each file is processed concurrently with all other files using a dedicated thread, and then each dedicated thread uses a ThreadPool thread (Task.Factory.StartNew) to starts a parallel loop (Parallel.ForEach) with unconfigured MaxDegreeOfParallelism. This setup ensures that the ThreadPool will be saturated so badly, that there is no hope that the availability of threads will ever match the demand. Most probably it will also result to a highly inefficient use of the storage device, especially if the hardware is a classic hard disk.
Your freezing problem may not be C# or code related
on the top left of your console window, on the icon .. right click
select Properties
remove the option of Quick Edit Mode and Insert Mode
you can google that feature, but essentially manifests in the problem you describe above
The formatting problem on the other hand does seem to be, here you need to create a class that serializes writes to the console window from a singe thread. a consumer/producer pattern would work (you could use a BlockingCollection to implement this quite easily)
As always, im quite the noob, as im sure you will see from both my code and question. For practice im currently writing an Xamarin.Android app for a game called Eve Online. People there mine resources from planets to make cash. These mines have to be reset at different intervals, and the real pros can have up to 30 characters doing it. Each character can have 5 planets, usually there are at least 2 mines (extractors) on each. So there could be 300 timers going on.
In my app you save your characters in an sqlite db, and every hour a intentservice runs through the API and checks your times and if their expired or not. This is how i do that:
public async Task PullPlanets(long KeyID, long CharacterID, string VCode, string CharName)
{
XmlReader lesern = XmlReader.Create("https://api.eveonline.com/char/PlanetaryColonies.xml.aspx?keyID=" + KeyID + "&vCode=" + VCode + "&characterID=" + CharacterID);
while (lesern.Read())
{
long planet = 0;
string planetName;
planet = Convert.ToInt64(lesern.GetAttribute("planetID"));
planetName = lesern.GetAttribute("planetName");
if ((planet != 0) && (planetName != null))
{
planets.Add(planet);
planetNames.Add(planetName);
await GetExpirationTimes(CharName, planet, planetName, KeyID, CharacterID, VCode);
}
}
lesern.Close ();
}
public async Task GetExpirationTimes(string CharName, long planetID, string planetName, long KeyID, long CharacterID, string VCode)
{
string planet = planetID.ToString();
XmlReader lesern = XmlReader.Create("https://api.eveonline.com/char/PlanetaryPins.xml.aspx?keyID=" + KeyID + "&vCode=" + VCode + "&characterID=" + CharacterID + "&planetID=" + planet);
while (lesern.Read())
{
string expTime;
expTime = lesern.GetAttribute("expiryTime");
if ((expTime != null) && (expTime != "0001-01-01 00:00:00"))
{
allInfo.Add (new AllInfo (CharName, planetName, Convert.ToDateTime (expTime)));
}
}
lesern.Close ();
SendOrderedBroadcast (stocksIntent, null);
}
}
After this, it sends the times back to my Activity, where they get added to an extractor. It seems to work pretty fine, although ive only been able to test with 2 characters with a total of 14 extractors so far. An alarmmanger in activity calls the service every hour, and it sends a notification. When user opens the activity, it pulls the list from service, sorts it, and displays it. I would welcome input on if this is the way to do it.
I do see a problem in the horizon, though. The Eve API blocks if an app surpases 30 API-calls per second. Im pretty sure someone with 30 characters would do that. So, im wondering if i should add something to delay each call if a certain number is passed? This is how i call the first XML call.
var table = db.Table<CharsList> ();
foreach (var e in table) {
long KeyIDOut = Convert.ToInt64(e.KeyID);
long CharIDOut = Convert.ToInt64(e.CharacterID);
string VCodeOut = e.VCode.ToString();
string navnOut = e.Name.ToString();
PullPlanets(KeyIDOut, CharIDOut, VCodeOut, navnOut);
}
CheckTimes ();
}
Is it viable to add a
if (table.Count > 10) {
foreach (var e in table) {
//start the first characters call
Thread.Sleep(100)
}
The service is intentservice and not on UI thread. I guess this would bring the calls under 30 a sec, but i have never used Thread.Sleep and fear what else could happen in my code. Are there other things that could help me not blow the limit? Can this code handle 300 extractors?
I believe you are generally right in your approach. I had to do a similar thing for a reddit client I was writing, except their limits is once a second or so.
The only problem I see with your setup is that assume that Thread.Sleep does sleep for the amount of time you give it. Spurious wakeups are possible in some cases, so what I would suggest is that you give it a smaller value, save the last time you accessed the service and then put a loop around the sleep call that terminates once enough time has passed.
Finally if you are going to be firing up a lot of intent services for a relatively short amount of work, you might want to have a normal service with a thread to handle the work - that way it will only have to be created once but it is still of the UI thread.
When invoking UpdatePerformanceCounters: In this updater all the counter names for the category and instance counters are the same - they are always derived from an Enum. The updater is passed a "profile" typically with content such as:
{saTrilogy.Core.Instrumentation.PerformanceCounterProfile}
_disposed: false
CategoryDescription: "Timed function for a Data Access process"
CategoryName: "saTrilogy<Core> DataAccess Span"
Duration: 405414
EndTicks: 212442328815
InstanceName: "saTrilogy.Core.DataAccess.ParameterCatalogue..ctor::[dbo].[sp_KernelProcedures]"
LogFormattedEntry: "{\"CategoryName\":\"saTrilogy<Core> DataAccess ...
StartTicks: 212441923401
Note the "complexity" of the Instance name.
The toUpdate.AddRange() of the VerifyCounterExistence method always succeeds and produces the "expected" output so the UpdatePerformanceCounters method continues through to the "successful" incrementing of the counters.
Despite the "catch" this never "fails" - except, when viewing the Category in PerfMon, it shows no instances or, therefore, any "successful" update of an instance counter.
I suspect my problem may be that my instance name is being rejected, without exception, because of its "complexity" - when I run this through a console tester via PerfView it does not show any exception stack and the ETW events associated with counter updates are successfully recorded in an out-of-process sink. Also, there are no entries in the Windows Logs.
This is all being run "locally" via VS2012 on a Windows 2008R2 server with NET 4.5.
Does anyone have any ideas of how else I may try this - or even test if the "update" is being accepted by PerfMon?
public sealed class Performance {
private enum ProcessCounterNames {
[Description("Total Process Invocation Count")]
TotalProcessInvocationCount,
[Description("Average Process Invocation Rate per second")]
AverageProcessInvocationRate,
[Description("Average Duration per Process Invocation")]
AverageProcessInvocationDuration,
[Description("Average Time per Process Invocation - Base")]
AverageProcessTimeBase
}
private readonly static CounterCreationDataCollection ProcessCounterCollection = new CounterCreationDataCollection{
new CounterCreationData(
Enum<ProcessCounterNames>.GetName(ProcessCounterNames.TotalProcessInvocationCount),
Enum<ProcessCounterNames>.GetDescription(ProcessCounterNames.TotalProcessInvocationCount),
PerformanceCounterType.NumberOfItems32),
new CounterCreationData(
Enum<ProcessCounterNames>.GetName(ProcessCounterNames.AverageProcessInvocationRate),
Enum<ProcessCounterNames>.GetDescription(ProcessCounterNames.AverageProcessInvocationRate),
PerformanceCounterType.RateOfCountsPerSecond32),
new CounterCreationData(
Enum<ProcessCounterNames>.GetName(ProcessCounterNames.AverageProcessInvocationDuration),
Enum<ProcessCounterNames>.GetDescription(ProcessCounterNames.AverageProcessInvocationDuration),
PerformanceCounterType.AverageTimer32),
new CounterCreationData(
Enum<ProcessCounterNames>.GetName(ProcessCounterNames.AverageProcessTimeBase),
Enum<ProcessCounterNames>.GetDescription(ProcessCounterNames.AverageProcessTimeBase),
PerformanceCounterType.AverageBase),
};
private static bool VerifyCounterExistence(PerformanceCounterProfile profile, out List<PerformanceCounter> toUpdate) {
toUpdate = new List<PerformanceCounter>();
bool willUpdate = true;
try {
if (!PerformanceCounterCategory.Exists(profile.CategoryName)) {
PerformanceCounterCategory.Create(profile.CategoryName, profile.CategoryDescription, PerformanceCounterCategoryType.MultiInstance, ProcessCounterCollection);
}
toUpdate.AddRange(Enum<ProcessCounterNames>.GetNames().Select(counterName => new PerformanceCounter(profile.CategoryName, counterName, profile.InstanceName, false) { MachineName = "." }));
}
catch (Exception error) {
Kernel.Log.Trace(Reflector.ResolveCaller<Performance>(), EventSourceMethods.Kernel_Error, new PacketUpdater {
Message = StandardMessage.PerformanceCounterError,
Data = new Dictionary<string, object> { { "Instance", profile.LogFormattedEntry } },
Error = error
});
willUpdate = false;
}
return willUpdate;
}
public static void UpdatePerformanceCounters(PerformanceCounterProfile profile) {
List<PerformanceCounter> toUpdate;
if (profile.Duration <= 0 || !VerifyCounterExistence(profile, out toUpdate)) {
return;
}
foreach (PerformanceCounter counter in toUpdate) {
if (Equals(PerformanceCounterType.RateOfCountsPerSecond32, counter.CounterType)) {
counter.IncrementBy(profile.Duration);
}
else {
counter.Increment();
}
}
}
}
From MSDN .Net 4.5 PerformanceCounter.InstanceName Property (http://msdn.microsoft.com/en-us/library/system.diagnostics.performancecounter.instancename.aspx)...
Note: Instance names must be shorter than 128 characters in length.
Note: Do not use the characters "(", ")", "#", "\", or "/" in the instance name. If any of these characters are used, the Performance Console (see Runtime Profiling) may not correctly display the instance values.
The instance name of 79 characters that I use above satisfies these conditions so, unless ".", ":", "[" and "]" are also "reserved" the name would not appear to be the issue. I also tried a 64 character sub-string of the instance name - just in case, as well as a plain "test" string all to no avail.
Changes...
Apart from the Enum and the ProcessCounterCollection I have replaced the class body with the following:
private static readonly Dictionary<string, List<PerformanceCounter>> definedInstanceCounters = new Dictionary<string, List<PerformanceCounter>>();
private static void UpdateDefinedInstanceCounterDictionary(string dictionaryKey, string categoryName, string instanceName = null) {
definedInstanceCounters.Add(
dictionaryKey,
!PerformanceCounterCategory.InstanceExists(instanceName ?? "Total", categoryName)
? Enum<ProcessCounterNames>.GetNames().Select(counterName => new PerformanceCounter(categoryName, counterName, instanceName ?? "Total", false) { RawValue = 0, MachineName = "." }).ToList()
: PerformanceCounterCategory.GetCategories().First(category => category.CategoryName == categoryName).GetCounters().Where(counter => counter.InstanceName == (instanceName ?? "Total")).ToList());
}
public static void InitialisationCategoryVerify(IReadOnlyCollection<PerformanceCounterProfile> etwProfiles){
foreach (PerformanceCounterProfile profile in etwProfiles){
if (!PerformanceCounterCategory.Exists(profile.CategoryName)){
PerformanceCounterCategory.Create(profile.CategoryName, profile.CategoryDescription, PerformanceCounterCategoryType.MultiInstance, ProcessCounterCollection);
}
UpdateDefinedInstanceCounterDictionary(profile.DictionaryKey, profile.CategoryName);
}
}
public static void UpdatePerformanceCounters(PerformanceCounterProfile profile) {
if (!definedInstanceCounters.ContainsKey(profile.DictionaryKey)) {
UpdateDefinedInstanceCounterDictionary(profile.DictionaryKey, profile.CategoryName, profile.InstanceName);
}
definedInstanceCounters[profile.DictionaryKey].ForEach(c => c.IncrementBy(c.CounterType == PerformanceCounterType.AverageTimer32 ? profile.Duration : 1));
definedInstanceCounters[profile.TotalInstanceKey].ForEach(c => c.IncrementBy(c.CounterType == PerformanceCounterType.AverageTimer32 ? profile.Duration : 1));
}
}
In the PerformanceCounter Profile I've added:
internal string DictionaryKey {
get {
return String.Concat(CategoryName, " - ", InstanceName ?? "Total");
}
}
internal string TotalInstanceKey {
get {
return String.Concat(CategoryName, " - Total");
}
}
The ETW EventSource now does the initialisation for the "pre-defined" performance categories whilst also creating an instance called "Total".
PerformanceCategoryProfile = Enum<EventSourceMethods>.GetValues().ToDictionary(esm => esm, esm => new PerformanceCounterProfile(String.Concat("saTrilogy<Core> ", Enum<EventSourceMethods>.GetName(esm).Replace("_", " ")), Enum<EventSourceMethods>.GetDescription(esm)));
Performance.InitialisationCategoryVerify(PerformanceCategoryProfile.Values.Where(v => !v.CategoryName.EndsWith("Trace")).ToArray());
This creates all of the categories, as expected, but in PerfMon I still cannot see any instances - even the "Total" instance and the update always, apparently, runs without error.
I don't know what else I can "change - probably "too close" to the problem and would appreciate comments/corrections.
These are the conclusions and the "answer" insofar as as it explains, to the best of my ability, what I believe is happening and posted by myself - given my recent helpful use of Stack Overflow this, I hope, will be of use to others...
Firstly, there is essentially nothing wrong with the code displayed excepting one proviso - mentioned later. Putting a Console.ReadKey() before program termination and after having done a PerformanceCounterCategory(categoryKey).ReadCategory() it is quite clear that not only are the registry entries correct (for this is where ReadCategory sources its results) but that the instance counters have all been incremented by the appropriate values. If one looks at PerfMon before the program terminates the instance counters are there and they do contain the appropriate Raw Values.
This is the crux of my "problem" - or, rather, my incomplete understanding of the architecture: INSTANCE COUNTERS ARE TRANSIENT - INSTANCES ARE NOT PERSISTED BEYOND THE TERMINATION OF A PROGRAM/PROCESS. This, once it dawned on me, is "obvious" - for example, try using PerfMon to look at an instance counter of one of your IIS AppPools - then stop the AppPool and you will see, in PerfMon, that the Instance for the stopped AppPool is no longer visible.
Given this axiom about instance counters the code above has another completely irrelevant section: When trying the method UpdateDefinedInstanceCounterDictionary assigning the list from an existing counter set is pointless. Firstly, the "else" code shown will fail since we are attempting to return a collection of (instance) counters for which this approach will not work and, secondly, the GetCategories() followed by GetCounters() or/and GetInstanceNames() is an extraordinarily expensive and time-consuming process - even if it were to work. The appropriate method to use is the one mentioned earlier - PerformanceCounterCategory(categoryKey).ReadCategory(). However, this returns an InstanceDataCollectionCollection which is effectively read-only so, as a provider (as opposed to a consumer) of counters it is pointless. In fact, it doesn't matter if you just use the Enum generated new PerformanceCounter list - it works regardless of whether the counters already exist or not.
Anyway, the InstanceDataCollectionCollection (this is essentially that which is demonstrated by the Win32 SDK for .Net 3.5 "Usermode Counter Sample") uses a "Sample" counter which is populated and returned - as per the usage of the System.Diagnostics.PerformanceData Namespace whichi looks like part of the Version 2.0 usage - which usage is "incompatible" with the System.Diagnostics.PerformanceCounterCategory usage shown.
Admittedly, the fact of non-persistance may seem obvious and may well be stated in documentation but, if I were to read all the documentation about everything I need to use beforehand I'd probably end up not actually writing any code! Furthermore, even if such pertinent documentation were easy to find (as opposed to experiences posted on, for example, Stack Overflow) I'm not sure I trust all of it. For example, I noted above that the instance name in the MSDN documentation has a 128 character limit - wrong; it is actually 127 since the underlying string must be null-terminated. Also, for example, for ETW, I wish it were made more obvious that keyword values must be powers of 2 and opcodes with value of less than 12 are used by the system - at least PerfView was able to show me this.
Ultimately this question has no "answer" other than a better understanding of instance counters - especially their persistence. Since my code is intended for use in a Windows Service based Web API then its persistence is not an issue (especially with daily use of LogMan etc.) - the confusing thing is that the damn things didn't appear until I paused the code and checked PerfMon and I could have saved myself a lot of time and hassle if I knew this beforehand. In any event my ETW event source logs all elapsed execution times and instances of what the performance counters "monitor" anyway.
I'll try to explain the issue with a simplified console application example, however the real project is a ASP.NET MVC3 application.
Having the following tables:
imagine the following scenario:
user creates a report (a line in TestReport, where Text is the report string content, and Ready is a bool flag, saying, if the report is ready to be processed); by default Ready is set to false, i.e. not ready.
user wants the report to be processed, so he submits it; Ready is set to true here.
The system gives an opportunity to recall the report back, if it has not been processed yet. So, when the report is recalled, Ready is set to false back. On the contrary, when the report is processed, a line in TestReportRef, referencing report by its Id, is created.
Now imagine that at one and the same moment
user wants to recall the report;
the report is added to the process list;
As soon as this can happen simultaneously, errors may occur. That is the report will have Ready == false and it'll be referenced in TestReportRef.
Here is a simple console example of how this may happen:
var dc = new TestDataContext('my connection string');
dc.TestReport.InsertOnSubmit(new TestReport
{
Text = "My report content",
Ready = true //ready at once
});
dc.SubmitChanges();
Action recallReport = () =>
{
var _dc = new TestDataContext(cs);
var report = _dc.TestReport.FirstOrDefault(t => t.Ready);
if (report != null && !report.TestReportRef.Any())
{
Thread.Sleep(1000);
report.Ready = false;
_dc.SubmitChanges();
}
};
Action acceptReport = () =>
{
var _dc = new TestDataContext(cs);
var report = _dc.TestReport.FirstOrDefault(t => t.Ready);
if (report != null && !report.TestReportRef.Any())
{
Thread.Sleep(1000);
_dc.TestReportRef.InsertOnSubmit(new TestReportRef
{
FK_ReportId = report.Id
});
_dc.SubmitChanges();
}
};
var task1 = new Task(recallReport);
var task2 = new Task(acceptReport);
task1.Start();
task2.Start();
task1.Wait();
task2.Wait();
foreach (var t in dc.TestReport)
{
Console.WriteLine(string.Format("{0}\t{1}\t{2}", t.Id, t.Text, t.Ready));
}
foreach (var t in dc.TestReportRef)
{
Console.WriteLine("ref id:\t" + t.FK_ReportId);
}
Thread.Sleep(1000); is added to be ensure, that tasks will check one and the same situation.
The given example may sound awkward, however, I hope, it should explain the issue I'm dealing with.
How can I avoid this? Making the repository singleton doesn't seem to be a good idea. Shall I use some shared mutex (one for all web requests) to separate write-operations only?
Or is there a pattern I should use in this kind of scenario?
This is only a simplified example of one of the scenarios I have. However, there are several scenarios in which it may run into a similar discrepancy. The best thing would be to make this kind of intersection impossible, I guess.
Why don't add a version column on the Report table? Task starts by tracking current version,when task end, if the version is the same that the tracked one, operation is ok, otherwise fail. If operation appear ok, update the version to version +1. This is a sort of optimistic lock; that implicitly suppose that conflicts may occur, but they are not so frequent.
UPDATE
If you are using linqto sql maybe you can have a check at the parameter UpdateCheck [Column(UpdateCheck=UpdateCheck.Always)]
This can be useful to handle concurrency in your case.
I have a function which is taking a lot of time to execute in a web application.
I have tested this with a profiler and by my logging.
I have other functions running in the same pageload.
What is a best way to display the rest of the values from those functions and keep this function in a thread and display it in a label when it finishes?
This function is used to get events in application which takes time.
private void getEventErrors()
{
EventLog eventLog = new EventLog("Application", ".");
getEvents(eventLog.Entries);
}
private void getEvents(EventLogEntryCollection eventLogEntryCollection)
{
int errorEvents = 0;
foreach (EventLogEntry logEntry in eventLogEntryCollection)
{
if (logEntry.Source.Equals("XYZ"))
{
DateTime variable = Convert.ToDateTime(logEntry.TimeWritten);
long eventTimeTicks = (variable.Ticks);
long eventTimeUTC = (eventTimeTicks - 621355968000000000) / 10000000;
long presentDayTicks = DateTime.Now.Ticks;
long daysBackSeconds = ((presentDayTicks - 864000000000) - 621355968000000000) / 10000000;
if (eventTimeUTC > daysBackSeconds)
{
if (logEntry.EntryType.ToString() == "Error")
{
errorEvents = errorEvents + 1;
}
}
}
}
btn_Link_Event_Errors_Val.Text = errorEvents.ToString(GUIUtility.TWO_DECIMAL_PT_FORMAT);
if (errorEvents == 0)
{
lbl_EventErrorColor.Attributes.Clear();
lbl_EventErrorColor.Attributes.Add("class", "green");
}
else
{
lbl_EventErrorColor.Attributes.Clear();
lbl_EventErrorColor.Attributes.Add("class", "red");
}
}
I have 3 functions in the pageload event, two to get the values from the DB and the other one is shown above.
Should both these functions be service calls?
What i wanted was, the page should load fast and if there is a function taking a lot of time it should run in the background and display when done and in the process if the user want to navigate to a new page it should kill it and move on.
If you have a function that is running in a separate thread in ASP.NET, you may want to consider moving it to a service. There are many reason for this
See this answer (one of many on SO) for why running long running tasks in ASP.NET is not always a good idea.
One option for the service is to use WCF. You can get started here. Your service could implement a method, say GetEvents() which you could use to pull your events. That way you won't tie up your page waiting for this process to complete (using AJAX of course). Also, this allows you to change your implementation of GetEvents() without touching your code on your website.