I have a progress bar on the front-end of my web app which gets its current percentage by listening for messages sent from a SignalR hub in the back-end. It tracks a long running process which has several stages with many iterations.
My initial set-up was to simply send a message every iteration. This caused issues however as the rate of iteration (and therefore message rate) was too fast for the front-end and the bar became very jumpy and buggy.
So instead I decided to use a Stopwatch object in the following way (where SendProgress() is the procedure that tells the hub to message the client):
int progressCount = 1;
var stopWatch = new System.Diagnostics.Stopwatch();
stopWatch.Start();
for(int i=0; i<taskCount; i++)
{
//PROCESS DONE HERE
if (stopWatch.Elapsed.TotalMilliseconds >= 500.0)
{
SendProgress(taskCount, progressCount, 0, 40);
stopWatch.Reset();
stopWatch.Start();
}
progressCount++;
}
Thus preventing messages from being sent faster than once every 500ms.
This worked well in terms of limiting the message rate, however I noticed a drop in performance, which after a little research I gather is due to using Stopwatch, which is inefficient.
What would be a better approach here? I also though of using Thread.Sleep() but that would just be adding artificial slowness to the algorithm which is, obviously, bad. Is there a way I can accurately control the message rate, without slowing things down too badly?
Run this in a console app to see if the logic applies to what you want to do.
Check the comments to track what's happening (and let me know if you need me to break it down further). Happy coding! ;)
static void Main(string[] args)
{
//set the percentage you want to show progress for (eg, every 20%).
const int updatePercentage = 20;
//this is just the total of the for loop for this example.
const int loopMax = 1000000;
//calculate what 20% of the total is, to set as the check point used to show the progress.
decimal loopCheck = Convert.ToDecimal(loopMax * (Convert.ToDecimal(updatePercentage) / 100));
for (int i = 1; i <= loopMax; i++)
{
//check if the mod of the current position meets the check point.
if ((i % loopCheck) == 0)
{
//show the progress...
Console.WriteLine($"You've reached the next {updatePercentage}% of the for loop ({i.ToString("#,##0")})");
}
}
Console.ReadLine();
}
First of all - I'm very low skilled programmer. I am building the foundation of a simple music app for my bachelor degree project. My question is regarding a internal clock method that is meant to increase an int value by 1 BPM times a minute.
I've created an internalClock class:
public class internalClock
{
// THIS METHOD WILL BE CALLED WHEN THE THREAD IS STARTED
public static void clockMethod()
{
int BPM = 135;
int clockTick = 1;
Form1 clockForm = new Form1();
// infinite loop
while (true)
{
if (clockTick == 8)
{
clockTick = 1;
}
else
{
clockTick++;
}
clockForm.metrobox.Text = clockTick.ToString();
Thread.Sleep(60 * 1000 / BPM);
}
}
}
This is how I managed to get an access to the RichTextBox itself:
public RichTextBox metrobox
{
get { return metroBox; }
set { metroBox = value; }
}
In the main 'Program.cs' I've written what's meant to start a separate thread to run the clockMethod until the program is closed:
// THREADING
// Create a thread
internalClock oClock = new internalClock();
Thread oClockThread = new Thread(new ThreadStart(internalClock.clockMethod));
// Start the internalClock thread
oClockThread.Start();
It's not updating the text in the RichTextBox. Also, if I call the clockMethod() without creating a separate thread for it - the application freezes. Sorry for my amateur question, I'm just getting started with C# (yeah.. my uni is useless). What am I doing wrong in my code?
So the above code has several problems, however I would encourage you to check out the Timer control that you can add to the form you want to do processing at a certain interval or in certain "ticks". MSDN Form Timer
With the timer you can remove that class you have and invoking a new thread, etc etc. I would read up on the Timer class in the given link and think about how you can re-work your application structure to fit that. The concepts for why that thread isn't working, etc, is frankly not that important for where you're at. I think you just need to focus for now on a tool that already does what you want it to do, which I believe is the Timer.
As a general note, you usually don't need to create a raw thread in .NET. As of .NET 4.0 you can access types called Tasks to perform multi-threaded logic and processing. If you find the need to do that later on, check that out. Task Type MSDN
My application could have up to roughly 100 requests for a batch job within a few milliseconds but in actuality, these job requests are being masked as one job request.
To fix this issue so that only one job request is just not feasible at the moment.
A workaround that I have thought is to program my application to fulfill only 1 batch job every x milliseconds, in this case I was thinking of 200 milliseconds, and ignore any other batch job that may come in within those 200 milliseconds or when my batch job have completed. After those 200 milliseconds are up or when the batch job is completed, my application will wait and accept 1 job request from that time on and it will not process any requests that may have been ignored before. Once my application accepts another job requests, it will repeat the cycle above.
What's the best way of doing this using .Net 4.0? Are there any boiler plate code that I can simply follow as a guide?
Update
Sorry for being unclear. I have added more details about my scenario. Also I just realized that my proposed workaround above will not work. Sorry guys, lol. Here's some background information.
I have an application that builds an index using files in a specified directory. When a file is added, deleted or modified in this directory, my application listens for these events using a FileSystemWatcher and re-indexes these files. The problem is that around 100 files can be added, deleted or modified by an external process and they occur very quickly, ie: within a few milliseconds. My end goal is to re-index these files after the last file change have occurred by the external process. The best solution is to modify the external process to signal my application when it has finished modifying the files I'm listening to but that's not feasible at the moment. Thus, I have to create a workaround.
A workaround that may solve my problem is to wait for the first file change. When the first file change have occurred, wait 200 milliseconds for any other subsequent file changes. Why 200 milliseconds? Because I'm hoping and confident that the external process can perform its file changes within 200 milliseconds. Once my application have waited for 200 milliseconds, I would like it to start a task that will re-index the files and go through another cycle of listening to a file change.
What's the best way of doing this?
Again, sorry for the confusion.
This question is a bit too high level to guess at.
My guess is your application is run as a service, you have your requests come into your application and arrive in a queue to be processed. And every 200 ms, you wake the queue and pop and item off for processing.
I'm confused about the "masked as one job request". Since you mentioned you will "ignore any other batch job", my guess is you haven't arranged your code to accept the incoming requests in a queue.
Regardless, you will generally always have one application process running (your service) and if you choose you could spawn a new thread for each item you process in the queue. You can monitor how much cpu/memory utilization this required and adjust the firing time (200ms) accordingly.
I may not be accurately understanding the problem, but my recommendation is to use the singleton pattern to work around this issue.
With the singleton approach, you can implement a lock on an object (the access method could potentially be something along the lines of BatchProcessor::GetBatchResults) that would then lock all requests to the batch job results object. Once the batch has finished, the lock will be released, and the underlying object, will have the results of the batch job available.
Please keep in mind that this is a "work around". There may be a better solution that involves looking into and changing the underlying business logic that causes multiple requests to come in for a job that's processing on demand.
Update:
Here is a link for information regarding Singleton (includes code examples): http://msdn.microsoft.com/en-us/library/ff650316.aspx
It is my understanding that the poster has some sort of an application that sits and waits for incoming requests to perform a batch job. The problem that he is receiving multiple requests within a short period of time that should actually have come in as just a single request. And, unfortunately, he is not able to solve this problem.
So, his solution is to assume that all requests received within a 200 ms timespan are the same, and to only process these once. My concern with this would be whether this assumption is correct or not? This entirely depends on the sending systems and the environment in which this is being used. The general idea to be able to do this would be to update a lastReceived date/time when a request is processed. Then when a new request comes in, compare the current date/time to the lastReceived date/time and only process it if the difference is greater than 200 ms.
Other possible solutions:
You said you could not modify the sending application so only one job request was sent, but could you add additional information to it, for instance a unique identifier?
Could you store the parameters from the last job request and compare it with the next job request and only process them if they are different?
Based on your Update
Here is an example how you could wait 200ms using a Timer:
static Timer timer;
static int waitTime = 200; //in ms
static void Main(string[] args)
{
FileSystemWatcher fsw = new FileSystemWatcher();
fsw.Path = #"C:\temp\";
fsw.Created += new FileSystemEventHandler(fsw_Created);
fsw.EnableRaisingEvents = true;
Console.ReadLine();
}
static void fsw_Created(object sender, FileSystemEventArgs e)
{
DateTime currTime = DateTime.Now;
if (timer == null)
{
Console.WriteLine("Started # " + currTime);
timer = new Timer();
timer.Interval = waitTime;
timer.Elapsed += new ElapsedEventHandler(timer_Elapsed);
timer.Start();
}
else
{
Console.WriteLine("Ignored # " + currTime);
}
}
static void timer_Elapsed(object sender, ElapsedEventArgs e)
{
//Start task here
Console.WriteLine("Elapsed # " + DateTime.Now);
timer = null;
}
As the title says.
I am looking for a way to check if a process is idle, there is the obvious running and not running but how to determine if it's not doing anything?
Thanks
it depends on how you define idle.
However you could create some sort of heuristic for defining a process as idle using the Process class. I'll assume that a process is 'idle' if it hasn't consumed more than thresholdMillis over a certain period of time
Process p = Process.GetProcessById(proc_id);
TimeSpan begin_cpu_time = p.TotalProcessorTime;
//... wait a while
p.Refresh();
TimeSpan end_cpu_time = p.TotalProcessorTime;
if(end_cpu_time - begin_cpu_time < TimeSpan.FromMillis(thresholdMillis))
{
//..process is idle
}
else
{
//..process is not idle
}
so depending on how you choose your threshold_millis value you will get different results. but this should be a decent heuristic for seeing if a process is idle.
Ideally you would probably use some sort of timer to periodically query and update the 'idleness' of a process.
Are you looking for something like GetLastInputInfo?
Of late, I'm becoming more health oriented when constructing my program, I have observed that most of programs take 2 or 3 minutes to execute and when I check on the task scheduler, I see that they consume 100% of CPU usage, can I limit this usage programatically in code? This will certainly enable me to run multiple programs at a given time.
Thanks,
Nidhi
This thread is over four years old, and it still annoys me that the accepted answer criticizes the question rather than answering it. There are many valid reasons you would want to limit the CPU time taken by your program, I can list a few off the top of my head.
It might seem like a waste not to use all free CPU cycles available, but this mentality is flawed. Unlike older CPUs, most modern CPUs do not run at a fixed clock speed - many have power saving modes where they drop the clock speed and cpu voltage when load is low. CPUs also consume more power when performing calculations than they do running NOOPs. This is especially relevant to laptops that require fans to cool the CPU when it is under high load. Running a task at 100% for a short time can use far more energy than running a task at 25% for four times as long.
Imagine you are writing a background task that is designed to index files periodically in the background. Should the indexing task use as much of the CPU as it can at a lower priority, or throttle itself to 25% and take as long as it needs? Well, if it were to consume 100% of the CPU on a laptop, the CPU would heat up, the fans would kick in, and the battery would drain fairly quickly, and the user would get annoyed. If the indexing service throttled itself, the laptop may be able to run with completely passive cooling at a very low cpu clock speed and voltage.
Incidentally, the Windows Indexing Service now throttles itself in newer versions of Windows, which it never did in older versions. For an example of a service that still doesn't throttle itself and frequently annoys people, see Windows Installer Module.
An example of how to throttle part of your application internally in C#:
public void ThrottledLoop(Action action, int cpuPercentageLimit) {
Stopwatch stopwatch = new Stopwatch();
while(true) {
stopwatch.Reset();
stopwatch.Start();
long actionStart = stopwatch.ElapsedTicks;
action.Invoke();
long actionEnd = stopwatch.ElapsedTicks;
long actionDuration = actionEnd - actionStart;
long relativeWaitTime = (int)(
(1/(double)cpuPercentageLimit) * actionDuration);
Thread.Sleep((int)((relativeWaitTime / (double)Stopwatch.Frequency) * 1000));
}
}
First of all, I agree with Ryan that the question is perfectly valid and there are cases where thread priorities are not at all sufficient. The other answers appear highly theoretical and of no practical use in situations where the application is properly designed but still needs to be throttled. Ryan offers a simple solution for cases in which a relatively short task is performed in high frequency. There are cases, however, when the task takes a very long time (say a minute or so) and you cannot or don't want to break it into smaller chunks between which you can do the throttling. For these cases the following solution might be helpful:
Rather that implementing throttling into the business code, you can design the algorithm itself to work at full steam and simply throttle the thread which runs the operation "from the outside". The general approach is the same as in Ryan's answer: Calculate a suspension time based on the current usage and suspend the thread for this timespan before resuming it again. Given a process which you want to throttle, this is the logic:
public static class ProcessManager
{
[Flags]
public enum ThreadAccess : int
{
TERMINATE = (0x0001),
SUSPEND_RESUME = (0x0002),
GET_CONTEXT = (0x0008),
SET_CONTEXT = (0x0010),
SET_INFORMATION = (0x0020),
QUERY_INFORMATION = (0x0040),
SET_THREAD_TOKEN = (0x0080),
IMPERSONATE = (0x0100),
DIRECT_IMPERSONATION = (0x0200)
}
[DllImport("kernel32.dll")]
static extern IntPtr OpenThread(ThreadAccess dwDesiredAccess, bool bInheritHandle, uint dwThreadId);
[DllImport("kernel32.dll")]
static extern uint SuspendThread(IntPtr hThread);
[DllImport("kernel32.dll")]
static extern int ResumeThread(IntPtr hThread);
[DllImport("kernel32.dll")]
static extern int CloseHandle(IntPtr hThread);
public static void ThrottleProcess(int processId, double limit)
{
var process = Process.GetProcessById(processId);
var processName = process.ProcessName;
var p = new PerformanceCounter("Process", "% Processor Time", processName);
while (true)
{
var interval = 100;
Thread.Sleep(interval);
var currentUsage = p.NextValue() / Environment.ProcessorCount;
if (currentUsage < limit) continue;
var suspensionTime = (currentUsage-limit) / currentUsage * interval;
SuspendProcess(processId);
Thread.Sleep((int)suspensionTime);
ResumeProcess(processId);
}
}
private static void SuspendProcess(int pid)
{
var process = Process.GetProcessById(pid);
if (process.ProcessName == string.Empty)
return;
foreach (ProcessThread pT in process.Threads)
{
IntPtr pOpenThread = OpenThread(ThreadAccess.SUSPEND_RESUME, false, (uint)pT.Id);
if (pOpenThread == IntPtr.Zero)
{
continue;
}
SuspendThread(pOpenThread);
CloseHandle(pOpenThread);
}
}
private static void ResumeProcess(int pid)
{
var process = Process.GetProcessById(pid);
if (process.ProcessName == string.Empty)
return;
foreach (ProcessThread pT in process.Threads)
{
IntPtr pOpenThread = OpenThread(ThreadAccess.SUSPEND_RESUME, false, (uint)pT.Id);
if (pOpenThread == IntPtr.Zero)
{
continue;
}
var suspendCount = 0;
do
{
suspendCount = ResumeThread(pOpenThread);
} while (suspendCount > 0);
CloseHandle(pOpenThread);
}
}
}
The benefit of this solution is that the checking interval becomes independent of the duration of your "long running task". Furthermore, business logic and throttling logic are separated. The suspense/resume code is inspired by this thread. Please note that disposal and ending the throttling needs to be implemented in the solution above, it is not production code.
That's not your concern... It's the job of the operating system to distribute processor time between running processes. If you'd like to give other processes first crack at getting their stuff done, then simply reduce the priority of your own process by modifying the Process.PriorityClass value for it.
See also: Windows Equivalent of ‘nice’
You could write a Governor class that throttles the CPU usage. This class would contain a utility method that should be called on a regular basis (e.g. calling this utility function within a while loop of your function) by your CPU bound function. The governor would check if the amount of time elapsed exceeded a particular threshold, and then sleep for a period of time so as to not consume all the CPU.
Here's a simple Java implementation off the top of my head (just so you get the idea) that will throttle the CPU usage to 50% if you have a single threaded CPU bound function.
public class Governor
{
long start_time;
public Governor()
{
this.start_time = System.currentTimeMillis();
}
public void throttle()
{
long time_elapsed = System.currentTimeMillis() - this.start_time;
if (time_elapsed > 100) //throttle whenever at least a 100 millis of work has been done
{
try { Thread.sleep(time_elapsed); } catch (InterruptedExceptione ie) {} //sleep the same amount of time
this.start_time = System.currentTimeMillis(); //reset after sleeping.
}
}
}
Your CPU bound function would instantiate a Governor, and then just call throttle on a regular basis within the function.
Thank all of you for answering. I've been working on this and the exe it runs for a few hours and want to share to help others. I wrote a class I'm going to set and forget in a WPF app that'll encrypt and push data to the cloud, but I couldn't ever have it ever interfere with the timing of the WPF app and what the WPF app needs in the way of resources, which I am also going to add a flag to disable when the WPF app is in it's highest resource consumption state. I've already highly threaded this WPF with the TPL. This solution has both the priority set of the process
myProcess.PriorityClass = ProcessPriorityClass.Idle;
and the CPU percentage limited.
then in my mainDisplay.xaml.cs I'll use
ProcessManagement.StartProcess(5);
in MainWindow()
And there is no window popping up when that exe is run
RedirectStandardOutput = true,
UseShellExecute = false,
CreateNoWindow = true
in the object initalizer
internal class ProcessManagement
{
private static int CpuPercentageLimit { get; set; }
public static void StartProcess(int cpuPercent)
{
CpuPercentageLimit = cpuPercent;
var stopwatch = new Stopwatch();
while (true)
{
stopwatch.Reset();
stopwatch.Start();
var actionStart = stopwatch.ElapsedTicks;
try
{
var myProcess = new Process
{
StartInfo =
{
FileName = #"D:\\Source\\ExeProgram\\ExeProgram\\bin\\Debug\\ExeProgram.exe",
RedirectStandardOutput = true,
UseShellExecute = false,
CreateNoWindow = true
}
};
myProcess.Start();
myProcess.PriorityClass = ProcessPriorityClass.Idle;
myProcess.Refresh();
myProcess.WaitForExit();
var actionEnd = stopwatch.ElapsedTicks;
var actionDuration = actionEnd - actionStart;
long relativeWaitTime = (int)((1 / (double)CpuPercentageLimit) * actionDuration);
var sleepTime = (int)((relativeWaitTime / (double)Stopwatch.Frequency) * 1000);
Thread.Sleep(sleepTime);
myProcess.Close();
}
catch (Exception e)
{
// ignored
}
}
}
}
In my application, there is ample time, like 24/7/365, to upload lots of data, including thousands of images, but the UI also needs to stay active when used and when the system runs, nothing else can be running.
If you have a multi core processor, you can set the Affinity on each process to only use which cores you want it to use. This is the closest method I know of. But it will only allow you to assign percentages that are a factor of 50% on a dual core, and 25% on a quad core.
You can run your program in a thread with a lower threadpriority, the rest is up to your operating system. Having a process eat up 100% of your CPU is not bad. My SETI is usually taking up all my remaining CPU time without bothering my other programs. It only gets a problem when your thread gets priority over more important programs.
According to MSDN, you can only set a thread priority, i.e.
var t1 = new Thread(() => doSomething());
t1.Priority = ThreadPriority.BelowNormal;
t1.Start();
where doSomething is the function you want to create a thead for. The priority can be one of the ThreadPriority enumeration members Lowest, BelowNormal, Normal, AboveNormal, Highest - for a description see the MSDN link above. Priority Normal is the default.
Note that CPU usage also depends on how many cores and logical processors your physical CPU has *) - and how the threads and processes are assigned to those cores (the assignment to a dedicated processor is called "processor affinity" - if you want to know more about that, see this StackOverflow question).
*) To find that out, open the task manager (via Ctrl+Alt+Delete - select "task manager"), go to Performance and select CPU there: Below the utilization graph you can see "Cores" and "Logical processors". A core is a physical unit built into the CPU, while a logical processor is just an abstraction, which means the more cores your CPU consists of, the faster it can process parallel tasks.
I honestly think rather than worry about trying to limit CPU utilization by your app, you should focus more of your energies on profiling the application to uncover and correct bottlenecks and inefficiencies that may exist.
If you code is running at all, it is at 100%
I suppose slipping in some sleeps might have an effect.
I have to wonder about that 2-3 minute figure. I've seen it too, and I suppose it's loading and initializing lots of stuff I probably don't really need.
This is something I have come across a lot of times with complex integrations (for example a daily update of products, stock and pricing on an ecomm system).
Writing the integrations as efficiently as possible is always good, using DB server power where you can instead for iterating objects in the code but at the end of the day these things WILL take up processor time and you may want to run them on a server that is doing other things too.
Sharing is caring ;-)
A good approach to avoid coding a Governor or Throttling mechanism is to use the power of a web server. Expose the integration as a "local API call" and run it in IIS (for instance). There you have multiple throttling options and affinity masks you can apply to the application pool. This can then be easily adjusted "on the fly" to give a good balance and monitor closely.
If there is no other task running, is it wrong for your app to use all the cpu capacity that is available? It is available, as in it's there and it is free to use. So use it!
If you somehow limit the cpu usage of your task, it will take longer to complete. But it will still take the same number of cpu cycles, so you gain nothing. You just slow down your application.
Don't do it. Don't even try it. There's no reason why you should.
I think what you need to do is to understand the performance problem in your application instead of trying to put a cap on the CPU usage.
You Can use Visual Studio Profiler to see why you application is taking 100% CPU for 2-3 minutes in the first place. This should reveal the hot spot in your app, and then you can be able to address this issue.
If you are asking in general regarding how to do resource throttling in windows, then you can look at the "Task" objects, Job objects allows you to set limits such as Working set, process priority...etc.
You can check out the Job objects documentation here
http://msdn.microsoft.com/en-ca/library/ms684161(VS.85).aspx
Hope this helps.
Thanks