How to measure (sysstat/pidstat) and limit (cpulimit/nice/taskset) CPU usage of a Linux process

# install
$ sudo apt-get install sysstat | sudo yum install sysstat

pidstat [OPTIONS] [ interval [ count ] ]
'interval' amount of time in seconds between each report (defaults to 0, from boot)
'count' number of reports generated at 'interval' seconds apart
'-p pid' by pid or ALL
'-C name' by process name
'-d' I/O stats
'-r' paging activity
'-l' show command line and args
'-T CHILD|TASKS' also shows child/tasks processes

# measure the average CPU usage of a Linux process
$ pidstat -p 13203
02:09:02 PM PID %usr %system %guest %CPU CPU Command
02:09:07 PM 9280 15.40 11.40 0.00 26.80 0 tincd
...
# same for all processes
$ pidstat -p ALL | wc -l
# or by process name
$ pidstat -C "mysql"

# CPU Usage of ALL CPUs every 1 secs for 3 secs
$ sar -u 1 3
01:27:32 PM CPU %user %nice %system %iowait %steal %idle
01:27:33 PM all 0.00 0.00 0.00 0.00 0.00 100.00
...

# same as above but for individual CPU/Core
$ sar -P 1 1 3

# memory free/used
$ sar -r 1 3
07:28:06 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact
07:28:07 AM 6209248 2097432 25.25 189024 1796544 141372 0.85 1921060 88204
...

# swap space used
$ sar -S 1 3
07:31:06 AM kbswpfree kbswpused %swpused kbswpcad %swpcad
07:31:07 AM 8385920 0 0.00 0 0.00
...

# overall I/O activities
$ sar -b 1 3
01:56:28 PM tps rtps wtps bread/s bwrtn/s
01:56:29 PM 346.00 264.00 82.00 2208.00 768.00
...

# same as above but by individual block device
$ sar -p -d 1 1
01:59:45 PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
01:59:46 PM sda 1.01 0.00 0.00 0.00 0.00 4.00 1.00 0.10
01:59:46 PM sda1 1.01 0.00 0.00 0.00 0.00 4.00 1.00 0.10
...

# run queue and load average
$ sar -q 1 3
06:28:53 AM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked
06:28:54 AM 0 230 2.00 3.00 5.00 0
...

# network statistics
$ sar -n DEV
01:11:13 PM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s
01:11:14 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:11:14 PM eth0 342.57 342.57 93923.76 141773.27 0.00 0.00 0.00
...

# sar data using log file and start time
$ sar -q -f /var/log/sa/sa23 -s 10:00:01

# monitor disk I/O of individual disks
$ iostat
avg-cpu: %user %nice %system %iowait %steal %idle
2.11 0.08 3.20 3.65 0.00 90.96
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 55.77 1403.98 2138.71 4139635 6306020

# generate sar report in CSV
$ sadf -d /var/log/sa/sa29 -- -B
localhost;601;2006-03-29 19:10:00 UTC;63.42;30.71;267.35;0.45
localhost;600;2006-03-29 19:20:00 UTC;27.25;23.02;281.88;0.26

from sar examples, pidstat examples and sysstat tutorial

  • Limit CPU usage per process. Use cpulimit program runs as a user-space daemon which sends SIGSTOP and SIGCONT signals to a specified Linux process to adjust its CPU usage dynamically. CPU % is per-CPU on a SMP system.
$ apt-get install cpulimit | yum install cpulimit (needs EPEL)

# limit by pid or process name
$ sudo cpulimit -p 8645 -l 10
$ sudo cpulimit -e -l 20

from @Xmodule and @nixCraft

  • Modify scheduling priority: Use nice command to run a program with modified scheduling priority / nicenesses. Nicenesses range at least from -20 (resulting in the most favorable scheduling) through 19 (the least favorable). The default behavior is to increase the niceness by 10.
    See also VeryNice, a Arch daemon to dynamically adjust nice levels.
$ /bin/nice -n 3 <command></command>

# change the priority of process ID’s 1024 and 66, and all processes owned by users daemon and root
$ renice +1 1024 -u daemon root -p 66

from nice/renice.

  • taskset@man retrieve or set a process’s CPU affinity
'-p' select process by pid
'-a' applying the user requested changes to all of the threads
'-c' CPUs to bind, eg: '0,1,2'

$ taskset -pac 0,1,2 [pid]

from How to Change CPU Affinity of CPU Intensive Multithreaded Applications

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s