Notes from the Linux Performance Monitoring talk at Velocity 2015 - Part 1, Part 2

http://www.brendangregg.com/linuxperf.html

USE

  • Utilization
  • Saturation
  • Errors

Observability Tools

  • atop (atop uses the linux kernel event interface rather than sampling on screen updates, so it is better for viewing systems affected by short-lived processes)
  • htop
  • vmstat -Sm 1
  • iostat -xmdz 1
  • mpstat -P ALL 1
  • free -m
  • sar -n DEV 1
  • strace -tttT # very disruptive of system performance, slows system significantly
  • tcpdump
  • pidstat -t 1
  • pidstat -d
  • swapon -s
  • lsof
  • sar -n TCP,ETCP,DEV 1
  • collectl
  • dstat
  • strace 2>&1 | head -n 100 # since there's no strace -c N
  • ss
  • iptraf
  • slabtop
  • pcstat
  • perf
  • tiptop
  • rdmsr
  • perf-tools/execsnoop

Benchmarking tools

  • unixbench
  • imbench
  • sysbench
  • lmbench
  • fio
  • pchar
  • iperf

Tuning tools

  • sysctl
  • ulimit
  • chcpu

Static tools

Tracing

  • ftrace
  • iosnoop
  • iolatency
  • opensnoop
  • tpoint
  • funccount
  • funcgraph
  • kprobe
  • bytehist
  • stap