一.前言

linux下有大量的系统命令vmstat、iostat等可以反映系统的总体io情况，但是不能监测具体进程的io情况，本文将介绍两种方法：

1.如果内核版本大于2.6.20，通过 /proc/pid/io 便可以获取进程的io信息。

2.通过echo 1 > /proc/sys/vm/block_dump ，来把 block 读写(WRITE/READ/DIRTY)状况 dump 到日志里，通过 dmesg 命令来查看。

二.  /proc/pid/io

(详细解释：http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/proc.txt;hb=HEAD)

示例：

test:/tmp# dd if=/dev/zero of=/tmp/test.dat &

[1] 3828

test:/tmp # cat /proc/3828/io

rchar: 323934931           // 读出的总字节数，read或者pread()中的长度参数总和(pagecache中统计而来，不代表实际磁盘的读入)

wchar: 323929600           // 写入的总字节数，write或者pwrite中的长度参数总和

syscr: 632687              // read()或者pread()总的调用次数

syscw: 632675              // write()或者pwrite()总的调用次数

read_bytes: 0              // 实际从磁盘中读取的字节总数   (这里if=/dev/zero 所以没有实际的读入字节数)

write_bytes: 323932160     // 实际写入到磁盘中的字节总数

cancelled_write_bytes: 0   // 由于截断pagecache导致应该发生而没有发生的写入字节数(可能为负数)

检测工具dstat ( ) 的top_io 模块就是利用此功能实现。dstat的详细说明可以到这里

三.block_dump具体操作步骤是：#sysctl vm.block_dump=1

or

# echo 1 > /proc/sys/vm/block_dump

然后就可以通过 dmesg 就可以观察到各个进程 IO 活动的状况了：

#dmesg -c

kjournald(542): WRITE block 222528 on dm-0

kjournald(542): WRITE block 222552 on dm-0

bash(18498): dirtied inode 5892488 (ld-linux-x86-64.so.2) on dm-0

bash(18498): dirtied inode 5892482 (ld-2.5.so) on dm-0

dmesg(18498): dirtied inode 11262038 (ld.so.cache) on dm-0

dmesg(18498): dirtied inode 5892496 (libc.so.6) on dm-0

dmesg(18498): dirtied inode 5892489 (libc-2.5.so) on dm-0

通过一些脚本分析这些数据就可以得到各个进程的IO情况。下文便描述了这种方法(不同版本的linux下这里dmesg日志格式可能不同)

How to find per-process I/O statistics on Linux

Newer Linux kernels have per-process I/O accounting and you can use the iotop tool to find out what’s performing I/O, but in many cases I’m trying to find the source of an I/O problem in an older kernel. I found sort of a hack-ish way to do that today, while trying to figure out why a system was basically unresponsive.

I founda post on Stack Overflowthat showed a way you can get per process I/O statistics from the kernel even in older kernels. I adapted this to my needs, and wrote a little script.

Here’s how you use it. First, get it:

wget

Then turn on kernel messages about I/O:

echo 1 > /proc/sys/vm/block_dump

This makes the kernel start writing messages about every I/O operation that takes place. Now all you have to do is get those messages and feed them into my script:

while true; do sleep 1; dmesg -c; done | perl iodump

Wait a little while, then cancel the script. The results should look something like the following:

:~# while true; do sleep 1; dmesg -c; done | perl iodump

^C# Caught SIGINT.

TASK                   PID      TOTAL       READ      WRITE      DIRTY DEVICES

firefox               4450       4538        251       4287          0 sda4, sda3

kjournald             2100        551          0        551          0 sda4

firefox              28452        185        185          0          0 sda4

kjournald              782         59          0         59          0 sda3

pdflush                 31         30          0         30          0 sda4, sda3

syslogd               2485          2          0          2          0 sda3

firefox              28414          2          2          0          0 sda4, sda3

firefox              28413          1          1          0          0 sda4

firefox              28410          1          1          0          0 sda4

firefox              28307          1          1          0          0 sda4

firefox              28451          1          1          0          0 sda4

I deliberately generated a bunch

of I/O by deleting my Firefox history and cache.

这里还有个网上找的脚本分析dmesg 的输出的：

#!/bin/bash

#/etc/init.d/rsyslog stop

/etc/init.d/syslog stop

echo 1 > /proc/sys/vm/block_dump

dmesg -c >/dev/null

sleep 60

#dmesg | awk '/(READ|WRITE|dirtied)/ {process[$2]++} END {for (x in process) {print process[x], x;} }' |sort -nr |awk '{print $2" "$1}' | head -n 10

dmesg | awk '/WRITE/ {process[$2]++} END {for (x in process) {print process[x], x;} }' |sort -nr |awk '{print $2" "$1}' | head -n 10

echo 0 > /proc/sys/vm/block_dump

/etc/init.d/syslog start

#/etc/init.d/rsyslog start