Wednesday, 4 September 2013

Linux, 'du' and 'df' tools report different space utilization

The 'df' command reports how many disk blocks are used, whilst 'du' traverses the filesystem and reports the actual number of blocks used (directory by directory), including any relating to files in use by processes.
In most cases, space utilisation values returned from 'df' and 'du' will be consistent. However, the potential exists for a running process to delete a large file, say. In this instance, according to 'du', the large file no longer exists, so the blocks associated with the deleted file are not reported. With the process still running, and with an open file descriptor still held against the deleted file, 'df' continues to track and report all disk blocks used, including those associated with the deleted (phantom) file. In this situation, the disk space associated with the deleted file will only be relinquished back to the system when the process completely releases the deleted file's descriptor or the process terminates (either gracefully or killed).
The solution is to identify and stop (or kill) the process that continues to hold a file descriptor open for the deleted file. To do so, run the lsof command (/usr/sbin/lsof | grep deleted) as root to identify the holding process, for example:

# lsof | grep deleted
COMMAND     PID   USER    FD  TYPE  DEVICE      SIZE       NODE NAME
cannaserv  3825  canna    0u   CHR   136,0                    2 /dev/pts/0 (deleted)
vmware     4295   root    6u   REG   253,0      6770   13074503 /tmp/vmware-root/ui-4295.log (deleted)
vmware-re  4316   root    6u   REG   253,0      6770   13074503 /tmp/vmware-root/ui-4295.log (deleted)
vmnet-nat  4448   root    0u   CHR   136,0                    2 /dev/pts/0 (deleted)
vmware-se  4454   root    0u   CHR   136,0                    2 /dev/pts/0 (deleted)
gdm-binar  4506   root    0u   CHR   136,0                    2 /dev/pts/0 (deleted)
gconfd-2   5392   root   12wW  REG   253,0       609   13090818 /tmp/gconfd-root/lock/0t1188207163ut519551u0p5392r346479926k3219784492 (deleted)
vmware-vm  5822   root   57u   REG   253,0   6520832   13074477 /tmp/vmware-root/ram0 (deleted)
vmware-vm 16487   root   57u   REG   253,0  11153408   13074520 /tmp/vmware-root/ram0 (deleted)
kdeinit   17991   root   17u   REG   253,0     26712   13074524 /tmp/kde-root/khtmlcacheM7jXYb.tmp (deleted)
kdeinit   17991   root   18u   REG   253,0      5631   13074501 /tmp/kde-root/khtmlcacheZlJmda.tmp (deleted)
kdeinit   17991   root   21u   REG   253,0     44718   13074514 /tmp/kde-root/khtmlcacheH5m4lc.tmp (deleted)

In the example above, the 7th column in the output denotes the size of deleted files (in bytes). The 9th column denotes which file remains held open. The 1st and second columns denotes the process and pid that still holds the open file descriptor.

No comments:

Post a Comment