Unix Tip: Fragmentation and Unix file systems

Send in your Unix questions today!


See additional Unix tips and tricks

People who have had bad experiences on Windows disks that performed poorly after they had become very fragmented often wonder if they need to apply some kind of periodic analysis and defragmentation on their Unix systems as they transition into a Unix admin or user role. For the most part, the answer is no. Most Unix file systems don't become fragmented the way FAT and NTFS file systems do. But, rather than give a quick yes/no answer, let's take a look at what fragmentation, and why Unix file systems are not as prone to fragmentation as their Windows cousins and what you need to do when a Unix file system is fragmented.

What Causes Fragmentation?

Fragmentation comes about when a system cannot or will not allocate enough contiguous disk space to store an entire file in a single location on a disk. Instead, the file ends up being broken into a number of pieces that are written to various locations on the disk and the file system must then maintain some kind of file system structure that keeps track of where all the related file pieces are stored.

When a file is written to disk, it isn't always possible to write it to the disk in consecutive blocks. The disk may be too full to provide a single location that is large enough to accommodate the file. It takes longer to read a file when it is not stored in consecutive blocks since the disk's read-write head has to move more to gather the individual pieces of the file. While fragmentation is a performance issue, it is less of a problem on a system with an affective buffer cache with read-ahead. On such file systems, the file system may be fetching portions of a file while the user or the application is still occupied with the first few blocks.

Fragmentation on Windows

To see fragmentation on a Windows 2000 system, open Programs -> Accessories -> System Tools -> Disk Defragmenter. Then, click on the Analyze button. This will bring up a graphical display in which the disk areas containing fragmented files are shown in red and, optionally (i.e., if you click on View Report) a list of fragmented files, their sizes and number of fragments. Depending on the degree of fragmentation, the tool will recommend that you defragment the disk. You can then click on Defragment to assemble the files in a non-fragmented fashion. This operation may take quite a while to run, but may make a noticeable difference in your disk's performance.

Fragmentation is more of a problem on FAT file systems than NTFS, largely because the FAT32 file system predates many of the innovations in file system design that have made them work more efficiently.

Fragmentation on Unix

Most modern Unix file systems attempt to fragmentation at a minimum, though they do this in various ways. The ufs file systems used by Solaris and nearly all BSD variants of Unix attempt to keep fragmentation to a minimum by writing related data blocks within the same cylinder group. This reduces seek time when the files are accessed. And, while a large block size is used to improve throughput, a smaller unit of storage -- referred to as a fragment -- is used to store portions of files that don't require a full block. This reduces the wasted space within the file system and one variety of fragmentation sometimes referred to as "internal fragmentation".

The ext2 and ext3 file systems most often used on Linux systems also attempt to keep fragmentation at a minimum. These file systems keep all blocks in a file close together. How they do this is by preallocating disk data blocks to regular files before they are actually used. Because of this, when a file increases in size, several adjacent blocks are already reserved, reducing file fragmentation. It is, therefore, seldom necessary to analyze the amount of fragmentation on a Linux system, never mind actually run a defragment command. An exception exists for files that are constantly appended to as the reserved blocks will only last so long.

Using fsck

To see fragmentation on a Unix system, you would generally use a command such as fsck that reports the percentage of fragmentation on the file system. An example from a fsck output from a Solaris UFS file system is shown here:

** /dev/rdsk/c1d0s4
** Currently Mounted on /mail
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
77 files, 1004062 used, 2094449 free (81 frags, 261796 blocks,  0.0%
fragmentation)

You will often see the result of fsck operations, such as the message shown below from a Linux system, on system reboot.

/dev/rwd0e: 22096 files, 299456 used, 506110 free (12078 frags, 61754 blocks,
1.5% fragmentation)

Defragmentation

While you're unlikely to see many Unix systems with fragmentation reaching higher than 5%, it's good to know what you can do to defragment a file system if and when you run into this situation. The classical method is to back up the file system with a program such as dump or ufsdump, rebuild the file system with a command such as newfs or mkfs, and then reload the file system from the backup. On a large file system, this operation can take several hours to run.

Larger file systems and those that have plenty of free space are less likely to incur any significant amount of fragmentation. Planning for file system growth when you install a system will, therefore, keep your file systems performing when for a long time.