Linux swap space mini-HOWTO versions: 2002-Oct-25, 2002-Nov-01,04,07,08,10,11,12,17 2003-Jun-13 Randy Dunlap <rddunlap AT osdl.

org> ====================================================================== Copyright (c) 2002-2003 Randy Dunlap. This document may be distributed only subject to the terms and conditions set forth in the LDPL (Linux Documentation Project License) at "http://www.tldp.org/COPYRIGHT.html". 1. Introduction Linux uses swap space as "extra" memory for pages of application memory that are not being actively used by the application but have been modified (written to). The swap space size plus RAM size is the total amount of virtual memory for the system. When most of the system's real memory is in use, and there is a need for more, some data will be moved into swap to free real RAM memory for use by applications or for kernel use, such as for driver buffers, files, or network packets. This is called swapping out. When the data that is in swap space needs to be used it is swapped back in from swap space. The rate at which data is swapped to and from one or more swap spaces can be monitored with the 'vmstat' command's swap-in (si) and swap-out (so) columns. All of this is triggered by a high demand for memory. That can include running many/large programs, but even file or network I/O require allocating memory and can trigger the kernel to scavenge for more memory, which can trigger swapouts. Linux kernel code and data are not swappable and are never moved to swap. User code never needs to be written to swap space because it already exists on disk and can be read in from there if it is required again. User data can be written to swap space and read back in when needed. A possible alternative to swap space that is allocated statically is "swapd", a "dynamic swapping manager for Linux." It is available from URL::http://ftp.linux.hr/pub/swapd/ . One person reports using swapd exclusively on several Linux systems, without static swap space. He reports that it works reliably and consumes almost no resources. swapd is tuned by editing the "swapd.conf" file. Example file: memlimit 131072 pause 10000 swapsize 65536 timeout 8640 2. How much swap space to use That's tough to say. If your system (and applications) are using lots of memory (more than the amount of physical memory in your system), then you needs lots of swap space. Of course you may decide that your system's environment shouldn't support (use) swap space, either because of performance issues, or you don't have any [fast] storage devices available for swapping, or

because you have large RAM and will never use it all up, so you should never see a need for swapping. Typical total swap space size is equal to your system's RAM size. Early versions of Linux 2.4 (through 2.4.9) liked swap space size of 2 * system RAM size, but that's no longer the case. One negative about providing too much swap space is that errant programs may run for an extended time and use up RAM and swap space. This probably causes system slowness due to excessive swapping, and could lead to the wrong process being killed by the OOM killer when the kernel runs out of memory (see the OOM Killer section below). This failure can be prevented by not defining huge amounts of swap space unless you know that it is required. In other words, swap space can be an area for system slowness, so allocate it wisely. Swap space usage can be monitored with the 'free' command or by reading the /proc/swaps and /proc/meminfo files. The install programs of most Linux distributions do a reasonable job of allocating swap space for small to medium systems. 3. Swap space limits [Historical:] Linux 2.4.10 and later, and Linux 2.5 support any combination of swap files or swap devices to a maximum number of 32 of them. Prior to Linux 2.4.10, the limit was any combination of 8 swap files or swap devices. On x86 architecture systems, each of these swap areas has a limit of 2 GiB. The current swap space limits are: (1) both Linux 2.4.x and 2.5.x kernels support swap spaces of up to 64 GiB in size (2) Linux 2.4.x supports 32 swap spaces and 2.5.x supports 32 swap spaces (3) mkswap(8) needs fixes for creating swap spaces larger than 2 GiB merged back to the util-linux package from one or more distro packages; this work is pending. Some distro versions of mkswap(8) support more than 2 GiB swap spaces. 4. OOM (Out-of-Memory) killer If the Linux VM can't find memory to allocate when it's needed, it puts in-use user data pages on the swap-out queue, to be swapped out. If the VM can't allocate memory and can't swap out in-use memory, the Out-of-memory killer may begin killing current userspace processes. The rationale for OOM killing is described in the Linux-MM docs [2]. 5. Setting up swap space Managing swap with Linux is easy and flexible. Linux's swap spaces can be turned on and off without a reboot and even while they are in use. Swap space can be allocated in its own partition (maybe safer) or in files in a live filesystem. Logical Volumes can be used as swap devices.

There are two (2) versions (formats) of swap space supported by Linux 2.1 (development kernel) and later (through 2.4). Linux 2.0 requires "version 0" swap space. Linux 2.1.117 and later support both version 0 and version 1 swap space. Linux 2.5 only supports version 1 swap space. 'mkswap' can format swap space in either format. See 'man mkswap' for details. To use a file in a filesystem for swap space, choose the swap space size that you want for this swap area (up to 2 GiB on x86) and then enter (as described in 'man mkswap'): dd bs=1024 count=1M if=/dev/zero of=/path/to/swapfile.n This writes a 1 GiB file (1 MiB * 1024) to "/path/to/swapfile.n". You can do this multiple times to use more swap files on large-memory systems. Swap files cannot be sparse; they must be fully allocated before using them. Of course, you can also allocate swap space in its own partition(s). This usually requires a bit more planning of disk space allocation. Swap space in partitions or in files is described in the /etc/fstab file ('man 5 fstab'). Swap spaces can be given different priorities ('man 2 swapon') that determine how the swap spaces will be used. Higher priority swap spaces will be exhausted first and swap spaces with equal priorities will be striped. The priority can be specified at swap space activation with the 'swapon' command or in the /etc/fstab file. For example: # swap space in device partitions /dev/hda2 none swap pri=5,defaults /dev/hde2 none swap pri=5,defaults # swap space in swap files /work/swapfile.1 none swap pri=5,defaults /work/swapfile.2 none swap pri=5,defaults 0 0 0 0 0 0 0 0

After swap space is allocated, it must be formatted (initialized) with swap header information. This only needs to be done one time. mkswap /dev/hda2 mkswap /dev/hdb1 mkswap /work/swapfile.1 mkswap /work/swapfile.2 Swap space is enabled during the system init procedure (scripts) by running "swapon" ('man swapon'). Swap space that is in filesystems is also enabled at this time if the filesystem volumes are already mounted. If you add some swap space after system init, you can use the 'swapon' command to begin using it. For example: swapon /dev/hdf1 swapon /work/swapfile.3 Using multiple swap partitions or swap files on different disks has an effect of "striping" swap requests across those disks, similar to RAID-0 striping. It's generally a good idea to have multiple swap areas (of the same priority) if you have multiple devices on independent channels. This lets the kernel swap in parallel. It can also be a good idea to keep swap on less-used devices/channels, so heavy non-swap I/O is not

hindered by swap I/O. In Linux 2.4 and earlier, swap files are less robust than swap devices (partitions) because of the need to read metadata from the filesystem for swap files, making them prone to OOM deadlocks when allocating pages and buffer_heads to use in swapping. This has changed in Linux 2.5. Using a swap file has no disadvantage compared to swapping to a device (partition). The kernel doesn't need to allocate any memory to get a swapcache page to disk. This is interesting because swap files are much easier to administer (add, remove, resize) and easier to stripe. It is now feasible (in the Linux 2.5.40 timeframe) to eliminate swap devices (partitions) completely and not be penalized in performance. If you have two disks then it is very sensible to create a swap file on each one and perform an equal-priority stripe between them. This will give the best raw swap I/O bandwidth, but it could cause additional seeking between swap and regular file data. Dedicating an entire disk to swap will obviously reduce the seeking problem. But really, if your application is dependent on swap performance, you need more RAM. Swap should be viewed as a lightweight background optimization to make unused pages available for other work, rather than as a cure for an underprovisioned machine. 6. Disabling swap space Swap space is usually turned off (disabled) by system shutdown by using the 'swapoff' command ('man swapoff'). swapoff /work/swapfile.1 swapoff /work/swapfile.2 swapoff /dev/hdb1 swapoff /dev/hda2 or by using 'swapoff -a', which disables all swap partitions that are listed in the /etc/fstab file (this does not apply to swap files, only to swap partitions). 7. Other uses of swap space Swap space is used by the "tmpfs" filesystem as overflow storage whenever cache memory usage needs to be reduced. tmpfs also supports limit checking (maximum space usage). See its documentation in the kernel source tree in the file "linux/Documentation/filesystems/tmpfs.txt". tmpfs is also used to support "shmfs". shmfs is a filesystem used to support POSIX shared memory. glibc 2.2 and later expect tmpfs to be mounted at /dev/shm for POSIX shared memory open and unlink. With both tmpfs and shmfs you probably want to limit how big they can grow. Use a mount option of size=xxxM to do so. For example, in /etc/fstab: tmpfs shmfs /tmp /dev/shm tmpfs tmpfs size=750M,mode=2777 defaults 0 0 0 0

Swap devices (partitions) can also be used for "software suspend" (swsusp) in Linux 2.5 and as a destination for saving a kernel crash dump in versions of Linux which support these features. Using a swap file(s) also allows you to share swap space with other OSes as described in the Linux Swap Space Mini-HOWTO [3] and on the linux-kernel mailing list [4]. CREDITS Thanks to everyone who commented on initial and updated versions of this mini HOWTO, including but not limited to: Andries Brouwer, Bernd Eckenfels, Mark Hahn, Troels Walsted Hansen, Bill Irwin, Dave Jones, Pavel Machek, Andrew Morton, Vince Mulhollon, Wladimir Mutel, Ha Shao, Shane Shrybman, and Ed Tomlinson. Some of their comments are used or quoted here. TRANSLATIONS The original English version of this mini-HOWTO is available at "http://www.xenotime.net/linux/swap-mini-howto.txt". A Hungarian translation by Gabor Micsko is available at "http://www.hup.hu/modules.php?name=News&file=article&sid=1976". A Polish translation by Jacek Politowski is available at "http://jacek.rallypl.eu.org/tlumaczenia/swap-mini-howto.txt". REFERENCES 1. Joe Knapka, "Outline of the Linux Memory Management System". URL::http://home.earthlink.net/~jknapka/linux-mm/vmoutline.html 2. Rik van Riel: "Linux-MM docs: the OOM killer" URL::http://linux-mm.org/docs/oom-killer.shtml 3. Swap-Space, Linux Swap Space Mini-HOWTO. Updated: July 2002. How to share your Linux swap partition with Windows. URL::http://www.tldp.org/HOWTO/mini/Swap-Space.html 4. URL::http://marc.theaimsgroup.com/?l=linux-kernel&m=103623636324081&w=2 ### end ###