dd (Unix)

From Wikipedia, the free encyclopedia

dd is a common Unix program whose primary purpose is the low-level copying and
conversion of raw data. dd is an application that will "convert and copy a file" [1]
according to the referenced manual page for Version 7 Unix and is most likely inspired
from DD found in IBM JCL, and the command's syntax is meant to be reminiscent of
this;[2] in JCL, "DD" stands for Data Description. dd is used to copy a specified number of
bytes or blocks, performing on-the-fly byte order conversions, as well as more esoteric
EBCDIC to ASCII conversions.[3] dd can also be used to copy regions of raw device files,
e.g. backing up the boot sector of a hard disk, or to read fixed amounts of data from
special files like /dev/zero or /dev/random.[4]

It is jokingly said to stand for "disk destroyer", "data destroyer", or "delete data", since,
being used for low-level operations on hard disks, a small mistake, such as reversing the
if and of parameters, can possibly result in the loss of some or all data on a disk.[3]

Contents
1 Usage
2 Output messages
3 ATA Disks over 128 GiB
4 Recovery-oriented variants of dd
5 See also
6 References
7 External links

Usage
The command line syntax of dd is significantly different from most other Unix programs,
and because of its ubiquity it is resistant to recent attempts to enforce a common syntax
for all command line tools. Generally, dd uses an option=value format, whereas most
Unix programs use either -option value or --option=value format. Also, dd's input is
specified using the "if" (input file) option, while most programs simply take the name by
itself. It is rumored to have been based on IBM's JCL, and though the syntax may have
been a joke,[2] there seems never to have been any effort to write a more Unix-like
replacement.

Example use of dd command to create an ISO disk image from a CD-ROM:

dd if=/dev/cdrom of=/home/sam/myCD.iso bs=2048 conv=sync

Note that an attempt to copy the entire disk image using cp may omit the final block if it
is an unexpected length; dd will always complete the copy if possible.

Using dd to wipe an entire disk with random data:

dd if=/dev/urandom of=/dev/hda

Using dd to duplicate one hard disk partition to another hard disk:

dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=noerror

The noerror conversion option means to keep going if there is an error (though a better
tool for this would be ddrescue).

Duplicate a disk partition as a disk image file on a remote machine over a secure ssh
connection:

dd if=/dev/sdb2 | ssh user@host "dd of=/home/user/partition.image"

Overwrite the first 512 bytes of a file with null bytes:

dd if=/dev/zero of=path/to/file bs=512 count=1 conv=notrunc

The notrunc conversion option means do not truncate the output file — that is, if the
output file already exists, just replace the specified bytes and leave the rest of the output
file alone. Without this option, dd would create an output file 512 bytes long.

To duplicate a disk partition as a disk image file on a different partition:

dd if=/dev/sdb2 of=/home/sam/partition.image bs=4096 conv=noerror

Create a 1 GiB file containing only zeros (bs=blocksize, count=number of blocks):

dd if=/dev/zero of=file1G.tmp bs=1M count=1024

To zero out a drive:

dd if=/dev/zero of=/dev/sda

To make sure that the drive is really zeroed out:

dd if=/dev/sda | hexdump -C | head

The output of this command will resemble the following if the drive is blank:

00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
201f78000
16841664+0 records in
16841664+0 records out
8622931968 bytes (8.6 GB) copied, 1247.05 s, 6.9 MB/s

If the drive is blank, one line of blank bytes will be printed, followed by a '*' signifying
repeated blank lines, followed by a line indicating the address of the line which ends the
repetition, followed by the statistics which are printed after the output. The numbers in
the statistics above are illustrative. If the drive is not entirely blank, there will be more
than one line of data output.

To duplicate the first 2 sectors of the floppy:

dd if=/dev/fd0 of=/home/sam/MBRboot.image bs=512 count=2

To create an image of the entire master boot record (including the partition table):

dd if=/dev/sda of=/home/sam/MBR.image bs=512 count=1

To create an image of only the boot code of the master boot record (without the partition
table):

dd if=/dev/sda of=/home/sam/MBR_boot.image bs=446 count=1

To make drive benchmark test and analyze read and write performance:

dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file

dd if=/home/sam/1Gb.file bs=64k | dd of=/dev/null

To make a file of 100 random bytes:

dd if=/dev/urandom of=/home/sam/myrandom bs=100 count=1

To convert a file to uppercase:

dd if=filename of=filename conv=ucase

To search the system memory:

dd if=/dev/mem | hexdump -C | grep 'some-string-of-words-in-the-file-you-forgot-to-save-before-you-hit-t

Image a partition to another machine:

On source machine:
dd if=/dev/hda bs=16065b | netcat < targethost-IP > 1234
On target machine:
netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

Sending a SIGINFO signal (or a USR1 signal on Linux) to a running `dd' process makes it
print I/O statistics to standard error and then resume copying:

$ dd if=/dev/zero of=/dev/null& pid=$!

$ kill -USR1 $pid
18335302+0 records in 18335302+0 records out 9387674624 bytes (9.4 GB) copied,
34.6279 seconds, 271 MB/s
Create a 1 GiB sparse file or resize an existing file to 1 GiB without overwriting:

dd if=/dev/zero of=mytestfile.out bs=1 count=0 seek=1G

Some implementations understand x as a multiplication operator in the block size and

count parameters:

dd bs=2x80x18b if=/dev/fd0 of=/tmp/floppy.image

where the "b" suffix indicates that the units are 512-byte blocks. Unix block devices use
this as their allocation unit by default.

For the value of bs field, following decimal number can be suffixed:

w means 2
b means 512
k means 1024
M specifies multiplication by 1024*1024
G specifies multiplication by 1024*1024*1024

Hence bs=2*80*18b means, 2*80*18*512=1474560 which is the exact size of 1440 KiB
floppy disk

To mount that image: mount -o loop floppy.image /mntpoint

Output messages
The GNU variant of dd as supplied with Linux does not describe the format of the
messages displayed on stdout on completion, however these are described by other
implementations e.g. that with BSD.

Each of the "Records in" and "Records out" lines shows the number of complete blocks
transferred + the number of partial blocks, e.g. because the physical medium ended
before a complete block was read.

ATA Disks over 128 GiB

Seagate documentation warns, "Certain disc utilities, such as DD, which depend on low-
level disc access may not support 48-bit LBAs until they are updated."[5] 48-bit LBA is
required for ATA harddrives over 128 GiB in size. However, in Linux, dd uses the kernel to
read or write to raw device files.[6] Support for 48-bit LBA was not present before version
2.4.23 of the kernel.[7][8]

Recovery-oriented variants of dd
Open Source unix-based programs for rescue include dd_rescue
(http://www.garloff.de/kurt/linux/ddrescue/) and dd_rhelp
(http://www.kalysto.org/utilities/dd_rhelp/index.en.html) , which work together, savehd7
(http://seed7.sourceforge.net/scrshots/savehd7.htm) , or GNU ddrescue
(http://www.gnu.org/software/ddrescue/ddrescue.html) .
Antonio Diaz Diaz (the developer of GNU ddrescue) compares[9] the variants of dd for the
task of rescuing:

The standard utility dd does a linear read of the drive, so it can take a long time
or even fry the drive without rescuing anything if the errors are at the
beginning of the drive. Kurt Garloff's dd_rescue does basically the same thing
as dd, only more efficiently. LAB Valentin's dd_rhelp is a complex shell script
that runs Garloff's dd_rescue many times, trying to be strategic about copying
the drive, but it is very inefficient.

dd_rhelp first extracts all the readable data, and saves it to a file, inserting zeros
where bytes cannot be read. Then it tries to re-read the invalid data and update this
file.
GNU ddrescue can be used to copy data directly to a new disk if needed, just like
Linux dd.

dd_rhelp or GNU ddrescue will yield a complete disk image, faster but possibly with some
errors. GNU ddrescue is generally much faster, as it is written entirely in C++, whereas
dd_rhelp is a shell script acting as a frontend to dd_rescue. Both dd_rhelp and GNU
ddrescue aim to copy data fast where there are no errors, then copy in smaller blocks
and with retries where there are errors. GNU ddrescue is easy to use with default options,
and can easily be downloaded and compiled on Linux-based Live CDs such as Knoppix,
and can be used with SystemRescueCD.

GNU ddrescue example [10]

# first, grab most of the error-free areas in a hurry:

ddrescue -n /dev/old_disk /dev/new_disk rescued.log
# then try to recover as much of the dicey areas as possible:
ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log

There are large differences in how disk errors are processed by various kernels.
Operating systems such as FreeBSD, NetBSD, OpenBSD, Solaris, as well as different Linux
kernels (i.e. hda vs. sda (<2.6.20)) behave differently. Also, Linux lacks "raw" disk devices
like *BSD has, which makes it less desirable for low-level data recovery. Non-raw devices
read larger blocks than requested, obscuring the actual location where the error
occurred. You may wish to use "dmesg |tail -n8" to see the error messages on the
console.

See also
List of Unix programs
Backup
Disk cloning
Disk image
Loopback

References
1. ^ Bell Laboratories. "dd man page"
 (http://www.orangetide.com/Unix/V7/usr/man/man1/dd.1) .
http://www.orangetide.com/Unix/V7/usr/man/man1/dd.1. Retrieved 2009-02-25.
2. ^ a b Eric S. Raymond. "dd" (http://www.catb.org/jargon/html/D/dd.html) .
http://www.catb.org/jargon/html/D/dd.html. Retrieved 2008-02-19.
3. ^ a b Sam Chessman. "How and when to use the dd command?"
(http://www.codecoffee.com/tipsforlinux/articles/036.html) . CodeCoffee.
http://www.codecoffee.com/tipsforlinux/articles/036.html. Retrieved 2008-02-19.
4. ^ "Dd - LQWiki" (http://wiki.linuxquestions.org/wiki/Dd) . LinuxQuestions.org.
http://wiki.linuxquestions.org/wiki/Dd. Retrieved 2008-02-19.
5. ^ Windows 137GB (128
(http://web.archive.org/web/20070316080228/http://www.seagate.com/support/kb/disc/tp/137gb.pdf)
GiB) Capacity Barrier - Seagate Technology (March 2003)
6. ^ This is verifiable with strace. "Talk:Dd (Unix) / Disks over 128 GiB"
(https://secure.wikimedia.org/wikipedia/en/wiki/Talk:Dd_(Unix)#Disks_over_128_GiB) . (Self-
published pseudonymously on Wikipedia).
https://secure.wikimedia.org/wikipedia/en/wiki/Talk:Dd_(Unix)#Disks_over_128_GiB.
Retrieved 2009-12-07.
7. ^ "Talk:Dd (Unix) / Disks over 128 GiB"
(https://secure.wikimedia.org/wikipedia/en/wiki/Talk:Dd_(Unix)#Disks_over_128_GiB) . (Self-
published pseudonymously on Wikipedia).
https://secure.wikimedia.org/wikipedia/en/wiki/Talk:Dd_(Unix)#Disks_over_128_GiB.
Retrieved 2009-12-07.
8. ^ "ChangeLog-2.4.23" (http://www.kernel.org/pub/linux/kernel/v2.4/ChangeLog-2.4.23) .
www.kernel.org. http://www.kernel.org/pub/linux/kernel/v2.4/ChangeLog-2.4.23. Retrieved
2009-12-07.
9. ^ "Interview With GNU DDRescue's Antonio Diaz Diaz" (http://blue-
gnu.biz/content/interview_gnu_ddrescue_039_s_antonio_diaz_diaz) . Blue-GNU. http://blue-
gnu.biz/content/interview_gnu_ddrescue_039_s_antonio_diaz_diaz. Retrieved 2008-12-06.
10. ^ "Damaged Hard Disk" (http://www.cgsecurity.org/wiki/Damaged_Hard_Disk) .
www.cgsecurity.org. http://www.cgsecurity.org/wiki/Damaged_Hard_Disk. Retrieved 2008-
05-20.

External links
dd
(http://www.opengroup.org/onlinepubs/9699919799/utilities/dd.html)
: convert and copy a file – Commands & Utilities Reference, The Single UNIX®
Specification, Issue 7 from The Open Group
dd (http://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html) :
manual page from GNU coreutils.
dd for Windows (http://www.chrysocome.net/dd) .
savehd7 - Save a potentially damaged harddisk partition
(http://seed7.sourceforge.net/scrshots/savehd7.htm)
GNU ddrescue (http://www.gnu.org/software/ddrescue/ddrescue.html) .
Manual for GNU ddrescue (http://www.manpagez.com/info/ddrescue) .
Softpanorama dd page (http://www.softpanorama.org/Tools/dd.shtml) .
DD at Linux Questions Wiki (http://wiki.linuxquestions.org/wiki/Dd) .
Retrieved from "http://en.wikipedia.org/wiki/Dd_(Unix)"
Categories: Hard disk software | Data recovery | Standard Unix programs | Unix SUS2008
utilities

This page was last modified on 12 November 2010 at 21:43.

Text is available under the Creative Commons Attribution-ShareAlike License;
additional terms may apply. See Terms of Use for details.
Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit
organization.