Beginners Guides: Hard Drive Data Recovery

If you've been using computers for a decent amount of time there is a good chance someone has told you that data stored on a hard drive is not exactly safe. I'm here to assure you that this is indeed true. Never mind the fact that unlike tapes or CDs or other methods of storage, hard drives are mechanical, active devices and are thus subject to comparatively rapid breakdown. No, the real threat to hard drives are the people that use them, by which I mean you and me. Hard drives, being the dynamic storage devices that they are, are extremely easy to erase in any number of amusing and simple to achieve ways... as are USB hard drives and flash memory cards (recovery tips for that media is detailed here). Also recently added; How to fix a 1TB hard drive that suddenly changes to 0.0GB, or 32MB in size. Working as a computer tech during the glory days of Windows XP, you get rather used to using FDISK and other hard drive utilities to prepare and repair customer's drives, which leads to a certain over confidence. That attitude can lead straight to disaster, sort of like giving a 12 year old boy the keys to an ATV. Picture this if you will; there I was, two or three sentences and a screen shot away from finishing a 5000+ word article on computer upgrades. All I had to do was fire up FDISK on a dual boot Windows PC system and grab a few screen shots. I figured I'd write a little blurb on how to partition a drive, making sure to tell the readers not to mess with FDISK if they were not sure what they were doing Yes, there's going to be some irony here. So anyway, I wanted to get some more screen shots of the actual partitioning screen, but I did not have a blank hard drive handy. I figured I could use my NTFS formatted XP drive (which FDISK perceived as a blank drive) to start the "process," grab the screen shots and then cancel the partitioning. No problem. Except for one little thing I had forgotten that FDISK, in the process of checking the disk before it prompts you for the size of the partition, writes information to certain areas of the hard drive. This data writes over whatever might have been there before. Meanwhile, there I was, watching the '%complete' counter and wondering why a little red warning flag kept going off in my brain? I restarted WinXP and waited for it to boot, and waited... and waited... Oops.

Primary Partition Gone?

The hard drive that suffered the data loss

was a 17Gb Fujitsu drive with two 5Gb XP NTFS partitions (Home and Professional) and 6Gb of unused space. Both XP partition were unbootable after the incident. After transferring the drive to a Windows 2000 computer so I could use disk manager, (to load disk manager on XP or 2000, right click 'my computer' select 'manage' then 'disk manager') this is what I saw. The Primary partition where my 5000+ word article was saved, is seen as unformatted and cannot be read by the OS. The second XP partition could not be booted, but was seen as formatted and I could transfer files easily from it using explorer. Unfortunately, all the data I needed was on the first partition. What to do? Well there are a few tricks you can use to get data back from the brink of an abyss like the one I've created for myself here. First though, we should understand exactly what a file system is, and how it controls access to your data on a computer.

An overview of file systems A file system is a method an operating system uses to arrange data and free space on a hard drive or other storage device so it can be written to and read from. File systems create partitions which are areas of free space than can be addressed by the file system and seen as a logical drives (C: D: etc.) to be written to and read from. The two file systems used by the various Windows operating systems are NTFS (NT File System) and FAT (File Allocation Table). FAT is an earlier file system, used first in DOS as FAT-16, then later in Windows 9x/ME as FAT-32. The only major difference between FAT-16 and -32 is in the amount of data they can address. FAT-16 can only use up to 2GB of space on each logical drive, FAT-32 on the other hand can create partitions of up to 32GB in size. Later Microsoft operating systems like Windows 2000/XP/Vista/7 are fully compatible with FAT, even if it is not the default method they use to store files. NTFS is used in Windows NT/2000/XP/Vista/7 and provides a more secure and efficient method of file storage. In addition to allowing security to be implemented on individual files, NTFS also stores backup copies of essential disk information to aid in recovering from disaster. Both file systems use the Master Boot Record (MBR) and partition table, found in the first sector of each hard drive or storage device. The MBR and partition table determine which partition(s) on the disk are bootable, and locate and pass control to that partition to boot the operating system. If the MBR or partition table are damaged, the drive will become unbootable, and may appear to be blank if the partition information has been erased.

Fixing NTFS Partitions

The first sector of NTFS partitions is

reserved for the partition boot sector. This contains the information that allows the OS to read the partition. Without it, the partition cannot be accessed. By its nature, NTFS keeps a backup copy of the boot sector on the last sector of the partition which can allow recovery programs to restore it. The FAT equivalent of this is also called the boot sector, and resides on the first sector of the partition. The difference is that FAT does not keep a backup copy of this information, making recovery much more difficult... The first file stored on an NTFS partition is the Master File Table(MFT) which is essentially a listing of the names, properties and locations of all the other files in the partition. This is referenced by the operating system to access individual files. NTFS stores a backup copy of this file. Data restoration software will attempt to access or restore a copy of the MFT in order to access files on the partition. FAT partitions use something similar, called predictably enough the File Allocation Table (FAT). The FAT is also backed up on the disk, and can be restored by software. The major disadvantage of the FAT as compared to the MFT is that it needs to be located on a specific area of the partition to function, so if that area of the disk is damaged, recovery can be difficult. When a file is deleted (removed from the recycle bin within Windows), both file systems simply mark the file as deleted. The data is not actually removed from the drive, but rather the space it takes up on the disk is now considered to be free. Consequently, if you delete a file accidentally, you have an excellent chance of being able to restore it provided you do not write more information to the disk. In my situation, I had two NTFS partitions on the effected disk. When I ran FDISK, it wrote garbage information over certain areas of this disk, including areas of both partitions. As a result, the first partition (the one with my article on it) had lost its partition boot sector, meaning it could not be accessed normally by an operating system. The second partition had merely had crucial system files overwritten, and was unbootable, but still fully accessible once I transferred the disk to another computer. Thankfully there is a way to fix all of this, and get the data back! First, a small disclaimer: All the processes described from here on are strictly for resolving software issues with your data, like accidentally deleting partitions or files. If your hard drive has a physical problem, if it is making strange noises, shaking, rattling or smoking, nothing here will apply. Turn your computer off, unplug the drive and call a data recovery service if your files are vital.

Steps to Data Recovery

The number one rule to follow when you have lost data is to not write anything more to the affected hard drive! This rule stands true for any situation...

If you have deleted a partition by accident, do not create another partition, just leave it blank. If you have deleted files from the recycling bin that you realize you need, do not (if possible) save anything to the drive. The reason for this is that hard drives do not actually erase anything, not data or partitions. When you erase a file from the operating system, it is just marked on the drive as having been deleted. When the system needs to store more data on the drive, it will consider files on the drive marked 'deleted' as being empty space, and cheerfully copy over them. If that happens then you're in big trouble. The same rule applies twice over for partitions; since partition information just presents the operating system with a way of addressing the space available on the drive. If you wipe out a partition everything from it will seem to be gone. So if there is no partition information, no data can be read by the operating system. This does not mean that your data it is not there however, only that you can't see it. Datarecovery programs have no such handicap. What I had done in my zeal was to allow FDISK to test the integrity of the drive, which it does by writing a pattern of data to certain areas. Of course, in my case, many of these areas contained partition information and/or critical system files. The result was one missing partition, due to a destroyed boot sector, and one unbootable (but still readable) XP installation. The good news? I got it (almost) all back. Here's how. The first, and best thing to do in a data-loss situation is to make sure no more data is written to the drive. Obviously, if you have just the one partition and it's fried, you can't boot normally to the operating system. The best option in this situation is to transfer the drive to another computer, preferably one using the same file-system as your damaged partition (i.e. the same operating system, or a newer version). See the PCSTATS Beginners Guides sectionfor information on how to move your hard drive to another computer. Transferring the HDD to another computer has the dual benefit of preventing the drive from being written to accidentally, and potentially allowing you to retrieve information from the disk just by using Windows Explorer to look through file structures. If you have damaged or erased essential operating system files, but the partition information is still intact Windows will not boot. The HDD can still be read from a different operating system which is one way out of the doom and gloom. This was the case with one of the two XP partitions on the disk I mangled, as I was able to fully access it after moving the hard disk physically to another computer.

File recovery programs

If you do not have the means to physically transfer the hard disk, resist the temptation to re-install your OS. There are several software tools available which will enable you to boot your computer with an alternative operating system and then help you try to recover the files.

The simplest way to gain access to the files on your hard drive with a toasted OS is boot your computer with a DOS boot disk and then use a DOS compatible file recovery program such as Testdisk, detailed below. Note that if you have a single hard drive with a single partition that is no longer bootable, file recovery becomes instantly more difficult. Most recovery programs will need a place to copy recovered data, and if you are using the same drive which has the lost data on it you have no guarantee that you will not be destroying more data than you save. It's a far better idea to either install a new hard drive onto the current system and put a new OS on that, or find another computer to transfer the lost hard drive to. That said, there are several programs such as 'Winternals Disk Commander' and 'ERD Commander', that will boot your system straight into DOS or an alternate OS, then perform file recovery. None of these programs are free however. If you have installed your hard drive into another computer, or if you have put a new drive with a separate OS into your current machine in order to boot, you now have a couple of advantages: Firstly, You can attempt to access your lost data normally through Windows File Explorer. This will not work if the partition information has been changed, since the OS will not 'see' the logical drives. Secondly, You can safely play with recovering your files, since you now have a completely separate hard drive on which to put recovered data without compromising the source (lost) drive. Freeware Recovery Programs If there is one problem with the area of data recovery software, it's that companies know that a functional recovery program is something that people will pay good money for. Hence freeware and non-crippled shareware programs are thin on the ground. There are a few options available though, so on with the list. Please read these through carefully before deciding the next step you will take.

FINDNTFS Freeware
FINDNTFS (available here ) is a rather effective free program to locate and recover NTFS (NT File System, The default method of storing files on Windows 2k/XP) files. It is available in several versions, including one that will run from a DOS boot, and thus can be used when the Windows OS is not bootable. FINDNTFS is capable of several things besides finding and copying lost files, but that is what we will focus on for now. To use FINDNTFS, boot your system into DOS using a boot disk with the findntfs.exe file on it. To obtain a list of NTFS files and directories on the drive you are attempting to recover from, type 'FINDNTFS # 1 1 1 c:\recoverlog.txt files' at the command prompt. The "#" should be replaced with the number of the hard drive you are reading from. If you have only one drive, it will be '1', if there is more than one drive in the system the physical hard drive with the 'c:' logical drive on it will be '1'.

This command tells The FINDNTFS program to search the entire specified disk for NTFS files, and output the file list to a text file on the C: drive.

Note that you can save the text file under any name on any drive, as long as it has the '.txt' extension. Do not save the log file onto a drive you are trying to recover data from however. Once saved to another drive you can view the log file, which should look something like this.

If everything went well, you should have a reassuring, if somewhat jumbled list of the available NTFS files on the drive you selected. Search the document for the filenames that are most essential, and then scroll up until you see the directory that they are in. You will need to do this, since the directory order that FINDNTFS sees will not necessarily be the same as you had before the data was lost. Note down the number of the directory before proceeding on with the next step.

NTFS reader for DOS

To recover files using FINDNTFS, you need to use the 'copy' command. A limitation of the program is that it will only save recovered files into the directory where the FINDNTFS executable is located, so make sure you have enough space available to hold your restored files. Type 'FINDNTFS # (replace "#" with thr drive you are recovering from, as before) 1 1 1 copy #'. This is the directory number holding the files you wish to restore. You may enter up to 10 directory numbers. If you do not enter a number, the program will attempt to restore and copy all NTFS files on the selected disk.

FINDNTFS will copy the selected directories and files to the directory containing the FINDNTFS.exe file. Check the recovered files to make sure they are not corrupted. Sadly, FINDNTFS has one major limitation. It cannot copy NTFS files onto DOS readable (FAT) partitions, and therefore is not able to recover files without an NTFS supporting operating system running (Windows NT/2K/XP). Thus it is not possible to recover files directly with this program using a DOS boot disk. NTFS reader for DOS Freeware NTFS reader for DOS (available here ) can copy NTFS files onto DOS (FAT) partitions. In fact, that is the only thing it does, but what a useful thing... If your NTFS disk is unbootable, you can put this program on a floppy and copy files from the disk onto the floppy. Very good for rescuing essential documents that need to be completed. The limitation of this program is that it cannot read from partitions that have damaged boot sectors, or from drives with damaged partition tables, as it needs to be able to see the NTFS partition before it copies data from it. The company that made this software, 'Active@ Data Recovery Service' (www.ntfs.com), offers a commercial version, 'Active@ Partition Recovery', which adds the ability to search the drive for lost partitions as well as copy data, making it a complete recovery tool.

Note that NTFS reader will also work under Windows 9x/ME but not on 2K or XP, due to restrictions these Operating systems place on accessing drives directly.

TESTDISK, The Holy Grail

TestDisk is a DOS and Windows tool (also available for Linux) produced by CGsecurity.org that can be used to locate and recover

lost partitions (FAT and NTFS) by repairing the partition table or replacing partition boot sectors with the backup copies. Download the latest DOS version here (TestDisk & PhotoRec 6.11), the Windows NT/XP/2000/2003/Vista versions are here (TestDisk & PhotoRec 6.11) or if you prefer it is also available for Linux (ver 6.11.3), and Mac OS X (ver 6.11.3). The providers of TestDisk have created a handy Wiki for it right here, and also for PhotoRec here. PhotoRec is a good tool for recoving damaged, formatted or erased photos and images from a flash media's filesystem, even if it "has been severely damaged or re-formatted" say the company. More this topic in PCSTATS Guide to flash memory data recovery. Anyhow, let's get on with the show; back to TestDisk and my lost hard drive partition... Using the TestDisk program I was able to make my first partition (the one with the articles) accessible by restoring the backup partition boot sector, enabling the operating system on the computer to 'see' the partition again. I was happy. Testdisk is not overly difficult to use, but it does require a bit of attention. First, please read the documentation (located in the 'doc' directory) for an overview. Upon starting Testdisk, you get a screen listing your available physical drives at the top.

Highlight the drive you wish to recover and select the 'analyze' option.

It will show the current partition structure and upon hitting 'enter,' will start searching the drive to see whether the actual partitions match. Make a note of this. If you have erased your partition table, nothing will be shown here.

Testdisk Backs up Lost Data

Once this search is complete, hit enter again. It's a good idea to run the 'search!' option to do a more comprehensive search of the drive. If you initially had no partitions shown, because of a damaged or wiped partition table, Testdisk will now hopefully have rediscovered the partitions. Verify the information and select 'write' to save the new information onto the disk. If removing your partitions was the only damage you did, you should now be up and running again. If you damaged the partition boot sector on your partition, (as I did) you should select the 'advanced' option, then 'boot.'

Testdisk will compare the boot sector to the backup boot sector. If they are identical, it can do nothing more, but if they are different it will ask you if you wish to overwrite the boot sector with the information from the backup.

This operation made my 'lost' partition with the articles on it accessible again. Testdisk is an extremely useful tool for partition recovery. Though it lacks a graphical interface and can only be run from DOS, it is capable of restoring lost information in minutes.

Undeleting Files in Windows XP

The majority of this article deals with recovering lost data from vanished partitions and accidentally wiped drives. This is all well and good, but what happens if you simply want a couple of files back that you deleted? Gone from the recycling bin? All is not lost. A variety of applications are available that can dig into your hard drives history and recover deleted items.

As you read earlier in this article, a file that is deleted from a Windows drive is not actually erased from the disk. Instead it is marked as having been deleted. The next time Windows is writing data to the disk, it will regard these deleted files as free space that can be copied over. This is why it's important to keep disk operations to a minimum when you are trying to recover data. It's also why you should install and operate your data recovery software on a different partition than the files you are trying to save whenever possible. Undelete programs simply scan your drive for files marked with the delete symbol and offer you a chance to restore them. Simple and pretty much foolproof as long as too much time has not passed since the original file deletion. You'll likely be surprised at how far back the files you find with these utilities go. Let's take a look at some freeware file restoration programs: Restoring deleted files with PC Inspector file recovery This versatile freeware program is capable of many recovery tasks, including retrieving deleted files. Let's take a look at how to use it to recover those documents your toddler accidentally wiped out. Note that PC Inspector can only recover files on a FAT file system, and does not work with NTFS formatted drives. Start PC-Inspector.

Choose the 'recover deleted files' option.

Highlight the logical drive (c:, d:, e:, etc.) that you wish to look for deleted files on and hit 'ok.'

Now expand the 'deleted' entry. The right pane shows a directory tree from the root of the logical drive you are searching. The green files are deleted files that you can attempt to recover, while green folders are deleted folders. Look through until you find what you are looking for, or use the 'find...' command in the 'object' menu to search for specific files. Once you have located the file or files you wish to recover, right click them and choose 'save to.' Enter a location on the disk and the file(s) will be restored to that location. For a really simple and effective way to recover deleted files, it doesn't get much better than Restoration, that's next.

Commercial Data Recovery Utilities

Restoring deleted files with Restoration. Restoration is a really simple and effective way to recover deleted files This incredibly simple tool will search any NTFS or FAT32 drive and recover a list of deleted files that can be restored. Let's look at how to use it:

Start Restoration.

Choose the drive you wish to scan in the 'drives' drop down box, and click 'search by deleted files.'

A list of deleted files is created. To restore one or more files, highlight them and click 'restore by copying' then choose a target directory. Note that unlike PC inspector, restore does not sort the deleted files by folder, it simply dumps them all into a single list, which can make it harder to find what you are looking for. If the file you want is not in the list, try searching again with the 'include used clusters by other files' option checked. This will include files which have been partially overwritten in the list. Note that this may well mean that your file will be corrupted or unreadable. There are many excellent commercial recovery packages out there, and to help you get started I've listed about a dozen software data recovery applications on the last page of this Guide with links to where you can download and try the programs yourself. Commercial Data Recovery Utilities

There are many excellent commercial recovery packages out there. The majority of these are designed to access the disk through an operating system, in which case you will need to have your affected drive transferred to another computer, or at least have a separate drive with a new OS on your original system. Commercial recovery programs generally use the 'virtual recovery technique, which involves creating an 'image' of the disk to be restored in memory and then transferring files from that image to an alternate hard disk. Two good examples of this type of program are 'Active File Recovery' and R-TT.com's R-Studio. I did not comprehensively test any of the listed programs, but when I first lost my data, I used several demo and preview versions of the following software to ascertain that my data was actually still there. One standout was R-Studio, whose demo version allows the recovery of files up to 64K in size, allowing me to move my articles safely off the drive before commencing the recovery in earnest. Highly recommended. From previous work experience, I can also recommend 'Winternals Disk Commander', though they do not provide a preview of the software. Following is a partial list of commercially available partition/file recovery and undeletion software we suggest you check out if the freeware we have already mentioned hasn't done the trick. Commercial recovery software list.
R-Studio Disk Commander File Rescue 2.5 GetDataBack for FAT GetDataBack for NTFS Norton Utilities/SystemWork s Undelete Hard Drive Mechanic Back2Life DFSee/ Fast File Recovery Undelete 3.0 Active UNERASER File Scavenge File Restore File Recover 2000 Undelete Fast File Undelete Data Recovery For NTFS http://www.r-tt.com http://www.winternals.com/products/repairandrecovery/diskcommander.as p http://www.filerescueplus.com http://www.runtime.org

http://www.symantic.com http://www.executive.com/consumer/undelete/undelete.asp http://www.highergroundsoftware.com/6.html http://www.highergroundsoftware.com/downloads2.htm http://www.simtel.net/pub/pd/57588.html http://www.dfsee.com http://savemyfiles.com/fastfile.htm http://www.pcconnection.com/scripts/productdetail.asp?product_id=30988 4 http://www.uneraser.com/undelete.htm http://www.quetek.com/prod02.htm http://www.winternals.com/products/repairandrecovery/filerestore.asp http://www.filerecover.com http://www.quantumsoft.co.uk/undelete.stm http://www.dtidata.com/products_ff_undelete.asp http://www.dtidata.com http://www.restorer2000.com/r2k.htm

Restoring Hard Drive Capacity - When a 1TB hard drive thinks it's 32MB

There is one other relatively new hard drive problem that can cause (apparent) data loss - the so-called "1TB hard drive showing up as 32MB" error. Many people have experienced this issue with hard drives over 750GB in capacity, but unlike a lot of data recovery situations discussed in this Beginners Guides, it's usually easy to fix...

Restore factory Hard Drive Capacity When HDD Shows up as 32MB

The loss of factory hard drive capacity seems to be an increasingly common occurrence with very large capacity (ie. 1TB) hard drives, but it's not specific to any one hard drive manufacturer. It can happen to brand new drives or existing hard drives full of data. In either case the data portion of the drive isn't generally affected. If a new or existing hard drive in your computer suddenly pops up with a capacity of 32MB, or another value equally small compared to the true drive size, don't freak out, the data should still be there. Here's what typically happens; a large capacity hard drive of between 500GB, 750GB, 1TB and 1.5TB suddenly appears to lose most of its capacity for no apparent reason. In many situations the capacity of the hard drive shrinks to 0.0GB or 32MB (and occasionally 32GB) and becomes inaccessible to the Windows Vista.

Disk 1 is actually 1TB in size, but here it shows up as 32MB. What's happening, to put it simply, is that the portion of the hard drive responsible for telling the computer how big it is, is mis-reporting that value. We don't have a good technical explanation for what causes this issue to randomly strike PC users, but suffice

to say a portion of the drive firmware known as the LBA48, HPA and DCO records becomes corrupted.

Select the correct hard drive which is mis-reporting its capacity from the list. Our experience with the loss of factory hard drive capacity bug came whilst setting up a new PC with two fresh 1TB Western Digital Caviar Black SATA hard drives. Both SATA drives had been formatted and were working fine. After hard power-off reboot one of the 1TB disks suddenly became a lot smaller than it actually was. Instead of reporting it's true capacity of 1000GB, or 1TeraByte, the Western Digital Caviar Black Edition SATA hard drive appeared to both the Microsoft Windows Vista operating system and the motherboard BIOS with a false capacity of 32MB. Somehow 999.968GB of storage space had disappeared into thin air! The new drive capacity of 32MB just seemed a little too specific to be random, so we did a little investigating. What we found is that quite a few PC users have experienced the exact same situation a new or existing large capacity hard drive suddenly looks to the computer like it's only 32MB, or 32GB in size. If the hard drive was previously filled with 1000GB worth of data you can imagine the shock of seeing it reduced to 32MB! Incidently, the 32MB figure seems to be drawn from the hard drives' onboard cache memory. How to restore the 32MB drive size back to full capacity CAUTION: As with all Hard Drive recovery situations, if your data is critical don't attempt data recovery yourself, send the HDD to a professional data recovery service. We've tried the proceedures outlined below ourselves with success, but PCSTATS cannot

forsee every possible issue. Following the steps on this page could result in data loss. Proceed at your own risk. Always back up your data on a regular basis. The fix (for a blank hard drive which doesn't have any data on it) is pretty simple, first connect the hard drive which has lost its factory capacity and reports itself as being only 32MB large to a computer that works. If it's an IDE hard drive, just make sure it's the only hard drive on the IDE cable. Serial ATA hard drives don't require any special steps, just plug in the SATA data cable and make sure the computer isn't trying to boot of this drive. Next, go to blog.Atola.com and download a program called Atola Technology HDD Restore Capacity Tool.

Launch the program and select the hard drive from the menu which is mis-reporting its capacity, then confirm the correct hard drive has been selected and hit "Restore Capacity". The program will essentially correct the portion of the hard drive's firmware that contains the drive capacity information.

Next, shut down and physically turn off the computer and wait about 10 seconds. Then power the PC back up. If everything worked as it should, the hard drive which previously showed up as 32MB will have been restored to the correct factory capacity. In our situation that means the 1TB Western Digital Caviar Black hard drive was back to its true 1000GB capacity.

Disk 1 is back to its true capacity of 1TB (or literally 931GB).

According to the maker of this program, the HDD Restore Capacity Tool only "changes hard drive firmware settings, it does not read to or write write from the user data portion of the disk." We haven't tried Atola's HDD Restore Capacity Tool on a drive full of data for the simple reason that it's difficult to replicate the problem which causes a drive to mis-report its capacity as 32MB in the first place. However, from what we've read the program should work in the same fashion and your data should not be affected.