An Overview of Secure Data Deletion

2 days ago, Avast made a post on their blog describing their successful efforts to recover data from 20 old phones that they bought from eBay. Some details of their forensic analysis of the phones were provided the next day. Avast describes the inadequacy of deleting files “the regular way” and plugs its own app, which it claims allows the secure deletion of files. When Avast mentions deleting files “the regular way,” they simply mean using your operating system’s (e.g. Linux, Windows, Android) normal delete function to delete your files. Whether you simply hit the delete key/button or followed up by emptying your recycle bin/trash, it’s still just regular deletion.

Deleting a file this way is much like having the address of a house removed from the US Postal Service’s database. Although the location is not referenced by an address, the house is still standing, and anyone curious enough about the old structure can break in and look through any secrets you might have left before moving out. When you delete a file on your phone, computer, or other storage media the regular way, all the operating system does is remove any reference to the file. The file, however, is still there, stored in the unallocated (free) space of your drive until it is overwritten by new data, just like your abandoned house is still standing until it is bulldozed and a new house is built over it.

Many people delete files the regular way before giving away, discarding or selling their devices. The result is that their sensitive data is easily recoverable by the new owner or anyone who likes to go dumpster diving for personal information. You don’t need to be a forensics expert to know how to do this. If you want to try this out for yourself, put any file (e.g. a picture, a PDF document, a Powerpoint presentation) on any drive of your choice. It can be a flash drive, external hard drive, or your SSD. If you want to save time, though, use a small drive. After you’ve saved the file on the drive, delete the file, then download any of the “File recovery” programs listed here. Many of these programs are free. If you’re a Windows user short on time, try the portable version of Recuva, which doesn’t even require installation. Follow the wizard, and chances are, you will have your deleted file back within 5 minutes. It isn’t too hard to imagine what would happen if the file was an old tax return and an identity thief was running the program on your drive.

Given how easy it is to recover deleted data, it is imperative that anyone who values the security of their data take measures to render such data unrecoverable. As technology changes, the meaning of “secure deletion” evolves with it. Two types of technology are commonly used by consumers for data storage. The first, magnetic storage (hard drives), is still common in desktops. laptops, and in external storage media. The second is flash memory, which is commonly used in SSDs, USB flash drives, and cell phones.

Securely Deleting Data from Hard Drives

Techniques for securely deleting (or rendering unrecoverable) data from hard drives have been known for years. Here is a list of some of those techniques:

1. Overwrite the data on the drive. For example, use dban to erase entire hard drives prior to disposal, or a program like Eraser to overwrite individual files or the unallocated space of the drive. Although there are many different standards that require varying numbers of passes (number of overwrites), some believe that a single pass is enough.

In 2008, someone hosted a contest called “The Great Zero Challenge” to prove that even professional data recovery firms could not recover data from a hard drive that had been overwritten just once with zeros. The contest host’s website is now dead, but I believe nobody accepted the challenge to recover data from such a hard drive by the deadline.

2. Use full disk encryption on the hard drive. Even if the drive is not securely overwritten afterwards, any data that can be recovered is gibberish unless the passphrase used was weak or a weakness in the encryption algorithm is later discovered.

3. Physically destroy the hard drive. Methods that are commonly suggested include drilling holes in the platters, disassembling the drive and using the platters as Frisbees, using a hammer to smash the drive, or putting a few bullets through it.

4. Use the internal “secure erase” command built into ATA hard drives. A program developed by the Center for Magnetic Recording Research implements this command, but it may not work on all computers.

5. Degaussing: This technique reduces/removes the magnetic field on the hard drive. However, it requires specialized and expensive equipment.

None of these methods are perfect. For example, overwriting only works on the hard drive’s working sectors. Hard drive blocks that become bad when being used by the consumer are added to the G-List. Data on these sectors cannot be erased even by dban and similar hard drive overwriting programs (though the Secure Erase command will erase the G-List), but it may be recoverable with special equipment. In some cases, even physical damage may not be enough.

Nevertheless, for the everyday user, a combination of methods can be enough. A very conscientious user may use full disk encryption during normal use of the drive, overwrite the hard drive one or more times with dban when the drive becomes unnecessary, run Secure Erase after dban, then physically destroy the drive, leaving almost no chance that there will be recoverable data. If the drive physically fails before software overwriting can be performed, a combination of encryption and physical destruction may still suffice.

Securely Deleting Data from Flash Memory

The problem of securely deleting or otherwise preventing the recovery of data from flash memory still remains. Currently, much data is stored on USB flash drives and memory cards (e.g. SD, MMC, microSD, etc). Despite my best efforts, I could not find much information on the effectiveness of certain techniques of data erasure on non-SSD flash memory (if you do find such information, feel free to leave me a comment). For example, some claim that overwriting is effective, while others disagree, but neither side seems to have offered hard evidence. Perhaps Avast can help by listing the types of memory used by the phones they tested and posting the results of data recovery efforts on ones that have been wiped with their utility, but I won’t get my hopes up.

Fortunately, some research has been done on SSDs. Researchers tested methods of securely erasing data from SSDs and published their results in a paper titled “Reliably Erasing Data from Flash-Based Solid State Drives”. Let’s look at what this research says about the effectiveness of some of the techniques I described in the hard drive section on SSDs:

1. Overwriting is somewhat effective. In most cases, overwriting twice was enough to sanitize a drive. However, in some cases, overwriting was not completely effective. In one striking example, 1% (1 GB) of data remained even after 20 overwrites of the entire drive.

2. There are 2 types of encryption in SSDs. The first type is hardware-based encryption implemented by the drive manufacturer. In this case, erasure is performed by deleting the key rather than the data. This is the equivalent of locking a door and then throwing away the key. However, it is only effective if the key is truly gone; i.e. the SSD’s key store is properly sanitized. The papers’ authors call this “unduly optimistic,” not to mention impossible to test.

The second type of encryption is software-based encryption; the use of programs such as DiskCryptor, TrueCrypt, and dmcrypt-LUKS to encrypt the drive as a whole, rather than individual files. While the paper hints that overwriting may not be effective due to “digital remnants” that are not accessible, properly implemented full disk encryption should prevent any unencrypted data from being written to the SSD, thus rendering the data unrecoverable even if it can’t be overwritten.

3. Degaussing was ineffective against SSDs.

What does this mean for us? Because some of the methods that were effective on hard drives are ineffective on at least some types of flash memory, we should be careful about transferring or otherwise disposing of flash memory that hasn’t been completely physically destroyed. If you’re relying on software-based encryption to protect your data, make sure to fully encrypt the entire drive before you write any unencrypted data to it, because you may not be able to get rid of this unencrypted data even with many overwrites.

Avast is using the post to plug their Anti-Theft app, which they claim securely erases data by overwriting. As we saw, however, overwriting data on flash memory may not guarantee its destruction. Until I see more studies, I would probably feel safer smashing my phone with a hammer rather than selling it on eBay.

Leave a Reply

Your email address will not be published. Required fields are marked *