Securely Erase Data: A Guide to Effective Techniques and Tools

[fox26]

I have read in several places, I don't remember exactly where, that to effectively and securely erase stored date from most modern digital storage devices, it is necessary (or at least this is one often-used technique) to repeatedly, in several, maybe dozens, of passes, write randomly selected strings of 0's and 1's over the data. My Bitdefender antivirus app now has what is called a "file shredder" to do just that. (I imagined from the name that it used this technique, and a call to Bitdefender indicated that it probably did.)

My question is: Why is it necessary to use such a time-consuming technique, when, it seems, just writing all 0's (or all 1's) would be quicker and even more effective? It is true that the ultimate limit set by quantum mechanics on the accuracy with which the state of every atom, free electron, etc. in the storage device after writing the 0's could be known would allow, at least in some situations and for some storage locations, the determination of whether a 0 or a 1 had been stored there before writing the 0's, but this could not be done with present technology, or any likely future technology. I could be wrong about this. Am I wrong, or were instead the descriptions I read of erasing techniques, and the Bitdefender phone tech person, both wrong, or, more unlikely, are the erasing techniques used generally, including by Bitdefender, obviously needlessly complicated and time-consuming?

 

[DrClaude]

For magnetic storage devices, such as hard disks, there is residual magnetism that can be measured. So even if all bits are set to 0 or 1, with the proper tools one can still read the information that was there before. Cycling many times between 0's and 1's reduces this residual magnetism, making it harder to find the previous state of each bit.

I do not know if the same is true of other types of memory, like SSD, but I assume it could be. You have to remember that each bit is not held by a single atom/molecule, but by a huge bunch of them, such that writing a new bit doesn't change completely the state of the underlying medium (most atoms/molecules change their state, not all of them).

As for Bitdefender's approach of using random strings, I don't know if this is more efficient than just cycling 0's and 1's.

 

[jedishrfu]

When data is written to a hard disk, it is streamed to a magnetic head while the disk spins underneath it. An actuator moves the head from track to track as it writes your data. Each track has a predefined width that the head travels in. Because of vibration the head doesn't travel in a simple circle but randomly wiggles back and forth a bit.as the track is written.

https://en.wikipedia.org/wiki/Hard_disk_drive

Whether you writing zeros or ones there will still be some residual data because the head never passes over the same spot on the track to completely overwrite/erase the prior data written there meaning it could be read back so multiple writes of random data obscure it more completely.

The other part is that when you delete files on your hard disk they aren't actually deleted. The file is unlinked from the filename to disk sector mapping but the data will still be there for awhile (hence recovery programs can often bring back deleted files in the near term). In the longer term, the OS will begin to reuse the deleted tracks once its cycled thru its free tracks.

Imagine having a column of names on a whiteboard new names are added to the bottom of the list and deleted names are simply erased. Once you hit the bottom of the whiteboard writing new names then you go back to the top and fill in the blanked/erased rows with new names. That's kind of how a hard disk works. THis strategy also benefits the hard disk in not reusing the some areas more than others which could lead to faster hard disk failures as the head may scrap the disk surface slightly with each pass and and disk shock.

http://www.explainthatstuff.com/harddrive.html

SSDs and USB sticks are electronic memory based with different operating characteristics from hard drives. I don't think they have the same issue when overwrites are done. However, they do practice the notion of minimal writes to a given location to preserve the SSD.

https://en.wikipedia.org/wiki/Solid-state_drive

 

[fox26]

DrClaude and jedishrfu,

Thank you for your answers. Since I read them I've been looking online for more articles about erasing data, and found, as is often the case online, considerable differences of opinion about what techniques are best in different situations. Without going into tedious details, the consensus, to the extent it exists, seems to be that to erase moderately sensitive data on magnetic drives that are going to be resold or junked, several, but not dozens, of passes writing randomly selected 0's and 1's is desirable and also sufficient. For highly confidential information, destruction of the drives is advisable. For personal computers being resold or junked, maybe just writing all 0's or all 1's is good enough.

Actually what I had in mind by my "most modern digital storage devices" were SSD's and USB flash drives. For them, although it's not entirely clear, writing all 0's or all 1's may always be sufficient in those cases where it can be done, but often the construction and programming of such drives makes the overwriting of certain data on them about impossible, so different techniques, such as encryption of the data before storage or destruction of the drives, should be used. You probably already know more about this than I do.

Wiggling of the write head on a hard disk is something I hadn't thought about.

 

[jedishrfu]

I think you summarized the state of data erasure well.

If the computer has been used to conduct online banking, shopping, home finance, tax preparation... then I'd be inclined to remove the hard drive.

There are folks who buy junk computers just to see what they can find on the hard drive.

I wouldn't want to risk that scenario so I remove the drive and destroy it instead.

 

[Rive]

For magnetic storage devices, such as hard disks, there is residual magnetism that can be measured.

There was some experiments where overwritten data could be restored based on residual magnetism, but that was back in the MFM era. Around 30 (or more) years ago.
These days, it's true that with complicated instruments remnants of previous data can be identified, but that's all. It can be identified, and things stops at this point. Like a skeleton can be identified - it is a skeleton, see?

For highly confidential information, destruction of the drives is advisable.

Overwriting is completely safe for sensitive data too. What makes the difference is, that for sensitive data the staff who does the overwriting is not to be trusted, so it is a requirement to identify the wreck of the drive before disposing it.

The very point is, that for magnetic storage it has to be physical overwriting. Just logical 'delete' won't do.

 

[Vanadium 50]

I think you need to decide who you want to keep from reading your stuff. A random tech-savvy criminal? Or a major world government? That will determine the steps you need to take in protecting it. For me, a secure erase followed by a few hammer blows is plenty.

The more interesting question is what to do with a failed drive that used to have sensitive data on it.

 

[rkolter]

For my failed drives, I crack the drive open using a hammer, and for drives with platters, I grind the surfaces down with a belt sander. I physically break up the rest of the drive and put the pieces into a crock pot full of salty brine overnight. I let them cool in the brine after that until it's trash day, then throw the parts out. Any working drive I am disposing of, I tend to follow the above steps for as well. I work in IT. The last thing I want is someone one-upping me on drive security.

I second Vanadium's response - in general you are probably good with even a basic secure erase that overwrites the drive with random data 1-3 times. If you're a bit paranoid, or a perfectionist, overwrite ten times.

I would add one more thing - if you are truly worried about your data being taken, encrypt your drive - both because if the drive is stolen it cannot be read, and because if you later wipe the drive, the data you are obscuring is encrypted - if someone did get the data, it wouldn't be much good to them.

 

[Svein]

CCleaner has a Drive Wiper tool.
So does the Paragon Hard Disk Suite.
I seem to remember several freeware tools that do the same thing.

 

[OCR]

CCleaner has a Drive Wiper tool.

I don't believe CCleaner can erase your Local Disk, though...?
It won't on my computer, anyway...

.

 

[Rive]

Then get a Linux stick and use the BADBLOCKS program in destructive mode.

Better to have some cooling for the drive, because it'll get some heat...

 

[Vanadium 50]

I don't believe CCleaner can erase your Local Disk, though...?

No Windows program can erase the boot drive since the code it needs to run is on that drive.

 

[OCR]

No Windows program can erase the boot drive since the code it needs to run is on that drive.

... Those were my thoughts too.

 

[Gaserpas]

I don't know why Svein recommends the CCleaner tool , only those who have used it know that it is a waste of time. I've used a tool that's helpful but it's not freeware.

 

[Filip Larsen]

Another approach to allow for secure erasing data on a drive is to employ encryption on the full drive or selected files from the start and then securely erase the key when the disc or files are to be erased.

For devices such as SSD that may not overwrite the same physical memory block when you overwrite the logical blocks belonging to a file, encryption (and physical destruction) is a much more secure choice for erasing data than overwriting. Also, some USB drives come with built-in encryption and secure reformatting.

 

[FactChecker]

Physically destroying the drive is an order of magnitude faster than over-writing the drive even once. Over-writing several times would take a very long time. Unless you are worried that a professional spy will be trying to get your data, you can do a reasonable job of "destroying" it in 10 minutes with a sledge hammer or a drill.

A year ago, I had to discard 15 to 20 old drives. If I over-wrote them, I would still be working on it. (an exageration)

 

[Svein]

I don't know why Svein recommends the CCleaner tool , only those who have used it know that it is a waste of time. I've used a tool that's helpful but it's not freeware.

Well, I only said that CCleaner had the option to wipe a drive. I also said that Paragon Hard Disk Suite has that option, which I am currently using - it takes 4.5 hours to wipe a 74Gbyte drive!

 

[phyzguy]

I don't think anyone has answered the question with respect to solid state drives and USB "thumb" drives. On these devices, the data is stored in floating gate non-volatile memory cells. Unlike magnetic drives, I don't think there is any way to recover the data from these devices once it is erased. So a one-pass simple erasure (with all 1's, all 0's, or any random pattern) should be sufficient. Does anyone know differently?

 

[Filip Larsen]

So a one-pass simple erasure (with all 1's, all 0's, or any random pattern) should be sufficient. Does anyone know differently?

I would image that a general flash drive tool would have difficulty ensuring that any bad blocks on the drive has been securely erased. A bad block would not normally be readable, but if bad block management (BBM) is only implemented in the host computer driver and not the on on-chip controller (I am not aware if all flash drives have on-chip BBM, though I think most have) then it seems plausible that special hardware readers may be able to read sensitive data that has "gotten stuck" in those bad blocks. For drives with on-chip BBM special data forensic equipment may still be able to recover content of some of the bad blocks on the drive.

 

[sysprog]

A good 2015 (revised from 2006 version) overview: https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-88r1.pdf

 

[FactChecker]

The only completely sure way:

 

[sysprog]

@ Fact Checker, it's true that the most reliable way to destroy data is to destroy the integrity of the physical medium that holds it, e.g. by extreme heat, perhaps by lowering it into a molten metal crucible as the video clip you posted illustrates, or maybe by tossing it into a magma pool of an active volcano ...

 

[Gaserpas]

I don't know why Svein recommends the CCleaner tool , only those who have used it know that it is a waste of time. I've used a tool that's helpful but it's not freeware.

I saw this on Google yesterday.maybe you can get what you want ...

 

[sysprog]

I don't know why Svein recommends the CCleaner tool , only those who have used it know that it is a waste of time. I've used a tool that's helpful but it's not freeware.

I saw this on Google yesterday.maybe you can get what you want ...

It looks to me like you're panning well-established freeware and plugging a trial of something that is less well-established. The freeware version of CCleaner is able to implement the same erasure options as are listed at the site at the location a link to which you posted.

From a https://www.ccleaner.com/docs/ccleaner/ccleaner-settings/changing-ccleaner-settings:

CCleaner has four methods of secure deletion: a Simple Overwrite (1 pass), DOD 5220.22-M (3 passes), NSA (7 passes), and Gutmann (35 passes). A 'pass' refers to how many times CCleaner writes over the spot on the hard drive. The more times CCleaner writes to that spot, the harder the file will be to recover by any means. The drawback is that it will take CCleaner longer to complete the job.​