Any tools or processes for deleting slack space from donated storage media?

Slack space poses some questions as far as "donor intent" concerning digital materials on storage media donated to archives. A donor may believe this information was deleted and not be aware it is forensically accesible.

I was curious if there are any specific tools or processes for deleting/wiping slack space on donated disks & drives. Also interested in any policies or papers for how archives and archivists are handling the issue.

Jefferson Bailey

digital-preservation
digital-archiving
born-digital
digital-forensics

Comments

Andy Jackson: Could you clarify what you mean by 'slack space'?
Jefferson Bailey: Nick provides a good definition in the comment to David's answer below (Thanks Nick!). There is a passable explanation here: http://www.techrepublic.com/blog/security/computer-forensics-finding-hidden-data/232. There are ways to overwrite slack space information when deleting a file, but it isn't how most users delete files and can be time consuming. Was just curious if there was a forensic tool to identify and delete only slack space info, i.e. the area of a block/cluster allocated for deletion/rewrite by the file system but not yet overwritten.

Answer by David Tolmie

I'm not entirely sure what you mean by "slack" space.

If you're looking to securely erase the entire disk/drive, DBAN is widely used and highly effective. http://www.dban.org/

I'm less aware of tools to only wipe free space, which is what I think of when I hear slack space. If you have access to a Mac computer running newer versions of OSX, Disk Utility provides the ability to erase just free space. When you select a volume, then select Erasae Free Space, it will provide you a slider to choose how securely to erase the free space and explain what actions it will perform to do so. Disk Utility can also be used to wipe entire drives securely.

If you're wiping flash media, don't get too excessive with the number of times you have the tool overwrite the device (three should suffice), as flash media is more prone to wearing out from excessive writes than magnetic media.

Comments

Nick Krabbenhoeft: Storage media divide a bytestream into chunks and allocate these chunks to clusters on a disk. If the media uses 512 byte clusters and an object is 513 bytes, the object will be split over two clusters, one with 512 bytes and the second with only 1 byte. The remaining 511 bytes in the second cluster are empty. This is the slack space. The problem arises that storage media are typically not overwritten when an object is deleted. The only thing that's deleted is the reference to the location of the object. The supposedly empty 511 bytes from above could still have significant amounts of data.
Joe: @NickKrabbenhoeft : if that's your concern, if you're willing to accept the risks of a media refresh, you'd be better off copying the content to new media in a format that removes that space. (I believe that the TAR format does, without introducing compression or other parts that might be problematic).
Nick Krabbenhoeft: @Joe A media refresh would accomplish the task, but some archives are interested in preserving the original content on the original media, but only the content the donor believed they were giving.

Answer by Jakob

On Windows one can use SDelete to wipe out selected folders, files, and/or empty space. by the way this is useful also to save space when virtualizing a Windows machine because the image of the storage media can better be compressed.

Comments

Umber Ferrule: This is a command line utility offered by Microsoft.

Answer by Umber Ferrule

Windows also has File Shredder and Eraser.

Both are free and both discussed on Gizmo's Freeware pages.

Comments

Answer by Wing Tang Wong

Whole Disk Method:

Re-editing my edited answer, since my original answer contained what I wanted to convey. But to re-word:

1) The 'dd' command to create a large space consuming file containing zero(s) is to fill up all the unused, but were potentially previously used space. Ie, this cleans up the potential hiding spaces on the device. This does not clean up the current slack space associated with live files. Ie, a 512 byte file on a filesystem with 32KB clusters would represent some 31.5KB of potential slack space data, etc.

2) Once the free space on the disk has been zero'd out, you would then copy the existing files to another location on the drive. However, to avoid the slack space issue, you may want to consider which program you use to copy your files from one location to another. Presuming a program like "rsync" is only copying data that is part of the live file, you can just rsync the data from one folder to another folder on the device. Since all of the other free space has been zero'd out, there is no new data containing slack space that will be generated.

3) Once your files have been copied, delete the original files and re-perform step #1 to wipe out the newly created "dirty" unused space.

Once you complete the above steps, you will have a drive with the data files you want to send/leave, and all slack space will have been zeroed out.

You HAVE to do a full disk zero wipe of the free space to ensure that no new data bearing slack space is created.

So, updated instructions:

# Go into the mounted filesystem
cd /Volume/MountedFileSystem/

# Wipe the free space on the disk
dd if=/dev/zero of=./junk-file-containing-zeroes.dat bs=1000000
rm junk-file-containing-zeroes.dat

# create a tar file containing just the data you want to send.
tar cfPp ./datafiles.tar ./

# Delete the original files containing dirty slackspace
rm YourFilesAndFoldersButNotThatTarFile

# Wipe up that new dirty space you just created!! (Yes, this does another full disk free space wipe)
dd if=/dev/zero of=./junk-file-containing-zeroes.dat bs=1000000
rm junk-file-containing-zeroes.dat

# Put the data files back in place
tar xfPp ./datafiles.tar

If you have a 2nd device you can tardump into, you can avoid the 2nd wipe, but the instructions above assumes you want to retain certain live data files on the device.

Inidividual File Method:

Note, in the event you don't want to do a full disk wipe, this is what you can do, for each individual file. For example, I have the following file:

Note the size:

-rw-r--r--   1 wingwong  staff  364 Apr 22 11:07 dirty-data.txt

Note the inode:

26909293 -rw-r--r--  1 wingwong  staff  364 Apr 22 11:07 dirty-data.txt

Let's clean a spot on the disk for that file(this needs to be bigger than the size of the original file and needs to be big enough to fill out to the next block size on the storage device):

dd if=/dev/zero of=./clean-data.dat bs=1000000 count=2

Note the inode and size of the clean data file:

-rw-r--r--   1 wingwong  staff  1000000 Apr 22 11:11 clean-data.dat
26909297 -rw-r--r--  1 wingwong  staff  1000000 Apr 22 11:11 clean-data.dat

Now, we copy the data from our file into the clean data space:

$ dd if=./dirty-data.txt of=./clean-data.dat 
0+1 records in
0+1 records out
364 bytes transferred in 0.099726 secs (3650 bytes/sec)

Note the inode of the clean data file, it remains the same as it was before:

26909297 -rw-r--r--  1 wingwong  staff  364 Apr 22 11:12 clean-data.dat
26909293 -rw-r--r--  1 wingwong  staff  364 Apr 22 11:07 dirty-data.txt

But the data size is the same as the original dirty-data.txt file:

-rw-r--r--   1 wingwong  staff  364 Apr 22 11:12 clean-data.dat
-rw-r--r--   1 wingwong  staff  364 Apr 22 11:07 dirty-data.txt

At this point, clean-data.dat contains the data from dirty-data.txt, minus the slack space data.

You can now just remove the original dirty-data.txt file and rename the new file to take its place.

Comments

anarchivist: This is wiping the entire disk, not just the slack space.
Wing Tang Wong: No, this is wiping the un-used space on the disk. Live data is not impacted. Note the 'dd' isn't going to the block device, but to a junk file on the filesystem.
anarchivist: Got it - it still doesn't address the problem of wiping the entirety of it. Can you clarify and/or emphasize this in your answer?
Wing Tang Wong: Updated answer.
Wing Tang Wong: Rewrote my answer with two methods: the original whole disk free space method, and the individual file method. Note that the individual file method is quicker and doesn't require a full disk write process, but also doesn't prevent creation of new dirty slack space that's hiding on the disk. So, user beware.

Answer by anarchivist

First, I'm presuming that your process would something like the following -

Either:

Wipe slack space
Image disk

Or:

Image disk
Wipe slack space

The first case is risky; the second case is arguably too much work for most people or institutions. If your concerns about slack space are strong enough to acknowledge the significance of the risk of inadvertently making the data in slack space available, I would strongly recommend looking at logical disk imaging instead.

There's a great question with answers on this site already that addresses why you'd choose forensic or logical disk imaging. Realistically, it's much easier to just get a logical transfer of the assets than worrying about deleting slack space across a heterogeneous set of disk images that may have differing block sizes or file systems.