------------------------------------------------------------------------------- FUSE Encrypted File System (micro-how-to) The EncFS file system is an ordinary directory containing encrypted files using a encryption hashed' filenames. this 'directory' is mounted to another directory (using fuse) where it will appear in its decrypted form. That is the encrypted files are just ordinate binary files and with the correct times and tree structure. Only the names and contents of the files remain hidden. The directory tree and rough file sizes are preserved! Advantages * The encrypted can be stored on any normal file system, again without decryption. That includes SAN, USB, or DVDs. You do not even need to worry that the 'remote' file services have access to your data as their system never sees the decrypted data or passwords. By default they could see the encfs config file, unless separated (see below). * Hardlinks work if the underlying filesystem allows their use. * You can copy, update or synchronised the directory with another copy of that directory (say on another machine) without needing to decrypt it or even have the password! As such automated backups can still be performed securely, even by other people. * Similarly you can use Rsync 'delta difference' to syncronize between multiple copies of the same encrypted file system!!!! In fact ANY file system copy or transfer method can be used. The files remain... files. * You can syncronize with 'the cloud' without worry about some unknown SysOp looking at your files, regardless of what the their policy is or is not. * You can (with special techniques) merge two or more (I have a dozen) encrypted directories trees into the same top level (or faked top levels) of the same encrypted data directory, though there is a small chance of filename clashes. That is one encrypted filename is used by two seperate repositories, but is is highly unlikely. This is only on the top level as sub-directory contents remain separate, as each sub-directory can only belong to one repository. * You can add 'chaff' to the encrypted directory with (faked hashed file names and directories) to prevent someone knowng if all the data was decrypted or by which password. Some files may never be decryptable, or left behind by 'lost' password. (prevent rubber hose attacks) Disadvantages * Some overhead (but what scheme hasn't). * Directory Structure and rough (not exact) files sizes in each directory is visible. The directory structure could have and should have be hidden in a similar way to other file systems. That is a directory is just a special file containing a list of file pointers. * The 'encrypted hash filenames' make it obvious the directory contains an EncFS (or similar) filesystem. The presence of a encfs config file would only confirm this (see config separation below). * Most of the encrypting information (apart from password) including iteration count, and salt, is visible in the encfs config file at the top level of the encfs directory tree. This provides valuable information to a hacker. Better to keep that seperate (and private). Keep it on a USB as a required but separate physical key. (See config separation below) * Some management of this to add chaff to sub-directores is probably needed. * Removing excess and unwanted 'chaff' in a encfs directory is next to imposible, unless a 'chaff dataset' is used. The only way to separate chaff from real data is to decrypt and move all known and valid data to a new encfs directory. ------------------------------------------------------------------------------- Other related apps BoxFetch for Windows -- GUI app for Windows, and Mobile phones Note it doe not support all modes, just more common one CloudFetch for Android -- Encfs with data stored in the cloud Dropbox, SpiderOak, SparkleShare and Ubuntu One https://play.google.com/store/apps/details?id=az.cloudfetch ------------------------------------------------------------------------------- Basic Installation and setup of EncFS install sudo apt-get install encfs create and/or mount encfs ~/encrypted/ ~/decrypted unmount fusermount -u ~/decrypted As it is a FUSE based file system, it is completely user level controlled, which little kernel requirements and no superuser access. The EncFS is a 'pass-thru' file system that encrypts the individual files stored in the fake "decrypted" file system as individual files in the "encrypted" directory. This means that the files sizes, directory structure, permissions and times is visible in the encrypted directory tree. Though the file names themselves are encrypted. However as the file system is still files, it can be incrementally backed up as files, using normal file system backup utilities rather than requiring the whole partition to be ghosted. The file system also does not require a pre-allocated block of disk space (fix sized file, or partitions), but will grow and shrink as needed in the normal file system. That is because files are encrypted to files. You can NOT of course use it to encrypt the whole home, data, system, or swap memory of the computer for complete protection. This is better achieved using a lower level disk block encryption such as the linux dmcrypt. But then few people need to protect there whole computer, just the contents of a few files. ------------------------------------------------------------------------------- Case-insensitive Filesystems To store EncFS on a file system that is case insensitive for lookup, (VFAT, NTFS, HFS+, etc), create it using expert mode and select B32Block or B32Stream as the filename cipher to use. ------------------------------------------------------------------------------- Encrypted Filenames and Hardlinks The "encfsctl" program provides utilities to encode and decode file paths. (given the encfs config file ".encfs6.xml" and the password). These are not all documented. A password will always be needed. But more importantly the encfs filesystem does not need to be mounted in decrypted form, to convert between a encrypted and unencrypted filename, or decrypt one specific file stored in the encfs filesystem. Option Examples... encfsctl decode encfs_dir encfs_filename decrypts filename (not contents) to its unencrypted filename That is is this encrypted files real filename. encfsctl encode encfs_dir filename encrypts the given filename to the encrypted file path (result includes the includes encfs_dir prefix) encfsctl ls encfs_dir decrypt and lists all filenames in a directory (note top level gets listed at this time) encfsctl cat encfs_dir filename Show contents of the file encfsctl export encfs_dir dest_dir decrypts all the files found in the encrypted directory to the dest_dir This means you can lookup and rename the encrypted filenames without needing actually mount the encfs filesystem. This can be used to generate a listof encrypted filenames. For example generate a 'exclude list' of encrypted filenames which you don't want backed up or copied to some remote server (EG: temorary or unimportant files or directories). However remember the filename encryption is dependant on the whole file path, unless you specifically turn off "Filename Initialization Vector Chaining". As such you can not simply move a encrypted file into a different sub-directory in the encrypted tree, as the filename in that sub-directory will be different. To move a file you need to comletely rename all parts of its file path, including its actual filename. NOTE: If 'External IV Chaining' is turned on in the ".encfs6.xml" file (off by default) then the file contents will also depend on the file path in the encrypted directory. In this situation, hardlinks will no longer work, and renaming a directory will also require a re-encryption of all the files in that directory. This can be a very massive overhead, especially for very large files, which is why it is not enabled by default. Newer patched version... Environment variable ENCFS_PASSWORD_FD with a -S option will have encfsctl read passwords from the given fd (file descrptor) number rather than from stndard input. If the patch is present then 'encode' will produce a relative path rather than an absolute path, making its reverse compatible of 'decode' See http://code.google.com/p/encfs/issues/detail?id=33 Less obvious decode filename usage... Another less obvious way of matching a real filepath to its encrypted filepath is to look at the 'openfiles list' of the "encfs" process that is mounting the file system. For example... encfs ... /tmp/encrypted /tmp/decrypted # Do the mount pid=$(pgrep encfs) # find the process lsof -p $pid > t1 # list open files exec 5 t2 # list open files exec 5<&- # close filename to encrypt comm -3 t1 t2 | sed 's%.*/tmp/encrypted%%' # compare open file lists Results CvtrBM5uitlmTte2Hxrk0J-T So "README" is actually the encfs filename "CvtrBM5uitlmTte2Hxrk0J-T" Filename Encryption Algorithm (not reproducible yet) The filename is encrypted using MAC-16 ^ IV and key then the string... MAC-16 + encrypted_filename is converted to base32 or base64 depending on the filesystem case sensitivity. This is then base64 encoded which is modified such that the characters './' are replaced with ',-' respectiveally. see http://code.google.com/p/encfs/source/browse/trunk/encfs/base64.cpp line 122 A checksum (MAC-16) is created. The reverse is the same except the MAC checksum is compared. See sources... http://code.google.com/p/encfs/source/browse/trunk/encfs/base64.cpp http://code.google.com/p/encfs/source/browse/trunk/encfs/BlockNameIO.cpp#145 http://code.google.com/p/encfs/source/browse/trunk/encfs/StreamNameIO.cpp#102 Shell equivelent (to be worked out)... Shell Decrypt... ------------------------------------------------------------------------------- encfs and encfsctl will read passwords from TTY by default BUT... -S, --stdinpass Read password from standard input. OR from the file descriptor number in ENCFS_PASSWORD_FD --extpass=program lets you call a password gathering program such as "ssh-askpass" with the environment variable "RootDir" pointing to the encrypted directory. Trailing newline is striped, 2k passwords at most. Extra FUSEmount flags (after a -- option) -- -c request kernel-level caching of file data (speed up) -- -n /dev/xxx specify this device in 'df' command output instead of reporting it as being "encfs" ------------------------------------------------------------------------------- Remote encrypted data directories. Store the encrypted data on a remote system and mount it using "mount.cifs" (windows home) or "fuse-sshfs" (ssh filesystem) or even "dropbox" to syncronize between local and remote copies. Note you don't need encfs config file on the encrypted filesystem stored on the remote filesystem, though you will need it to decrypt that filesystem on any machine that you want to make use of it. This requires two layered FUSE mounts... sudo yum install fuse-sshfs sshfs user@storage:raw_data ~/raw_data -o "uid=$(id -u),gid=$(id -g)" ENCFS6_CONFIG=~/lib/encfs6.xml encfs ~/raw_data ~/data ... fusermount -u ~/data fusermount -u ~/raw_data ------------------------------------------------------------------------------- EncFS also allows reversed encryption. That is keep the local data unencrypted, but create a 'EncFS' encrypted image of the same data, to copy to a remote system for distribution or backup. Making a copy of the reversed encrypted EncFS directory and then using 'normal' EncFS to decrypting it should work perfectly fine. A copy of the ".encfs6.xml" configuration (not the password) is also transfered (unchanged) to ensure that this is directly posible without any extra effort. This also allow you to generate a EncFS encryption of your data, which can then be used to generate rsync backup 'snapshots' on a remote, and posibilty insecure server. You then do not need encryption for normal use. The rsync delta differences used for file transfer continue to work as normal. Example... Setup (copy to) the encrypted backup (using reverse) mkdir /tmp/backup_encrypt encfs --reverse /home/me /tmp/backup_encrypt ...password questions... rsync -av --delete /tmp/backup_encrypt/ /external-drive/backup_enc fusermount -u /tmp/backup_encrypt cp /home/me/.encfs6.xml /external-drive/backup_enc The last is a precaution, I do not recommend keeping the config with the data in this way. The encrypted files is now stored in /external-drive/backup_enc What-ever and Where-ever that may be. To directly decrypt the encrypted files mkdir /tmp/backup_decrypt encfs /external-drive/backup_enc/ /tmp/backup_decrypt ...password questions... # decrypted filesystem is now mounted fusermount -u /tmp/backup_decrypt WARNING: Encfs doesn't like the temporary files that programs like rsync and unison create. For each file rsync transfers, it first saves a temporary version under a slightly different name, then when transfer is complete it moves the file to it's intended name. As encfs --reverse can't decrypt these files it raises an error. The solution is to tell programs like this to use another directory for temporary/partial files. To get rsync working I made a directory '/tmp/rsync' and then tried rsync again with the -T option: rsync -av -T /tmp/rsync homedir_crypt/ homedir_crypt2/ ------------------------------------------------------------------------------- Interleaved EncFS Filesystems, for Plausible Deniablity NOTE the use of a --anykey can be used to create two separate encrypted trees but store them using the same 'data directory' with the same 'configuration file'. encfs crypt_dir decrypted_default EnvFS Password: normal_default_pass-phrase encfs --anykey crypt_dir decrypted_other EnvFS Password: different_pass-phrase Basically it turns off the 'key validation' against the password hash check in the ".encfs6.xml" file. Also EncFS will only make filenames that successfully decrypt visible in the decrypted mount. If the directory mounts and contains no data, then you know you have the password wrong. You will get no error about wrong password when using --anykey, because you turned off that check. As such a double check the initial password should be done when decrypting, OR by looking that at least one file (if available) has appeared in the decrypted directory. If you modify the "" in the .encfs6.xml it will also make the original password used to create the config useless, and thus a normal encfs mounting will never succeed. That is you must then always use the "--anykey" flag to mount your working encrypted filesystem. If more than two filesystems (using two passwords and same config, or same password and different config, see below), is present in the same encrypted filesystem, the files and directories will be interleaved with different sets of encrypted ilenames. Note however that sub-directories will remain separated, (preserving the original directory structure). This will make it obvious that not all the data was decrypted if you compare just the top level of a decrypted tree against and the encrypted tree, and some sub-directory with a lot of data does not match. The "encfsctl" utility makes this even easier using the 'showcruft' option. That is to find the files NOT de-codable by a specific password use... encfsctl showcruft crypt_dir EncFS Password: any_pass-phrase However if you also add chaff, and make it known that there is a a large percentage of chaff in the encrypted directory, (using readme files or extra comments in the encfs config file), there is no way anyone can tell if what is left un-decrypted is actually chaff or more encrypted data. The simplest way to make chaff that looks like encfs dtata, would be to mount the directory with a junk password, copy random files and directories, then un-mount and forget that password. You could also trash that config file and use a different config for each password you use. WARNING: If you forget a particular password, it is VERY difficult to determine which top-level files or directories belonged to the now lost password. A better but more difficult method would be the random creation of random files and sub-directories throughout the encrypted filesystem. This would also hide the original tree structure somewhat, but the administration of such chaff, could be a serious problem. ------------------------------------------------------------------------------- Remove the encfs config files from data directory Also allows use of multiple config files As a extra precaution, or to hide which ".encfs6.xml" file (and the meta-data) that you are using, you can move that file to another location, separating it from the encrypted data it decodes. Its location can be passed to encfs command using the ENCFS6_CONFIG environment variable. For example... ENCFS6_CONFIG="encfs_config_file" encfs /tmp/encfs_dir /tmp/decoded_dir This means you can now select from a different configuration file (with different password to use for a specific set of encrypted data. It also means you can now use different configuration files and different password to access different data from the same source encrypted data directory (see previous). However while the encrypted filesystem is mounted the environment variable is visible in the process list. But then so is the actual "encfs" command you used. However if you make the ENCFS6_CONFIG point to a UNIX named-pipe file, you can then cat the config file into the encfs command, then remove the pipe file. This means other users will still have no idea what config file was used for the mount. This works and was confirmed by Valient Gough, valient_AT_gmail_DOT_com as the config file only open/read/close by "encfs", one time only. Once it is read, the config is no longer needed, though the decrypted data remains accessable. As config files are now separate to the encrypted filesystem, you can store them separately on a portable USB stick. That means a user not only needs a password, but also a physical 'key' (the USB holding the config) to mount the encfs filesystem. Also as separate config files are able to be used you can use a seperate config for different passwords in a 'interleaved' encrypted filesystem. This means changing one password (and its config) will then not effect the password and config of another interleaved filesystem. My key storage "ks" system does this for EncFS as part of keeping configs, user passwords, and encrypted data, completely separate from each other. See "Key Storage" below. ------------------------------------------------------------------------------- Encrypting a Secret as a EncFS filename! As a random off hand thought, using EncFS filenames to hold information. This could be used as programed method of storing secret information, using a internal password, and an existing external encfs config file, to decrypt the actual password. For example store a password some software program needs to access something else. This lets you hardcode a password into a program without making it easy for a hacker to recover. especially if they have the program but not the encfs config file! The secret information stored as an encrypted file name which is created in the EncFS directory using ANY password key (using --anykey), and the current configuration file. Once setup you do not need to mount the EncFS directory to get the file name (the secret) or even have a file in the encfs directory tree (you will however need a EncFS tree or more specifically a encfs config file to use for the encryption/decryption process). That is with a 'config file' and any 'password' you can decrypt a string using the EncFS filename crypting method. First get the encrypted file name. ls crypt_dir > ~/t encfs --anykey crypt_dir some_where EnvFS Password: file_pass-phrase touch some_where/SECRET_INFO_AS_A_FILENAME fusermount -u some_where cd crypt_dir mv `ls crypt_dir | diff - ~/t` # the encrypted_filename with that data The encrypted_filename is at least as long as the original filename, usually slightly longer due to the encryption block size used. But you don't actually need the file anymore, just the string. rm crypt_dir/encrypted_filename Now to get the secret from the string you can decrypt it as a filename. The file itself does not have to exist, but we need a EncFS directory or at least a config. encfsctl decode crypt_dir encrypted_filename EncFS Password: file_pass-phrase SECRET_INFO_AS_A_FILENAME Note it may be better and easier to do this using OpenSSL encryption, and a base64 hashing instead. But it is an interesting, and novel idea, that is not as 'common' and OpenSSL encryption. ------------------------------------------------------------------------------- Encrypt Filesystem on the Encrypted Filesystem This should be very possible, but probably would not provide any real benefit in terms of security. You just need two passwords to decrypt things down to the final level. Basically this is not a good idea. ------------------------------------------------------------------------------- EncFS Key Storage System ("ks") Protect Encrypted Filesystems with Encrypted 'Key' Files (similar to LUKS) A better technique with far less overhead, is to use the user password, to decrypt a 'key file' that holds the real password used to decrypt the data. This extra level of indirection allows real data to be encrypted with a strong, random generated keys, while storing that key and all the other configuation information in a completely separate area. The user does not need to personally know that hard to remember purely random master password, or any details of the encryption. Not only that but by using 'key files', the users password can be changed easilly simply by re-encrypting the 'key file' without needing to decrypt, copy, and re-encrypt the actual data, which can take a very long time. The hard to remember master password remains the same. You can even give different users their own 'key file' with their own separate password, to decrypt the same data. Individual The users do not need to share the password, and administaions can control access via each users individual 'key file'. This two-stage password indirection is used by LUKS disk encryption and is presented in the paper... "TKS1, An anti-forensic, two level, and iterated key setup scheme." http://clemens.endorphin.org/TKS1-draft.pdf Also see.. Dual password encryption with EncFS http://magazine.redhat.com/2007/06/13/ And my own "Key Store" ("ks") script that provides this method for EncFS. http://www.ict.griffith.edu.au/anthony/software/#ks My 'Key Store' program stores the 'key files' in a completely separate area such as a removable USB stick, so that you can arrange for some physical security of the access keys. The key files also hold all the other details of the encryption: such as the actual ".encfs6.xml" configuration file to use, the location of the encrypted data, optionally where it should be mounted (if fixed). It also stores the command needed to access (mount or display) the data, allowing any type of encryption to be used, not just EncFS. The key files also provide a way of encrypting smaller individual files such as a file holding your web passwords, for easy access when needed, but without needing a large separate encrypted data store (EncFS). That is store that data where the EncFS Config file data would live in the full setup. -------------------------------------------------------------------------------