------------------------------------------------------------------------------- EncFS - Encrypted File System (micro-how-to) The EncFS file system is an ordinary directory containing encrypted files using a encryption hashed' filenames. This 'directory' is mounted to another directory (using fuse) where it will appear in its decrypted form. That is the encrypted files are just ordinary binary files and with the correct times and tree structure. Only the names and contents of the files remain hidden. The directory tree, metadata, and rough file sizes are preserved unencrypted! Advantages * The encrypted can be stored on any normal file system, again without decryption. That includes SAN, USB, or DVDs. You do not even need to worry that the 'remote' file services have access to your data as their system never sees the decrypted data or passwords. By default they could see the encfs config file, unless separated (see below). * Hardlinks work if the underlying filesystem allows their use. * You can copy, update or synchronised the directory with another copy of that directory (say on another machine) without needing to decrypt it or even have the password! As such automated backups can still be performed securely, even by other people. * Individual files can be randomaly accessed, like Disk/Block encryption. For example you can jump into the middle of a very large encrypted video file! There is no need to decrypt the whole file from teh beginning. * Similarly you can use Rsync 'delta difference' to syncronize between multiple copies of the same encrypted file system! In fact ANY file system copy or transfer method can be used. The files remain... files. * You can syncronize with 'the cloud' without worry about some unknown SysOp looking at your files, regardless of what the their policy is or is not. Do not however upload the 'configuration file ".encfs.xml" * You can (with special techniques) merge two or more (I have a dozen) encrypted directories trees into the same top level (or faked top levels) of the same encrypted data directory. There is a small chance of filename clashes. That is one encrypted filename is used by two seperate repositories, but is is highly unlikely. This is can only happen on the top level as all sub-directories contents remain separate, as each sub-directory can only belong to one specific encfs repository. * You can add 'chaff' to the encrypted directory with (faked hashed file names and directories) to prevent someone knowng if all the data was decrypted or by which password. Some files may never be decryptable, or may have been created and left behind by 'lost' password. This allows you to prevent rubber hose attacks if you make it obvious such chaff data exists. Disadvantages * VERY light overhead (all schemes need some overhead) * The 'encrypted hash filenames' make it obvious the directory contains an EncFS (or similar) filesystem. The presence of a encfs config file would simply confirm this (unless you store that config elsewhere). * Directory Structure is the same, apart from ".encfs.conf" on the top level. Unless extra chaff (which fails to decrypt) is added. * If interleaving two encfs on top of each other, only the top level will be shared between the two files systems. The individual sub-directories will remains seperate to each of the interleaved encfs encryptions. * File metadata (apart from filename and file size) remains exactly the same. That is: File inode, permissions, ownership, group, date, etc * File sizes are roughly the same but not exactly. The encrypted files is a few bytes larger, due to encryption block rounding upward, and individual file salting. * The encrypting information (normally in ".encfs.conf") , including iteration count, and salt, is visible in the encfs config file at the top level of the encfs directory tree. That is the 'public' parts of the encryption process, leaving just the password missing. This file can also be interfered with. Suggested improvements * Remove access to the "configuration file", perhaps by using the users password to define its filename, and decrypt it in some 'standard' way. Once it is decrypted the information it provides can be used to decrypt the rest of the file system. * Ability to add and manage chaff, to be automatically added as needed. This can be used to hide just how much information is actually present and 'fuzz' teh directory structure to some extent. ------------------------------------------------------------------------------- Implementations and related applications FUSE for linux -- the original implementation of EncFS All modes supported, can do reverse encfs, config seperation, and even network mounted encrypted stores (like dropbox) BoxFetch for Windows -- GUI app for Windows, and Mobile phones Note it doe not support all modes, just more common one CloudFetch for Android -- Encfs with data stored in the cloud Dropbox, SpiderOak, SparkleShare and Ubuntu One https://play.google.com/store/apps/details?id=az.cloudfetch EncFSAnywhere A Java app that allows your browser to look at EncFS filesystems stored on a cloud service such as DropBox https://bitbucket.org/marcoschulte/encfsanywhere ------------------------------------------------------------------------------- Basic Installation and setup of EncFS, using FUSE install sudo apt-get install encfs create and/or mount encfs ~/encrypted/ ~/decrypted unmount fusermount -u ~/decrypted or umount ~/decrypted As it is a FUSE based file system, it is completely user level controlled, which little kernel requirements and no superuser access. The EncFS is a 'pass-thru' file system that encrypts the individual files access via a fake "decrypted" file system, as individual files in the "encrypted" directory. This means that the files sizes, directory structure, permissions and times is visible in the encrypted directory tree. Though the file names and file data is encrypted. However as the encrypted file system are still plain files, it can be incrementally backed up as files, or syncronized across multiple machines, using normal file system backup utilities rather than requiring the whole files system to be treated as a single unit, or un-encrypted for the data transfer. The file system also does not require pre-allocated disk space (such as a fix sized file, or seperate partition), but will grow and shrink as needed in a normal file system. That is because files are encrypted to files. You can NOT really use it to encrypt the whole home, data, system, or swap memory of the computer for complete protection. This is better achieved using a lower level disk block encryption such as the linux dmcrypt. But then few people need to protect there whole computer, just the contents of a few files. Even then there are ways around this (see "home_encryption.txt") ------------------------------------------------------------------------------- Case-insensitive Filesystems To store EncFS on a file system that is case insensitive for lookup, (VFAT, NTFS, HFS+, etc), create it using expert mode and select B32Block or B32Stream as the filename cipher to use. ------------------------------------------------------------------------------- Encrypted Filename Handling, and internals The "encfsctl" program provides utilities to encode and decode file paths. (given the encfs config file ".encfs6.xml" and the password). These are not all documented. A password will always be needed. But more importantly the encfs filesystem does not need to be mounted in decrypted form, to convert between a encrypted and unencrypted filename, or to decrypt one specific file stored in the encfs filesystem. Option Examples... encfsctl decode encfs_dir encfs_filename decrypts filename (not contents) to its unencrypted filename That is is this encrypted files decrypted filename. encfsctl encode encfs_dir filename encrypts the given filename to the encrypted file path (result includes the includes encfs_dir prefix) encfsctl ls encfs_dir decrypt and lists all filenames in a directory (note top level gets listed at this time) encfsctl cat encfs_dir filename Show contents of the file encfsctl export encfs_dir dest_dir decrypts all the files found in the encrypted directory to the dest_dir This means you can lookup and even rename (move) the encrypted filenames without needing actually mount the encfs filesystem. This can be used to generate a listof encrypted filenames. For example generate a 'exclude list' of encrypted filenames which you don't want backed up or copied to some remote server (EG: temorary, unimportant, of constantally changing files or directories). However remember the filename encryption is dependant on the whole file path, unless you specifically turn off "Filename Initialization Vector Chaining". As such you can not simply move a encrypted file into a different sub-directory in the encrypted tree, as the filename in that sub-directory will be different. To move a file you need to comletely rename all parts of its file path, including its actual filename. NOTE: If 'External IV Chaining' is turned on in the ".encfs6.xml" file (off by default) then the file contents will also depend on the file path in the encrypted directory. In this situation, hardlinks will no longer work, and renaming a directory will also require a re-encryption of all the files in that directory. This can be a very massive overhead, especially for very large files, which is why it is not enabled by default. Newer patched version... Environment variable ENCFS_PASSWORD_FD with a -S option will have encfsctl read passwords from the given fd (file descrptor) number rather than from stndard input. If the patch is present then 'encode' will produce a relative path rather than an absolute path, making its reverse compatible of 'decode' See http://code.google.com/p/encfs/issues/detail?id=33 Less obvious decode filename usage... Another less obvious way of matching a real filepath to its encrypted filepath is to look at the 'openfiles list' of the "encfs" process that is mounting the file system. For example... encfs ... /tmp/encrypted /tmp/decrypted # Do the mount pid=$(pgrep encfs) # find the process lsof -p $pid > t1 # list open files exec 5 t2 # list open files exec 5<&- # close filename to encrypt comm -3 t1 t2 | sed 's%.*/tmp/encrypted%%' # compare open file lists Results CvtrBM5uitlmTte2Hxrk0J-T So "README" is actually the encfs filename "CvtrBM5uitlmTte2Hxrk0J-T" Filename Encryption Algorithm (not reproducible yet) The filename is encrypted using MAC-16 ^ IV and key then the string... MAC-16 + encrypted_filename is converted to base32 or base64 depending on the filesystem case sensitivity. This is then base64 encoded which is modified such that the characters './' are replaced with ',-' respectiveally. see http://code.google.com/p/encfs/source/browse/trunk/encfs/base64.cpp line 122 A checksum (MAC-16) is created. The reverse is the same except the MAC checksum is compared. See sources... http://code.google.com/p/encfs/source/browse/trunk/encfs/base64.cpp http://code.google.com/p/encfs/source/browse/trunk/encfs/BlockNameIO.cpp#145 http://code.google.com/p/encfs/source/browse/trunk/encfs/StreamNameIO.cpp#102 Shell equivelent (to be worked out)... Shell Decrypt... ------------------------------------------------------------------------------- encfs and encfsctl will read passwords from TTY by default BUT... -S, --stdinpass Read password from standard input. OR from the file descriptor number in ENCFS_PASSWORD_FD --extpass=program lets you call a password gathering program such as "ssh-askpass" with the environment variable "RootDir" pointing to the encrypted directory. Trailing newline is striped, 1/2k passwords at most (though manual says it is 2K) Program may be a shell pipeline. --extpass='command_1 | command_2' Remember for passwords any character could be used including newlines, though typically newlines and nulls are not used for obvious reasons. Caution is recommended. Extra FUSEmount flags (after a -- option) -- -c request kernel-level caching of file data (speed up) -- -n /dev/xxx specify this device in 'df' command output instead of reporting it as being "encfs" ------------------------------------------------------------------------------- Remote encrypted data directories. Store the encrypted data on a remote system and mount it using "mount.cifs" (windows home) or "fuse-sshfs" (ssh filesystem) or even "dropbox" to syncronize between local and remote copies. Note you don't need encfs config file on the encrypted filesystem stored on the remote filesystem, though you will need it to decrypt that filesystem on any machine that you want to make use of it. This requires two layered FUSE mounts... sudo yum install fuse-sshfs sshfs user@storage:raw_data ~/raw_data -o "uid=$(id -u),gid=$(id -g)" ENCFS6_CONFIG=~/lib/data_encfs6.xml encfs ~/raw_data ~/data ... fusermount -u ~/data fusermount -u ~/raw_data ------------------------------------------------------------------------------- EncFS also allows reversed encryption. That is keep the local data unencrypted, but create a 'EncFS' encrypted image of the same data, to copy to a remote system for distribution or backup. Making a copy of the reversed encrypted EncFS directory and then using 'normal' EncFS to decrypting it should work perfectly fine. A copy of the ".encfs6.xml" configuration (not the password) is also transfered (unchanged) to ensure that this is directly posible without any extra effort. This also allow you to generate a EncFS encryption of your data, which can then be used to generate rsync backup 'snapshots' on a remote, and posibilty insecure server. You then do not need encryption for normal use. The rsync delta differences used for file transfer continue to work as normal. Example... Setup (copy to) the encrypted backup (using reverse) mkdir /tmp/backup_encrypt encfs --reverse /home/me /tmp/backup_encrypt ...password questions... rsync -av --delete /tmp/backup_encrypt/ /external-drive/backup_enc fusermount -u /tmp/backup_encrypt cp /home/me/.encfs6.xml /external-drive/backup_enc The last is a precaution, I do not recommend keeping the config with the data in this way. The encrypted files is now stored in /external-drive/backup_enc What-ever and Where-ever that may be. To directly decrypt the encrypted files mkdir /tmp/backup_decrypt encfs /external-drive/backup_enc/ /tmp/backup_decrypt ...password questions... # decrypted filesystem is now mounted fusermount -u /tmp/backup_decrypt WARNING: Encfs doesn't like the temporary files that programs like rsync and unison create. For each file rsync transfers, it first saves a temporary version under a slightly different name, then when transfer is complete it moves the file to it's intended name. As encfs --reverse can't decrypt these files it raises an error. The solution is to tell programs like this to use another directory for temporary/partial files. To get rsync working I made a directory '/tmp/rsync' and then tried rsync again with the -T option: rsync -av -T /tmp/rsync homedir_crypt/ homedir_crypt2/ ------------------------------------------------------------------------------- Interleaved EncFS Filesystems, for Plausible Deniablity NOTE the use of a --anykey can be used to create two separate encrypted trees but store them using the same 'data directory' with the same 'configuration file' (though it does not even need to be the same config file). encfs crypt_dir decrypted_default EncFS Password: normal_default_pass-phrase encfs --anykey crypt_dir decrypted_other EncFS Password: different_pass-phrase Basically it turns off the 'key validation' against the password hash check in the ".encfs6.xml" file. That 'hash' will no longer matter. Also EncFS will only make filenames that successfully decrypt visible in the decrypted mount. If the directory mounts and contains no data, then you know you have the password wrong. You will not get an error about wrong password when using --anykey, because you turned off that check. As such a double check of the initial password should be done when decrypting, by ensuring that at least one file (if available) has appeared in the decrypted directory. If you modify the "" in the ".encfs6.xml" it will also make the original password used to create the config useless, and thus a normal encfs mounting will never succeed. That is you must then always use the "--anykey" flag to mount your working encrypted filesystem. If your password is wrong the mount will still succeed, but no files will be visible. If more than two filesystems (using two passwords and same config, or even different configs - see below), is stored in the same encrypted filesystem, the files and directories will be interleaved with different sets of encrypted ilenames. Note however that sub-directories of each filesystem will remain separated, (preserving the original directory structure). This will make it obvious that not all the data was decrypted if you compare just the top level of a decrypted tree against and the encrypted tree, and some sub-directory with a lot of data does not match up. Of couse that sub-directory may be just fake 'chaff'. The "encfsctl" utility makes this even easier using the 'showcruft' option. That is to find the files NOT de-codable by a specific password use... encfsctl showcruft crypt_dir EncFS Password: any_pass-phrase However if you also add chaff, and make it known that there is a large percentage of chaff in the encrypted directory, (using readme files or extra comments in the encfs config file). That way there is no way anyone can tell if what is left un-decrypted is actually chaff or more encrypted data. The simplest way to make chaff that looks like encfs data, would be to mount the directory with a junk password, copy random files and directories, then un-mount and forget that password. You could also trash that config file and use a different config for each real filesystem you use. WARNING: If you forget a particular password, it is VERY difficult to determine which top-level files or directories belonged to the now lost password. A better but more difficult method would be the random creation of random files and sub-directories throughout the encrypted filesystem. This would also hide the original tree structure somewhat, but the administration of such chaff, could become a serious problem. You should also 'touch' files and directories at random throughout the working life of the EncFS directory, as you don't want the chaff to be just all the 'old' files in the encrypted file system. ------------------------------------------------------------------------------- Remove the encfs config files from data directory (Also allows the use of multiple config files) As a extra precaution, or to hide which ".encfs6.xml" file (and the meta-data) that you are using, you can move that file to another location, separating it from the encrypted data it decodes. Its location can be passed to encfs command using the ENCFS6_CONFIG environment variable. An example of doing this was already done for example in 'remote mounted encrypted directories above. For example this stores the config file outside the encfs directory... ENCFS6_CONFIG="encfs_config_file" encfs /tmp/encfs_dir /tmp/decoded_dir However while the encrypted filesystem is mounted the environment variable is visible in the process list. But then so is the actual "encfs" command you used. However if you make the ENCFS6_CONFIG point to a UNIX named-pipe file, you can then cat the config file into the encfs command, then remove the pipe file. This means other users will still have no idea what config file was used for the currently mounted decrypted filesystem. This works and was confirmed by Valient Gough, valient_AT_gmail_DOT_com as the config file is only open/read/close by "encfs", one time only. Once it is read, the config is no longer needed, as it has the encryption key in memory. The decrypted data remains accessable to the encfs process even though the configuration file is nolonger available. Multiple config files.... As config files can be separated from the encrypted filesystem, you can store them separately on a portable USB stick, or in some other encrypted filesystem. That means a user not only needs a password, but also a physical 'key' (the USB holding the config) to mount the encfs filesystem, and only during the mount process. As separate config files are able to be used you can use a seperate config for each passwords in a 'interleaved' encrypted filesystem (see plausible denyability above). This means changing one password (and its config) will then not effect the password and config of another interleaved filesystem. My key storage "ks" system does this for EncFS as part of keeping configs, user passwords, and encrypted data, completely separate from each other. See "Key Storage" below. ------------------------------------------------------------------------------- Encrypting a Secret as a EncFS filename! (don't bother) As a random off hand thought, using EncFS filenames to hold information. This could be used as programed method of storing secret information, using a internal password, and an existing external encfs config file, to decrypt the actual password. For example store a password some software program needs to access something else. This lets you hardcode a password into a program without making it easy for a hacker to recover. especially if they have the program but not the encfs config file! The secret information stored as an encrypted file name which is created in the EncFS directory using ANY password key (using --anykey), and the current configuration file. Once setup you do not need to mount the EncFS directory to get the file name (the secret) or even have a file in the encfs directory tree (you will however need a EncFS tree or more specifically a encfs config file to use for the encryption/decryption process). That is with a 'config file' and any 'password' you can decrypt a string using the EncFS filename crypting method. First get the encrypted file name. ls crypt_dir > ~/t encfs --anykey crypt_dir some_where EnvFS Password: file_pass-phrase touch some_where/SECRET_INFO_AS_A_FILENAME fusermount -u some_where cd crypt_dir mv `ls crypt_dir | diff - ~/t` # the encrypted_filename with that data The encrypted_filename is at least as long as the original filename, usually slightly longer due to the encryption block size used. But you don't actually need the file anymore, just the string. rm crypt_dir/encrypted_filename Now to get the secret from the string you can decrypt it as a filename. The file itself does not have to exist, but we need a EncFS directory or at least a config. encfsctl decode crypt_dir encrypted_filename EncFS Password: file_pass-phrase SECRET_INFO_AS_A_FILENAME Note it may be better and easier to do this using OpenSSL encryption, and a base64 hashing instead. But it is an interesting, and novel idea, that is not very 'common', and thus harder for a attacker to figure out. ------------------------------------------------------------------------------- Encrypt Filesystem on the Encrypted Filesystem This should be very possible, but probably would not provide any real benefit in terms of security. You just need two passwords to decrypt things down to the final level. It just adds more overhead to an already secure encryption. Basically this is not a good idea. However storing the passwords for encrypted filesystems in a key storage system is a good idea.. See "key_storage.txt" or my own "ks" program which understands EncFS setup and mounting. =============================================================================== Other FS Encryption Programs ------------------------------------------------------------------------------- GoCryptFS - Encrypted File System written in GO language. Source https://github.com/rfjakob/gocryptfs Now pre-built in fedora repository Uses Authenticated Encryption (data verifiable). Reversed encryption (but read only), and hardlinks are supported. Specific Advantages... * Faster for reading, writing, deleting. * Per-directory IV is stored as a file in a file in each directory so that renaming a sub-directory does not involve renaming every file within that sub-directory However this file makes it obvious that GoCryptFS is being used. You can separate the config file from the file system (using "-config" option) and can even do away with that config entirely if you provide the master cryptographic password directly. The other major disadvantage is that as it is written in GO, the FUSE library isn't available on as many systems (Windows, Android). Introduction https://opensource.com/article/19/8/how-encrypt-files-gocryptfs To try -- encrypted directory interleaving ------------------------------------------------------------------------------- CryFS Flattens the File System so as to hide its structure. You can also seperate the config from the rest of the encrypted data, like EncFS. Disadvantages * All files, including small ones are stored in 32Kb file chunks That means a directory of small files will become a LOT larger, Also it means meta-data (modification times) are not a problem. * Scalability in the number of files may become unmanageable, when very large files are stored. But you can increase chunk sizes to compensate. * I found it VERY slow compared to the original file system, or even against encfs. This was a significant factor for me. Deletion of files and directories can be especially slow! * Hardlinks are not supported. No Info on * Interleaving of Encrypted Trees * What obvious indicators that CryFS is used. ------------------------------------------------------------------------------- SecureFS Also using newer Authenticated Encryption for data verification. Lite mode: Similar to EncFS with file system correspondence. Each file has its own encryption key No info on if hardlinks is allowed, or is DirIV is an issue on directory renaming Full Mode (create --format 2): Directory structure stored as a B-Tree. But not the files. Basically it will flatten the directory structure. It handles Sparse files (zeroed disk blocks) Base 32 filename encoding is used (to allow case insensitive Filesystems) No info on * Hardlinks (at least in lite mode) * Directory IV and renaming issues. * Interleaving of Encrypted Trees * is Attributes kept? (dates, executable flag) PROBLEM: The config file must be a real seekable file, (fixed in v0.8.2 onwards) ------------------------------------------------------------------------------- eCryptFS In many ways this is very similar to EncFS, except each file is encrypted completely separally to each other file. Filename encryption is optional, but like EncFS file metadata (time stamps, rough size, and permissions) are visible. It is now a standard option for Ubuntu Linux, which can only be a plus, and allows for good user interfaces to be developed. I have not yet experimented with the filesystem, though the use of the kernal key ring means the cryptographic key may be more exposed while the user is logged in. Also while using your login password may be okay for your home directory, I would not recomend that for more personal data filesystems. I am also wondering how home encryption, and public directory access is handled. EG: personal web pages, .ssh login keys, as noted above. -------------------------------------------------------------------------------