------------------------------------------------------------------------------- Using OpenSSL for encryption and key generation This command has a lot of features, from random data generators, password hashing, encrypting data and files, hash checksums, through to encrypted SSL connections to remote systems. Help http://www.madboa.com/geek/openssl/#encrypt-simple Otherwise for a list of encryption types use... openssl list -cipher-commands -1 And digests openssl list -digest-commands -1 ------------------------------------------------------------------------------- OpenSSL can read all arguments from a stream (good for programmed use) ( echo -n "enc -aes-256-cbc " echo -n "-in file.txt -out file.enc " echo -n "-pass pass:passwd" echo "" ) | openssl Note: "openssl" still outputs a prompt (EG: "OpenSSL> ") and warnings however which will need to be removed. ------------------------------------------------------------------------------- File/Stream Encryption Example (using newer pbkdf2 key derivation)... # encrypt file.txt to file.enc using 256-bit AES in CBC mode openssl enc -aes-256-cbc -salt -md sha256 -pbkdf2 -iter 500000 \ -in file.txt -out file.enc You can optionally add '-e' to ensure it is encryption, but that is the default. Similarly while the '-salt' is the default, it wasn't always and is better to specifically specify either '-salt' or '-nosalt'. Defaults from v1.1.1: -salt To decrypt add: '-d' to the exact same options used to encrypt the data stream/files. # decrypt binary file.enc openssl enc -d -aes-256-cbc -salt -md sha256 -pbkdf2 -iter 500000 \ -in file.enc -out file.txt You can list the all the cyphers available using openssl enc -ciphers To list hashing digests use openssl dgst -list Other options of note... -none No encryption, salt, or file magic. Data is passed through as is. -z Compress the plaintext before encryption (see Caveat below) -a ASCII Armour using base64 of the encrypted output (or '-base64') -A As '-a' but with inserting extra new lines (one line output) In ASCII armour mode ('-base64', '-a' or '-A') the "Salted__" file magic will start with "U2FsdGVkX1" with next character is slightly variable, according to the effect of the random salt. ------------------------------------------------------------------------------- Openssl -z Compression acts VERY weird According to the 'enc' manual '-z' will compress plain text, before encrypting if openssl was built with zlib. In other words though the input is compressed, the output should still look like a normal Salted__ encrypted file. My version (linux fedora 31, v1.1.1g) does have zlib included, but the results do not correspond with what the manpage suggest should happen. If I encrypt something using '-z' the file ends up slightly BIGGER, though the input (plain text) should compress very well. It decrypts (using the same options, plus '-d') just fine however. openssl enc -aes-256-cbc -salt -pbkdf2 -iter 500000 -z < in > out Looking at the file I can see the 'Salted__' file magic in the output but there is an extra 8 bytes of data infront of the file magic! That is completely different to what is documented! Further adding to the weirdness, if I add '-base64' I would expect the output to be exactly the same, just with base64 encoding (ASCII Armoured in GPG parlance). But instead the output remains a binary file, but completely different and only slightly bigger again (not really enough to take into account the 3 to 4 byte mapping increase base64 provides. Again it still decrypts fine, with the same options. Again this seems to show the expected output is being modified AFTER it was encrypted not before! Now I did find out there is an extra configuration setting that is needed when openssl is complied to enable '-z' compression to work, but what I am seeing does not correspond to anything that manual suggests should happen. Posted on https://stackoverflow.com/questions/62112663/ ------------------------------------------------------------------------------- Salting The 'salt' is a random 8 binary bytes (64 bits), that is stored with the encrypted file so that every time you encrypt a file with a specific password or key the resulting file will be different. However it also adds 8 character 'file magic' of "Salted__" to the start of the encrypted data stream. Which identify the encryption as using a newer format. For example... echo "data" | openssl aes-256-cbc -pass pass:keyvar Salted__........ Where the '.' is the 8 binary bytes of salt, after that encrypted data blocks follow. Length of this small file will be: 8 magic + 8 salted + 16 (one data block, padded) or 32 bytes in total. All encrypted files are generally a multiple of 16 bytes Unsalted... If '-nosalt' is used then no salt is added, but also no file magic identifier. This is the old method, (PBKDF1 rather than PBKDF1.5). The file in this case is just a pure AES encryption of data, without any header at all. However without salt a fixed password/file will always produce the same deterministic encrypted result. In this case one block, or 16 bytes. Which makes it easy to generate 'raindow tables' of common passwords to try decrypting files with. Here we use '-a' outputs the result in base64 encoding. echo "data" | openssl aes-256-cbc -a -pass pass:keyvar -nosalt HX1CW6wu9h+PUeWhrpJiDA== WARNING: To decode a old '-nosalt' encryption, you need the -nosalt option. Though openssl should be able to determine salt/nosalt status of a file. ------------------------------------------------------------------------------- Passphrase handling in OpenSSL From command line... -pass pass:mySillyPassword From file -pass file:/path/to/secret/password.txt From File descriptor -pass fd:N From stdin -pass stdin Only the last two methods can be deemed to be 'secure' from other users on the machine it is run on. See including "password from pipeline" above The from a "file:" method can be secure using named pipes, but only if the named pipe file is secure. Watch out for newlines! ------------------------------------------------------------------------------- Password in same pipeline By including the password as part of the pipeline you avoid the need to place the password on the command line, environment variable, or reading it from a TTY, or other file decriptor. Great for shell scripts. Note the use of "echo -n" for encrypting the data, but a return is needed for decrypting the 'base64' encrypted data. Otherwise it you could get errors such as "wrong final block length" or "error reading input file" # encrypt passwd="ThePassword" secret="This is a Secret" encrypted=`( echo "$passwd"; echo -n "$secret"; ) | openssl enc -aes-256-cbc -pbkdf2 -iter 500000 -pass stdin -base64` # Result (in base64) echo "$encrypted" # output is variable but it is something like... # U2FsdGVkX192RszlbKdgyY7wk/3Y68FOiTur96QpKkhbirU56vn/olVqeee+NOMp # Note the start "U2FsdGVkX1" is the file magic, "Salted__", and will # always be the same. The next group (decoding 8 bytes) is the salt used # to randomise the password hashing and thus the encryption. # decrypt message=`( echo "$passwd"; echo "$encrypted"; ) | openssl enc -d -aes-256-cbc -pbkdf2 -iter 500000 -pass stdin -base64` echo $message # outputs: "This is a Secret" ------------------------------------------------------------------------------- Version changes.... The default message digest ("-md") changed from 'md5' to 'sha256' in v1.1.0 this means people relying on the defaults for their encryptions before this version will need to specifically set "-md md5" to decrypt. Before openSSL v1.1.1, the user password used the fast PBKDF1.5 password with the digest to convert it to a cryptographic key and IV. As such it is prone to brute force dictionary attacks, even with salting. After this version OpenSSL gives warnings that you should use the new option it provided "-pbkdf2" and "-iter" to hash the users password that many times. The default iteration count '10000' is however very low. It is recommended you use a much higher iteration count, so that it takes anywhere from 1 to 5 seconds to hash the users password into the correct cryptographic key. However only the 'salt' (to randomise the password) is stored in the output file, and some means of remembering the options used, especially the iteration count which should increase with time. This is even more important as default options will likely change in the future. See "keepout" which provides a wrapper and extra header to the encrypted file to save the openssl options needed to later decrypt the file. https://antofthy.gitlab.io/software/#keepout From Is there a standard for OpenSSL-interoperable AES encryption? https://crypto.stackexchange.com/questions/3298/ Using password-based encryption and decryption depends critically on using the same parameters and especially the same hash in the PBKDF for both operations. As noted above, the default hash used by the enc command changed in 1.1.0 to SHA256 versus MD5 in lower versions. Thus data encrypted using the default hash in lower versions won't decrypt with the default in 1.1.0 or vice versa. if data was encrypted with the default (MD5) on an older version, decrypt in 1.1.0 (and presumably higher when released) with -md md5 if data was encrypted with the default (SHA256) on 1.1.0 (or presumably higher), decrypt in an older version (at least back to 0.9.8, possibly before) with -md sha256 To proactively prevent a problem, specify -md consistently for both encryption and decryption even when it's redundant. ------------------------------------------------------------------------------- Key derivation See "key_derivation.txt" in this directory. You can look at the cryptographic key abd iv generated from combining the users password with a salt (and later a PBKDF2 iteration count), using either the "-P" option (which aborts the conversion) or "-p". For example here is a encryption (aborted by the "-P" option) so openssl will generate a random salt. It then shows the result of combining that salt with the users password to generate the cryptograhic key and iv. openssl enc -aes-256-cbc -pass pass:MYPASSWORD -P *** WARNING : deprecated key derivation used. Using -iter or -pbkdf2 would be better. salt=C0DD5FFEBC0FB3F1 key=D534F056E85A5FFA3D7B45CF4137D2CCCA2CB4DE9623C3F31E6723E02B0A59DD iv =9548317A164D545516F556693E69138F Note the warnings generated by "openssl" in the newer versions of openssl. Here we repeat this above but this time we supply the salt to use. This is equivelent to reading the salt from an encrypted file so openssl can re-generate the same cryptographic key and iv from the users password, so it can now decryptthe file. openssl enc -aes-256-cbc -S C0DD5FFEBC0FB3F1 -pass pass:MYPASSWORD -P *** WARNING : deprecated key derivation used. Using -iter or -pbkdf2 would be better. salt=C0DD5FFEBC0FB3F1 key=D534F056E85A5FFA3D7B45CF4137D2CCCA2CB4DE9623C3F31E6723E02B0A59DD iv =9548317A164D545516F556693E69138F With PBKDF2 enabled the generation of the key and iv is iterated many times (10000 by default) to slow it down and make brute-force attacks a lot harder. Of course we get a different result. openssl enc -aes-256-cbc -S C0DD5FFEBC0FB3F1 -pbkdf2 \ -pass pass:MYPASSWORD -P salt=C0DD5FFEBC0FB3F1 key=90DCA77E8B9FAE8B794415CD4152100436C4E8D81616D76231BA66D877F00414 iv =952FB80BB2B5690DA7D301FE05B97359 This iteration count is very low for todays super fast computers. It should be increased until the time taken to generate the key and iv is around a second or more. It should also get larger as time goes on. time openssl enc -aes-256-cbc -S C0DD5FFEBC0FB3F1 -pbkdf2 -iter 2000000 \ -pass pass:MYPASSWORD -P salt=C0DD5FFEBC0FB3F1 key=02EB03D0714D20761AA7C77A8E4E547412FB976DC3BA36EAFA20668203A88945 iv =486AD1B80EBB9708F7A1C4717298FD4F real 0m1.697s user 0m1.693s sys 0m0.003s For aes-128 Key = MD5(passwd + salt) IV = MD5(Key + Password + Salt) Note that this password hashing function is simple (and thus fast) It does not provide any protection in the form of a slow variable hash. For aes-256 the key size is 32 so a longer sequence is needed Hash0 = '' Hash1 = MD5(Hash0 + Password + Salt) Hash2 = MD5(Hash1 + Password + Salt) Hash3 = MD5(Hash2 + Password + Salt) Hash4 = MD5(Hash3 + Password + Salt) ... The hash is then split to generate the 'Key' and 'IV' needed to decrypt. Key = Hash1 + Hash2 IV = Hash3 See EVP_BytesToKey() function http://www.openssl.org/docs/crypto/EVP_BytesToKey.html A different hash could be used, with the sequence continuing until enough bits have been hashed to set both Key + IV in sequence. ASIDE: PKCS5 Password based encryption #5 (also know as PBKDF1) PBKDF1.5 Addition of a salt to the password hashing, one iteration PBKDF2 is the iterative hashing method to derive Key and IV HMAC-SHA1 hashing function used for PBKDF ------------------------------------------------------------------------------- Error Interpretation from log... sshd[31784]: error: RSA_public_decrypt failed: error:0407006A:lib(4):func(112):reason(106) look up openssl errstr 0407006A error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1: block type is not 01 The RSA_padding_add_PKCS1_type_1(3) manpage then says... block methods for signatures ------------------------------------------------------------------------------- OpenSSL 3.0.x -S When you specify a salt using -S the "Salted__" file magic string is no longer written or read by openssl ------------------------------------------------------------------------------- In Python... Specific Python Module for handling OpenSSL enc https://www.example-code.com/python/openssl_enc_decrypt.asp How to decrypt OpenSSL AES-encrypted files in Python? https://stackoverflow.com/questions/16761458/ See example in... store/crypto/python_openssl_enc.py But it is not working! =============================================================================== Other uses of openssl... =============================================================================== Random Data Generation # write 128 random bytes of base64-encoded data to stdout openssl rand -base64 128 # write 1024 bytes of binary random data to a file openssl rand -out random-data.bin 1024 # seed openssl with semi-random bytes from browser cache cd $(find ~/.mozilla/firefox -type d -name Cache | head -n 1) openssl rand -rand $(find . -type f -printf '%f:') -base64 1024 # get 32 bytes from /dev/urandom and base64 encode them head -c 32 /dev/urandom | openssl enc -base64 Primes # Generate a 64 bit prime number openssl prime -generate -bits 64 # check a number is prime openssl prime 17339130802889096893 ------------------------------------------------------------------------------- Checksums and Hashing # This is exactly like "md5sum" command but with different output format openssl dgst -md5 filename MD5(filename)= 81eda7985e99d28acd6d286aa0e13e07 # This is exactly like "sha1sum" command but with different output format openssl dgst -sha1 filename SHA1(filename)= e4eabc78894e2c204d788521812497e021f45c08 You can also output the hash as -binary or -hex (default) Unfortunateally you can not output it in base64. But you can pipe it back into openssl to convert to base64 echo -n "$text" | openssl dgst -md5 -binary | openssl enc -base64 4cuww4ea+DRyRvEsVZqGtQ== ASIDE: The echo is a bash builtin, so from a script "text" is not visible in the process list. You can convert that into a 22 character random filename... echo -n "text" | openssl dgst -sha1 -binary | dd bs=1 skip=2 count=18 2>/dev/null | openssl enc -base64 | tr '+/' ',-' oIyrM,ccAsZR28g6R00yxnbq This filename is similar to what is generated by EncFS but it is NOT the same scheme that EncFS uses, which includes a checksum. :-( ------------------------------------------------------------------------------- Unix Password handling # Generate the old UNIX des encrypted password # Give previously encrypted password as -salt for testing openssl passwd # generate md5 encrypted password openssl passwd -1 ------------------------------------------------------------------------------- SSL conection to a remote server To talk to the wever via HTTPS using 'openssl'... =======8<-------- :::prompt:::> openssl s_client -connect www.example.com:443 CONNECTED(00000003) ...lots of output about server and its certificate... --- HEAD / HTTP/1.0 <- You type this for the doc <- and a blank line HTTP/1.1 200 OK Date: Tue, 27 Oct 2009 05:23:53 GMT Server: Apache/2.0.52 (CentOS) Accept-Ranges: bytes Content-Length: 1532 Connection: close Content-Type: text/html; charset=iso-8859-1 closed =======8<-------- For SMTP (mail) # port 25/TLS; use same syntax for port 587 openssl s_client -connect remote.host:25 -starttls smtp # port 465/SSL openssl s_client -connect remote.host:465 Other servers # https: HTTP over SSL openssl s_client -connect remote.host:443 # ldaps: LDAP over SSL openssl s_client -connect remote.host:636 # imaps: IMAP over SSL openssl s_client -connect remote.host:993 # pop3s: POP-3 over SSL openssl s_client -connect remote.host:995 ------------------------------------------------------------------------------- Setup a HTTPS server!!! This is not recommended, and a certificate should be provided. # the -www option will sent back an HTML-formatted status page # to any HTTP clients that request a page on port 4433 # openssl s_server -cert mycert.pem -www # the -WWW option "emulates" a simple web server. Pages will be # resolved relative to the current directory. This example # is listening on the https port, rather than the default # port 4433 setting (requires root to open the 443 port). # openssl s_server -accept 443 -cert mycert.pem -WWW ===============================================================================