What is considered "best practice" for encrypting certain sensitive or personally identifiable data in a SQL database (under PCI, HIPAA, or other applicable compliance standards)?
There are many questions here regarding individual aspects of a solution, but I have not seen any that discuss the approach at a high level.
After looking around for quite some time, I came up with the following:
Use CryptoAPI and Rijndael
Generate IV and store it with the encrypted data
Use DPAPI (Machine scope) to "protect" the symmetric key
Store the symmetric key in the registry or a file or the database, split the key and store parts in multiple places for added protection
do not decrypt the data unless it is really needed, i.e. not upon read from the database. Instead, hold cipher text in memory.
Is this adequate? Outdated? Audit-safe? Reckless?
Your approach is good, with a few adjustments in my eyes (I code for PCI compliance generally):
Use CryptoAPI and Rijndael
Use Rijndael/AES256 at a minimum, regardless of other APIs
Generate IV and store it with the encrypted data
Good
Use DPAPI (Machine scope) to "protect" the symmetric key
Not sure if it matters. I'd just keep the IV next to the data that's encrypted, or if you're really paranoid on some other medium. Ensure that the IV is not accessible to the public.
Store the symmetric key in the registry or a file or the database, split the key and store parts in multiple places for added protection
Storing in multiple places will not help you if someone steals your media. It's a bit overkill to split the key up all over heck, but definitely do NOT store it with your IV and/or ciphertext. That'd be bad.
do not decrypt the data unless it is really needed, i.e. not upon read from the database. Instead, hold cipher text in memory.
Definitely. Holding cipher text in memory in fine, but don't pass it around anywhere, and don't decrypt except when you absolutely must, and even then don't EXPOSE the entire unencrypted dataset - only what is needed from it at the minimum. Also, do not hold the key in memory if possible - a memory dump could expose it.
Additions:
Whatever database you store your cipher text in, restrict read access entirely to the proc(s) that select for a given identifier. Do not allow read access to the tables that store this data to ANYONE, even the SA account. This way, a person who breaks into your system will have a hard time pulling down your cipher texts without knowing what IDs to look for. Do the same for any table(s) referencing the identifier on the ciphertext table. DO NOT ALLOW BLANKET READS OF THESE TABLES!
Restrict database access by IP
Never persist any unencrypted plaintext in memory over state. Allow it to be dereferenced/garbage collected as soon as the request is completed.
Restrict the server(s) running this code to as few users as possible.
Possibly combine encryption methods for a stronger ciphertext (AES + Blowfish for example)
Hope these help. Some of them are my personal opinions but remain PCI compliant to the best of my knowledge.
I saw that one of the previous comments mentioned that it doesn't matter if you use CryptoAPI. I just wanted to point out that CryptoAPI is FIPS 140-2 compliant, while Bouncy Castle and the built-in managed classes (all the ones with "Managed" at the end of their names in the System.Security.Cryptography namespace) are not. If you have a requirement for FIPS compliance, it's probably easiest to for you to use CryptoAPI.
I would add:
Keeping the IV hidden is not important. It's OK if the IV is public. Just use good IVs, which means, use a cryptographic-strong random number generator so that your IVs are indistinguishable from random.
Storing the encryption key separate from the data that it encrypts.
Add authentication to your encryption. For example, add an HMAC keyed with a second symmetric encryption key, covering the ciphertext. If you don't use some form of authenticated encryption, then your ciphertext could be modified, and you have no way of knowing (AES will decrypt garbage just fine.) You want any tampering of the ciphertext to be noticed.
Taken more generic list of best practices, from OWASP (Cryptographic Storage Cheat Sheet):
Use strong approved cryptographic algorithms
Do not implement an existing cryptographic algorithm on your own
Only use approved public algorithms such as AES, RSA public key cryptography, and SHA-256 or better for hashing
Do not use weak algorithms, such as MD5 or SHA1
Avoid hashing for password storage, instead use Argon2, PBKDF2, bcrypt or scrypt
Use approved cryptographic modes
In general, you should not use AES, DES or other symmetric cipher primitives directly. NIST approved modes should be used instead. Quote from Nist: "The approved algorithms for encryption/decryption are symmetric key algorithms: AES and TDEA."
Use strong random numbers
Ensure that any secret key is protected from unauthorized access
Also, according to this Cisco article:
DES is to be avoided and so is RSA-768, -1024
RSA-2048 and RSA-3072 are acceptable
AES-CBC mode is acceptable, while
AES-GCM mode is part of the Next Generation Encription.
Related
In most cases I need to encrypt a string with a password and then send/save it somewhere. Later I want do decrypt it with the password. I am not encrypting nuclear missile codes or medical patient data! The ideal would be 2 functions:
string Encrypt(string plainText, string password);
string Decrypt(string cipherText, string password);
I had a look at the crypto documentation... Oh boy! So I try to code the above calls myself (see a proof of concept using AES Managed and Base64 encoded payload). I am no crypto expert, why do I have to code that? I probably did somethings wrong...
To derive the key from the password the interface requires a salt. Can I use the password as salt? Can I re-use the IV as salt? Maybe not, but I don't want to add another parameter.
Can I use a fixed IV? Same plaintext and password should result in different cipher text, so I have to supply the IV for decryption in the payload.
Can I use a salt for the key and keep the IV constant instead? Feels wrong.
Creating a nonce and deriving IV and key salt from it is a valid approach?
If .Net would support the GCM mode would I still have this problems?
The .NET crypto API exposes a general purpose encryption library, containing object oriented approaches to implement cryptographic algorithms. Of course, to use these algorithms and algorithm implementations you need to have a good grasp on cryptography, which you currently lack.
This general purpose library is required to implement the various protocols that exist out there. Usually a single algorithm doesn't fulfill a specific use case (encrypt a string using a password, returning a different string, in your case). So a protocol needs to be chosen or devised that does fulfill that use case. This protocol may e.g. define a container format such as CMS or PGP, which can for instance be used to encrypt emails (the use case).
You're directly trying to apply cryptographic algorithms to solve your use case. That's not going to work. You need a pre-made protocol, preferably with a pre-made API.
Note that there are many different use cases, many different protocols and even more opinions on how to create and implement those correctly. Libsodium / NaCl for instance defines a small container format called SecretBox that does take some of the work from you.
However, it would of course be rather impossible to implement TLS on top of NaCl, as the functionality / algorithms are just not there. Again, .NET needs a generic crypto library like the .NET API for others to implement their protocols.
So either you'll have to byte the bullet and try to create your own protocol or you take an existing one and take an educated guess if it is secure (hopefully the protocol has been reviewed / updated a few times). Stay away from single person projects without additional contributors (like the many sample codes out there without review).
For your own protocol, yes, there are mistakes such as not storing the salt with the ciphertext. You need a random - or at least unique - salt to be secure, reusing the password for that is certainly not secure. Don't let it become a single person project itself and either borrow a protocol or have it reviewed.
OK, quickly then:
To derive the key from the password the interface requires a salt. Can I use the password as salt? Can I re-use the IV as salt? Maybe not, but I don't want to add another parameter.
No, the salt needs to be unique and preferably random; the password / salt combination should be unique (it should not repeat, not even in time, or over different domains).
Can I use a fixed IV? Same plaintext and password should result in different cipher text, so I have to supply the IV for decryption in the payload.
No, unless the key changes value each time (see above). For CBC the IV should be unpredictable unless you use a fresh key each time.
Can I use a salt for the key and keep the IV constant instead? Feels wrong.
That's possible, as long as you don't repeat the salt.
Creating a nonce and deriving IV and key salt from it is a valid approach?
That depends on very specific details. In other words, I would not try it if you don't exactly know what you're doing.
If .Net would support the GCM mode would I still have this problems?
Absolutely, and in a sense your problems would be worse if you'd use GCM, as using GCM with the same key and IV is completely broken.
Remember, GCM is just an algorithm, not a protocol, it cannot solve your use case by itself.
When I choose password encryption, I have found that Rijndael algorithm is one of the best encryption which cannot be cracked through brute force attack.
So I have choosen Rijndael algorithm for user's password encryption.
Now I have identified that, hashing (Irreversible) is more secure than encryption (Reversible) [Please correct me if I am wrong]
Here my question is,
Can I go with the existing implementation Rijndael algorithm
If I should not do encryption, Which one should be a best hashing algorithm.
I have referred the following website when implementing Rijndael algorithm.
http://msdn.microsoft.com/en-us/library/system.security.cryptography.rijndael(v=vs.110).aspx
How to generate Rijndael KEY and IV using a passphrase?
http://www.obviex.com/samples/encryption.aspx
OWASP lists some good practices for password storage.
You basically apply a protection_function to convert the credential to a protected form:
[protected form] = [salt] + protect([protection func], [salt] + [credential]);
You also add a salt so two versions of the same credential have a different stored form.
They also list the order in which you should choose hashing functions (yes, hashing is better than encrypting so that the password cannot be reverse engineered, even by the website owner). Argon2 and PBKDF are generally good choices for a protection_function.
Read the rest of the guide too. Also this related Security SE post about why AES (i.e. Rijndael) encrypted password storage is worse than even a not-so-strong hash (#Salvador's comment).
The problem with encryption is, that when an attacker get the key, he can decrypt all passwords of the database in no time, and therefore knows the original passwords which can be tried on other sites.
Since hashing is irreversible (there is no way to get back the original password), an attacker cannot use the hashes, even if he has control over the server. The same goes for the owner of the site.
Today recommended algorithms are BCrypt, PBKDF2 and SCrypt, all of them have a cost factor which controls the necessary time to calculate a single hash. The longer it needs, the more difficult it will be to brute-force.
I have an encoding application written in C# where users can optionally encrypt messages. I had been using the class in this answer, and it turns out I'm in good company because I found several places online that use the exact same code (one of which is Netflix's Open Source Platform).
However, comments to that answer (as well as later edits to that answer) led me to believe that this method was insecure. I opted to use the class in this answer to the same question instead.
How secure is AES encryption if you use a constant salt? How easily can this method be broken? I admit that I have very little experience in this area.
AES is a block cipher. A block cipher's input is a key and a block of plaintext. A block cipher is usually used in a block cipher mode of operation. All secure modes of operation use an Initialization Vector or IV. Otherwise identical plaintext would encrypt to identical ciphertext (for the same key), and this is leaking information.
Salt is not used by AES or modes of operation. It's usually used as input for Key Derivation Functions (KDFs), especially Password Based Key Derivation Functions (PBKDFs). Dot NET's Rfc2898DeriveBytes implements the PBKDF2 function as defined in - you'd guess it - RFC 2898: "PKCS #5: Password-Based Cryptography Specification Version 2.0".
If you use a static salt in a PBKDF2 then you would get the same key as output (for the same number of iterations). Now if you would ever leak the resulting key then all your ciphertext would be vulnerable. And if you would use multiple passwords then an attacker would be able to build a rainbow table; the PBKDF2 work factor would become less important; the attacker can simply build one table and then try all the resulting keys on all possible ciphertexts.
So, as the salt is not actually used for AES it doesn't make much of a difference for the security. It is however still a horrible sin, even worse than using the default iteration count for PBKDF2 / Rfc2898DeriveBytes.
Note that horrible security sins are committed by a large number of people on a daily basis. That there are many many many persons that get it wrong doesn't tell you that you are in "good company". That there are 289 upvotes just tells you that SO answers about cryptography should not be trusted based on vote count.
Salt is there for a reason.
This enables same input to be encrypted differently.
If an attacker would really insist, he can find some patterns that repeat themselves in encryption without salt, and eventually can get to your key more easily.
Still the attcker would have to work very hard.
Using constant salt equals to not using salt at all.
And it is highly recommended to use it, as it has no effect on the decryption process.
In my research I have found mixed messages on this subject so I'm looking for expertise to explain the best approach to encrypting variable amounts of data.
Requirements:
[Edit: Adding additional requirement #3 in response to comment]
I would like to use RSA for the public/private key encryption scheme
so I can distribute the public key to an application that should
encrypt data but should not know how to decrypt it
I need to support data lengths from 16 characters (credit card
number) to kilobytes (serialized objects) and beyond. Most of the
data I encrypt will be small (credit cards, addresses, etc).
This is for encrypting data at rest.
Options I'm Aware Of:
RSA-ONLY: Use RSACryptoServiceProvider to encrypt all data using public key.
Iterate through the data in blocks that are less than the key size
minus padding.
HYBRID: Use AesCryptoServiceProvider to encrypt the data, calling
.GenerateKey() and .GenerateIV() to generate a random key and IV.
Then use RSACryptoServiceProvider to encrypt the above key and IV
and prepend or append that to the data.
It seems to my the Hybrid approach gives me the best of both worlds. Strong block cipher (AES) and distributed public key (RSA).
What are the pros and cons of these approaches? What is the standard? Surprisingly I have not found much opinion or information on the subject and would appreciate any references you might have.
Bonus:
I am rolling my own for various reasons including corporate licensing restrictions but I'm curious if there is a good standard opensource approach for C#.
in most cases RSA is used to encrypt a symetric key (you don't really need to encrypt the IV, but hey...)
if you use RSA for encryption of data (instead of a key) you might run into the ECB (Electronic Code Book mode) problem that is known in the context of symetric block cyphers: for a given key, a clear-text is always mapped to the same cypher-text ... that alone doesn't help in breaking the encryption, but it can leak information since an attacker can identify which data packages contain the same clear-texts
i'd choose the hybrid approach, because it's suitable for arbitrary sized data, and won't be prone to this information leak unless you choose ECB for the mode of operation (CBC - Cypher Block Chaining mode - should do)
If you just want to use RSA to store a small amount of data, smaller than the number of bits in the key, you can pad the input data with random numbers. There are several padding schemes listed at https://en.wikipedia.org/wiki/RSA_(cryptosystem)#Padding
I was told that there's an encryption library I can use and there's a couple that I can choose from (eg. AES, RSA, etc). I also read something about keys. Are keys something you just generate so you can encrypt and decrypt a series of texts? Do you have to purchase that key?
Also, is there a best practice that I need to be aware of in encrypting and decrypting? Is encrypting a password recommended? Would performance be affected?
You are correct. Base64 encoding is a world away from actually encrypting your data. The former simply converts the data to be representable using 64 unique characters, obfuscating the data at best, while the latter actually converts your data into a representation that can only make sense once it is decrypted using the proper key. Do not ever base64 encoding if you want to keep something a secret.
Are keys something you just generate so you can encrypt and decrypt a series of texts?
Yes.
Do you have to purchase that key?
No, you generate the keys yourself.
Is encrypting a password recommended?
Most definitely. You should always encrypt passwords whenever possible.
Would performance be affected?
When encrypting data, you're using more CPU cycles than you would have otherwise, so performance is affected, but it really depends on what algorithm you use, the amount of data, etc.
Here are some linke that might help you out:
Some info on encryption in .NET
MSDN Article
More on Encryption
Start reading here:
http://msdn.microsoft.com/en-us/library/system.security.cryptography.aspx
Oh and yes - encrypting a password is recommended in most systems (do a search for hash and salt).
A common practice would be using Protected Configuration feature.
Encrypting and Decrypting Configuration Sections