My question is about what I actually need to store in the DB re encrypted values and how this can be used for the nonce for GCM specifically.
For reference these two answers provide sample code for encrypting data:
Cbc and gcm.
As I understand it, CBC requires the IV to be totally random. I understand that for CBC (and for all encryption really) the Key/IV should always be unique. If it is repeated it is 'bad' for CBC and a fundamental flaw if using GCM. GCM however does not need it to be totally random as long as it never repeats.
On these assumptions, we plan to have a 'key' somewhere in the app (i.e. separate the key from the encrypted data in the DB). We will roll the key every 'n' months and only use it for 'x' number of encryption before generating a new one (limit scope of use etc), however, we will be using the same 'key' for two or more records. Hence the unique key/iv comes into play.
Note: We need to be able to read out the data again later and 'use' it. This is not one way encryption.
In designing the DB we were going to use columns similar to the following.
DB columns for CBC mode (I have seen something similar on stack overflow... sorry can't find link):
ID (primary key, int)(unique)
Encrypted Value
Salt
Iterations
(Other... standard things like created date etc etc)
Db Columns for Gcm mode:
ID (primary key, int)(unique)
Encrypted Value
Associated Text
(Other... standard things like created date etc etc)
IV/Nonce Handling Question
For the CBC mode, we were just going to use e.g. Rfc2898DeriveBytes and get the IV from that and not store it.
Re the GCM mode, the idea here was to do the following:
Save the record first and 'get' the id of the row
convert the row id into a byte[16]... e.g. (example only...)
var rowID = 123456789.ToString();
var tempByte = new List<byte>(16);
tempByte.AddRange
(123456789.ToString()
.Select(Convert.ToByte));
if (rowID.Length < 16)
{
for (var i = 0; i < (16 - rowID.Length); i++)
{
// Note: something to fill the rest.....
var someNewByteFromSecureRandom = new[] {Convert.ToByte(1)};
tempByte.Add(someNewByteFromSecureRandom[0]);
}
}
Save the encrypted data into the row.
This way, the nonce is always going to be unique for the GCM encryption. Even if we always use the same key, the key/iv will never repeat. (Note: this approach would as we understand it not work for CBC because of its requirement that the iv be truely random and not a counter)
Basically, is there a problem with either of these approaches? (Ignoring the obvious dual save (insert/update) of the record when using the GCM approach outline to get the nonce). Are we saving something we should not? Are we not saving something we should?
Presumption: you are only protecting data at rest, i.e. active attacks on the data while the database is being utilized are not considered.
CBC requires the IV to appear random to an attacker, CBC could use an encrypted counter (the row ID) as IV;
to use a row ID as a nonce, you need to start the IV (rightmost bytes) with the row ID (converted into a 12 byte unsigned big endian number) - the leftmost bytes should be kept empty for maximum compatibility
CBC doesn't require PBKDF2 (so no salt or iteration count) if you already have a key - you could just store a random IV instead of the salt
"Associated Text" has no meaning on its own. GCM takes Additional Associated Data - or AAD - as input parameter. So the AAD is just any data in your database that you want to include in the authentication tag. GCM already includes the IV into the authentication tag so you don't have to include the row ID;
CBC is enough for data at rest as nobody should be able to change the data (that would be an active attacker, which is ruled out for data at rest); that said, GCM does catch any wrong key/wrong data error so it could be used to simplify error handling for your application.
Note that using a nonce of 12 bytes will limit the plaintext to 2^36 bytes (64 GiB). That should not be a problem for database entries (I hope). You could throw an error or runtime exception if you ever encounter such a large plaintext though.
Related
I am using bouncy castle library for encryption and decryption in c# and I was wondering how to deal with plain text smaller than block size?
here is what I have done so far:
AesFastEngine engine = new AesFastEngine();
GcmBlockCipher cipher = new GcmBlockCipher(engine);
AeadParameters param = new AeadParameters(new KeyParameters(key), 128, iv, null);
cipher.Init(true, parameters);
byte[] encData = new byte[plain.Length];
cipher.ProcessBytes(plain, 0, plain.Length, encData, 0);
when the plain data is smaller then block size it does nothing.
Unfortunately the Bouncy Castle and Oracle / Java implementations are not online. That is, the online properties of the underlying CTR mode encryption aren't kept. Online in this context means that bytes are directly encrypted/decrypted when they arive. This may have to do with how the encryption is handled and how the authentication tags are handled.
AES-CTR can be implemented in multiple ways. You can either first encrypt the counter and then directly XOR with plaintext/ciphertext when it arrives. You can also first buffer the plaintext and then, once you have a full block, create the counter, encrypt it and then XOR a full block of plaintext. This had advantages in the sense that it more closely resembles other modes of operation such as CBC. Furthermore you may not have to buffer the key stream in memory all that time.
The authentication tag can also be handled differently. Here there are basically three options. You could simply regard the authentication tag as a separate entity to the ciphertext. This lets you keep the online properties of CTR mode and should, in my opinion, be the preferred option. You could also see it as part of the ciphertext, but in that case you lose the online properties during decryption; you would need to know where the ciphertext ends before you can handle the last number of bytes that make up the authentication tag. So you'd need to buffer at least the size in bytes of the authentication tag. Finally, still during decryption, you might only want to return plaintext bytes after verification of the plaintext bytes. In that case you'd need to buffer the entire ciphertext and return the plaintext in one go.
As the authentication tag issues are only for decryption it is likely that Bouncy will just buffer because of the way CTR is implemented. You'd indeed have to call doFinal - as Robert already mentioned in the comments - to retrieve the last block of ciphertext as well as the authentication tag. It could be that the encryption is not yet performed because the encryption routine is kept somewhat symmetrical to the decryption routine as well.
Is there an existing secure implementation to achieve following:
Guid original = Guid.NewGuid();
Guid inverted = MysteryImplementation(original, salt); // salt is some sort of input
Guid shouldBeOriginal = MysteryImplementation(inverted, salt);
Assert.AreEqual(original, shouldBeOriginal, "MysteryImplementation did no work");
EDIT:
As this got down voted (although I'm a bit unsure why), I think more background is needed:
In a place far far away, there is an application in which primary keys are stored as GUIDs. In this application these GUIDs are exposed to web clients.
In my pursuit of improving status quo I had an idea to map these GUIDs with user session data in order mitigate the risk of an accidental/malicious leakage of primary keys. Mapping these GUIDs has the added benefit that it would also allow easier implementation of working-copies for those objects that the GUIDs refer to.
Those were the reasons why I decided to start looking for "secure" way to map GUIDs.
To answer comments:
-Mapping should preserve the global uniqueness when compared to all other GUIDs (I wouldn't want those mapped GUIDs to collide with existing GUIDs).
-"Secure" in this context means that it should be impossible to figure out the original GUIDs without knowing the cipher key (a typical crypto req, which I think translates that the mapped GUIDs should have normalized distribution).
You can easily do this:
Guid original = Guid.NewGuid();
byte[] encrypted = Encrypt(original, key);
Guid decrypted = Decrypt(encrypted, key);
Any symmetric encoding algorithm will do, from ROT13 on up. However, that's not what you asked for. What you asked for is an algorithm that has two properties:
The encrypt and decrypt algorithms are exactly the same.
The encrypted form of a GUID is also a valid globally unique identifier.
There are plenty of algorithms where the encryption and decryption processes are different but not actually that many where they are exactly the same. The simplest algorithm where encryption and decryption are the same is:
Generate a crypto-strength random one-time pad of the same length as the plaintext.
XOR the plaintext with the pad to produce the ciphertext.
To decrypt, XOR the ciphertext with the pad.
However that algorithm does not necessarily maintain the property that the ciphertext is a valid GUID.
Can you explain why it is that you need the ciphertext to be a valid GUID? The property that a GUID has to have is that it needs to be globally unique; how are you planning on guaranteeing global uniqueness? What stops you from encrypting one GUID that you generated into another GUID that someone else has generated unbeknownst to you?
More generally, can you explain what problem you are trying to solve in the first place? Nine times out of ten that I see someone trying to use cryptography, they're using it for the wrong purpose.
Yes. Those mystery algorithms are called symmetric ciphers. What you call salt is just the key for the algorithm.
However, it might be a bit harder getting a GUID back from that since encryption algorithms usually operate either on streams or blocks of data and by modifying a GUID you compromise it's GU property.
any symetric encrytion will do
http://msdn.microsoft.com/en-us/library/as0w18af(v=vs.110).aspx
I'm writing an encryption sequence for sensitive data in our database.
Currently I'm taking a GUID based on the UserId, and putting that through a hash. Then, I run the hash through a Rfc2898DeriveBytes to get Key and IV which I use to encrypt the data using the Rijndael function.
My code looks like this:
var salt = new byte[] { 1, 2, 23, 234, 37, 48, 134, 63, 248, 4 };
const int iterations = 1000;
using (var rfc2898DeriveBytes = new Rfc2898DeriveBytes(GenerateHash("2525"), salt, iterations)) {
_key = rfc2898DeriveBytes.GetBytes(32);
_iv = rfc2898DeriveBytes.GetBytes(16);
}
I then pass the _key and _iv along to decrypt or encrypt the data.
My goal is to have each user always have access to their unique key through every session. That being said, what can be randomized and still maintain this function? Do I always have to use the same salt and the same IV to get the data I want?
Rfc2898DeriveBytes is an implementation of PBKDF2. Obviously RFC 2898 is a reference to the standard where this Password Based Key Derivation Function has been defined. Note that the standard is broader than just the KDF; it's full title is "PKCS #5: Password-Based Cryptography Specification, Version 2.0".
PBKDF2 is a successor of PKCS#5 v1 which defined PBKDF / PBKDF1. The 1 was only added after PBKDF2 came into being. The class PasswordDeriveBytes is an implementation of PBKDF1. It should not be used anymore because both the KDF is outdated but also because Microsoft screwed up the implementation severely; it may repeat output keying material if more than the output of the underlying hash - SHA-1 so 20 bytes - is requested.
Besides being used as KDF, PBKDF2 can also be used as password hashing function, where the hash instead of the password is stored in a database. That way passwords can be verified, while the password cannot easily be retrieved even if the hash data is retrieved by an adversary. This is described in the followup RFC 8018 which contains the 2.1 version of the protocol.
Internally, PBKDF2 is just a repetition of a hash function over the password and salt. The iteration count is the work factor; it specifies how much work you (and adversaries) have to do before one hash is calculated. The salt makes sure that rainbow table attacks are impossible, and that identical passwords (of different users) don't lead to the same hash.
Due to a design error which requires the full amount of work to be repeated if more than one hash output is required, it is not recommended to request more data from it than the output of the hash function. In that case it is better to use another method to expand the output keying material (bytes), e.g. HKDF-Expand.
Observations on the code in the question:
The GenerateHash method is spurious, Rfc2898DeriveBytes will do this for you;
You should use something less predictable than a UID to create a key; the data should not be directly available to an attacker as this would completely defeat the purpose of PBKDF2;
If you want to use the same set of UID + salt + iterations for multiple encryption operations, then you should generate a random IV and prepend it to the ciphertext, having a non-random IV completely defeats the purpose of the IV;
You can change the salt to get multiple keys, but you would have to go through the PBKDF2 function for each and every encryption.
Just a general hint, only use the resulting key to encrypt data specific keys created out of a secure random function. Then you don't even need to bother about an IV, and you may be able to "re-encrypt" by decrypting the data specific key, and encrypting that with a new key.
Is there any way to perform private key encryption in C#?
I know about the standard RSACryptoServiceProvider in System.Security.Cryptography, but these classes provide only public key encryption and private key decryption. Also, they provide digital signature functionality, which uses internally private key encryption, but there are not any publicly accessible functions to perform private key encryption and public key decryption.
I've found this article on codeproject, which is a very good start point for performing this kind of encryption, however, I was looking for some ready-to-use code, as the code in the article can hardly encrypt arbitrary-long byte arrays containing random values (that means any values, including zeroes).
Do you know some good components (preferably free) to perform private key encryption?
I use .NET 3.5.
Note: I know this is generally considered as bad way of using asymmetric encryption (encrypting using private key and decrypting using public key), but I just need to use it that way.
Additional Explanation
Consider you have
var bytes = new byte[30] { /* ... */ };
and you want to use 2048bit RSA to ensure no one have changed anything in this array.
Normally, you would use digital signature (ie. RIPEMD160), which you then attach to the original bytes and send over to the receiver.
So, you have 30 bytes of original data, and additional 256 bytes of digital signature (because it is a 2048bit RSA), which is overall of 286 bytes. Hovewer, only 160 bits of that 256 bytes are actually hash, so there is exactly 1888 bits (236 bytes) unused.
So, my idea was this:
Take the 30 bytes of original data, attach to it the hash (20 bytes), and now encrypt these 50 bytes. You get 256 bytes long message, which is much shorter than 286 bytes, because "you were able to push the actual data inside the digital signature".
ECDSA Resources
MSDN
Eggheadcafe.com
c-plusplus.de
MSDN Blog
Wiki
DSA Resources
CodeProject
MSDN 1
MSDN 2
MSDN 3
Final Solution
If anyone is interested how I've solved this problem, I'm going to use 1024bit DSA and SHA1, which is widely supported on many different versions of Windows (Windows 2000 and newer), security is good enough (I'm not signing orders, I just need to ensure that some child can't crack the signature on his iPhone (:-D)), and the signature size is only 40 bytes long.
What you are trying to design is known as a "Signature scheme with message recovery".
Designing a new signature scheme is hard. Designing a new signature scheme with message recovery is harder. I don't know all the details about your design, but there is a good chance that it is susceptible to a chosen message attack.
One proposal for signature schemes with message recovery is RSA PSS-R. But unfortunately, this proposal is covered with a patent.
The IEEE P1363 standarization group, once discussed the addition of signature schemes with message recovery. However, I'm not sure about the current state of this effort, but it might be worth checking out.
Your Public key is a sub-set of your private key. You can use your private key as a public key as it will only use the components of the full key it requires.
In .NET both your private & public keys are stored in the RSAParameters struct. The struct contains fields for:
D
DP
DQ
Exponent
InverseQ
Modulus
P
Q
If you're at the point where the data is so small that the digital signature is huge in comparison, then you have excess signature. The solution isn't to roll your own algorithm, but to cut down what's there. You definitely don't want to try to combine a key with the hash in an amateurish way: this has been broken already, which is why we have HMAC's.
So here's the basic idea:
Create a session key using a cryptographically strong RNG.
Transmit it via PKE.
Use the session key to generate an HMAC-SHA1 (or HMAC-RIPEMD160, or whatever).
If the size of the hash is absurdly large for the given data, cut it in half by XORing the top with the bottom. Repeat as needed.
Send the data and the (possibly cut-down) hash.
The receiver uses the data and the session key to regenerate the hash and then compares it with the one transmitted (possibly after first cutting it down.)
Change session keys often.
This is a compromise between the insanity of rolling your own system and using an ill-fitting one.
I'm wide open to constructive criticism...
I get it now, after reading the comments.
The answer is: don't do it.
Cryptographic signature algorithms are not algorithms from which you can pick and choose - or modify - steps. In particular, supposing a signature sig looks something like encrypt(hash), orig + sig is not the same as encrypt(orig + hash). Further, even outdated signature algorithms like PKCS v1.5 are not as simple as encrypt(hash) in the first place.
A technique like the one you describe sacrifices security for the sake of cleverness. If you don't have the bandwidth for a 256 byte signature, then you need one of:
a different algorithm,
more bandwidth, or
a smaller key.
And if you go with (1), please be sure it's not an algorithm you made up! The simple fact is that crypto is hard.
I want to use encryption algorithm available in .Net Security namespace, however I am trying to understand how to generate the key, for example AES algorithm needs 256 bits, that 16 bytes key, and some initialization vector, which is also few bytes.
Can I use any combination of values in my Key and IV? e.g. all zeros in Key and IV are valid or not? I know the detail of algorithm which does lots of xors, so zero wont serve any good, but are there any restrictions by these algorithms?
Or Do I have to generate the key using some program and save it permanently somewhere?
I want to store data in database after encryption, the secure profile data like username, password, phone number etc, and the key will be available to database user mentioned in connection string only, and to the administrator.
You really ought to do this the correct way :)
1) Use a securely generated random IV
2) Use a securely generated random key
3) Don't use ECB mode - EVER
AesManaged aes = new AesManaged();
aes.GenerateKey();
aes.GenerateIV();
The code above will correctly and securely generate a random IV and random key for you.
Sounds like you need to read into the Rfc2898DeriveBytes class.
Rfc2898DeriveBytes.GetBytes();
It has a method(above) that allows you to tailor the size of byte arrays that are fed into the .Key and .IV properties on a symmetric encryption algorithm, simply by feeding an int value. The MS official 70-536 book suggests doing this pro-grammatically by dividing the KeySize property / 8.
I.e TripleDes or AESManaged. Whatever you use, the algorithm itself will have some pre-reqs that will need meeting first. I.e satisfying the key size conditions. The RunTime will automatically fill the properties and fields and etc the best and most strongest values for you. But the IV and Key needs to come from you. This how you can do the following:
RijndaelManaged myAlg = new RiRijndaelManaged();
byte[] salt = Encoding.ASCII.GetBytes("Some salt value");
Rfc2898DeriveBytes key = new Rfc2898DeriveBytes("some password", salt);
myAlg.Key = key.GetBytes( myAlg.KeySize / 8);
myAlg.IV = key.GetBytes( myAlg.BlockSize / 8);
// myAld should now fully set-up.
Above you can see what I mean by doing it pro-grammatically, as it should pretty much
do it all for you, without you even really having to bat an eye-lid as to meeting it's pre-reqs.
The Microsoft 70-536 book states that the .Key properties expect the byte arrays you supply
to them in bytes and not bits. The RFC class works in bytes where as an algorithms KeySize property works in bits. 1 byte = 8 bits. Can you see where this is going ... ?
This should give you an idea as to why the above sample peice of code is done the way it is! I studied it and it makes pretty darn good sense to me!
The above answer should allow you to create your algorithm object with supplied password and a static salt value that can be hard code at both ends. Only thing you need to do is worry about how you going to make sure that the byte arrays stored at .Key and .IV are safely transported to a recipient so that can successfully decrypt the message you encrypted. By safely reconstructing the same algorithm object.
OBTW:
AESManaged has a keysize req': 128Bits = 16 Bytes !!!
(8*8 = 64, 64Bit / 8bits per Byte = 8 Bytes) Therefore
64*2 = 128Bit, 8*2, ==> 16bytes key size !
256Bit = 32Bytes !!!!
According to the 70-536 official training kit book, Aes is limited to having keysize of 128bits in size. 256bits,192 and 128 key size for example can be used with the Rijndael class.
You could on the other hand completely forget all that crap and simply use .GenerateKey and GenerateIV methods instead to save you all the hassle of sorting out a pre-shared and agreed password and static salt values. Your only concern is figuring out a way of storing and retrieving the key and IV byte arrays. Binary Formatter? .
If you are using encryption to exchange data then you will need a key exchange protocol, but you don't make one yourself instead use one off-the-shelf like TLS or SSL.
If you use encryption to store data then you generate the IV using CryptGenRandom (or its .net equivalent RandomNumberGenerator.GetBytes) and save it along the document (in clear, no need to protect the IV). You never write down the key, the key is provided by the user. Usualy you derive the key from a password phrase using CryptDeriveKey, or its .Net equivalent PasswordDeriveKey.CryptDeriveKey.
Update
To store a secret in the database that is available only to the user and an administrator you need to use 3 keys:
one to encrypt the data with (call it the DK key)
one user key to encrypt the DK key (call it UK)
one administrator key to encrypt the DK key (call it AK)
In theory you encrypt the data with DK and then encrypt the DK with UK and save it, and encrypt the DK with AK and save it. This way the user can use again the UK to decrypt the DK and then decrypt the data, and the administrator can use the AK to decrypt the DK and then decrypt the data. The big problem is the fact that the system is always automated, so the system needs access to the administrator's key which means is not truly a persnal key of the administrator, but instead is a system key (it cannot be used for purposes of non-repudiation for instance).
As a heads up, knowledge of what IV is or how to use AES from C# and how cryptography algorithm work will get you exactly 0 (zero) traction in solving this kind of problems. The issue is never what IV and key to use, the issue is always key provisioning. For actual crypto operations, just use the built-in support from the database, see Cryptography in SQL Server. I can easily argue that the only facility you need is TDE (Transparent Data Encryption) to protect against accidental loss of media.
Generate a random letters / hex code in a specific length.
This function (taken from here) return a random key in a specific length:
private static string CreateSalt(int size)
{
//Generate a cryptographic random number.
RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
byte[] buff = new byte[size];
rng.GetBytes(buff);
// Return a Base64 string representation of the random number.
return Convert.ToBase64String(buff);
}
Use System.Security.Cryptography.RandomNumberGenerator to generate random bytes:
var rnd = new System.Security.Cryptography.RandomNumberGenerator.Create();
var key = new byte[50];
rnd.GetBytes(key);
It really depends on what you ned to do with the key.
If the key is to be generated by the computer (and can be any random value) I generally take a SHA256 of a couple GUIDs. This is about as random as you're going to get without a hardware random number generator.
You can use keys with all 0s but obviously it won't be very secure.