Let's say I have a MySQL database into which users can enter some personal data like postal addresses via a php website. The users can NOT log into this site to later verify what they have entered. The enterprise they have entered their data for (voluntarily of course) then can use this data to maybe send the users actual mail (you know, postal services and such) or emails (of course the users agreed beforehand that they want to receive emails). The database serves merely as a store for the data and I want it to be a bit safe. If anyone breaks into the database, retrieving the email address and the actual first and last name (many email addresses contain both anyway) might not do much harm, but knowing where people live could be too much of a giveaway.
The enterprise is accessing the database via a C# front-end that targets stored procedures within the database to do stuff, including searching for users based on their email address.
From what I have gathered through searching I could think of the following procedure to handle the personal data in a more secure way (than saving them as plain text in the database)
before submitting the sensitive information to the stored procedures the plain text gets encrypted with a key while still in php, so all the MySQL server logs see is encrypted data
the front-end uses the same key to make the data humanly readable again when it is displayed to the enterprise's users (They need to access this private information and the user is comfortable with the enterprise doing so, that's the whole point of this scheme)
My line of thinking is: These are not passwords that are stored so I don't need all the password hashing trickery (As I understand it, when securely saving a password in a database, you use a one way algorithm, so you can never reverse engineer the password straight from the database but have to hash every password you want to try and test that against the desired database entry to see if you chose the right password) but can instead go for a simple encrypt/decrypt, because I don't want to brute force every address out of the database.
There are a few rough edges that raise concerns for me:
I need to somehow provide the key I want to encrypt by to php. Usually this is done via a library or external php document, like you provide the database connection information in a separate php file which resides in a folder on the server which is not accessible from the web (The server will say access denied if you try to access it) Is this a good practice and can I be sure that this key-file is really safe?
I need to provide the key to the front-end as well. This should be done separately in a (maybe encrypted?) config file for the front-end. Is it wise to have the key in two places, albeit for two different systems? The key must never change or else part of the data will be lost!
Somehow I have the feeling that if someone knows how to access the database he/she probably figured out where the connection data was and how that was to be accessed. Oh look, here is an encryption key, I wonder what that does. How likely is it that if the database access is breached the encryption key is in the open as well rendering all efforts to give a little extra of privacy to the users void?
If I wanted to add another bit of "extra security" and encrypted the email address as well I would have to encrypt every email address I want to search for from the php or the front-end, right?
Having searching procedures using ´RLIKE´ will break on encrypted fields, won't they? So to retain searching for parts of an email address I cannot encrypt the email entry, right?
I will have to change my database fields to binary to accommodate encrypted data or make them bigger and base64-encode them, won't I?
Is there an encryption/decryption algorithm ready to use in PHP 7.0.7 and C#? (I don't worry that much about C#) One that is reasonably secure while not bloating my tiny texts to massive chunks of binary? I don't know if that is of any consequence, but if I use for example a 256 bit key, that's 32 bytes. If the street part of the address is shorter that 32 characters, will the encryption work? Will there be cumbersome padding involved?
All in all I feel that the security gain is minute compared to the measures I have to take in my php files as well as in the code for the front-end. The perceived security gain might be bigger ("Woha! They are saving our data encrypted! They sure know what they are doing!"). Having strict and restrictive privileges for certain types of users (for example revoke ´SELECT´ commands) should in all be more helpful, shouldn't it?
Edit for #Luke Joshua Park:
Thank you for your detailed answer.
I suppose by API server you mean the webserver my php's reside on? This indeed should be separate from the database server. Both servers are hosted within a university's network but can be accessed from the internet.
I can follow the authentication path to the point where every user from within the enterprise (small-ish project at said university, maybe a bad choice of wording) has a database user with sensibly set grants. But users from outside using the php only send data to be stored in the database (ideally with a common but seperate database-user with grants set accordingly), and never retreive (their own) data. Using authentication would mean they first had to create an account (which is not needed) and how do they authenticate themselves for creating the unwanted account?
It is good that you're asking these questions before implementing a solution, cryptography is difficult to get right and a sound understanding is required before you start.
I'll answer your questions quickly first, but the more important part is what follows.
Not really. See below.
Yes, in most cases, keys should be kept on the device they are created on, wherever possible.
Provided your API server(s) doesn't also have the database on it, relatively unlikely.
Yes.
Yes.
Yes. But don't base64 them. Wasted space and processing power for no benefit.
You're asking the wrong questions. An algorithm isn't "for" a language. You just need to pick the right algorithm/block mode/padding depending on your needs.
For the most part the questions you are asking are irrelevant. Believe it or not, your issue is more to do with authentication than it is to do with encryption.
The first thing you need to understand is that a server breach is a server breach. Bad stuff is going to happen regardless of how much cryptography you throw at it. But we can minimize damage where possible.
Your database software should be running on a separate server/instance/whatever from your API server. Encryption/decryption should only take place on your API server. This has the benefit that both your API server and database server would have to be breached in order to decrypt the data (to access the keys). How you store the keys on your API server isn't all that important provided they aren't in your webroot or something silly like that.
Everything past this is authentication. You need a strong authentication system to determine who can send you information and who can retrieve information from you. Communication to and from your API server should obviously be encrypted with TLS at all times. You might consider TLS client authentication as a way to ensure the entity requesting data from you is who they say they are. Usually client authentication can't really be used on the web, but if you're interacting with "enterprises" in a more private way, then client authentication is an excellent choice.
In summary:
Separate your API server from your database server. Encryption keys should only ever be on the API server. See this repository for a collection of encryption examples from PHP to just about any other language.
Use TLS for all ingoing and outgoing communication.
Focus on authentication. TLS Client authentication is a good option.
Related
I'm currently facing a few issues which I need advice on.
I apologise in advance if the questions don't make much sense or if they have been answered before (I found somewhat relevant questions, but either didn't understand the answers or did not think they applied to my situation). Any guidance in this field or answers to the question would be extremely helpful.
Situation: We have a standalone app rolled out to customers, which uses AWS Userpools for user accounts.
Issue number 1, storing the AWS (and other) API keys:
I've read around and quickly realised the most secure location to store them is server side.
We have an FTP server set up currently used for auto updates which I assume can also be used for retrieving the API keys.
Issue with this is that we still need to store the information to access the server somewhere securely.
What would be the best approach for storing and accessing API keys on server side?
Once API keys have been retrieved can we store them in memory to not have to access server again?
Issue Number 2, feature flags:
Due to different pay levels we have implemented feature flags, these flags are retrieved from a database when connecting to the app online.
However if they need to get on the app whilst offline they need to be stored locally in a way where they can not be modified.
Is it safe enough to encrypt the data and store it in a file locally and decrypt it on app startup?
Issue Number 3, making sure the user is ok to use app if offline:
As we want the user to not be able to log in after a set amount of days in offline mode without reconnecting I imagine I should be going down the route of using the AWS Userpool Tokens. How would one safely store these tokens to persist through app closure and is there any risk of storing them locally?
I imagine encrypting the server access information and feature flags in files is a way to go, however then we need an encryption key in order to decrypt these files. Where would one store that?
Note: I'm more concerned about the API keys rather than the feature flags.
If I have a stored key file used to decrypt the encrypted input coming into my application, and I store that key file as an embedded resource so that it is embedded in the assembly when deploying, how difficult would it be for someone to reverse engineer the application and retrieve the key file?
Also, the application is deployed through ClickOnce "Online only" mode which I imagine would also make it more difficult to reverse engineer? (I'm not exactly sure of the workings of ClickOnce but I wasn't able to find the dll's/assemblies on my local machine after running the application in online only mode...).
Update:
As Ralf essentially answered the main question below in his comment (answer: it's not really safe at all), here's some more information so that you knowledgeable people can possibly suggest a better security model.
The encryption will be used to encrypt the login password for my application, to be used in a SSO setup (the user will first sign on to a different system and then by clicking a link will be able to directly open my application without having to enter in their login details).
The encrypted data will be sent as a base-64 string URL parameter in the link that will launch the my click-once application.
I will also be developing the application that will create the encrypted data for the URL parameter (clarification: not the first application the user will sign in to for the SSO, I will only be creating a small tool to convert the plain text password into an encrypted base64 string).
It's only an internal application so bullet proof security isn't essential and ease of deployment is more important, but it'd be nice to know what the best practices and different options that are available.
Whether in clear text or encrypted, you don't want to store the password. When you get a password, all you should do is pass it to your server app where you compare it against the salted hash of the password you have in the DB. Even if you don't think security is that important, you need to take care with the password because people frequently reuse passwords across different systems. I know they shouldn't but they do.
If you want to implement a single sign on (SSO), create a sign on token on the server side and pass it back to the client, either encrypted or signed (HMAC is a good choice for signing). This is an unforgeable token because you need to know the encryption key or shared secret for the HMAC and that data is only known on your servers. So you have your SSO and all data involving the SSO is managed on servers so there's no data leakage or chance of spoofing.
As long as the aplication can be launched, the files have to be somewhere on the computer. You just have to know where to look. The reverse-engineering may be ugly but it is always possible. The computer has to be able to understand what he is supposed to do so you only have to extraxt the information one is looking for. Therefore the security of your application should never depend on the difficulty of reverse-engineering! I believe that a secure application should be open-source anyways.
You propably need a diffent security model. The important thing here is that you know against what you want to protect the data. If you simply want that you know the data is sent by the server and not somebody else (man in the middle attack) you could use digital signatures instead.
If you do not want anybody to read any data sent between server and client you should use some sort of ssl implementation to create an encrypted channel. Then you only need to watch out that the public key of the server is not altered on the client. This may be done by an certificate of an official CA but unfortunately those are usualy not for free.
I'm building an application that syncs data between users' Exchange Server accounts (version 2007-2013 supported) and the application.
The application can't use impersonation (at least not in the typical case) as users could be on any number of domains and exchange servers.
I know I'm going to have to ask for their username/email-address and password initially. However, I really don't want to be responsible for storing these credentials if I don't have to (even if they are encrypted, I'd rather not).
I'm not sure what questions to ask, so I'm going with these:
How does Exchange Server authenticate? Do the user's credentials get sent directly to the server as they are, or are the hashed together before being sent across the wire? If they are hashed, how can I get/generate this hash for re-use on successive authentications?
Does Exchange Server send some sort of authentication token that can be re-used later (and forever, until password change or invalidation)?
If you know of a solution to the problem, that the answers to these questions won't address, please do provide it instead.
Active directory federation services is exactly for such tasks. You can read about it there.
As mentioned by Kirill, ADFS 2.0 is one of the best solution for your task. You can also look into other SSO implementations as well. Though the main goal of SSO implementation is to maintain single Login state for multiple application (thereby reducing multiple Login prompt for each application), some of your application goals seems relevant. Please do a thorough research on all the tradeoffs before heading to the sso implementation since there is a small degree of complexity involved during implementation. SSO suits best if you are considering integration of multiple application in the future with the exchange server.
To answer some of your questions (in the same order - considering an SSO scenario with ADFS 2.0):
The authentication to exchange server will be done via ADFS 2.0 (Which provides security tokens (STS service) - to your application after authenticating with AD/ main Directory service). All the communication is encrypted and token signing certificates are used for Integrity and confidentiality.
The lifetime of Security tokens sent by ADFS 2.0 can be configured and reused as required. Please see this blog post for more details.
Also you can configure the ADFS 2.0 (Federation Service) to send only the relevant claim values (like username and email address) to the application, thereby improving the data security.
The System.Net.CredentialCache should work to suite your needs. The WebCredentials is a wrapper for the System.Net.NetworkCredential. Depending on the connection type/domain ect you should be able to utilize System.Net.CredentialCache.DefaultNetworkCredentials or System.Net.CredentialCache.DefaultCredentials
perhaps you should take a look at this Links Connecting to EWS by using the EWS Managed API , Connect to Exchange - Getting Started Tutorial? hopfully it will give you a new idea how to solve your problem :)
because if i understand the information correctly you maybe over think problem but i haven't any experiences so i could also absolute wrong
Bottom Line
If you can't configure anything on the server, there's no automatically generated token to use. It's unfortunate, but you're facing the same general problem that web browsers have--saving the password.
It's worth noting that any authentication needs to be over SSL (an https connection) to prevent a third party listening in on the authentication.
Password storage thoughts:
My suggestion is then to be somewhat creative when storing the password. You can use a keyed encryption algorithm, and then use a hash to generate the key, letting you arbitrarily choose what goes into the key. You would want at least 3 pieces of information going into this: something unique to the device, something unique to the app, and something unique to the exchange server.
For example:
a unique id given by the device (it doesn't matter whether or not this value is app-specific or not, merely that it is consistent)
a (long) string of information compiled into the app, possibly keyed to installation specific values, say the time when the app was first used
something unique to the destination, like the DNS name and perhaps some more specific server info
If you're willing to provide the option to the user, you could have an authorization PIN of some kind that would also be added to the data.
All this data gets put together in one byte array and hashed. The hash (or part of it, or it twice, depending on the hash size vs. the key length) is then used as the key for the encryption of the password.
You could also include some check information along with the password to be able to check client side whether or not the password was decrypted correctly. (If the wrong data is hashed, the wrong key is generated, and the wrong result comes from the decryption).
It's worth noting that all the information to be used for putting into the hash needs to be stored on the device, which is why I would suggest a Pin to authorize the usage of the account.
I am looking forward to have the less information as possible on my application resulting in pulling most of the data from a website or server and I would like to hear by advices and recommendations from everyone with experience in the field to help me improve my application.
Currently my authentication consists of:
To start the application the user have to input their username and password (without a valid user and pass the app does not work) and click login.
The credentials will be sent to my login page using HTTPS, the script only accept SSL connection and a personalized User-Agent name.
The login page will verify the credentials and send back a session and some initial data.
The session is reused to gather more data over time or as needed.
Side note: the SSL is 256 bits, the session auto expire after a few minutes
For the above basic authetication and data pulling would you recommend me anything else to implement ?
Should I change anything ?
2nd Layer of protection
Now I would like to implement more security by encrypting all the data sent from/to my application, my question:
What should I use to encrypt and decrypt the data, a pair of private and public keys held on both sides or RIJNDAEL's method ?
What is the proper way or what parts of information should I leave on the client and server or how should I form the knowledge of the common passwords or keys ?
For example if I was using a pair of RSA keys, I would need to leave 1 private key at the client and 1 public key since you cannot decrypt any data with the public key on c# while you can do it on the server and for Rijndael would need to have the IV and the key on both side.
What is the proper way to handle these ?
I will be very glad with practical reading material, comments, examples, suggestions, advices :)
UPDATE:
No one else wants to say anything ? I was hoping to get more input on this question and also tough that this was a fairly used method around ...
Sounds like you've considered the security of the app side of the equation at length. I'd suggest you turn your thinking to the website part of the equation.
When I look at security, I always look at two things:
Transport security--is your data protected while in transit? Sounds like it is, assuming you are using sufficient key length for the SSL cert on the server. Also, you can force the website do client cert validation as part of the SSL handshake. That ensures that no one can spoof the client app and convince your website to share information.
Payload security--should you encrypt the data payload? Is there a chance someone may be able to break into your web server, or better yet, via the DNS poisoning or some other method, convince your app to connect to a malicious server with a valid (for that cn), but different HTTPs certificate? Should you choose to encrypt the payload, you can piggy back the whole thing onto the same certificates you already use. Just make sure that the certs have data encryption bits turned on, and you can use private/public keys from the certs to encrypt the payload. So, if the malicious user replaced the cert, not only do they need to spoof the cn and the chain of trust, but also have the right public key from the app to decrypt the data you are encrypting with your private key and signing with server's public key.
Some other questions to ponder:
You are saying that the session is reused? Does it not expire? If not, you'd want to make it expire.
Can you leverage network security? Can you use a VPN tunnel or IP ACL to limit who can even gain access to the web server?
What about keyloggers? Passwords can be intercepted. The second authentication factor can be something a user has, like a key card or a fingerprint, or an RSA SecurId. If you don't want to go that far, you can present a user with a "site seal"-- an image that they have to recognize as associated with their account. Maybe even present several images and let them choose one that they've chosen during the signup process. You can also make them solve a small puzzle--something that will distinguish a human from a machine (CAPTCHA-type).
All these points minimize info stored on the client side--just a cert or two and a key to go with that cert.
The topic of security is vast, and we can start a whole discussion here on different implementation details. The above is just some points to think about.
Remember, that everything has cost. Security costs usability and CPU cycles. Proper balance is key, but that of course is up to you.
Before you build Fort Knox, make sure someone is going to want to live there :)
Im not sure about what authentification method I should use for my webservice. I've searched on SO, and found nothing that helped me.
Preliminary
Im building an application that uploads data from a local database to a server (running my webservice), where all records are merged and stored in a central database. I am currently binary serializing a DataTable, that holds a small fragment of the local database, where all uninteresting stuff is already filtered out. The byte[] (serialized DataTable), together with the userid and a hash of the users password is then uploaded to the webservice via SOAP. The application together with the webservice already work exactly like intended.
The Problem
The issue I am thinking about is now: What is if someone just sniffs the network traffic, 'steals' the users id and password hash to send his own SOAP message with modified data that corrupts my database?
Small update: Not to be misunderstood: I dont worry about a syntactic/validation problem. All data that arrives at the webservice is of course validated, and I unit-tested that intensively. I meant 'attackers could semantically corrupt the database': e.g. a user can edit only his submitted records. An attacker could make use of that fact, and masquerade hisself as some user and edit his uploaded data.
I just dont want that people with some technical understanding can just dump the database with garbage in another users name.
Options
The approaches to solving that problem, I already thought of, are:
Using ssl + certificates for establishing the connection:
I dont really want to use ssl, I would prefer a simpler solution. After all, every information that is transfered to the webservice can be seen on the website later on. What I want to say is: there is no secret/financial/business-critical information, that has to be hidden. I think ssl would be sort of an overkill for that task.
Encrypting the byte[]:
I think that would be a performance killer, considering that the goal of the excercise was simply to authenticate the user.
Hashing the users password together with the data:
I kind of like the idea: Creating a checksum from the data, concatenating that checksum with the password-hash and hashing this whole thing again. That would assure the data was sent from this specific user, and the data wasnt modified.
The actual question(s)
So, what do you think is the best approach in terms of meeting the following requirements?
Rather simple solution (As it doesnt have to be super secure; no secret/business-critical information transfered)
Easily implementable retrospectively (Dont want to write it all again :) )
Doesnt impact to much on performance
What do you think of my prefered solution, the last one in the list above?
Is there any alternative solution I didnt mention, that would fit better?
Am I worried about nothing? Is it enough to just send the users id and password hash with every SOAP message?
You dont have to answer every question in detail. Just push me in the right direction. I very much appreciate every well-grounded opinion.
Thanks in advance!
You absolutely must use HTTPS. SSL is by far the simplest secuirty system you could implement, and it only costs $30 per year. Do not reinvent the wheel! After all how much is your time really worth? You can't just call an "encryption function". To implement this protocol properly you have to worry about block cipher modes, initialization vectors, a string2key (s2k) function, and finally a way to authenticate the server and/or client (asymmetric cyrpto/PKI...) . In short the vast majority of programmers have absolutely no idea what goes into creating a truly secure protocol.
Further more it is absolutely impossible to create a secure session and authentication without SSL. This is coming from the OWASP top 10 A3:Broken Authentication and Session Management.
Hashing the users password together with the data
What you are describing here is a Hash Message Authentication Code or HMAC. There is no point in doing this if you are just sending the username and password over the line in clear text. The whole point of an hmac is that you are using a secret, and the password isn't a secret unless you use SSL.
If you are sending the password hash over the wire to authenticate then you really fucked up. The whole point of hashing a password is to slow down the attacker after he has used sql injection to obtain another users password hash from the database, if you are using a message digest to authenticate then the attacker won't have to break the hash. It is as if you are storing passwords in clear text.