We Keep Coding

Cross-Platform encryption for Database

Let's say I have a MySQL database into which users can enter some personal data like postal addresses via a php website. The users can NOT log into this site to later verify what they have entered. The enterprise they have entered their data for (voluntarily of course) then can use this data to maybe send the users actual mail (you know, postal services and such) or emails (of course the users agreed beforehand that they want to receive emails). The database serves merely as a store for the data and I want it to be a bit safe. If anyone breaks into the database, retrieving the email address and the actual first and last name (many email addresses contain both anyway) might not do much harm, but knowing where people live could be too much of a giveaway.
The enterprise is accessing the database via a C# front-end that targets stored procedures within the database to do stuff, including searching for users based on their email address.
From what I have gathered through searching I could think of the following procedure to handle the personal data in a more secure way (than saving them as plain text in the database)
before submitting the sensitive information to the stored procedures the plain text gets encrypted with a key while still in php, so all the MySQL server logs see is encrypted data
the front-end uses the same key to make the data humanly readable again when it is displayed to the enterprise's users (They need to access this private information and the user is comfortable with the enterprise doing so, that's the whole point of this scheme)
My line of thinking is: These are not passwords that are stored so I don't need all the password hashing trickery (As I understand it, when securely saving a password in a database, you use a one way algorithm, so you can never reverse engineer the password straight from the database but have to hash every password you want to try and test that against the desired database entry to see if you chose the right password) but can instead go for a simple encrypt/decrypt, because I don't want to brute force every address out of the database.
There are a few rough edges that raise concerns for me:
I need to somehow provide the key I want to encrypt by to php. Usually this is done via a library or external php document, like you provide the database connection information in a separate php file which resides in a folder on the server which is not accessible from the web (The server will say access denied if you try to access it) Is this a good practice and can I be sure that this key-file is really safe?
I need to provide the key to the front-end as well. This should be done separately in a (maybe encrypted?) config file for the front-end. Is it wise to have the key in two places, albeit for two different systems? The key must never change or else part of the data will be lost!
Somehow I have the feeling that if someone knows how to access the database he/she probably figured out where the connection data was and how that was to be accessed. Oh look, here is an encryption key, I wonder what that does. How likely is it that if the database access is breached the encryption key is in the open as well rendering all efforts to give a little extra of privacy to the users void?
If I wanted to add another bit of "extra security" and encrypted the email address as well I would have to encrypt every email address I want to search for from the php or the front-end, right?
Having searching procedures using ´RLIKE´ will break on encrypted fields, won't they? So to retain searching for parts of an email address I cannot encrypt the email entry, right?
I will have to change my database fields to binary to accommodate encrypted data or make them bigger and base64-encode them, won't I?
Is there an encryption/decryption algorithm ready to use in PHP 7.0.7 and C#? (I don't worry that much about C#) One that is reasonably secure while not bloating my tiny texts to massive chunks of binary? I don't know if that is of any consequence, but if I use for example a 256 bit key, that's 32 bytes. If the street part of the address is shorter that 32 characters, will the encryption work? Will there be cumbersome padding involved?
All in all I feel that the security gain is minute compared to the measures I have to take in my php files as well as in the code for the front-end. The perceived security gain might be bigger ("Woha! They are saving our data encrypted! They sure know what they are doing!"). Having strict and restrictive privileges for certain types of users (for example revoke ´SELECT´ commands) should in all be more helpful, shouldn't it?
Edit for #Luke Joshua Park:
Thank you for your detailed answer.
I suppose by API server you mean the webserver my php's reside on? This indeed should be separate from the database server. Both servers are hosted within a university's network but can be accessed from the internet.
I can follow the authentication path to the point where every user from within the enterprise (small-ish project at said university, maybe a bad choice of wording) has a database user with sensibly set grants. But users from outside using the php only send data to be stored in the database (ideally with a common but seperate database-user with grants set accordingly), and never retreive (their own) data. Using authentication would mean they first had to create an account (which is not needed) and how do they authenticate themselves for creating the unwanted account?

It is good that you're asking these questions before implementing a solution, cryptography is difficult to get right and a sound understanding is required before you start.
I'll answer your questions quickly first, but the more important part is what follows.
Not really. See below.
Yes, in most cases, keys should be kept on the device they are created on, wherever possible.
Provided your API server(s) doesn't also have the database on it, relatively unlikely.
Yes.
Yes.
Yes. But don't base64 them. Wasted space and processing power for no benefit.
You're asking the wrong questions. An algorithm isn't "for" a language. You just need to pick the right algorithm/block mode/padding depending on your needs.
For the most part the questions you are asking are irrelevant. Believe it or not, your issue is more to do with authentication than it is to do with encryption.
The first thing you need to understand is that a server breach is a server breach. Bad stuff is going to happen regardless of how much cryptography you throw at it. But we can minimize damage where possible.
Your database software should be running on a separate server/instance/whatever from your API server. Encryption/decryption should only take place on your API server. This has the benefit that both your API server and database server would have to be breached in order to decrypt the data (to access the keys). How you store the keys on your API server isn't all that important provided they aren't in your webroot or something silly like that.
Everything past this is authentication. You need a strong authentication system to determine who can send you information and who can retrieve information from you. Communication to and from your API server should obviously be encrypted with TLS at all times. You might consider TLS client authentication as a way to ensure the entity requesting data from you is who they say they are. Usually client authentication can't really be used on the web, but if you're interacting with "enterprises" in a more private way, then client authentication is an excellent choice.
In summary:
Separate your API server from your database server. Encryption keys should only ever be on the API server. See this repository for a collection of encryption examples from PHP to just about any other language.
Use TLS for all ingoing and outgoing communication.
Focus on authentication. TLS Client authentication is a good option.

Storing credentials in an encrypted file

I'm working on a couple of projects that need to store user credentials for third-party applications, such as Paypal, Facebook developer creds etc. I've read a few books about different kinds of coding, including ASP.NET, WPF, jQuery, and all have nice examples on how to access the preceding services with own secrets, and also all of them use the exact phrase "in a real project you would store these in an encrypted file". None, however, give an example how to do so.
I have little (read: none) experience in encryption, but based on my understanding, I would need to encrypt a username and a password using some sort of key (salt?) and save them in a file. I would also want to be able to use these credentials on my apps so I would need to store the key (salt?) in my code.
Now my question is: How is it safer to store the decryption key, which is still plain text, in the program code, than the actual credentials?
Wouldn't the malicious user be able to decrypt my password-file as soon as he gets the key?
--EDIT--
I really mean my own credentials I need to store to log in to third party applications, not credentials of my users. For example I need to identify my self and/or my app to Google, so that users can log in to my app using their own Google account.
--EDIT 2--
To clarify, this is what I'm talking about. This screenshot is from asp.net PayPal tutorial:
Any quick pointers on good practises here?

You misunderstood the part about storing encrypted password* + salt: this is done when your system needs to validate someone else's credentials. In this situation storing password hash and the salt is more secure than storing the credentials, encrypted or not, because an attacker would have no way of getting the password back, even if he manages to get his hands on both the hash and the salt.
Storing decryption key in plain text is never a good option, because an attacker would get your users' passwords as soon as he gets access to the key.
There is no good solution to persisting your own credentials, encrypted or not. Your component that talks to 3-rd party services should use developer APIs from these providers. For example, PayPal provides two sets of APIs which you can use to access your account without having to store your password.
If you need to store a small amount of secret information in an encrypted form, use registry APIs to store the data in a key known to your application, and accessible from the user running your server-side component. This secret would be safe, as long as hackers do not hack the password for the account under which your service is running.
* Technically, password is not encrypted, it's hashed, i.e. there is no reliable way to turn the result of conversion back to the original value.

Your suspicions are correct. If the user has access to the key they can just go and decrypt the username and passwords themselves.
The two options you have are
Make it difficult enough to get the password that the reword of getting the password is not worth the effort to find it. This approach is done via things like Code Obfuscateors, I would not recommend this to someone starting out. Its not easy to get it right and it only takes one person who things it is "worth the effort" for it to break.
Don't give the user the information ever. Instead of storing the usernames and passwords in the program have your program call out to a server you own, then that server is what makes the request using the credentials. This approach is the more reliable one and is "unbreakable" (as long as your server is secure) but costs more because you now need to keep a server up and running that can handle the load of your entire userbase.

Can I avoid storing MS Exchange credentials while still being able to authenticate (against EWS)?

I'm building an application that syncs data between users' Exchange Server accounts (version 2007-2013 supported) and the application.
The application can't use impersonation (at least not in the typical case) as users could be on any number of domains and exchange servers.
I know I'm going to have to ask for their username/email-address and password initially. However, I really don't want to be responsible for storing these credentials if I don't have to (even if they are encrypted, I'd rather not).
I'm not sure what questions to ask, so I'm going with these:
How does Exchange Server authenticate? Do the user's credentials get sent directly to the server as they are, or are the hashed together before being sent across the wire? If they are hashed, how can I get/generate this hash for re-use on successive authentications?
Does Exchange Server send some sort of authentication token that can be re-used later (and forever, until password change or invalidation)?
If you know of a solution to the problem, that the answers to these questions won't address, please do provide it instead.

Active directory federation services is exactly for such tasks. You can read about it there.

As mentioned by Kirill, ADFS 2.0 is one of the best solution for your task. You can also look into other SSO implementations as well. Though the main goal of SSO implementation is to maintain single Login state for multiple application (thereby reducing multiple Login prompt for each application), some of your application goals seems relevant. Please do a thorough research on all the tradeoffs before heading to the sso implementation since there is a small degree of complexity involved during implementation. SSO suits best if you are considering integration of multiple application in the future with the exchange server.
To answer some of your questions (in the same order - considering an SSO scenario with ADFS 2.0):
The authentication to exchange server will be done via ADFS 2.0 (Which provides security tokens (STS service) - to your application after authenticating with AD/ main Directory service). All the communication is encrypted and token signing certificates are used for Integrity and confidentiality.
The lifetime of Security tokens sent by ADFS 2.0 can be configured and reused as required. Please see this blog post for more details.
Also you can configure the ADFS 2.0 (Federation Service) to send only the relevant claim values (like username and email address) to the application, thereby improving the data security.

The System.Net.CredentialCache should work to suite your needs. The WebCredentials is a wrapper for the System.Net.NetworkCredential. Depending on the connection type/domain ect you should be able to utilize System.Net.CredentialCache.DefaultNetworkCredentials or System.Net.CredentialCache.DefaultCredentials

perhaps you should take a look at this Links Connecting to EWS by using the EWS Managed API , Connect to Exchange - Getting Started Tutorial? hopfully it will give you a new idea how to solve your problem :)
because if i understand the information correctly you maybe over think problem but i haven't any experiences so i could also absolute wrong

Bottom Line
If you can't configure anything on the server, there's no automatically generated token to use. It's unfortunate, but you're facing the same general problem that web browsers have--saving the password.
It's worth noting that any authentication needs to be over SSL (an https connection) to prevent a third party listening in on the authentication.
Password storage thoughts:
My suggestion is then to be somewhat creative when storing the password. You can use a keyed encryption algorithm, and then use a hash to generate the key, letting you arbitrarily choose what goes into the key. You would want at least 3 pieces of information going into this: something unique to the device, something unique to the app, and something unique to the exchange server.
For example:
a unique id given by the device (it doesn't matter whether or not this value is app-specific or not, merely that it is consistent)
a (long) string of information compiled into the app, possibly keyed to installation specific values, say the time when the app was first used
something unique to the destination, like the DNS name and perhaps some more specific server info
If you're willing to provide the option to the user, you could have an authorization PIN of some kind that would also be added to the data.
All this data gets put together in one byte array and hashed. The hash (or part of it, or it twice, depending on the hash size vs. the key length) is then used as the key for the encryption of the password.
You could also include some check information along with the password to be able to check client side whether or not the password was decrypted correctly. (If the wrong data is hashed, the wrong key is generated, and the wrong result comes from the decryption).
It's worth noting that all the information to be used for putting into the hash needs to be stored on the device, which is why I would suggest a Pin to authorize the usage of the account.

Application pulling data from site?

I am looking forward to have the less information as possible on my application resulting in pulling most of the data from a website or server and I would like to hear by advices and recommendations from everyone with experience in the field to help me improve my application.
Currently my authentication consists of:
To start the application the user have to input their username and password (without a valid user and pass the app does not work) and click login.
The credentials will be sent to my login page using HTTPS, the script only accept SSL connection and a personalized User-Agent name.
The login page will verify the credentials and send back a session and some initial data.
The session is reused to gather more data over time or as needed.
Side note: the SSL is 256 bits, the session auto expire after a few minutes
For the above basic authetication and data pulling would you recommend me anything else to implement ?
Should I change anything ?
2nd Layer of protection
Now I would like to implement more security by encrypting all the data sent from/to my application, my question:
What should I use to encrypt and decrypt the data, a pair of private and public keys held on both sides or RIJNDAEL's method ?
What is the proper way or what parts of information should I leave on the client and server or how should I form the knowledge of the common passwords or keys ?
For example if I was using a pair of RSA keys, I would need to leave 1 private key at the client and 1 public key since you cannot decrypt any data with the public key on c# while you can do it on the server and for Rijndael would need to have the IV and the key on both side.
What is the proper way to handle these ?
I will be very glad with practical reading material, comments, examples, suggestions, advices :)
UPDATE:
No one else wants to say anything ? I was hoping to get more input on this question and also tough that this was a fairly used method around ...

Sounds like you've considered the security of the app side of the equation at length. I'd suggest you turn your thinking to the website part of the equation.
When I look at security, I always look at two things:
Transport security--is your data protected while in transit? Sounds like it is, assuming you are using sufficient key length for the SSL cert on the server. Also, you can force the website do client cert validation as part of the SSL handshake. That ensures that no one can spoof the client app and convince your website to share information.
Payload security--should you encrypt the data payload? Is there a chance someone may be able to break into your web server, or better yet, via the DNS poisoning or some other method, convince your app to connect to a malicious server with a valid (for that cn), but different HTTPs certificate? Should you choose to encrypt the payload, you can piggy back the whole thing onto the same certificates you already use. Just make sure that the certs have data encryption bits turned on, and you can use private/public keys from the certs to encrypt the payload. So, if the malicious user replaced the cert, not only do they need to spoof the cn and the chain of trust, but also have the right public key from the app to decrypt the data you are encrypting with your private key and signing with server's public key.
Some other questions to ponder:
You are saying that the session is reused? Does it not expire? If not, you'd want to make it expire.
Can you leverage network security? Can you use a VPN tunnel or IP ACL to limit who can even gain access to the web server?
What about keyloggers? Passwords can be intercepted. The second authentication factor can be something a user has, like a key card or a fingerprint, or an RSA SecurId. If you don't want to go that far, you can present a user with a "site seal"-- an image that they have to recognize as associated with their account. Maybe even present several images and let them choose one that they've chosen during the signup process. You can also make them solve a small puzzle--something that will distinguish a human from a machine (CAPTCHA-type).
All these points minimize info stored on the client side--just a cert or two and a key to go with that cert.
The topic of security is vast, and we can start a whole discussion here on different implementation details. The above is just some points to think about.
Remember, that everything has cost. Security costs usability and CPU cycles. Proper balance is key, but that of course is up to you.
Before you build Fort Knox, make sure someone is going to want to live there :)

SSL + Jquery + Ajax

I starting too look at a bit of security into my site. My site I would consider a very low security risk as it has really no personal information from the user other than email.
However the security risk will go up a bit as I am partnering with a company and the initial password for this companies users will be the same password they use essentially to get onto the network and every piece of software.
So I have up my security( what is fine by me...I wanted to get around to this anyways).
So one of my security concerns is this.
A user logs in.
form submit(non ajax is done).
Password is hashed & Salted and compared to one in the database.
Reject or let them proceed.
So this uses no jquery or ajax but is just asp.net mvc and C#. Still if my understanding is right the password is sent in clear text.
So if a use SSL and I would not need to worry about that is this correct?
If that is true is that all I need?
Second the user can change their password at anytime. This is done through ajax. So when the password is sent it is sent in clear text( and I can verify this by looking at firebug).
So if I have SSL enabled on this page is that all I need or do I need to do more?
So I am just kinda confused of what I need to make the password being sent to the server(both ajax and full post ways secure).
I am not sure if I need to do more then SSL or if that is enough and if it is not enough what is the next layer of security?

If you use HTTPS/SSL for your page, your AJAX requests will (by default) be handled the same way, so yes this would eliminate all the clear-text issues and you'll get all the security benefits of SSL with your AJAX requests as well.
I would say for most sites this provides a sufficient level of protection, as much as is sane anyway. Keep in mind that nothing is 100% secure (go to any mainstream conference, the security talks will scare you :), it may be 100% secure so far, but it rarely remains that way. That being said, SSL is the most effective security mechanism/deterrent readily available, and it's the best route to go.
Also, SSL is the cheapest in terms of development time, you may not need to change not one line of code in your app to get it working, just make sure to keep your certificates valid/up-to-date.

if you are using SSL your data sent accross the server is well taken care by SSL let it be a ajax hit or simple hit