I am developing .Net Core Web API 2.2 project and trying to protect it best I can. This application will be connected to SQL database plus it will be sending emails from the server, and therefore I would like to figure out what is the good way of protecting my sensitive data (such as connection string, database password or even email password for SMTP account).
I have read that it is bad practice storing your passwords in a plain text in your file somewhere and one of the best practices is to use some Microsoft Azure functionality (where you provide some key and it returns you the actual password) that I have not yet used. Furthermore I do not have any subscription with Azure, and for the time being I would like not to go that direction.
Another method proposed by some of you guys was to store all the password to Environmental Variables and simply reference it in the application. I am currently exploring this option, as my app will be hosted on a 'virtual windows server' where I do not have direct access to, and thus it's difficult (without direct access) to get there and set up environmental variables (not even sure if that would be possible).
Finally, so far the best option (in case it will not be possible to use the variables mentioned above), was to actually store connections and passwords directly to appsettings.json file, but to hash them and decrypt on run-time. This option for me is surely feasible; however I wanted to ask (even though this might be quite subjective) you guys, whether this is a correct approach or there is something I have missed that could help me better to protect my application from external threats.
Any suggestions or advices would be more than appreciated as I do not really know now how to proceed.
P.S. I am using VSTS repository to store all the application code, which might be probably (I am guessing) the reason why people suggest to at least hash your passwords when storing them in appsettings.json
The appsettings.json file should never be used for secrets simply because it's committed to source control. That alone makes it a bad choice. However, there is also no capability to encrypt anything in appsettings.json. You could, I suppose, encrypt your secrets via some other means and merely place the ciphertext in appsettings.json manually after the fact, but then you would need some facility to decrypt the secret later, when then means exposing your means of encryption (i.e. your private key), which kind of defeats the entire point. Long and short, don't use appsettings.json.
Environment variables are a compromise solution. Since you manually set them on the server (not in your source control) and they can be made to only be accessible to certain users (restricted access), you get a modicum of security. However, they are also stored plaintext, which means if someone is able access the server to view them, all security is out the window. Environment variables can also be set as part of your CI\D pipeline in DevOps (formerly VSTS), so direct access to the server is not necessarily a prerequisite as long as the service account doing the deploy has the necessary access.
Azure Key Vault is the recommended approach because it's the only built-in config provider that supports encryption, meaning your secrets are encrypted at rest and pretty much secure end-to-end. However, there's nothing uniquely special about Azure Key Vault other than its ready availability. You can conceivably use any type of service that lets store secrets securely; you may just have to write your own config provider to target it.
Related
I'm new in ASP.Net core, I try to develop ASP.Net core MVC Web API. I need connect to my database. I no idea where I should put my connection string. I put it in appsettings.json file. But it will expose my database password when i deploy the API to the non Azure hosting. I try secret manager but it seem like just for development stage，not sure I understand right or not. I try to put in Environment variable but it not work when in my IIS hsoting.
May i know how I can secure my sensitive data when deploy to Non Azure hosting. Where should I put?
There is analog of Azure KeyVault it free and open source.You can store there your connection strings and others sensitive data.Also there is client library for .net
Your main choices are:
JSON (unencrypted)
Environment variable (unencrypted)
Azure Key Vault (encrypted)
As you correctly ascertained, JSON will be plain-text and is undesirable as a result, mostly because of source control. If you're working on a closed-source solution, this is less of an issue, though. Once deployed on the server, file permissions can keep the details from prying eyes. User secrets is just JSON. It's better in that it's kept out of your project (so it doesn't get committed to your source control), but it's still stored unencrypted, and yes, it's only for development.
Azure Key Vault is the only built-in option that allows encryption, and thus, is the most secure option. You can use Azure Key Vault whether or not your app is actually hosted in Azure. However, it's not free (though, it's also not very expensive).
Environment variables are what's most commonly used outside of Azure Key Vault. While unencrypted, they're stored on the server (outside your project) and are only visible to users with sufficient privileges on that server. If a malicious actor is able to gain access to an account with privileges to view the environment variables, it could be argued that you have far worse problems than exposing database credentials, already.
If the environment variables are not being seen by your app, there's two things to look at:
Ensure that you've added them as system variables and not user variables. User variables are only accessible by the user logged in while adding them (i.e. you), which is almost assuredly not the same user the web server is running as. Alternatively, you can log in add the web server user to add the variables as user variables. This is common when running under a service account. It adds a questionable degree of extra security as only that one account can see the values, not any admin level account (smaller attack surface).
If using IIS, you must edit the advanced settings of the App Pool, and enable "Load User Profile" from there.
I use "Manage Web secrets" to save db and any other crucial data secret from other developers and team mates
This will help you isolate, various platforms as well
you can read more at
https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-3.1&tabs=windows
I have to save keys out of project and repository. For this, I want to use User secrets. But it is written there,
Call AddUserSecrets only when the app runs in the Development environment, as shown in the following example
And I can't understand or find a cause. Why can't I use it in the Production environment?
You can find it in the link you provided to the User Secrets documentation:
The Secret Manager tool doesn't encrypt the stored secrets and
shouldn't be treated as a trusted store. It's for development purposes
only. The keys and values are stored in a JSON configuration file in
the user profile directory.
I think the short answer is that you probably could if you wanted to but that it is not what it is intended for.
My understanding is that the primary purpose of User Secrets is to keep credentials out of source control. In the days before GitHub and the cloud, most developers just stuck any and all credentials in the web.config and it was mostly ok. Then lots of people started using public repositories and AWS and all of a sudden https://www.zdnet.com/article/trufflehog-high-entropy-key-hunter-released-to-the-masses/
There are now a great many different tools out there for managing secrets, which one best suits your needs is a much harder question, but you could consider:
Are you using access controlled source control?
Are you cloud or on-prem for build and deploy?
Who has read access to the live servers?
How sensitive is the data you are storing?
What other applications are running on the server?
I was just poking around in the CreateDefaultBuilder method and found this, which is perhaps relevant:
if (hostingEnvironment.IsDevelopment())
{
Assembly assembly = Assembly.Load(new
AssemblyName(hostingEnvironment.ApplicationName));
if (assembly != (Assembly) null)
config.AddUserSecrets(assembly, true);
}
Obviously you don't have to use the default version and you could add secrets for all the environments, but there it is.
This is a development time only tool. Storing any kind of secret in a file is risky, because you may accidentally check it in. In production, you can for example use environment variables to hold secrets (or any other more secure mechanism.)
While environment variables are one of the most used options in web development there are some reasons why this may not be the best approach:
1.The environment is implicitly available to the process and it's hard to track access. As a result, for example, you may face with situation when your error report will contain your secrets
2.The whole environment is passed down to child processes (if not explicitly filtered). So your secret keys are implicitly made available to any 3rd-party tools that may be used.
All this are one of the reasons why products like Vault become popular nowadays.
So, you may use environment variables, but be aware.
User secrets are basically a JSOn File somewhere in your user directory. That works well on your dev pc. But on a production system, the values should usually be injected through more production ready configuration system(s), like Environment Variables, appsettings.json or a azure keyvault. Envs and appsettings are already activated per default.
Let's say I have a MySQL database into which users can enter some personal data like postal addresses via a php website. The users can NOT log into this site to later verify what they have entered. The enterprise they have entered their data for (voluntarily of course) then can use this data to maybe send the users actual mail (you know, postal services and such) or emails (of course the users agreed beforehand that they want to receive emails). The database serves merely as a store for the data and I want it to be a bit safe. If anyone breaks into the database, retrieving the email address and the actual first and last name (many email addresses contain both anyway) might not do much harm, but knowing where people live could be too much of a giveaway.
The enterprise is accessing the database via a C# front-end that targets stored procedures within the database to do stuff, including searching for users based on their email address.
From what I have gathered through searching I could think of the following procedure to handle the personal data in a more secure way (than saving them as plain text in the database)
before submitting the sensitive information to the stored procedures the plain text gets encrypted with a key while still in php, so all the MySQL server logs see is encrypted data
the front-end uses the same key to make the data humanly readable again when it is displayed to the enterprise's users (They need to access this private information and the user is comfortable with the enterprise doing so, that's the whole point of this scheme)
My line of thinking is: These are not passwords that are stored so I don't need all the password hashing trickery (As I understand it, when securely saving a password in a database, you use a one way algorithm, so you can never reverse engineer the password straight from the database but have to hash every password you want to try and test that against the desired database entry to see if you chose the right password) but can instead go for a simple encrypt/decrypt, because I don't want to brute force every address out of the database.
There are a few rough edges that raise concerns for me:
I need to somehow provide the key I want to encrypt by to php. Usually this is done via a library or external php document, like you provide the database connection information in a separate php file which resides in a folder on the server which is not accessible from the web (The server will say access denied if you try to access it) Is this a good practice and can I be sure that this key-file is really safe?
I need to provide the key to the front-end as well. This should be done separately in a (maybe encrypted?) config file for the front-end. Is it wise to have the key in two places, albeit for two different systems? The key must never change or else part of the data will be lost!
Somehow I have the feeling that if someone knows how to access the database he/she probably figured out where the connection data was and how that was to be accessed. Oh look, here is an encryption key, I wonder what that does. How likely is it that if the database access is breached the encryption key is in the open as well rendering all efforts to give a little extra of privacy to the users void?
If I wanted to add another bit of "extra security" and encrypted the email address as well I would have to encrypt every email address I want to search for from the php or the front-end, right?
Having searching procedures using ´RLIKE´ will break on encrypted fields, won't they? So to retain searching for parts of an email address I cannot encrypt the email entry, right?
I will have to change my database fields to binary to accommodate encrypted data or make them bigger and base64-encode them, won't I?
Is there an encryption/decryption algorithm ready to use in PHP 7.0.7 and C#? (I don't worry that much about C#) One that is reasonably secure while not bloating my tiny texts to massive chunks of binary? I don't know if that is of any consequence, but if I use for example a 256 bit key, that's 32 bytes. If the street part of the address is shorter that 32 characters, will the encryption work? Will there be cumbersome padding involved?
All in all I feel that the security gain is minute compared to the measures I have to take in my php files as well as in the code for the front-end. The perceived security gain might be bigger ("Woha! They are saving our data encrypted! They sure know what they are doing!"). Having strict and restrictive privileges for certain types of users (for example revoke ´SELECT´ commands) should in all be more helpful, shouldn't it?
Edit for #Luke Joshua Park:
Thank you for your detailed answer.
I suppose by API server you mean the webserver my php's reside on? This indeed should be separate from the database server. Both servers are hosted within a university's network but can be accessed from the internet.
I can follow the authentication path to the point where every user from within the enterprise (small-ish project at said university, maybe a bad choice of wording) has a database user with sensibly set grants. But users from outside using the php only send data to be stored in the database (ideally with a common but seperate database-user with grants set accordingly), and never retreive (their own) data. Using authentication would mean they first had to create an account (which is not needed) and how do they authenticate themselves for creating the unwanted account?
It is good that you're asking these questions before implementing a solution, cryptography is difficult to get right and a sound understanding is required before you start.
I'll answer your questions quickly first, but the more important part is what follows.
Not really. See below.
Yes, in most cases, keys should be kept on the device they are created on, wherever possible.
Provided your API server(s) doesn't also have the database on it, relatively unlikely.
Yes.
Yes.
Yes. But don't base64 them. Wasted space and processing power for no benefit.
You're asking the wrong questions. An algorithm isn't "for" a language. You just need to pick the right algorithm/block mode/padding depending on your needs.
For the most part the questions you are asking are irrelevant. Believe it or not, your issue is more to do with authentication than it is to do with encryption.
The first thing you need to understand is that a server breach is a server breach. Bad stuff is going to happen regardless of how much cryptography you throw at it. But we can minimize damage where possible.
Your database software should be running on a separate server/instance/whatever from your API server. Encryption/decryption should only take place on your API server. This has the benefit that both your API server and database server would have to be breached in order to decrypt the data (to access the keys). How you store the keys on your API server isn't all that important provided they aren't in your webroot or something silly like that.
Everything past this is authentication. You need a strong authentication system to determine who can send you information and who can retrieve information from you. Communication to and from your API server should obviously be encrypted with TLS at all times. You might consider TLS client authentication as a way to ensure the entity requesting data from you is who they say they are. Usually client authentication can't really be used on the web, but if you're interacting with "enterprises" in a more private way, then client authentication is an excellent choice.
In summary:
Separate your API server from your database server. Encryption keys should only ever be on the API server. See this repository for a collection of encryption examples from PHP to just about any other language.
Use TLS for all ingoing and outgoing communication.
Focus on authentication. TLS Client authentication is a good option.
If I have a stored key file used to decrypt the encrypted input coming into my application, and I store that key file as an embedded resource so that it is embedded in the assembly when deploying, how difficult would it be for someone to reverse engineer the application and retrieve the key file?
Also, the application is deployed through ClickOnce "Online only" mode which I imagine would also make it more difficult to reverse engineer? (I'm not exactly sure of the workings of ClickOnce but I wasn't able to find the dll's/assemblies on my local machine after running the application in online only mode...).
Update:
As Ralf essentially answered the main question below in his comment (answer: it's not really safe at all), here's some more information so that you knowledgeable people can possibly suggest a better security model.
The encryption will be used to encrypt the login password for my application, to be used in a SSO setup (the user will first sign on to a different system and then by clicking a link will be able to directly open my application without having to enter in their login details).
The encrypted data will be sent as a base-64 string URL parameter in the link that will launch the my click-once application.
I will also be developing the application that will create the encrypted data for the URL parameter (clarification: not the first application the user will sign in to for the SSO, I will only be creating a small tool to convert the plain text password into an encrypted base64 string).
It's only an internal application so bullet proof security isn't essential and ease of deployment is more important, but it'd be nice to know what the best practices and different options that are available.
Whether in clear text or encrypted, you don't want to store the password. When you get a password, all you should do is pass it to your server app where you compare it against the salted hash of the password you have in the DB. Even if you don't think security is that important, you need to take care with the password because people frequently reuse passwords across different systems. I know they shouldn't but they do.
If you want to implement a single sign on (SSO), create a sign on token on the server side and pass it back to the client, either encrypted or signed (HMAC is a good choice for signing). This is an unforgeable token because you need to know the encryption key or shared secret for the HMAC and that data is only known on your servers. So you have your SSO and all data involving the SSO is managed on servers so there's no data leakage or chance of spoofing.
As long as the aplication can be launched, the files have to be somewhere on the computer. You just have to know where to look. The reverse-engineering may be ugly but it is always possible. The computer has to be able to understand what he is supposed to do so you only have to extraxt the information one is looking for. Therefore the security of your application should never depend on the difficulty of reverse-engineering! I believe that a secure application should be open-source anyways.
You propably need a diffent security model. The important thing here is that you know against what you want to protect the data. If you simply want that you know the data is sent by the server and not somebody else (man in the middle attack) you could use digital signatures instead.
If you do not want anybody to read any data sent between server and client you should use some sort of ssl implementation to create an encrypted channel. Then you only need to watch out that the public key of the server is not altered on the client. This may be done by an certificate of an official CA but unfortunately those are usualy not for free.
In many Application's i had Cases where i should write Username and Password's in my Application inside the Class ,like HTTP Authentication ,FTP Authentication MSSQL Server Connection String also Provides Authentication Information's ,so which is the Best way to protect these Information's because someone could Decompile my Application easily maybe using Reflector and get these Information's which can be useful for some Attack's or something like that .
Bests
Can you explain more specifically what your general goal is? Usually there's better designs than hard coding passwords and authentication tokens.
There's really nothing you can do to protect those secrets if you are distributing your application. Even encrypting your config files, the application still needs the key to decrypt, so your attacker has all they need.
You can choose other designs though, such as prompting a user for their own unique password, and then making database calls against a web service, rather than distributing an application that connects directly to a central database. But you'd have to explain the scenario for a better recommendation.