We Keep Coding

Client-Server Secure communication within application

I want to create a game within the Unity game engine in C# with .NET where I can securely connect out to a private server, and read and write data from/to the server. I understand how to do this in a non-secure way, where I would setup a private SQL database with a webpage interface between the two.
My problem comes here, I know if I want the web interface to be public, I need credentials to connect to the server, such as an Auth key, or username and password. However, in order to use those, then the auth-key would need to ship with the game as a file, or would need to be written directly into the codebase. I know that users can decompile games, and access these files and get the credentials if I include them, which means that option is off the table as far as I am aware.
It is my understanding that even with this layer between the database and the client, someone could still find the url for the interface between them, and then send custom function calls to this url as I will be doing in the game program. Wouldn't this lead to the same conclusion as having the database be open?
If anyone has answers to this, or resources where I can learn about this process, please let me know! I have never done server-side programming before and have also never thought about security before in my life.

You're correct in that you should not embed authentication keys into your apps. Where you're falling down is in exposing your database to the world. Don't do this. Instead, hide it behind a web server that serves only json data files. This way you can take advantage of the web server's authentication and session protocols.
Since you're already working with C#, I would suggest looking into ASP.NET Core MVC and specifically about WebAPI. But I would probably recommend node.js for lightweight microservices.

What are the best practices for implementing API security with a publicly available mobile application without user authentication?

Context
I'm currently in the works of creating a public mobile application.
This application will be made available through the App Store and Google Play Store.
The user will be able to perform certain actions in the application, which will then be posted to an API.
The application will be made with Xamarin Forms.
Question
I want to secure this API to prevent anyone other than the App from making any post requests.
From what i have read the best way to go about this would be to implement an OATH2 flow.
However; i wouldn't like for a user to have to sign-in or register to any type of service.
How would i go about validating the post requests that come into the API and make sure that it's not someone that is impersonating my mobile application?
Things i have thought of:
Api-keys or clientsecrets. As it seems there are pretty easy to extract from a mobile app.
Certificate validation. (Couldn't find much on this for mobile client - server architecture)
Generating some type of checksum over the contents that i can validate serverside. However the code for this would have to be available client side or there would have to be an endpoint providing the checksum. But this would just move the problem imo.
?
Any and all critic and feedback is welcome.
i do apologize if there's anything obvious that i have missed during my research on this matter.
Thank you for your time.

Check the following for android: https://developer.android.com/training/safetynet/attestation
The SafetyNet Attestation API provides a cryptographically-signed attestation, assessing the device's integrity. In order to create the attestation, the API examines the device's software and hardware environment, looking for integrity issues, and comparing it with the reference data for approved Android devices. The generated attestation is bound to the nonce that the caller app provides. The attestation also contains a generation timestamp and metadata about the requesting app.
For iOS, you can check the: https://developer.apple.com/documentation/devicecheck
As mentioned in the documentation, non are foolproof and should be joined with other security measures, but it's better than nothing nonetheless.

The Difference Between WHO and WHAT is Accessing the API Server
From what i have read the best way to go about this would be to implement an OATH2 flow.
OAuth2 only identifies who is in the request, not what is doing the request.
This is a common misconception among developers of any seniority, therefore I will link you to an article I wrote that has a section dedicated to explain this. I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between who and what is accessing your API server, but I will extract here the main takes from it:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
So think about the who as the user your API server will be able to Authenticate and Authorize access to the data, and think about the what as the software making that request in behalf of the user.
Your Questions/Affirmations
I want to secure this API to prevent anyone other than the App from making any post requests.
And about GET request?
Depending on the data your mobile is asking to be delivered by the API server, the GET request may have the same degree of importance in terms of security then any POST request.
Let's say you app is doing GET requests to get sensible data like PII(Personal Identifiable Information), then without them being protected you are just opening your API server to be easily abused by attackers, and this results in the so called data breaches that can have a huge legal and financial impact for the business and their customers. For example in Europe with GDPR a data breach will have huge fines, specially when they conclude that the API was not properly protected to avoid automated data extraction.
Anyone can learn how to perform a MitM(Man in the Middle) attack to see how the mobile app does the API requests in order to automate them. I wrote the article
Steal that Api Key with a Man in the Middle Attack that illustrates how easy a MitM attack can be done:
In order to help to demonstrate how to steal an API key, I have built and released in Github the Currency Converter Demo app for Android, which uses the same JNI/NDK technique we used in the earlier Android Hide Secrets app to hide the API key.
So, in this article you will learn how to setup and run a MitM attack to intercept https traffic in a mobile device under your control, so that you can steal the API key. Finally, you will see at a high level how MitM attacks can be mitigated.
However; i wouldn't like for a user to have to sign-in or register to any type of service.
Well then you are left with only one way of protecting the API request, and is to know what is doing the request, and now you bought yourself a very hard problem to solve, but not an impossible one.
Your Solutions
API Keys or Client Secrets
Things i have thought of:
Api-keys or clientsecrets. As it seems there are pretty easy to extract from a mobile app.
They are indeed very easy to extract, but you can make them harder(but not impossible) to extract by static binary analysis as I explain in the article How to Extract an API key from a Mobile App with Static Binary Analysis:
The range of open source tools available for reverse engineering is huge, and we really can't scratch the surface of this topic in this article, but instead we will focus in using the Mobile Security Framework(MobSF) to demonstrate how to reverse engineer the APK of our mobile app. MobSF is a collection of open source tools that present their results in an attractive dashboard, but the same tools used under the hood within MobSF and elsewhere can be used individually to achieve the same results.
Spoiler alert is that the best way to hide secrets from the above article is by using the JNI/NDK interface to hide them in native code:
Using Android Studio 2.2 and higher, you can use the NDK to compile C and C++ code into a native library and package it into your APK using Gradle, the IDE's integrated build system. Your Java code can then call functions in your native library through the Java Native Interface (JNI) framework.
Well now you just use a MitM attack to extract them and we are back to square zero? Well not exactly, because more layers of defense you apply, more the attacker needs to peel and be willing to spend the time on it, but much more can be done to upset them.
Certificate Pinning
Certificate validation. (Couldn't find much on this for mobile client - server architecture)
You can't find because certificate pinning is usually only done by the mobile app as I show in the article Securing HTTPS with Certificate Pinning:
In order to demonstrate how to use certificate pinning for protecting the https traffic between your mobile app and your API server, we will use the same Currency Converter Demo mobile app that I used in the previous article.
In this article we will learn what certificate pinning is, when to use it, how to implement it in an Android app, and how it can prevent a MitM attack.
So why not doing the same sever side? Well because once the app is release anything on it is public, therefore the certificate private key you would to ship with the app to allow the API server to pin against would be easily extracted with static binary analysis or during runtime with an instrumentation framework like Frida
Inject your own scripts into black box processes. Hook any function, spy on crypto APIs or trace private application code, no source code needed. Edit, hit save, and instantly see the results. All without compilation steps or program restarts.
I can imagine you now looking where is the link to the my article with an example of using Frida? Sorry, but I don't have yet an article with a Frida example :(
Digitally Signing Requests with HMAC
Generating some type of checksum over the contents that i can validate serverside. However the code for this would have to be available client side or there would have to be an endpoint providing the checksum. But this would just move the problem imo.
Yes you are correct, the problem would have to have the code in client side, but having an endpoint providing the checksum would only move your problem into two backends, because then you would need to ensure that the checksum endpoint would only sign request from your mobile app ;)
HMAC:
In cryptography, an HMAC (sometimes expanded as either keyed-hash message authentication code or hash-based message authentication code) is a specific type of message authentication code (MAC) involving a cryptographic hash function and a secret cryptographic key. As with any MAC, it may be used to simultaneously verify both the data integrity and the authenticity of a message.
Other Solutions
How would i go about validating the post requests that come into the API and make sure that it's not someone that is impersonating my mobile application?
As I said before you should not limit this to POST requests, instead you should validate that any request to the API server is indedd from *what you expect, your genuine and untampered mobile app.
Android SafetyNet
This is often the one recommended, and is a very good starting point, but it's not designed to be used as a stand-alone defence, as per Google own words:
The goal of this API is to provide you with confidence about the integrity of a device running your app. You can then obtain additional signals using the standard Android APIs. You should use the SafetyNet Attestation API as an additional in-depth defense signal as part of an anti-abuse system, not as the sole anti-abuse signal for your app.
You can read my answer to the question Android equivalent of ios devicecheck for a more detailed explanation for what you need to pay attention for when implementing SafetyNet.
iOS DeviceCheck
Until recently the device check was very limited in is usefulness, but now with iOS 14 they also the app test feature to sign the requests sent to the API server, but it still have some limitations, but I have not had the time to full explore this new feature thus I will not be able to give the same detailed explanation I gave for what to watch for when implementing the iOS device check, but I can recommend you this article:
It’s important to know that App Attest is not a “is this device jailbroken?” check, as that has been proven over and over to be impossible to perform. Instead, it aims to protect server requests in order to make it harder for hackers to create compromised versions of your app that unlock premium features or inserts features like cheats. Note the word harder: as jailbreakers have physical access to their devices, nothing will completely safeguard you from fraud in this case.
If you have a large user base, Apple recommends you to gradually roll this feature as requests to attestKey are rate limited. After carefully rolling it out for existing users, you can guarantee that it will only be called for new users.
So while this is a very good improvement from Apple it's rate limited, therefore care must be taken in what requests we decide to use it and how frequently the attestations are performed.
Mobile App Attestation
This is a concept that you can build on top of the Android SafetyNet and iOS device check solutions, and you can read my answer to the question How to secure an API REST for mobile app? in order to understand more about how the Mobile App Attestation concept works. You want to read about it in the section "A Possible Better Solution".
Do you want to go Going the Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.

How to Grant Angular Authorization to my REST API Calls for Mobile App

I have a C# Azure Web API backend where data is retrieved from a front-end Ionic Mobile App (which is basically an Angular App)
The authorization of users is done via Ionic's cloud service, so they handle the heavy lifting of registering users via FB, Twitter, basic (username/password).
My question is, when I go to call services from my backend API, how can I make sure someone just doesn't read a hardcoded username/password inside of the internal javascript code to access the backend data?
I know it's pretty far fetched, but is there anyway for the API to know the request is actually coming from the app (Android and iOS) and not just from someone trying to insert data and comments from a web browser that is unauthorized?

Since you're calling the API from JavaScript that is available for end users, you can assume that your JavaScript and all the logic/credentials contained within are accessible to all.
There are fairly secure ways around this, and FB/Twitter and their ilk have implemented it (using OAuth). Essentially, on passing credentials to the API, a token is generated, which is then used for subsequent calls to the API instead of the credentials.
You can avoid people randomly firing off 'unauthorized' requests using nonces which are generated when you render the form, and can be used only once to submit the form in question. You can then time-limit the validity of the nonce on the API end. Unfortunately, it's not foolproof, but this will limit the damage of any sort of 'brute-force' attack that you might get.
Again, with any shared 'secret' (that would guarantee the origin of requests), you have to assume that anyone with enough willpower will be able to extract it from apps, thus any method you implement here will be 100% foolproof. Probably the best you can do is have a shared secret generated for each user on each device.

Short answer: you can't.
Long answer: you can (and must) validate the behaviour of a client but not the client itself.
For example we can take a look on Pokemon Go: after a few hours there were bots able to play, after a couple of weeks Niantic started assuming Machine Learning software engineer and encrypt its API using unknown6 algorithm for stopping the bots, but after a few days of hard working the bots came again online.
You can use all the secure method of this universe (whit an high expense) but if someone (that have good knowledge of software engineering) want emulate your client at the end I will reach his objective

How to force third party service respect the security?

I have to come up with an integration process to allow existing system to use external data providers. The system is a medical timetable web site, using ASP.NET MVC, that allows the patients to schedule their appointments to doctors.
As far as I go you can see on a figure below:
All the providers must expose my contract ISuperIntegration which will be develop by me. I won't be developing External service 1 and External service 2, they will be developed by other companies.
Here the issue comes: basing on the concept of that I could require the way providers should setup their services to communicate with my website properly, I want to forbid for another third party clients consume "External Service 1" and "External Service 2", Or at least make it difficult to do that.
Here is a list of stuff I am setting:
ISuperIntegration interface. It contains operations related to my domain such as GetSchedule, GetDoctors and so on.
Transport protocol. I don't want it to be complicated so I'm thinking about using HTTP.
And could define some general recommendations but they could be easily avoided.
At the moment I'm thinking of using HTTPS with certificate authentication. It would require the implementer to setup their infrastructure so my web site could properly consume the data.
If I would go with basic HTTP, the implementer would just leave their service to be easily consumed by anyone else, which I don't want.
I hope my question is clear. Will be happy to give any other explanations you want.
I'll really appreciate any your responses, commits. Thank you!

I'd always use HTTPS for things like this. Let's just say that's the cost of doing business. You simply cannot have anyone with a sniffer grab that kind of traffic out of the sky. There's a reason why all banking etc. use HTTPS for things that should be secure.
Apart from that, web services have pretty standard mechanisms for security, I'd recommend looking at OAuth over HTTPS. There are plenty of implementations for that.
If your talking about basic web sites, I'd use a standard security mechanism as well like group based security (which boils down to a username + password). Again, there are plenty of implementations for that.
Basically my main word of advice is: don't go inventing stuff when it comes to security. If you're not an expert, you're probably going to get it wrong, and end up with something that can be intercepted by a third party or (much) worse.

You have several options:
Basic authentication over HTTP.
PRO. Easy to implemet
CON. UserCredentials was going in clear text throuh the network
Implement WS-Security with WCF. For example, they can sign their requests.
PRO. Easy to implement with WCF
CON. Java clients can faced with problems
You can force clients to use HTTPS.
CON. You should setup your web server.

You are like Oracle, they want people to develop in Java language but they also want to forbid competitors to run the Java compiled code on non Oracle's virtual machines, or at least make it difficult to do that :)
So you can do the same by protecting your interface with patent or copyright law. However, I doubt that it is patentable or copyrightable :)

Considering the privacy sensitivity of the data, IMHO it must be encrypted while in transport. Hence HTTPS not HTTP.
Authentication of your service to those providing services to you: well essentially it's up to them, not up to you who they expose it to, similarly how they want it protected is their call. Now assuming you do have a way to make them do the right thing...
Client certificates aren't that expensive nor prohibitive in setup to get up and running. But you do need to register the client certificate (every time it is renewed!) with the server in order to get the needed authorisation (just recognizing it's a valid cert isn't enough: anybody can apply for a (validly signed) certificate ...) .
But all that is relatively painless and rather well documented around the web, and it can be done on almost any platform of choice.

As several people mentioned these earlier you can't guarantee that those external companies will expose your service with specific security settings, it's up to them.
If you are responsible for developing MVC application and WCF service you can only force someone to use specific security settings on the layer between your WCF services and those External 1 and 2 providers.
Btw, here is a good tutorial that can be useful if you want to improve your knowledge about how to configure WCF security.
How External Services expose your service it's up to them. Just image that this is normal web 'proxy' behavior.
Maybe the architecture which your company adopted it is not the best for this solution

SaaS development using .Net

I do not know how to explain this in technical terms. So let me begin with an example:
Story
I have an online e-commerce site www.ABCStore.com . I built this using MVC 4 (Razor) in Dot Net.
My friend has a travel agency for which his online site is www.DEFAgency.com . He got it built in Java.
Both our websites were up and running. One fine day I got a call from a company FicticiousServiceProvider and they asked me if I would be interested in getting customer feedback as a functionality on my website without having to write any code myself. What they offered was, I would have to include just a single line of code in the footer of my Masterpage(or layout page) and then the customers who log on to the site would see a small icon on the pages and would be able to provide their feedback.
The feedback will not be available directly to me. The FicticiousServiceProvider guys will analyze the data and provide them to me on a regular basis or on a need basis.
There were other services too which they offered.
I was really happy to have a functionality like that, specially without having to write any code. I tried it and it worked fine in my .Net website. My friend(with a java website) also added a single line to his code and it worked for him too.
My questions here are:
What is this process called ?
If I were FicticiousServiceProvider, how would I have developed this using .Net ? I mean, how to develop a functionality so that a consumer can consume the service using a single line provided by the service provider. Data transfer from my site in the form of feedback to the FicticiousServiceProvider is also happening, without me being able to see anything.
How was it possible for FicticiousServiceProvider to provide the functionality to a .Net app and a java app without any change in the line provided by them?
I have given the description from a consumers perspective. Please suggest from a developer's perspective. Many Thanks.

These things, like Google Analytics tracking code, are usually some kind of javascript injecton. It will use javascript to 'inject' a bit of code that sends a request to their servers (what their server side is coded in is irrelevant really). They then handle the request that includes the information they've gathered in javascript on the client side and store it, then use server side software to analyse that data to give out reports, etc..
So to try and answer your question separately.
I'd call the process javascript injection.
You would have to find the best way to send a request to your servers and handle that request. Could be done with ASP.Net MVC quite easily but any server side technology/code that can handle requests and send data to a store.
They use javascript which is separate to any server side code and works across browsers on the client side.