I'm trying to set up a WCF service protected by ADFS. I'm currently able to request a token and send it with the request using WIF and Thinktecture IdentityModel 4.5 with the following code:
static SecurityToken GetToken()
{
var factory = new WSTrustChannelFactory(
new UserNameWSTrustBinding(SecurityMode.TransportWithMessageCredential),
"https://fs2.server2012.local/adfs/services/trust/13/usernamemixed")
{
TrustVersion = TrustVersion.WSTrust13
};
if (factory.Credentials != null)
{
factory.Credentials.UserName.UserName = #"username";
factory.Credentials.UserName.Password = "password";
}
var rst = new RequestSecurityToken
{
RequestType = RequestTypes.Issue,
KeyType = KeyTypes.Symmetric,
AppliesTo = new EndpointReference(
"https://wcfservicecertificate/wcfservice/Service.svc/wstrust"),
};
var channel = factory.CreateChannel();
RequestSecurityTokenResponse rstr;
return channel.Issue(rst, out rstr);
}
With this I can call the WCF service by using ChannelFactory.CreateChannelWithIssuedToken:
var factory = new ChannelFactory<IService>(binding,
new EndpointAddress("https://wcfservicecertificate/wcfservice/Service.svc/wstrust"));
if (factory.Credentials != null)
{
factory.Credentials.SupportInteractive = false;
factory.Credentials.UseIdentityConfiguration = true;
}
var proxy = factory.CreateChannelWithIssuedToken(GetToken());
var result= proxy.GetData(2);
This works as expected but can only be used on (mobile) windows platforms. I would also like to be able to use the same principle on iOS and Android. Using this article I was able to request a security token from ADFS using the following code:
const string soapMessage =
#"<s:Envelope xmlns:s=""http://www.w3.org/2003/05/soap-envelope""
xmlns:a=""http://www.w3.org/2005/08/addressing""
xmlns:u=""http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"">
<s:Header>
<a:Action s:mustUnderstand=""1"">http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue</a:Action>
<a:To s:mustUnderstand=""1"">https://fs2.server2012.local/adfs/services/trust/13/UsernameMixed</a:To>
<o:Security s:mustUnderstand=""1"" xmlns:o=""http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"">
<o:UsernameToken u:Id=""uuid-6a13a244-dac6-42c1-84c5-cbb345b0c4c4-1"">
<o:Username>username</o:Username>
<o:Password Type=""http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText"">password</o:Password>
</o:UsernameToken>
</o:Security>
</s:Header>
<s:Body>
<trust:RequestSecurityToken xmlns:trust=""http://docs.oasis-open.org/ws-sx/ws-trust/200512"">
<wsp:AppliesTo xmlns:wsp=""http://schemas.xmlsoap.org/ws/2004/09/policy"">
<a:EndpointReference>
<a:Address>https://wcfservicecertificate/wcfservice/Service.svc/wstrust</a:Address>
</a:EndpointReference>
</wsp:AppliesTo>
<trust:KeyType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/SymmetricKey</trust:KeyType>
<trust:RequestType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</trust:RequestType>
<trust:TokenType>urn:oasis:names:tc:SAML:2.0:assertion</trust:TokenType>
</trust:RequestSecurityToken>
</s:Body>
</s:Envelope>";
var webClient = new WebClient();
webClient.Headers.Add("Content-Type", "application/soap+xml; charset=utf-8");
var result = webClient.UploadString(
address: "https://fs2.server2012.local/adfs/services/trust/13/UsernameMixed",
method: "POST",
data: soapMessage);
This results in a SAML2.0 token which I would like to send in a request to our WCF service in order to authenticate. There are various sources (including the article mentioned earlier) which state that this should be possible but I've yet to find a solution.
Any help would be appreciated.
You can use one of hybrid solutions which use SAML with OAuth or other authorization technologies. This is more secure against phising techniques. For SAML only approach, you can refer to following link: How to pass security tokenfrom one wcf service to another wcf service. It is said that you need to enable saveBootstrapTokens property on webconfig.
This link can be useful too: Availability of Bootstrap Tokens
This can easily be done without using WIF. Lets completely avoid WIF and the .Net framework and do it in Java for illustration purposes. First make a call to the Security Token Service using the template approach like you have done. You then need to extract the SAML from the response, Base64 encode it and stuff it in the Autorization header of the subsequent request to your protected WCF service. You may also need to do the same with a ProofKey if you are coding for Non-Repudiation. Also I'm only showing authentication using username/password for brevity as Certificate Authentication involves much more work - you have to hash (SHA1 )part of the message then encrypt the hash with the private key of the cert and then add this as a xml element to the original message etc...
Here is the java helper code:
import java.io.*;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.time.Instant;
import java.util.Map;
import java.util.Properties;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.Base64;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import javax.net.ssl.HttpsURLConnection;
public class SecurityService {
private String _username;
private String _password;
private String _stsUrl;
private String _samlAssertion;
private String _samlEncoded;
private String _binarySecret;
private String _workingDirectory;
private String _platformUrl;
private String _soapBody;
private Integer _responseCode;
private Integer _plaformResponseCode;
private String _response;
private String _platformResponse;
private String _xproofSignature;
private Map<String, String> _headerDictionary;
public void setUsername(String username) {
this._username = username;
}
public void setPassword(String password) {
this._password = password;
}
public void setStsUrl(String stsUrl) {
this._stsUrl = stsUrl;
}
public String getStsUrl() {
return _stsUrl;
}
public void setplatformUrl(String platformUrl) {
this._platformUrl = platformUrl;
}
public String getSamlAssertion() {
return _samlAssertion;
}
public String getSamlEncoded() {
return _samlEncoded;
}
public String getSoapBody() {
return _soapBody;
}
public Integer getResponseCode() {
return _responseCode;
}
public Integer getPlatformResponseCode() {
return _plaformResponseCode;
}
public String getResponse() {
return _response;
}
public String getPlatformResponse() {
return _platformResponse;
}
public String getXProofSignature() {
return _xproofSignature;
}
public String getBinarySecret() {
return _binarySecret;
}
public String gePlatFormUrl() {
return _platformUrl;
}
public void setHeaderDictionary(Map<String, String> headerDictionary){
this._headerDictionary = headerDictionary;
}
public Map<String, String> getHeaderDictionary(){
return _headerDictionary;
}
public SecurityService() throws Exception {
}
public SecurityService(Boolean useConfig) throws Exception {
if (useConfig) {
this._workingDirectory = System.getProperty("user.dir") + "\\app.config";
this.getProperties();
}
}
public void sendAuthenticatedGet() throws Exception {
URL obj = new URL(_platformUrl);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
// Add request header
con.setRequestProperty("Authorization", "Saml " + _samlEncoded);
con.setRequestProperty("X-ProofSignature", _xproofSignature);
_plaformResponseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
_platformResponse = response.toString();
}
public void sendAuthenticatedPost(String body) throws Exception {
URL obj = new URL(_platformUrl);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
//add request header
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json");
// Add request header
con.setRequestProperty("Authorization", "Saml " + _samlEncoded);
con.setRequestProperty("X-ProofSignature", _xproofSignature);
// Add Azure Subscription Key using generic Add Headers method
if (_headerDictionary != null) {
for (String key : _headerDictionary.keySet()) {
con.setRequestProperty(key, _headerDictionary.get(key));
}
}
_soapBody = body;
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
//wr.writeBytes(urlParameters);
wr.writeBytes(_soapBody);
wr.flush();
wr.close();
_responseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
_response = response.toString();
}
// HTTP POST request
public void sendPostToSts() throws Exception {
URL obj = new URL(_stsUrl);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
//add request header
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/soap+xml");
String body = getTemplateCertificate();
_soapBody = (((body.replace("[Created]", Instant.now().toString())).replace("[Expires]", Instant.now()
.plusSeconds(300).toString())).replace("[username]", _username)).replace("[password]", _password).replace("[stsUrl]", _stsUrl);
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
//wr.writeBytes(urlParameters);
wr.writeBytes(_soapBody);
wr.flush();
wr.close();
_responseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
_response = response.toString();
// Get Binary Secret
// <trust:BinarySecret></trust:BinarySecret>
final Pattern patternBinarySecret = Pattern.compile("<trust:BinarySecret>(.+?)</trust:BinarySecret>");
final Matcher matcherBinarySecret = patternBinarySecret.matcher(response.toString());
matcherBinarySecret.find();
_binarySecret = matcherBinarySecret.group(1);
// Get the SAML Assertion
final Pattern patternEncryptedAssertion = Pattern.compile("<trust:RequestedSecurityToken>(.+?)</trust:RequestedSecurityToken>");
final Matcher matcherEncryptedAssertion = patternEncryptedAssertion.matcher(response.toString());
matcherEncryptedAssertion.find();
_samlAssertion = matcherEncryptedAssertion.group(1);
byte[] proofKeyBytes = _binarySecret.getBytes("UTF-8");
String encoded = Base64.getEncoder().encodeToString(proofKeyBytes);
byte[] decoded = Base64.getDecoder().decode(encoded);
// SAML Stuff - Works beautifully
byte[] samlBytes = _samlAssertion.getBytes("UTF-8");
_samlEncoded = Base64.getEncoder().encodeToString(samlBytes);
_xproofSignature = this.encode(_samlAssertion, _binarySecret);
}
private static String readFile( String file ) throws IOException {
BufferedReader reader = new BufferedReader( new FileReader(file));
String line = null;
StringBuilder stringBuilder = new StringBuilder();
String ls = System.getProperty("line.separator");
try {
while( ( line = reader.readLine() ) != null ) {
stringBuilder.append( line );
stringBuilder.append( ls );
}
return stringBuilder.toString();
} finally {
reader.close();
}
}
// Embedded WS-Trust template for username/password RST
private static String getTemplate () {
return "<s:Envelope xmlns:s=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:a=\"http://www.w3.org/2005/08/addressing\" xmlns:u= \"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\"><s:Header><a:Action s:mustUnderstand= \"1\">http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue</a:Action><a:MessageID>urn:uuid:cfea5555-248c-46c3-9b4d- 54936b7f815c</a:MessageID><a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo><a:To s:mustUnderstand=\"1\">[stsUrl]</a:To><o:Security s:mustUnderstand=\"1\" xmlns:o=\"http://docs.oasis- open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\"><u:Timestamp u:Id=\"_0\"><u:Created>[Created] </u:Created><u:Expires>[Expires]</u:Expires></u:Timestamp><o:UsernameToken u:Id=\"uuid-e273c018-1da7-466e-8671-86f6bfe7ce3c- 17\"><o:Username>[username]</o:Username><o:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username- token-profile-1.0#PasswordText\">[password] </o:Password></o:UsernameToken></o:Security></s:Header><s:Body><trust:RequestSecurityToken xmlns:trust=\"http://docs.oasis- open.org/ws-sx/ws-trust/200512\"><wsp:AppliesTo xmlns:wsp=\"http://schemas.xmlsoap.org/ws/2004/09/policy \"><wsa:EndpointReference xmlns:wsa=\"http://www.w3.org/2005/08/addressing \"><wsa:Address>https://mbplatform/</wsa:Address></wsa:EndpointReference></wsp:AppliesTo><trust:RequestType>http://docs.oasis- open.org/ws-sx/ws-trust/200512/Issue</trust:RequestType><trust:TokenType>http://docs.oasis-open.org/wss/oasis-wss-saml-token- profile-1.1#SAMLV2.0</trust:TokenType></trust:RequestSecurityToken></s:Body></s:Envelope>";
}
private String encode(String key, String data) throws Exception {
Mac sha256_HMAC = Mac.getInstance("HmacSHA256");
SecretKeySpec secret_key = new SecretKeySpec(key.getBytes("UTF-8"), "HmacSHA256");
sha256_HMAC.init(secret_key);
return Base64.getEncoder().encodeToString(sha256_HMAC.doFinal(data.getBytes("UTF-8")));
}
private void getProperties() throws Exception {
Properties prop = new Properties();
String fileName = _workingDirectory;
InputStream is = new FileInputStream(fileName);
prop.load(is);
_username = prop.getProperty("app.username");
_password = prop.getProperty("app.password");
_platformUrl = prop.getProperty("app.platformUrl");
_stsUrl = prop.getProperty("app.stsUrl");
}
}
and here is example usage:
SecurityService mbss = new SecurityService(true);
mbss.sendPostToSts();
System.out.println("CONTACTING AZURE SECURITY TOKEN SERVICE");
System.out.println("\nSending 'POST' request to URL : " + mbss.getStsUrl());
System.out.println("\nPost parameters : \n" + mbss.getSoapBody());
System.out.println("\nResponse Code : " + mbss.getResponseCode());
System.out.println("\nHERE IS THE SAML RESPONSE\n");
System.out.println(mbss.getResponse());
System.out.println("\nHERE IS THE BINARY SECRET\n");
System.out.println(mbss.getBinarySecret());
System.out.println("\nHERE IS THE SAML ASSERTION\n");
System.out.println(mbss.getSamlAssertion());
System.out.println("\nHERE IS THE ENCODED SAML ASSERTION\n");
System.out.println(mbss.getSamlEncoded());
System.out.println("\nHERE IS THE X-PROOF SIGNATURE\n");
System.out.println(mbss.getXProofSignature());
System.out.println("\nNOW CONTACTING WCF SERVICES WITH SECURITY HEADER\n");
mbss.sendAuthenticatedGet();
System.out.println("\nSending 'GET' request to URL : " + mbss.gePlatFormUrl());
System.out.println("Response Code : " + mbss.getPlatformResponseCode());
System.out.println("\nHERE ARE THE RESULTS FOLKS...ENJOY\n");
System.out.println(mbss.getPlatformResponse());
Related
I got "The account name or password that you have entered is incorrect" error when trying login using this api endpoint: https://steamcommunity.com/login/dologin/
I am using the credentials I use when logging in via Steam app or Steam web, so I don't think I have a problem with my credentials.
Here code which code I use:
public bool DoLogin(string username, string password)
{
var data = new NameValueCollection { { "username", username } };
// First get the RSA key with which we will encrypt our password.
string response = Fetch("https://steamcommunity.com/login/getrsakey", "POST", data, false);
GetRsaKey rsaJson = JsonConvert.DeserializeObject<GetRsaKey>(response);
// Validate, if we could get the rsa key.
if (!rsaJson.success)
{
return false;
}
// RSA Encryption.
RSACryptoServiceProvider rsa = new RSACryptoServiceProvider();
RSAParameters rsaParameters = new RSAParameters
{
Exponent = HexToByte(rsaJson.publickey_exp),
Modulus = HexToByte(rsaJson.publickey_mod)
};
rsa.ImportParameters(rsaParameters);
// Encrypt the password and convert it.
byte[] bytePassword = Encoding.ASCII.GetBytes(password);
byte[] encodedPassword = rsa.Encrypt(bytePassword, false);
string encryptedBase64Password = Convert.ToBase64String(encodedPassword);
SteamResult loginJson = null;
CookieCollection cookieCollection;
string steamGuardText = "";
string steamGuardId = "";
// Do this while we need a captcha or need email authentification. Probably you have misstyped the captcha or the SteamGaurd code if this comes multiple times.
do
{
Console.WriteLine("SteamWeb: Logging In...");
bool captcha = loginJson != null && loginJson.captcha_needed;
bool steamGuard = loginJson != null && loginJson.emailauth_needed;
string time = Uri.EscapeDataString(rsaJson.timestamp);
string capGid = string.Empty;
// Response does not need to send if captcha is needed or not.
// ReSharper disable once MergeSequentialChecks
if (loginJson != null && loginJson.captcha_gid != null)
{
capGid = Uri.EscapeDataString(loginJson.captcha_gid);
}
data = new NameValueCollection { { "password", encryptedBase64Password }, { "username", username } };
// Captcha Check.
string capText = "";
if (captcha)
{
Console.WriteLine("SteamWeb: Captcha is needed.");
System.Diagnostics.Process.Start("https://steamcommunity.com/public/captcha.php?gid=" + loginJson.captcha_gid);
Console.WriteLine("SteamWeb: Type the captcha:");
string consoleText = Console.ReadLine();
if (!string.IsNullOrEmpty(consoleText))
{
capText = Uri.EscapeDataString(consoleText);
}
}
data.Add("captchagid", captcha ? capGid : "-1");
data.Add("captcha_text", captcha ? capText : "");
// Captcha end.
// Added Header for two factor code.
data.Add("twofactorcode", "");
// Added Header for remember login. It can also set to true.
data.Add("remember_login", "false");
// SteamGuard check. If SteamGuard is enabled you need to enter it. Care probably you need to wait 7 days to trade.
// For further information about SteamGuard see: https://support.steampowered.com/kb_article.php?ref=4020-ALZM-5519&l=english.
if (steamGuard)
{
Console.WriteLine("SteamWeb: SteamGuard is needed.");
Console.WriteLine("SteamWeb: Type the code:");
string consoleText = Console.ReadLine();
if (!string.IsNullOrEmpty(consoleText))
{
steamGuardText = Uri.EscapeDataString(consoleText);
}
steamGuardId = loginJson.emailsteamid;
// Adding the machine name to the NameValueCollection, because it is requested by steam.
Console.WriteLine("SteamWeb: Type your machine name:");
consoleText = Console.ReadLine();
var machineName = string.IsNullOrEmpty(consoleText) ? "" : Uri.EscapeDataString(consoleText);
data.Add("loginfriendlyname", machineName != "" ? machineName : "defaultSteamBotMachine");
}
data.Add("emailauth", steamGuardText);
data.Add("emailsteamid", steamGuardId);
// SteamGuard end.
// Added unixTimestamp. It is included in the request normally.
var unixTimestamp = (int)(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1))).TotalSeconds;
// Added three "0"'s because Steam has a weird unix timestamp interpretation.
data.Add("donotcache", unixTimestamp + "000");
data.Add("rsatimestamp", time);
// Sending the actual login.
using (HttpWebResponse webResponse = Request("https://steamcommunity.com/login/dologin/", "POST", data, false))
{
var stream = webResponse.GetResponseStream();
if (stream == null)
{
return false;
}
using (StreamReader reader = new StreamReader(stream))
{
string json = reader.ReadToEnd();
loginJson = JsonConvert.DeserializeObject<SteamResult>(json);
cookieCollection = webResponse.Cookies;
}
}
} while (loginJson.captcha_needed || loginJson.emailauth_needed);
// If the login was successful, we need to enter the cookies to steam.
if (loginJson.success)
{
_cookies = new CookieContainer();
foreach (Cookie cookie in cookieCollection)
{
_cookies.Add(cookie);
}
SubmitCookies(_cookies);
return true;
}
else
{
Console.WriteLine("SteamWeb Error: " + loginJson.message);
return false;
}
}
enter image description here
Is there another solution how to login to steam and get html?
Ok, so I checked your encrypting and it looks fine. It is a little bit of chaos in your code so i will explain it with my:
I prefer to use RestSharp and Newton Soft JSON to do this, so a little mandatory stuff in Class body:
private IRestClient restClientTemporary;
private string getKeysURL = "/login/getrsakey/";
private string loginWithKey = "/login/dologin/";
Create RestClient, and RestRequests:
restClientTemporary = new RestClient("https://steamcommunity.com");
var request = new RestRequest(getKeysURL, Method.POST);
request.AddParameter("username", "YourSteamLogin");
var resp = restClientTemporary.Execute(request);
GetRsaResult response = Newtonsoft.Json.JsonConvert.DeserializeObject<GetRsaResult>(resp.Content);
Then i used your code as method to encrypt:
public static string EncryptionSof(string password, GetRsaResult response)
{
RSACryptoServiceProvider rsa = new RSACryptoServiceProvider();
RSAParameters rsaParameters = new RSAParameters
{
Exponent = HexToByte(response.publickey_exp),
Modulus = HexToByte(response.publickey_mod)
};
rsa.ImportParameters(rsaParameters);
// Encrypt the password and convert it.
byte[] bytePassword = Encoding.ASCII.GetBytes(password);
byte[] encodedPassword = rsa.Encrypt(bytePassword, false);
return Convert.ToBase64String(encodedPassword);
}
And then used this method:
string password = EncryptionSof("admin123/*its your steam password i think*/", response);
Next step is make request to get login data:
var loginRequest = new RestRequest(loginWithKey);
loginRequest.AddParameter("username", "YourSteamLogin");
loginRequest.AddParameter("password", password);
loginRequest.AddParameter("rsatimestamp", response.timestamp);
loginRequest.AddParameter("remember_login", false);
//Captcha stuff if needed:
loginRequest.AddParameter("captchagid", 3086601225255895896);
loginRequest.AddParameter("captcha_text", "LHYJ2P");
And finally execute it:
var responseFinal = restClientTemporary.Execute(loginRequest);
In response i received everything i need in responseFinal.Content:
{
"success":true,
"requires_twofactor":false,
"login_complete":true,
"transfer_urls":[
"https:\\/\\/store.steampowered.com\\/login\\/transfer",
"https:\\/\\/help.steampowered.com\\/login\\/transfer"
],
"transfer_parameters":{
"steamid":"12344567898765432",
"token_secure":"xDDDDDDDDD",
"auth":"LOOOOOL",
"remember_login":false
}
}
GetRsaResult model looks like:
public class GetRsaResult
{
public bool success { get; set; }
public string publickey_mod { get; set; }
public string publickey_exp { get; set; }
public string timestamp { get; set; }
public string token_gid { get; set; }
}
Ah, and i forget about changing 16-string to byte[], its method found on Stackoverflow:
public static byte[] HexToByte(string hex)
{
return Enumerable.Range(0, hex.Length)
.Where(x => x % 2 == 0)
.Select(x => Convert.ToByte(hex.Substring(x, 2), 16))
.ToArray();
}
And you have to remember one thing You never want to send plaintext password to steam. First request /login/getrsakey/ is just only for get keys to encrypt password. Steam gives you key to encrypt your password. So, you will use your plaintext password (in my sample its admin123 ) in your program to encrypt it with keys you received from Steam.
In second request /login/dologin/ you must send encrypted password (result of EncryptionSoF method)
When my AWS Credentials File (see docs) is updated by an external process the AmazonSQSClient doesn't re-read it, SendMessageAsync fails with a security/token error.
We use a custom powershell script to refresh the local AWS cred's file periodically. The script works fine, the file is refreshed prior to the credentials expiring on AWS. However, if my app is running when the file is refreshed the new credentials are not re-read from the file, the "client" will show that the previous credentials are still in use.
The AWS docs list several AWSCredential providers but none of them seem to be the correct choice...I think..
Restarting the app works, the new credentials are read correctly and messages are sent until the next time the cred's file is updated.
using (var client = new AmazonSQSClient(Amazon.RegionEndpoint.EUWest1))
{
return client.SendMessageAsync(request);
}
I don't think there is a way for a running app to pick up the default credentials being refreshed in credentials file. There is a solution for Node.js loading credentials from a JSON file. You can create a similar solution in C#. You can also run a local DB to store credentials so whenever credentials file is updated DB table or JSON file is also updated. You will need to use access key and secret key in your SQS client constructor as opposed to using default credentials.
// Load these from JSON file or DB.
var accessKey = "";
var secretKey = "";
using (var client = new AmazonSQSClient(accessKey, secretKey, Amazon.RegionEndpoint.EUWest1))
{
return client.SendMessageAsync(request);
}
The following works "ok" but I've only tested it with one profile and the file watcher is not as timely as you'd like so I'd recommend you wrap your usage inside a Retry mechanism.
// Usage..
var credentials = new AwsCredentialsFile();
using (var client = new AmazonSQSClient(credentials, Amazon.RegionEndpoint.EUWest1))
{
return client.SendMessageAsync(request);
}
public class AwsCredentialsFile : AWSCredentials
{
// https://docs.aws.amazon.com/sdk-for-net/v2/developer-guide/net-dg-config-creds.html#creds-file
private const string DefaultProfileName = "default";
private static ConcurrentDictionary<string, ImmutableCredentials> _credentials = new ConcurrentDictionary<string, ImmutableCredentials>(StringComparer.OrdinalIgnoreCase);
private static FileSystemWatcher _watcher = BuildFileSystemWatcher();
private readonly System.Text.Encoding _encoding;
private readonly string _profileName;
public AwsCredentialsFile()
: this(AwsCredentialsFile.DefaultProfileName, System.Text.Encoding.UTF8)
{
}
public AwsCredentialsFile(string profileName)
: this(profileName, System.Text.Encoding.UTF8)
{
}
public AwsCredentialsFile(string profileName, System.Text.Encoding encoding)
{
_profileName = profileName;
_encoding = encoding;
}
private static FileSystemWatcher BuildFileSystemWatcher()
{
var watcher = new FileSystemWatcher
{
Path = Path.GetDirectoryName(GetDefaultCredentialsFilePath()),
NotifyFilter = NotifyFilters.LastWrite,
Filter = "credentials"
};
watcher.Changed += (object source, FileSystemEventArgs e) => { _credentials?.Clear(); };
watcher.EnableRaisingEvents = true;
return watcher;
}
public static string GetDefaultCredentialsFilePath()
{
return System.Environment.ExpandEnvironmentVariables(#"C:\Users\%USERNAME%\.aws\credentials");
}
public static (string AccessKey, string SecretAccessKey, string Token) ReadCredentialsFromFile(string profileName, System.Text.Encoding encoding)
{
var profile = $"[{profileName}]";
string awsAccessKeyId = null;
string awsSecretAccessKey = null;
string token = null;
var lines = File.ReadAllLines(GetDefaultCredentialsFilePath(), encoding);
for (int i = 0; i < lines.Length; i++)
{
var text = lines[i];
if (text.Equals(profile, StringComparison.OrdinalIgnoreCase))
{
awsAccessKeyId = lines[i + 1].Replace("aws_access_key_id = ", string.Empty);
awsSecretAccessKey = lines[i + 2].Replace("aws_secret_access_key = ", string.Empty);
if (lines.Length >= i + 3)
{
token = lines[i + 3].Replace("aws_session_token = ", string.Empty);
}
break;
}
}
var result = (AccessKey: awsAccessKeyId, SecretAccessKey: awsSecretAccessKey, Token: token);
return result;
}
public override ImmutableCredentials GetCredentials()
{
if (_credentials.TryGetValue(_profileName, out ImmutableCredentials value))
{
return value;
}
else
{
var (AccessKey, SecretAccessKey, Token) = ReadCredentialsFromFile(_profileName, _encoding);
var credentials = new ImmutableCredentials(AccessKey, SecretAccessKey, Token);
_credentials.TryAdd(_profileName, credentials);
return credentials;
}
}
}
I'm trying to access Azure Cosmos DB using Table API.
The challenge is, despite creating SharedKeyLite, server is still returning Unauthorized - seems like SharedKeyLite is not supported or I'm generating the signature or headers wrong.
Here is the code
static readonly string storageAccountName = "accountName";
static readonly string storageAccountKey = "xxxx";
static readonly string uri = "https://accountName.table.cosmosdb.azure.com/Contacts()";
static readonly string utc_date = DateTime.UtcNow.ToString("r");
static void Main(string[] args)
{
Console.WriteLine(GetResult().Result);
}
static async Task<string> GetResult()
{
// Set this to whatever payload you desire. Ours is null because
// we're not passing anything in.
Byte[] requestPayload = null;
var requestDateString = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
var requestUri = new Uri(uri);
DateTime now = DateTime.UtcNow;
//Instantiate the request message with a null payload.
using (var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, uri)
{ Content = (requestPayload == null) ? null : new ByteArrayContent(requestPayload) })
{
ConstructHeaders(httpRequestMessage.Headers, requestDateString);
string authorizationHeader = GenerateSharedKeyLite(storageAccountKey, storageAccountName, uri,requestDateString);
httpRequestMessage.Headers.Authorization = new AuthenticationHeaderValue("SharedKeyLite", authorizationHeader);
// Send the request.
using (HttpResponseMessage httpResponseMessage = await new HttpClient().SendAsync(httpRequestMessage))
{
string json = await httpResponseMessage.Content.ReadAsStringAsync();
return json;
}
}
}
These are the headers I"m adding, expansion of ConstructHeaders method.
Refer this link for request parameters
//Construct the headers
static void ConstructHeaders(HttpRequestHeaders headers, string now)
{
headers.Add("x-ms-date", now);
headers.Add("x-ms-version", "2017-04-17");
// If you need any additional headers, add them here before creating
// the authorization header.
headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
if (headers.Contains("DataServiceVersion"))
headers.Remove("DataServiceVersion");
headers.Add("DataServiceVersion", "3.0;NetFx");
if (headers.Contains("MaxDataServiceVersion"))
headers.Remove("MaxDataServiceVersion");
headers.Add("MaxDataServiceVersion", "3.0;NetFx");
}
And this is the method that creates the SharedKeyLite
//Created Shared Key Lite
static string GenerateSharedKeyLite(string accessKey, string account, string url, string date)
{
var uri = new Uri(url);
var canonicalizedResourceString = uri.PathAndQuery;
var queryStart = canonicalizedResourceString.IndexOf('?');
if (queryStart > -1)
{
if (queryStart < canonicalizedResourceString.Length - 1)
{
var path = canonicalizedResourceString.Substring(0, queryStart);
var parameters = HttpUtility.ParseQueryString(canonicalizedResourceString.Substring(queryStart + 1));
var sb = new StringBuilder();
foreach (var keyOri in parameters.Keys)
{
var value = parameters[keyOri];
var key = keyOri.ToLowerInvariant();
sb.Append("\n");
sb.Append(key);
sb.Append(":");
sb.Append(value);
}
canonicalizedResourceString = canonicalizedResourceString + sb.ToString();
}
else
{
canonicalizedResourceString = canonicalizedResourceString.Substring(0, canonicalizedResourceString.Length - 1);
}
}
canonicalizedResourceString = $"/{account}{canonicalizedResourceString}";
var stringToSign = $"{date}\n{canonicalizedResourceString}";
var signedSignature = string.Empty;
using (var hmac = new HMACSHA256(Convert.FromBase64String(accessKey)))
{
var outputBytes = hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign));
var signature = Convert.ToBase64String(outputBytes);
return $"{account}:{signature}";
}
}
Any Help? Ideally I want to perform the odata query using simple.odata, but first trying to make this work using HttpClient
Just copy your code and it works on my side. If you haven't modified your code, please make sure your storageAccountName and storageAccountKey are correct.
BTW, in method GenerateSharedKeyLite there's no need to add query parameters to canonicalizedResourceString for entity operation. You only need to add comp if you want to operate component info for table or service. See constructing-the-canonicalized-resource-string.
The query string should include the question mark and the comp parameter (for example, ?comp=metadata). No other parameters should be included on the query string.
How do I get Content-Disposition parameters I returned from WebAPI controller using WebClient?
WebApi Controller
[Route("api/mycontroller/GetFile/{fileId}")]
public HttpResponseMessage GetFile(int fileId)
{
try
{
var file = GetSomeFile(fileId)
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(new MemoryStream(file));
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = file.FileOriginalName;
/********* Parameter *************/
response.Content.Headers.ContentDisposition.Parameters.Add(new NameValueHeaderValue("MyParameter", "MyValue"));
return response;
}
catch(Exception ex)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex);
}
}
Client
void DownloadFile()
{
WebClient wc = new WebClient();
wc.DownloadDataCompleted += wc_DownloadDataCompleted;
wc.DownloadDataAsync(new Uri("api/mycontroller/GetFile/18"));
}
void wc_DownloadDataCompleted(object sender, DownloadDataCompletedEventArgs e)
{
WebClient wc=sender as WebClient;
// Try to extract the filename from the Content-Disposition header
if (!String.IsNullOrEmpty(wc.ResponseHeaders["Content-Disposition"]))
{
string fileName = wc.ResponseHeaders["Content-Disposition"].Substring(wc.ResponseHeaders["Content-Disposition"].IndexOf("filename=") + 10).Replace("\"", ""); //FileName ok
/****** How do I get "MyParameter"? **********/
}
var data = e.Result; //File OK
}
I'm returning a file from WebApi controller, I'm attaching the file name in the response content headers, but also I'd like to return an aditional value.
In the client I'm able to get the filename, but how do I get the aditional parameter?
If you are working with .NET 4.5 or later, consider using the System.Net.Mime.ContentDisposition class:
string cpString = wc.ResponseHeaders["Content-Disposition"];
ContentDisposition contentDisposition = new ContentDisposition(cpString);
string filename = contentDisposition.FileName;
StringDictionary parameters = contentDisposition.Parameters;
// You have got parameters now
Edit:
otherwise, you need to parse Content-Disposition header according to it's specification.
Here is a simple class that performs the parsing, close to the specification:
class ContentDisposition {
private static readonly Regex regex = new Regex(
"^([^;]+);(?:\\s*([^=]+)=((?<q>\"?)[^\"]*\\k<q>);?)*$",
RegexOptions.Compiled
);
private readonly string fileName;
private readonly StringDictionary parameters;
private readonly string type;
public ContentDisposition(string s) {
if (string.IsNullOrEmpty(s)) {
throw new ArgumentNullException("s");
}
Match match = regex.Match(s);
if (!match.Success) {
throw new FormatException("input is not a valid content-disposition string.");
}
var typeGroup = match.Groups[1];
var nameGroup = match.Groups[2];
var valueGroup = match.Groups[3];
int groupCount = match.Groups.Count;
int paramCount = nameGroup.Captures.Count;
this.type = typeGroup.Value;
this.parameters = new StringDictionary();
for (int i = 0; i < paramCount; i++ ) {
string name = nameGroup.Captures[i].Value;
string value = valueGroup.Captures[i].Value;
if (name.Equals("filename", StringComparison.InvariantCultureIgnoreCase)) {
this.fileName = value;
}
else {
this.parameters.Add(name, value);
}
}
}
public string FileName {
get {
return this.fileName;
}
}
public StringDictionary Parameters {
get {
return this.parameters;
}
}
public string Type {
get {
return this.type;
}
}
}
Then you can use it in this way:
static void Main() {
string text = "attachment; filename=\"fname.ext\"; param1=\"A\"; param2=\"A\";";
var cp = new ContentDisposition(text);
Console.WriteLine("FileName:" + cp.FileName);
foreach (DictionaryEntry param in cp.Parameters) {
Console.WriteLine("{0} = {1}", param.Key, param.Value);
}
}
// Output:
// FileName:"fname.ext"
// param1 = "A"
// param2 = "A"
The only thing that should be considered when using this class is it does not handle parameters (or filename) without a double quotation.
Edit 2:
It can now handle file names without quotations.
You can parse out the content disposition using the following framework code:
var content = "attachment; filename=myfile.csv";
var disposition = ContentDispositionHeaderValue.Parse(content);
Then just take the pieces off of the disposition instance.
disposition.FileName
disposition.DispositionType
With .NET Core 3.1 and more the most simple solution is:
using var response = await Client.SendAsync(request);
response.Content.Headers.ContentDisposition.FileName
The value is there I just needed to extract it:
The Content-Disposition header is returned like this:
Content-Disposition = attachment; filename="C:\team.jpg"; MyParameter=MyValue
So I just used some string manipulation to get the values:
void wc_DownloadDataCompleted(object sender, DownloadDataCompletedEventArgs e)
{
WebClient wc=sender as WebClient;
// Try to extract the filename from the Content-Disposition header
if (!String.IsNullOrEmpty(wc.ResponseHeaders["Content-Disposition"]))
{
string[] values = wc.ResponseHeaders["Content-Disposition"].Split(';');
string fileName = values.Single(v => v.Contains("filename"))
.Replace("filename=","")
.Replace("\"","");
/********** HERE IS THE PARAMETER ********/
string myParameter = values.Single(v => v.Contains("MyParameter"))
.Replace("MyParameter=", "")
.Replace("\"", "");
}
var data = e.Result; //File ok
}
As #Mehrzad Chehraz said you can use the new ContentDisposition class.
using System.Net.Mime;
// file1 is a HttpResponseMessage
FileName = new ContentDisposition(file1.Content.Headers.ContentDisposition.ToString()).FileName
I need a good example on WCF Streaming File Transfer.
I have found several and tried them but the posts are old and I am wokding on .net 4 and IIS 7 so there are some problems.
Can you gives me a good and up-to-date example on that.
The following answers detail using a few techniques for a posting binary data to a restful service.
Post binary data to a RESTful application
What is a good way to transfer binary data to a HTTP REST API service?
Bad idea to transfer large payload using web services?
The following code is a sample of how you could write a RESTful WCF service and is by no means complete but does give you an indication on where you could start.
Sample Service, note that this is NOT production ready code.
[ServiceContract]
[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
public class FileService
{
private IncomingWebRequestContext m_Request;
private OutgoingWebResponseContext m_Response;
[WebGet(UriTemplate = "{appName}/{id}?action={action}")]
public Stream GetFile(string appName, string id, string action)
{
var repository = new FileRepository();
var response = WebOperationContext.Current.OutgoingResponse;
var result = repository.GetById(int.Parse(id));
if (action != null && action.Equals("download", StringComparison.InvariantCultureIgnoreCase))
{
response.Headers.Add("Content-Disposition", string.Format("attachment; filename={0}", result.Name));
}
response.Headers.Add(HttpResponseHeader.ContentType, result.ContentType);
response.Headers.Add("X-Filename", result.Name);
return result.Content;
}
[WebInvoke(UriTemplate = "{appName}", Method = "POST")]
public void Save(string appName, Stream fileContent)
{
try
{
if (WebOperationContext.Current == null) throw new InvalidOperationException("WebOperationContext is null.");
m_Request = WebOperationContext.Current.IncomingRequest;
m_Response = WebOperationContext.Current.OutgoingResponse;
var file = CreateFileResource(fileContent, appName);
if (!FileIsValid(file)) throw new WebFaultException(HttpStatusCode.BadRequest);
SaveFile(file);
SetStatusAsCreated(file);
}
catch (Exception ex)
{
if (ex.GetType() == typeof(WebFaultException)) throw;
if (ex.GetType().IsGenericType && ex.GetType().GetGenericTypeDefinition() == typeof(WebFaultException<>)) throw;
throw new WebFaultException<string>("An unexpected error occurred.", HttpStatusCode.InternalServerError);
}
}
private FileResource CreateFileResource(Stream fileContent, string appName)
{
var result = new FileResource();
fileContent.CopyTo(result.Content);
result.ApplicationName = appName;
result.Name = m_Request.Headers["X-Filename"];
result.Location = #"C:\SomeFolder\" + result.Name;
result.ContentType = m_Request.Headers[HttpRequestHeader.ContentType] ?? this.GetContentType(result.Name);
result.DateUploaded = DateTime.Now;
return result;
}
private string GetContentType(string filename)
{
// this should be replaced with some form of logic to determine the correct file content type (I.E., use registry, extension, xml file, etc.,)
return "application/octet-stream";
}
private bool FileIsValid(FileResource file)
{
var validator = new FileResourceValidator();
var clientHash = m_Request.Headers[HttpRequestHeader.ContentMd5];
return validator.IsValid(file, clientHash);
}
private void SaveFile(FileResource file)
{
// This will persist the meta data about the file to a database (I.E., size, filename, file location, etc)
new FileRepository().AddFile(file);
}
private void SetStatusAsCreated(FileResource file)
{
var location = new Uri(m_Request.UriTemplateMatch.RequestUri.AbsoluteUri + "/" + file.Id);
m_Response.SetStatusAsCreated(location);
}
}
Sample Client, note that this is NOT production ready code.
// *********************************
// Sample Client
// *********************************
private void UploadButton_Click(object sender, EventArgs e)
{
var uri = "http://dev-fileservice/SampleApplication"
var fullFilename = #"C:\somefile.txt";
var fileContent = File.ReadAllBytes(fullFilename);
using (var webClient = new WebClient())
{
try
{
webClient.Proxy = null;
webClient.Headers.Add(HttpRequestHeader.ContentMd5, this.CalculateFileHash());
webClient.Headers.Add("X-DaysToKeep", DurationNumericUpDown.Value.ToString());
webClient.Headers.Add("X-Filename", Path.GetFileName(fullFilename));
webClient.UploadData(uri, "POST", fileContent);
var fileUri = webClient.ResponseHeaders[HttpResponseHeader.Location];
Console.WriteLine("File can be downloaded at" + fileUri);
}
catch (Exception ex)
{
var exception = ex.Message;
}
}
}
private string CalculateFileHash()
{
var hash = MD5.Create().ComputeHash(File.ReadAllBytes(#"C:\somefile.txt"));
var sb = new StringBuilder();
for (int i = 0; i < hash.Length; i++)
{
sb.Append(hash[i].ToString("x2"));
}
return sb.ToString();
}
private void DownloadFile()
{
var uri = "http://dev-fileservice/SampleApplication/1" // this is the URL returned by the Restful file service
using (var webClient = new WebClient())
{
try
{
webClient.Proxy = null;
var fileContent = webClient.DownloadData(uri);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
}