We Keep Coding

HttpClient not returning json value of URI

I am trying to use HttpClient to GET information from Jira, but I am unable to see any of the information. I want to be able to get all the bugs that match certain filters so that I can add them to a table in my program.
I have tried to access Jira with the rest api, but every time I do it says that the issue or project doesn't exist. The thing is that if I enter the URI into the bar at the top of my browser I can see the JSON text that I want. This leads me to believe that the reason my code is not returning these values is because of an authorization issue. I am using basic auth to send my credentials. I also want to add that I used cURL in cmd to test my credentials with basic auth and it worked.
public async Task<JiraModel> GetBugs()
{
using (var client = new HttpClient())
{
string url = "https://myurl.atlassian.net/rest/api/3/project/VCMF";
String username = "username";
String password = "apikey";
String encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("UTF-8").GetBytes(username + ":" + password));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Authorization", "Basic " + encoded);
client.BaseAddress = new Uri("https://myurl.atlassian.net/rest/api/3/project/VCMF");
var response = await client.GetAsync(url);
var content = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<JiraModel>(content);
}
}
I should be getting the json results in string form by the end of this length of code, but I keep getting a 404 error instead that for this code specifically says "No project could be found with key 'VCMF'".

The issue here is that you're creating the authorization header incorrectly.
The constructor you're using for AuthenticationHeaderValue class takes two arguments: scheme and parameter:
public AuthenticationHeaderValue(string scheme, string parameter)
{
}
The first argument should be the scheme (Basic in this case) and the second, the base64-encoded credentials:
So instead of:
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Authorization", "Basic " + encoded);
It should be:
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", encoded);
Hope this helps!

Twilio download recording to file throwing error on JsonConvert.DeserializeObject()

I'm trying to download my recordings on Twilio to a file on my servers local file system (so I can send them to another storage location), but following the code that I've found is throwing an error on the JsonConvert.DeserializeObject() call.
The code that I found is here (called "Retrieve the actual recording media"): https://www.twilio.com/docs/video/api/recordings-resource#filter-by-participant-sid
Here's the code:
static void Main(string[] args)
{
// Find your Account SID and Auth Token at twilio.com/console
const string apiKeySid = "SKXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
const string apiKeySecret = "your_api_key_secret";
const string recordingSid = "RTXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
const string uri = $"https://video.twilio.com/v1/Recordings/{recordingSid}/Media";
var request = (HttpWebRequest)WebRequest.Create(uri);
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes(apiKeySid + ":" + apiKeySecret)));
request.AllowAutoRedirect = false;
string responseBody = new StreamReader(request.GetResponse().GetResponseStream()).ReadToEnd();
var mediaLocation = JsonConvert.DeserializeObject<Dictionary<string, string>>(responseBody)["redirect_to"];
Console.WriteLine(mediaLocation);
new WebClient().DownloadFile(mediaLocation, $"{recordingSid}.out");
}
And here's my version:
var twilioRecordingUri = $"https://api.twilio.com/2010-04-01/Accounts/{recording.AccountSid}/Recordings/{recording.Sid}.mp3?Download=true";
var request = (HttpWebRequest)WebRequest.Create(new Uri(twilioRecordingUri));
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes($"{apiKeySid}:{apiKeySecret}")));
request.ContentType = "audio/mpeg";
//request.Accept = "audio/mpeg";
request.AllowAutoRedirect = false;
var responseBody = new StreamReader(request.GetResponse().GetResponseStream()).ReadToEnd();
var deserialized = JsonConvert.DeserializeObject<Dictionary<string, string>>(responseBody);
var mediaLocation = deserialized["redirect_to"];
new WebClient().DownloadFile(mediaLocation, $"{recording.Sid}.out");
But executing that code, it fails on the JsonConvert.Deserialize(), like I mentioned; it throws this generic Json error:
Unexpected character encountered while parsing value: �. Path '', line
0, position 0.
Hovering over my "responseBody" variable does show that it's a really long string of funky characters.
My thought was that I should be adding either the "Accept" or "Content-type" to "audio/mpeg" since that's the type of file that I'm trying to retrieve. But when checking Dev Tools at both the request and response headers, neither the Accept or Content-type ever get my audio/mpeg setting that I just specified.
What's wrong with this code here?
Edit: for anyone that noticed the download URL is different from Twilio's example, I found this page that had the updated URL: How do I get a call recording url in twilio when programming in PHP?

I'm only posting this answer to show what the "working" version looks like. It was #Grungondola 's answer that prompted me. Thanks goes to him (as well as the accepted answer).
private async Task DownloadRecording(RecordingResource recording, string fileName)
{
if (string.IsNullOrEmpty(fileName))
throw new ArgumentNullException("fileName is required when downloading a recording.");
var twilioRecordingUri = $"https://api.twilio.com/2010-04-01/Accounts/{recording.AccountSid}/Recordings/{recording.Sid}.mp3?Download=false";
var request = (HttpWebRequest)WebRequest.Create(new Uri(twilioRecordingUri));
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes($"{apiKeySid}:{apiKeySecret}")));
request.ContentType = "audio/mpeg";
request.AllowAutoRedirect = false;
var stream = request.GetResponse().GetResponseStream();
var virtualPath = HttpContext.Server.MapPath(fileName);
var fileStream = new FileStream(virtualPath, FileMode.Create);
await stream.CopyToAsync(fileStream);
fileStream.Close();
}

It looks like the call you're trying to make is to download an .mp3 file and not a Dictionary<string, string>, so it's likely hitting an error when attempting to deserialize a string into a type that it doesn't match. What you're probably seeing as a result is a Base64 string, especially based on your description. Without seeing at least a sample of the data, I can't know for sure, but I'd guess that you're downloading the raw .mp3 file instead of the file information with location (redirect_to).
If the result is a pure Base64 string, you should be able to convert it to an array of bytes and write that directly to a file with whatever filename you want. That should get you the mp3 file that you want.

Console application, first webrequest doesn't get response, later does, why?

I am developing console application for API integration. This follow fetching token in first request and then getting report in second request after passing token.
string token = GetToken(app_id); // API call, which is working fine and getting token
string reportquerystring = "path?token=" + token;
WebRequest req = WebRequest.Create(#reportquerystring);
req.Method = "GET";
req.Headers["Authorization"] = "Basic " +
Convert.ToBase64String(Encoding.Default.GetBytes("username:password"));
var resp = req.GetResponse() as HttpWebResponse;
using (Stream downloadstream = resp.GetResponseStream())
{
XmlDocument reportxml = new XmlDocument();
string filename = "location\\";
string reportxmlString = (new StreamReader(downloadstream)).ReadToEnd();
reportxml.LoadXml(reportxmlString);
string json = JsonConvert.SerializeXmlNode(reportxml);
System.IO.File.WriteAllText(filename + "data_" + app_id + ".txt", json);
}
Here when I run this code while debugging, on the first call of download report, response xml is empty, when I drag debugger again before call, in the same run, then it gets response properly. But until and unless I figure out the reason why first call to download report API is not working or how I can make it work, I can not proceed.
Any suggestions ?

Making POST API request via c# with hashed API Key

I'm trying to make a POST call (from a C# WPF app) to a web service on an internal intranet (so I can't give the exact URL sorry), which is basically a url shortening service.
The page gives the following instructions:
In order to use the API service, simply make HTTP POST requests to this URL:
https://...internalAddress.../api/<method>
For example, to create a snip, make an HTTP POST request to:
https://...internalAddress.../api/shorten
with these parameters:
api_key hash of a registered API key
url URL to be shortened
Now I have tried to implement this in a couple of different ways with what I've found via google / here, these are:
1:
string apiKey = "xxxx11112222333";
string urlForShortening = #"http://www.codeproject.com/Tips/497123/How-to-make-REST-requests-with-Csharp";
string destination = #"https://internalurl/api/shorten";
var httpWebRequest = (HttpWebRequest)WebRequest.Create(destination);
httpWebRequest.ContentType = "text/json";
httpWebRequest.Method = "POST";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write("{'api_key': '" + apiKey + "', 'url': '" + urlForShortening + "'}");
}
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var responseText = streamReader.ReadToEnd();
MessageBox.Show(responseText);
}
2: (Using the rest library created in the article found in the shortening link)
string apiKey = "xxxx11112222333";
string urlForShortening = #"http://www.codeproject.com/Tips/497123/How-to-make-REST-requests-with-Csharp";
string destination = #"https://internalurl/api/shorten";
RestClient client = new RestClient(destination, HttpVerb.POST,
"{'api_key': '" + apiKey + "', 'url': '" + urlForShortening + "'}");
var json = client.MakeRequest();
MessageBox.Show(json);
have also tried feeding in the jsonData in double quotes:
var jsonData = "{\"api_key\": \"" + apiKey + "\", \"url\": \"" + urlForShortening + "\"}";
The result from both methods I always get is:
{"status": 400, "message": "Missing API key"}
Can someone please shed some light on what I'm doing wrong?
From the brief, I think the key may need to be hashed in some form and not sure how to do this.

Turns out my whole implementation was wrong, I was trying to send the data as JSON / using the wrong classes instead of a vanilla HTTP POST.
I used Method 2 found in this article and it worked fine: HTTP request with post
ie.
using (var client = new WebClient())
{
var values = new NameValueCollection();
values["api_key"] = "xxxx11112222333";
values["url"] = #"http://www.codeproject.com";
string destination = #"https://internalurl/api/shorten";
var response = client.UploadValues(destination, values);
var responseString = Encoding.Default.GetString(response);
MessageBox.Show(responseString);
}
Thanks for your help though!

I assume that you are attempting to send json data in the request body for the POST call to the API.
You dont seem to be providing a valid json here though. This is what you are sending now:
{'api_key':'someApiKey'}
{'url':'someUrlForShortening'}
Use a json validator to ensure you have a valid json document before you attempt to send it to the API.
A valid json would be
{
"api_key":"someApiKey",
"url":"someUrlForShortening"
}

Why my Http client making 2 requests when I specify credentials?

I created RESTful webservice (WCF) where I check credentials on each request. One of my clients is Android app and everything seems to be great on server side. I get request and if it's got proper header - I process it, etc..
Now I created client app that uses this service. This is how I do GET:
// Create the web request
var request = WebRequest.Create(Context.ServiceURL + uri) as HttpWebRequest;
if (request != null)
{
request.ContentType = "application/json";
// Add authentication to request
request.Credentials = new NetworkCredential(Context.UserName, Context.Password);
// Get response
using (var response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
if (response != null)
{
var reader = new StreamReader(response.GetResponseStream());
// Console application output
var s = reader.ReadToEnd();
var serializer = new JavaScriptSerializer();
var returnValue = (T)serializer.Deserialize(s, typeof(T));
return returnValue;
}
}
}
So, this code get's my resource and deserializes it. As you see - I'm passing credentials in my call.
Then when debugging on server-side I noticed that I get 2 requests every time - one without authentication header and then server sends back response and second request comes bach with credentials. I think it's bad for my server - I'd rather don't make any roundtrips. How should I change client so it doesn't happen? See screenshot of Fiddler
EDIT:
This is JAVA code I use from Android - it doesn't do double-call:
MyHttpResponse response = new MyHttpResponse();
HttpClient client = mMyApplication.getHttpClient();
try
{
HttpGet request = new HttpGet(serviceURL + url);
request.setHeader(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
request.addHeader("Authorization", "Basic " + Preferences.getAuthorizationTicket(mContext));
ResponseHandler<String> handler = new BasicResponseHandler();
response.Body = client.execute(request, handler);
response.Code = HttpURLConnection.HTTP_OK;
response.Message = "OK";
}
catch (HttpResponseException e)
{
response.Code = e.getStatusCode();
response.Message = e.getMessage();
LogData.InsertError(mContext, e);
}

The initial request doesn't ever specify the basic header for authentication. Additionally, since a realm is specified, you have to get that from the server. So you have to ask once: "hey, I need this stuff" and the server goes "who are you? the realm of answering is 'secure area'." (because realm means something here) Just because you added it here:
request.Credentials = new NetworkCredential(Context.UserName, Context.Password);
doesn't mean that it's going to be for sure attached everytime to the request.
Then you respond with the username/password (in this case you're doing BASIC so it's base64 encoded as name:password) and the server decodes it and says "ok, you're all clear, here's your data".
This is going to happen on a regular basis, and there's not a lot you can do about it. I would suggest that you also turn on HTTPS since the authentication is happening in plain text over the internet. (actually what you show seems to be over the intranet, but if you do go over the internet make it https).
Here's a link to Wikipedia that might help you further: http://en.wikipedia.org/wiki/Basic_access_authentication

Ok, I got it. I manually set HttpHeader instead of using request.Credentials
request.Headers.Add(HttpRequestHeader.Authorization, "Basic " + Convert.ToBase64String(Encoding.UTF8.GetBytes(Context.UserName + ":" + Context.Password)));
Now I see only single requests as expected..

As an option you can use PreAuthenticate property of HttpClientHandler. This would require a couple of lines more
var client = new HttpClient(new HttpClientHandler
{
Credentials = yourCredentials,
PreAuthenticate = true
});
With using this approach, only the first request is sent without credentials, but all the rest requests are OK.