We Keep Coding

WebRequest async asp.net-mvc

I use code to get data from Atlassian Jira and put the data from the response into a container, how can I make this method asynchronous?
I have tried various methods with webrequest, but they all point to an error in the code, perhaps I do not understand how it works at all. You can suggest documentation or specify in the code what I can do with asynchrony for such a request.
public IEnumerable<JiraDataModel> GetData(string dateFrom, string dateTo)
{
int allTicketsCount = 0;
int devTicketsCount = 0;
int slaTicketsCount = 0;
List<JiraRequestUrl> urlArray = new List<JiraRequestUrl>();
urlArray.AddRange(new List<JiraRequestUrl>
{
new JiraRequestUrl{type = "all", //*data*//},
new JiraRequestUrl(){type = "dev",//*data*//});
try
{
foreach (JiraRequestUrl u in urlArray)
{
WebRequest request = WebRequest.Create(_jiraUrl + u.url);
request.ContentType = "application/json; charset=utf-8";
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes(_credentials)));
request.Headers.Add("maxResults", "100");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
JiraGetDataModel.RootObject DeserializedResponse = (JiraGetDataModel.RootObject)JsonConvert.DeserializeObject(responseString, typeof(JiraGetDataModel.RootObject));
if (u.type.Equals("all"))
{
allTicketsCount = DeserializedResponse.total;
}
if (u.type.Equals("dev"))
{
devTicketsCount = DeserializedResponse.total;
}
}
}

sorry, I didn't quite understand that I didn't insert my attempts to use async, I did it, and it seems to work, can I improve something in my code? added 2 lines
public async Task<IEnumerable<JiraDataModel>> GetData(string dateFrom, string dateTo)
{
int allTicketsCount = 0;
int devTicketsCount = 0;
int slaTicketsCount = 0;
List<JiraRequestUrl> urlArray = new List<JiraRequestUrl>();
urlArray.AddRange(new List<JiraRequestUrl>
{
new JiraRequestUrl{type = "all", //*data*//},
new JiraRequestUrl(){type = "dev",//*data*//});
try
{
foreach (JiraRequestUrl u in urlArray)
{
WebRequest request = WebRequest.Create(_jiraUrl + u.url);
request.ContentType = "application/json; charset=utf-8";
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes(_credentials)));
request.Headers.Add("maxResults", "100");
HttpWebResponse response = (HttpWebResponse)(await request.GetResponseAsync()request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
JiraGetDataModel.RootObject DeserializedResponse = (JiraGetDataModel.RootObject)JsonConvert.DeserializeObject(responseString, typeof(JiraGetDataModel.RootObject));
if (u.type.Equals("all"))
{
allTicketsCount = DeserializedResponse.total;
}
if (u.type.Equals("dev"))
{
devTicketsCount = DeserializedResponse.total;
}
}
}

Jira Rest Api Login in C# [duplicate]

I've written below C# code to login to JIRA Rest API:
var url = new Uri("http://localhost:8090/rest/auth/latest/session?os_username=tempusername&os_password=temppwd");
var request = WebRequest.Create(url) as HttpWebRequest;
if (null == request)
{
return "";
}
request.Method = "POST";
request.ContentType = "application/json";
request.ContentLength = 200;
request.KeepAlive = false;
using (var response = request.GetResponse() as HttpWebResponse)
{
}
When I execute this, application just goes on running without returning any response. Please suggest if this is the right way of calling JIRA Login using REST API

For basic authentication you need to send in the username and password in a base64-encoding. Guidelines can be found in the API examples on atlassians developer page:
https://developer.atlassian.com/display/JIRADEV/JIRA+REST+API+Example+-+Basic+Authentication
, if you are doing it in C# you need to send the encoded data in the header in the following format:
"Authorization: Basic [ENCODED CREDENTIALS]"
Here is a simple example:
public enum JiraResource
{
project
}
protected string RunQuery(
JiraResource resource,
string argument = null,
string data = null,
string method = "GET")
{
string url = string.Format("{0}{1}/", m_BaseUrl, resource.ToString());
if (argument != null)
{
url = string.Format("{0}{1}/", url, argument);
}
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.ContentType = "application/json";
request.Method = method;
if (data != null)
{
using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(data);
}
}
string base64Credentials = GetEncodedCredentials();
request.Headers.Add("Authorization", "Basic " + base64Credentials);
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
string result = string.Empty;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
}
return result;
}
private string GetEncodedCredentials()
{
string mergedCredentials = string.Format("{0}:{1}", m_Username, m_Password);
byte[] byteCredentials = UTF8Encoding.UTF8.GetBytes(mergedCredentials);
return Convert.ToBase64String(byteCredentials);
}
(JiraResource is just an enum I use to decide which part of the API to use)
I hope this will help!

Here is a simpler solution which works as required:
var mergedCredentials = string.Format("{0}:{1}", username, password);
var byteCredentials = Encoding.UTF8.GetBytes(mergedCredentials);
var encodedCredentials = Convert.ToBase64String(byteCredentials);
using (WebClient webClient = new WebClient())
{
webClient.Headers.Set("Authorization", "Basic " + encodedCredentials);
return webClient.DownloadString(url);
}

If you don't want to encode your credentials in every request here is how to do it using cookies.
When requesting the cookie you don't need to add any authorization on the headers. This method will accept a JSON string with the user name and password and the URL. It will return the cookie values.
public async Task<JiraCookie> GetCookieAsync(string myJsonUserNamePassword, string JiraCookieEndpointUrl)
{
using (var client = new HttpClient())
{
var response = await client.PostAsync(
JiraCookieEndpointUrl,
new StringContent(myJsonUserNamePassword, Encoding.UTF8, "application/json"));
var json = response.Content.ReadAsStringAsync().Result;
var jiraCookie= JsonConvert.DeserializeObject<JiraCookie>(json);
return jArr;
}
}
public class JiraCookie
{
public Session session { get; set; }
}
public class Session
{
public string name { get; set; }
public string value { get; set; }
}
When I call it using url: http://[baseJiraUrl]/rest/auth/1/session it returns the following JSON response:
{
"session" : -{
"name" : JSESSIONID,
"value" : cookieValue
}
Keep in mind the URL above is valid in the version of JIRA I'm using and may vary depending on which version you're using. Read the JIRA API documentation for the correct URL for the version you are using. I'm using the following:
https://docs.atlassian.com/software/jira/docs/api/REST/7.6.1/#auth/1/session
Remember you'll have to store your cookie and use it on every subsequent request.
Check out this answer on how add cookies to your HttpClient request: How do I set a cookie on HttpClient's HttpRequestMessage.
Once you're done with the cookie (logging out) simply send a delete http request with the same URL as the post.

I tweaked the RunQuery code so that it will run today (Apr 2018). The encrypt/decrypt referenced below is from the following link (I converted it to an extension method and threw values into environment).
https://stackoverflow.com/questions/10168240/encrypting-decrypting-a-string-in-c-sharp
I successfully execute the code from LinqPad - thus the Dump() command after RunQuery
private string _baseUrl = "https://xxxxxx.atlassian.net";
private string _username = "YourLogin";
void Main()
{
RunQuery(JiraResource.project).JsonToXml().Dump();
}
public enum JiraResource { project }
private const string restApiVersion = "/rest/api/2/";
protected string RunQuery( JiraResource resource, string argument = null, string data = null, string method = "GET")
{
string url = $"{_baseUrl}{restApiVersion}{resource}";
if (argument != null) url = $"{url}{argument}/";
var request = WebRequest.Create(url) as HttpWebRequest;
request.ContentType = "application/json";
request.Method = method;
if (data != null)
{
using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(data);
}
}
string base64Credentials = GetEncodedCredentials();
request.Headers.Add("Authorization", "Basic " + base64Credentials);
var response = request.GetResponse() as HttpWebResponse;
string result = string.Empty;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
}
return result;
}
private string GetEncodedCredentials()
{
var encryptedPassword = Environment.GetEnvironmentVariable("PassEncrypted");
var encryptionSalt = Environment.GetEnvironmentVariable("PassSalt");
var password = encryptedPassword.Decrypt(encryptionSalt);
var mergedCredentials = $"{_username}:{password}";
var byteCredentials = UTF8Encoding.UTF8.GetBytes(mergedCredentials);
return Convert.ToBase64String(byteCredentials);
}
public static class MyExtensions
{
public static XElement JsonToXml(this string jsonData, bool isAddingHeader = true)
{
var data = isAddingHeader
? "{\"record\":" + jsonData + "}"
: jsonData;
data = data // Complains if xml element name starts numeric
.Replace("16x16", "n16x16")
.Replace("24x24", "n24x24")
.Replace("32x32", "n32x32")
.Replace("48x48", "n48x48");
var result = JsonConvert.DeserializeXmlNode(data, "data");
var xmlResult = XElement.Parse(result.OuterXml);
return xmlResult;
}
}

For posting multipart content in Rest I use Tiny.RestClient.
var client = new TinyRestClient(new HttpClient(), "http://localhost:8090");
var strResult = await client.PostRequest("rest/auth/latest/session).
WithBasicAuthentication("username", "password")
ExecuteAsStringAsync();

static void Main(string[] args)
{
using (WebClient wc = new WebClient())
{
wc.Headers.Add("Authorization", "Basic " + GetEncodedCredentials());
string tasks = wc.DownloadString("yourjiraurl/search?jql=task=bug");
var taskdetails = JsonConvert.DeserializeObject<TaskDetails>(tasks);
}
}
static string GetEncodedCredentials()
{
string mergedCredentials = string.Format("{0}:{1}", "UserName", "Password");
byte[] byteCredentials = UTF8Encoding.UTF8.GetBytes(mergedCredentials);
return Convert.ToBase64String(byteCredentials);
}

Why this method is called twice and at the first time the return doesn't work?

A flash application uses AMF message format to call the method SyncAddressBook (c#),and in this method ,it calls other two WebApis using HttpWebRequest (post and get ).
But only when calling WebApi (the method SyncAddressBook ), the method can return to flash well, when calling WebApi (the method SyncAddressBook ), the method SyncAddressBook can not return to the one which called it (i didnot write this method here) ,instead, it return itself. and for the second time, the method SyncAddressBook returned with error to flash .
here is the method (I omit something)
public AMFGreetoAddressBookResult SyncAddressBook(AMFGreetoLogin loginArgs, AMFGreetoAddressBook addressBook)
{
var result = new ........
this.PostReceiversGroups(addressBook, accessKey);
this.GetReceivers(accessKey, out receivers);
result.ResultCode = true;
return result;
}
private Boolean PostReceiversGroups(AMFGreetoAddressBook addressBook, string accessKey)
{
Uri theUri = new Uri(athenaSiteUrl + "/api/receivers");
var httpWebRequest = (HttpWebRequest)WebRequest.Create(theUri);
httpWebRequest.KeepAlive = false;
httpWebRequest.Accept = "application/json";
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
httpWebRequest.Headers.Add(AuthenticationServiceCodeHeaderKey, AddressServiceAccessCode);
httpWebRequest.Headers.Add(uthenticationServiceAccessKeyHeaderKey, AddressServiceAccessKey);
httpWebRequest.Headers.Add(AuthenticationUserAccessKey, accessKey);
httpWebRequest.Host = theUri.Host;
var serializer = new JavaScriptSerializer();
string json = serializer.Serialize(addressBook);
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
byte[] bytes = encoding.GetBytes(json);
httpWebRequest.ContentLength = bytes.Length;
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(json);
}
try
{
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
Account acc = JsonConvert.DeserializeObject<Account>(result);
}
}
catch (Exception e)
{
throw new Exception("住所管理サービスの更新処理でエラーが発生しました。[PostReceiversGroups] :" + e);
}
return true;
}
private Boolean GetReceivers(string accessKey, out AMFGreetoReceiver[] receivers)
{
Uri theUri = new Uri(athenaSiteUrl + "/api/receivers");
var httpWebRequest = (HttpWebRequest)WebRequest.Create(theUri);
httpWebRequest.KeepAlive = false;
httpWebRequest.Accept = "application/json";
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "GET";
httpWebRequest.Headers.Add(AuthenticationServiceCodeHeaderKey, AddressServiceAccessCode);
httpWebRequest.Headers.Add(uthenticationServiceAccessKeyHeaderKey, AddressServiceAccessKey);
httpWebRequest.Headers.Add(AuthenticationUserAccessKey, accessKey);
httpWebRequest.Host = theUri.Host;
receivers = null;
try
{
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
string result = streamReader.ReadToEnd();
var ser = new JavaScriptSerializer();
receivers = ser.Deserialize<AMFGreetoReceiver[]>(result);
streamReader.Close();
httpResponse.Close();
}
}
catch (Exception e)
{
throw new Exception("Receivers取得処理でエラーが発生しました。[GetReceivers] :" + e);
}
return true;
}

oauth/token returns empty body

I am encountering a problem getting the access_token in client application using oauth.
The returned response has empty body though in API I can see the response is not empty.
tokenresponse = {
"access_token":"[ACCESSTOKENVALUE]",
"token_type":"bearer",
"expires_in":"1200",
"refresh_token":"[REFRESHTOKENVALUE]",
"scope":"[SCOPEVALUE]"
}
The method from API that returns the token http://api.sample.com/OAuth/Token:
public ActionResult Token()
{
OutgoingWebResponse response =
this.AuthorizationServer.HandleTokenRequest(this.Request);
string tokenresponse = string.Format("Token({0})", response!=null?response.Body:""));
return response.AsActionResult();
}
The client method that requests the token is:
public string GetAuthorizationToken(string code)
{
string Url = ServerPath + "OAuth/Token";
string redirect_uri_encode = UrlEncode(ClientPath);
string param = string.Format("code={0}&client_id={1}&client_secret={2}&redirect_uri={3}&grant_type=authorization_code",code, ClientId, ClientSecret, redirect_uri_encode);
HttpWebRequest request = HttpWebRequest.Create(Url) as HttpWebRequest;
string result = null;
request.Method = "POST";
request.KeepAlive = true;
request.ContentType = "application/x-www-form-urlencoded";
request.Timeout = 10000;
request.Headers.Remove(HttpRequestHeader.Cookie);
var bs = Encoding.UTF8.GetBytes(param);
using (Stream reqStream = request.GetRequestStream())
{
reqStream.Write(bs, 0, bs.Length);
}
using (WebResponse response = request.GetResponse())
{
var sr = new StreamReader(response.GetResponseStream());
result = sr.ReadToEnd();
sr.Close();
}
if (!string.IsNullOrEmpty(result))
{
TokenData tokendata = JsonConvert.DeserializeObject<TokenData>(result);
return UpdateAuthorizotionFromToken(tokendata);
}
return null;
}
The result variable is empty.
Please let me know if you have any idea what could cause this. Initially I assumed is because of the cookies so I tried to remove them from request.
Thanks in advance.

Dear just create webclient using following code and you will get json info in tokeninfo.I used it and simply its working perfect.
WebClient client = new WebClient();
string postData = "client_id=" + ""
+ "&client_secret=" + ""
+ "&grant_type=password&username=" + "" //your username
+ "&password=" + "";//your password :)
string soundCloudTokenRes = "https://api.soundcloud.com/oauth2/token";
string tokenInfo = client.UploadString(soundCloudTokenRes, postData);
You can then use substring that contains only token from tokeninfo.
To upload tracks on sound cloud.
private void TestSoundCloudupload()
{
System.Net.ServicePointManager.Expect100Continue = false;
var request = WebRequest.Create("https://api.soundcloud.com/tracks") as HttpWebRequest;
//some default headers
request.Accept = "*/*";
request.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3");
request.Headers.Add("Accept-Encoding", "gzip,deflate,sdch");
request.Headers.Add("Accept-Language", "en-US,en;q=0.8,ru;q=0.6");
//file array
var files = new UploadFile[] { new UploadFile(Server.MapPath("Downloads//0.mp3"), "track[asset_data]", "application/octet-stream") };
//other form data
var form = new NameValueCollection();
form.Add("track[title]", "Some title");
form.Add("track[sharing]", "public");
form.Add("oauth_token", "");
form.Add("format", "json");
form.Add("Filename", "0.mp3");
form.Add("Upload", "Submit Query");
try
{
using (var response = HttpUploadHelper.Upload(request, files, form))
{
using (var reader = new StreamReader(response.GetResponseStream()))
{
Response.Write(reader.ReadToEnd());
}
}
}
catch (Exception ex)
{
Response.Write(ex.ToString());
}
}

can I check if a file exists at a URL?

I know I can locally, on my filesystem, check if a file exists:
if(File.Exists(path))
Can I check at a particular remote URL?

If you're attempting to verify the existence of a web resource, I would recommend using the HttpWebRequest class. This will allow you to send a HEAD request to the URL in question. Only the response headers will be returned, even if the resource exists.
var url = "http://www.domain.com/image.png";
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "HEAD";
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
/* A WebException will be thrown if the status of the response is not `200 OK` */
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
Of course, if you want to download the resource if it exists it would most likely be more efficient to send a GET request instead (by not setting the Method property to "HEAD", or by using the WebClient class).

If you want to just copy & paste Justin's code and get a method to use, here's how I've implemented it:
using System.Net;
public class MyClass {
static public bool URLExists (string url) {
bool result = false;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try {
response = (HttpWebResponse)webRequest.GetResponse();
result = true;
} catch (WebException webException) {
Debug.Log(url +" doesn't exist: "+ webException.Message);
} finally {
if (response != null) {
response.Close();
}
}
return result;
}
}
I'll keep his observation:
If you want to download the resource, and it exists, it would be more efficient to send a GET request instead by not setting the Method property to "HEAD" or by using the WebClient class.

Below is a simplified version of the code:
public bool URLExists(string url)
{
bool result = true;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
try
{
webRequest.GetResponse();
}
catch
{
result = false;
}
return result;
}

If you are using a unc path or a mapped drive, this will work fine.
If you are using a web address (http, ftp etc) you are better off using WebClient - you will get a WebException if it doesn't exist.

public static bool UrlExists(string file)
{
bool exists = false;
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(file);
request.Method = "HEAD";
request.Timeout = 5000; // milliseconds
request.AllowAutoRedirect = false;
try
{
response = (HttpWebResponse)request.GetResponse();
exists = response.StatusCode == HttpStatusCode.OK;
}
catch
{
exists = false;
}
finally
{
// close your response.
if (response != null)
response.Close();
}
return exists;
}

I had the same problem to solve in asp.net core, I've solved with HttpClient
private async Task<bool> isFileExist(string url)
{
using (HttpClient client = new HttpClient())
{
var restponse = await client.GetAsync(url);
return restponse.StatusCode == System.Net.HttpStatusCode.OK;
}
}

My version:
public bool IsUrlExist(string url, int timeOutMs = 1000)
{
WebRequest webRequest = WebRequest.Create(url);
webRequest.Method = "HEAD";
webRequest.Timeout = timeOutMs;
try
{
var response = webRequest.GetResponse();
/* response is `200 OK` */
response.Close();
}
catch
{
/* Any other response */
return false;
}
return true;
}

WebRequest will waiting long time(ignore the timeout user set) because not set proxy, so I change to use RestSharp to do this.
var client = new RestClient(url);
var request = new RestRequest(Method.HEAD);
request.Timeout = 5000;
var response = client.Execute(request);
result = response.StatusCode == HttpStatusCode.OK;

Thanks for all answers.
And I would like to add my implementation which includes default state when we get errors, for specific cases like mine.
private bool HTTP_URLExists(String vstrURL, bool vResErrorDefault = false, int vTimeOut = 1200)
{
bool vResult = false;
WebRequest webRequest = WebRequest.Create(vstrURL);
webRequest.Timeout = vTimeOut; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)webRequest.GetResponse();
if (response.StatusCode == HttpStatusCode.OK) vResult = true;
else if (response.StatusCode == HttpStatusCode.NotFound) vResult = false;
else vResult = vResErrorDefault;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
{
var resp01 = (HttpWebResponse)ex.Response;
if (resp01.StatusCode == HttpStatusCode.NotFound)
{
vResult = false;
}
else
{
vResult = vResErrorDefault;
}
}
else
{
vResult = vResErrorDefault;
}
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
return vResult;
}

Anoter version with define timeout :
public bool URLExists(string url,int timeout = 5000)
{
...
webRequest.Timeout = timeout; // miliseconds
...
}

This works for me:
bool HaveFile(string url)
{
try
{
using (WebClient webClient = new WebClient())
{
webClient.DownloadString(url);
}
return true;
}
catch (Exception)
{
return false;
}
}