how to send a http basic auth post? - c#

I have been given the task to create a http post using basic auth.
I am developing in C# in an asp.net MVC application.
I have also been given this example.
{
POST /v2/token_endpoint HTTP/1.1
Authorization: Basic Y2xpZW50X2lkOmNsaWVudF9zZWNyZXQ=
Accept: application/json
Content-Type: application/x-www-form-urlencoded
User-Agent: Java/1.6.0_33
Host: api.freeagent.com
Connection: close
Content-Length: 127
grant_type=authorization_code&code=12P3AsFZXwXjd7SLOE1dsaX8oCgix&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foauth
}
My question is how do i code this in C#?
If more information is need just ask, thanks in advance
edit: I have made some progress but I have not added the grant_type
public void AccessToken(string code)
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(ApiKey + ":" + ApiSecret));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
client.Headers[HttpRequestHeader.UserAgent] = "Java/1.6.0_33";
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.Connection] = "close";
client.Headers["grant_type"] = "authorization_code";
var result = client.DownloadString(url);
}
So how do i add: grant_type=authorization_code&code=12P3AsFZXwXjd7SLOE1dsaX8oCgix&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foauth to the post?

You can find here two samples how to make basic auth request with WebRequest and WebClient classes:
http://grahamrhay.wordpress.com/2011/08/22/making-a-post-request-in-c-with-basic-authentication/
http://anishshenoy57.wordpress.com/2013/01/22/basic-http-authentication-using-c/
Basically basic auth it's just Base64(username:password), so it's easy to implement it.
UPDATE1
Here is a sample based on your method:
public void AccessToken(string code)
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(ApiKey + ":" + ApiSecret));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
client.Headers[HttpRequestHeader.UserAgent] = "Java/1.6.0_33";
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.Connection] = "close";
client.Headers["grant_type"] = "authorization_code";
string data = string.Format(
"grant_type=authorization_code&code={0}&redirect_uri=http%3A%2F%2Flocalhost%3A8080",
code);
var result = client.UploadString(url, data);
}
The only different in calling method in WebClient. DownloadString will do GET request, but for POST you need to use UploadString method

I too have struggled with this, but I have now made it to the other side!
The one that held me up the most was that if you specify a redirect URL in the Auth Token request, you must also specify it in the Access Token request with the same URL, even though it's not used.
I started off with someones partial wrapper, and have evolved it almost beyond recognition. Hopefully this wrapper (complete with auth and access token code) will help others. I've only used it to create invoices and invoice items so far, and the other functions are untested, but it should get you your tokens and to the point where you can debug the requests.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;
using System.Web.Script.Serialization;
namespace ClientZone.External
{
public class FreeAgent
{
public enum Methods
{
POST = 1,
PUT
}
public enum Status
{
Draft = 1,
Sent,
Cancelled
}
private string _subDomain;
private string _identifier;
private string _secret;
private string _redirectURI;
public static string AuthToken = "";
private static string _accessToken = "";
private static string _refreshToken = "";
private static DateTime _refreshTime;
public FreeAgent(string identifier, string secret, string subdomain, string redirectURI)
{
_subDomain = subdomain;
_identifier = identifier;
_secret = secret;
_redirectURI = redirectURI; // If this was specified in the Auth Token call, you must specify it here, and it must be the same
}
public bool GetAccessToken()
{
try
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(_identifier + ":" + _secret));
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.KeepAlive] = "true";
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded;charset=UTF-8";
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
System.Net.ServicePointManager.Expect100Continue = false;
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
client.Headers[HttpRequestHeader.AcceptLanguage] = "en-US,en;q=0.8";
var result = "";
if (!(_accessToken == "" && _refreshToken != ""))
{
string data = string.Format("grant_type=authorization_code&code={0}&redirect_uri={1}", AuthToken, _redirectURI);
result = client.UploadString(url, data);
}
else
{
// Marking the access token as blank and the refresh token as not blank
// is a sign we need to get a refresh token
string data = string.Format("grant_type=refresh_token&refresh_token={0}", _refreshToken);
result = client.UploadString(url, data);
}
JavaScriptSerializer json_serializer = new JavaScriptSerializer();
var results_list = (IDictionary<string, object>)json_serializer.DeserializeObject(result);
_accessToken = results_list["access_token"].ToString();
int secondsUntilRefresh = Int32.Parse(results_list["expires_in"].ToString());
_refreshTime = DateTime.Now.AddSeconds(secondsUntilRefresh);
if (results_list.Any(x => x.Key == "refresh_token"))
{
_refreshToken = results_list["refresh_token"].ToString();
}
if (_accessToken == "" || _refreshToken == "" || _refreshTime == new DateTime())
{
return false;
}
}
catch
{
return false;
}
return true;
}
private HttpStatusCode SendWebRequest(Methods method, string URN, string request, out string ResponseData)
{
if (_accessToken == "")
{
// The access token has not been retrieved yet
GetAccessToken();
}
else
{
if (_refreshTime != new DateTime() && _refreshTime < DateTime.Now) {
// The token has expired and we need to refresh it
_accessToken = "";
GetAccessToken();
}
}
if (_accessToken != "")
{
try
{
WebClient client = new WebClient();
string url = "https://api.freeagentcentral.com/v2/" + URN;
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36";
client.Headers[HttpRequestHeader.Authorization] = "Bearer " + _accessToken;
client.Headers[HttpRequestHeader.ContentType] = "application/json";
client.Headers[HttpRequestHeader.Accept] = "application/json";
if (method == Methods.POST || method == Methods.PUT)
{
string data = request;
var result = client.UploadString(url, method.ToString(), data);
ResponseData = result;
return HttpStatusCode.Created;
}
else
{
var result = client.DownloadString(url);
ResponseData = result;
return HttpStatusCode.OK;
}
}
catch (WebException e)
{
if (e.GetType().Name == "WebException")
{
WebException we = (WebException)e;
HttpWebResponse response = (System.Net.HttpWebResponse)we.Response;
ResponseData = response.StatusDescription;
return response.StatusCode;
}
else
{
ResponseData = e.Message;
if (e.InnerException != null)
{
ResponseData = ResponseData + " - " + e.InnerException.ToString();
}
return HttpStatusCode.SeeOther;
}
}
}
else
{
ResponseData = "Access Token could not be retrieved";
return HttpStatusCode.SeeOther;
}
}
private int ExtractNewID(string resp, string URN)
{
if (resp != null && resp.Trim() != "")
{
JavaScriptSerializer json_serializer = new JavaScriptSerializer();
var results_list = (IDictionary<string, object>)json_serializer.DeserializeObject(resp);
if (results_list.Any(x => x.Key == "invoice"))
{
var returnInvoice = (IDictionary<string, object>)results_list["invoice"];
if (returnInvoice["created_at"].ToString() != "")
{
string returnURL = returnInvoice["url"].ToString();
if (returnURL.Contains("http"))
{
return int.Parse(returnURL.Remove(0, ("https://api.freeagentcentral.com/v2/" + URN).Length + 1));
}
else
{
return int.Parse(returnURL.Remove(0, URN.Length + 2));
}
}
}
}
return -1;
}
public int CreateContact(string firstName, string lastName, string emailAddress, string street, string city, string state, string postcode, string country)
{
StringBuilder request = new StringBuilder();
request.Append("{\"contact\":{");
request.Append("\"first-name\":");
request.Append(firstName);
request.Append("\",");
request.Append("\"last-name\":");
request.Append(lastName);
request.Append("\",");
request.Append("\"email\":");
request.Append(emailAddress);
request.Append("\",");
request.Append("\"address1\":");
request.Append(street);
request.Append("\",");
request.Append("\"town\":");
request.Append(city);
request.Append("\",");
request.Append("\"region\":");
request.Append(state);
request.Append("\",");
request.Append("\"postcode\":");
request.Append(postcode);
request.Append("\",");
request.Append("\"country\":");
request.Append(country);
request.Append("\"");
request.Append("}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.POST, "contacts", request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
return ExtractNewID(returnData, "contacts");
}
else
{
return -1;
}
}
public int CreateInvoice(int contactID, DateTime invoiceDate, int terms, string reference, string comments, string currency = "GBP")
{
StringBuilder request = new StringBuilder();
request.Append("{\"invoice\":{");
request.Append("\"dated_on\": \"");
request.Append(invoiceDate.ToString("yyyy-MM-ddTHH:mm:00Z"));
request.Append("\",");
if (!string.IsNullOrEmpty(reference))
{
request.Append("\"reference\": \"");
request.Append(reference);
request.Append("\",");
}
if (!string.IsNullOrEmpty(comments))
{
request.Append("\"comments\": \"");
request.Append(comments);
request.Append("\",");
}
request.Append("\"payment_terms_in_days\": \"");
request.Append(terms);
request.Append("\",");
request.Append("\"contact\": \"");
request.Append(contactID);
request.Append("\",");
if (currency == "EUR")
{
request.Append("\"ec_status\": \"");
request.Append("EC Services");
request.Append("\",");
}
request.Append("\"currency\": \"");
request.Append(currency);
request.Append("\"");
request.Append("}}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.POST, "invoices", request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
return ExtractNewID(returnData, "invoices");
}
else
{
return -1;
}
}
public bool ChangeInvoiceStatus(int invoiceID, Status status)
{
string returnData = string.Empty;
HttpStatusCode resp = SendWebRequest(Methods.PUT, "invoices/" + invoiceID.ToString() + "/transitions/mark_as_" + status.ToString().ToLower(), string.Empty, out returnData);
return false;
}
public int CreateInvoiceItem(int invoiceID, string itemType, float price, int quantity, float taxRate, string description)
{
StringBuilder request = new StringBuilder();
request.Append("{\"invoice\":{");
request.Append("\"invoice_items\":[{");
request.Append("\"item_type\": \"");
request.Append(itemType);
request.Append("\",");
request.Append("\"price\": \"");
request.Append(price.ToString("0.00"));
request.Append("\",");
request.Append("\"quantity\": \"");
request.Append(quantity);
request.Append("\",");
request.Append("\"sales_tax_rate\": \"");
request.Append(taxRate.ToString("0.00"));
request.Append("\",");
request.Append("\"description\": \"");
request.Append(description);
request.Append("\"");
request.Append("}]");
request.Append("}");
request.Append("}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.PUT, "invoices/" + invoiceID.ToString(), request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
// Invoice items is an update call to an invoice, so we just get a 200 OK with no response body to extract an ID from
return 0;
}
else
{
return -1;
}
}
}
}
To use this in an MVC setup you need to detect in the Index whether the login is necessary or not. If it is, you tell FreeAgent to come back to another ActionResult (I've used Auth) to catch the redirect and store the code.
public ActionResult Index()
{
if (MyNamespace.External.FreeAgent.AuthToken == "")
{
return Redirect("https://api.freeagent.com/v2/approve_app?redirect_uri=" + Request.Url.Scheme + "://" + Request.Url.Authority + "/Invoice/Auth" + "&response_type=code&client_id=1234567890123456789012");
}
else
{
return View();
}
}
public ActionResult Auth()
{
if (Request.Params.Get("code").ToString() != "")
{
ClientZone.External.FreeAgent.AuthToken = Request.Params.Get("code").ToString();
}
return View("Index");
}
Then to use the functions all you need is
MyNameSpace.External.FreeAgent FA = new ClientZone.External.FreeAgent("abcdefghhijklmnopqrstu", "1234567890123456789012", "acme", "http://localhost:52404/Invoice/Auth");
int newID = FA.CreateInvoice(54321, InvoiceDate, 30, "", "", "GBP");
if (newID != -1)
{
FA.CreateInvoiceItem(newID, unit, rate, quantity, number, description);
}
There is some UK/EU specific coding in the CreateInvoice. I'm not sure how custom or standard this is, but I've left it in as it's easier to delete it than recreate it.

Related

HttpClient.SendAsync is throwing "A task was canceled" issue

We are making a PUT call to the service layer to update an entity information
HttpClient.SendAsync method is throwing "A task was canceled" issue even though the API call is successfully made to the backend service layer. I could see the logs from service layer.
Initially I thought it could be related to timeout, so I increased the timeout from 10 seconds to 30 seconds, still the program is waiting for 30 seconds and gets timed out with the error "A task was canceled". But the api call was successfully completed in Service Layer within even 5 seconds
Please find below my code
protected override void Execute(CodeActivityContext context)
{
string uuid = UUID.Get(context);
Console.WriteLine(uuid + ": Http Api Call - STARTED");
string endPoint = EndPoint.Get(context);
string contentType = ContentType.Get(context);
string acceptFormat = AcceptFormat.Get(context);
HttpMethod httpMethod = HttpApiMethod.Get(context);
string clientCertificatePath = ClientCertificatePath.Get(context);
string clientCertificatePassword = ClientCertificatePassword.Get(context);
double httpTimeOut = HttpTimeout.Get(context);
Dictionary<string, string> requestHeaders = RequestHeaders.Get(context);
Dictionary<string, string> pathParams = PathParams.Get(context);
Dictionary<string, string> queryParams = QueryParams.Get(context);
string requestBody = RequestBody.Get(context);
WebRequestHandler webRequestHandler = new WebRequestHandler();
webRequestHandler.MaxConnectionsPerServer = 1;
if (clientCertificatePath != null && clientCertificatePath.Trim().Length > 0)
{
X509Certificate2 x509Certificate2 = null;
if (clientCertificatePassword != null && clientCertificatePassword.Trim().Length > 0)
{
x509Certificate2 = new X509Certificate2(clientCertificatePath.Trim(),
clientCertificatePassword.Trim());
}
else
{
x509Certificate2 = new X509Certificate2(clientCertificatePath.Trim());
}
webRequestHandler.ClientCertificates.Add(x509Certificate2);
}
HttpClient httpClient = new HttpClient(webRequestHandler)
{
Timeout = TimeSpan.FromMilliseconds(httpTimeOut)
};
if (acceptFormat != null)
{
httpClient.DefaultRequestHeaders.Add("Accept", acceptFormat);
}
HttpResponseMessage httpResponseMessage = InvokeApiSync(httpClient, endPoint,
httpMethod, contentType,
requestHeaders, pathParams,
queryParams, requestBody,
uuid
);
HttpResponseMessageObject.Set(context, httpResponseMessage);
ResponseBody.Set(context, httpResponseMessage.Content.ReadAsStringAsync().Result);
StatusCode.Set(context, (int)httpResponseMessage.StatusCode);
Console.WriteLine(uuid + ": Http Api Call - ENDED");
}
private HttpResponseMessage InvokeApiSync(HttpClient httpClient,
string endPoint,
HttpMethod httpMethod,
string contentType,
Dictionary<string, string> requestHeaders,
Dictionary<string, string> pathParams,
Dictionary<string, string> queryParams,
string requestBody,
string uuid)
{
if (pathParams != null)
{
ICollection<string> keys = pathParams.Keys;
if (keys.Count > 0)
{
foreach (string key in keys)
{
endPoint = endPoint.Replace(":" + key, pathParams[key]);
}
}
}
if (queryParams != null)
{
List<string> keys = new List<string>(queryParams.Keys);
if (keys.Count > 0)
{
endPoint = string.Concat(endPoint, "?", keys[0], "=", queryParams[keys[0]]);
for (int index = 1; index < keys.Count; index++)
{
endPoint = string.Concat(endPoint, "&", keys[index], "=", queryParams[keys[index]]);
}
}
}
try
{
HttpRequestMessage httpRequestMessage = new HttpRequestMessage(httpMethod, endPoint);
if (requestHeaders != null)
{
foreach (string key in requestHeaders.Keys)
{
httpRequestMessage.Headers.Add(key, requestHeaders[key]);
}
}
if (httpMethod.Equals(HttpMethod.Put) || httpMethod.Equals(HttpMethod.Post))
{
StringContent reqContent = null;
if (requestBody != null)
{
reqContent = new StringContent(requestBody);
}
else
{
reqContent = new StringContent("");
}
reqContent.Headers.ContentType = new MediaTypeHeaderValue(contentType);
httpRequestMessage.Content = reqContent;
}
HttpResponseMessage httpResponseMessage = httpClient.SendAsync(httpRequestMessage).Result;
return httpResponseMessage;
}
catch (Exception exception)
{
Console.WriteLine(uuid + " : HttpApi Error Has Occurred");
Console.WriteLine(uuid + " : " + exception.Message);
Console.WriteLine(uuid + " : " + exception.StackTrace);
throw exception;
}
}
Any help would be much appreciated. Thanks.!

URL not opening correctly on another browser asp.net mvc

I have an asp.net mvc application were i send sms messages with URL to clients. message and URL is sent correctly but when i click on the link it does not open correctly, example there is no data passed from database, example if you click the following url, there is no data passed to view: http://binvoicing.com/InvoiceAddlines/PaymentPreview?originalId=11&total=585.00
Here code:
public ActionResult SendSms(string cellnumber, int? id, string rt, string tenant, decimal? total, string userloggedin)
{
StreamReader objReader;
WebClient client = new WebClient();
int payid = 0;
int paypalaid = 0;
string UrL = "";
string pass = "mypassword";
//string cell = cellnumber;
string user = "username";
var pay = from e in db.PayfastGateways
where e.userId == userloggedin
select e;
var paya = pay.ToList();
foreach (var y in paya)
{
payid = y.ID;
}
var pal = from e in db.PaypalGateways
where e.userId == userloggedin
select e;
var payla = pal.ToList();
foreach (var y in payla)
{
paypalaid = y.ID;
}
string url = Url.Action("PaymentPreview", "InvoiceAddlines", new System.Web.Routing.RouteValueDictionary(new { originalId = id, total = total }), "http", Request.Url.Host);
if (payid == 0 && paypalaid == 0)
{
UrL = "";
}
else
{
UrL = url;
}
string mess = " Dear Customer, please click on the following link to view generated invoice, you can also pay your invoice online." + UrL;
string message = HttpUtility.UrlEncode(mess, System.Text.Encoding.GetEncoding("ISO-8859-1"));
string baseurl =
"http://bulksms.2way.co.za/eapi/submission/send_sms/2/2.0?" +
"username=" + user + "&" +
"password=" + pass + "&" +
"message=" + message + "&" +
"msisdn=" + cellnumber;
WebRequest wrGETURL;
wrGETURL = WebRequest.Create(baseurl);
HttpUtility.UrlEncode("http://www.vcskicks.com/c#");
try
{
Stream objStream;
objStream = wrGETURL.GetResponse().GetResponseStream();
objReader = new StreamReader(objStream);
objReader.Close();
}
catch (Exception ex)
{
ex.ToString();
}
ViewBag.cellnumber = cellnumber;
ViewBag.id = id;
ViewBag.rt = rt;
ViewBag.tenant = tenant;
ViewBag.total = total;
ViewBag.UrL = UrL;
return View();
}
SMS send url like http://binvoicing.com/InvoiceAddlines/PaymentPreview?originalId=11&total=585.00
Here Is my PaymentPreview method:
public ActionResult PaymentPreview(int? originalId, decimal? total)
{
TempData["keyEditId"] = originalId;
ViewBag.ind = originalId;
decimal totals = 0;
totals = (decimal)total;
var line = from e in db.InvoiceAddlines
where e.AddlineID == originalId
select e;
var addlines = line.ToList();
foreach (var y in addlines)
{
decimal? de = Math.Round(totals);
string tots = de.ToString();
y.Total = tots;
db.SaveChanges();
}
return View(db.InvoiceAddlines.ToList());
}
Hope someone can assist, thanks.
I think there is something wrong with the way URL is being built. See if this works -
Change this part of your code
string url = Url.Action("PaymentPreview", "InvoiceAddlines", new System.Web.Routing.RouteValueDictionary(new { originalId = id, total = total }), "http", Request.Url.Host);
To
string url = Url.Action("PaymentPreview", "InvoiceAddlines", new { originalId = id, total = total })

Unity C# GraphQL Post Request Body Authentication

I am working on trying to connect to my GraphQL server my developers have setup from within Unity. I have found some scripts to help with this process, however, I am still not able to connect because i need to be logged into the system hosting graphql to be able to query externally, I cannot just use the endpoint url.
My developer has told me to do a POST request to mysystem/login and in the request body add {email: string, password: string}. I have tried a few different things and nothing is working. I have to log into the mysystem/login with email and password, then i will be able to connect to mygraphql endpoint from the code below.--i was assuming this portion would go where I have the //smt authentication notes. Any help on how to setup the post request and where it should be or how it should work would be much appreciated.
using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using SimpleJSON;
using UnityEngine;
using UnityEngine.Networking;
namespace graphQLClient
{
public class GraphQuery : MonoBehaviour
{
public static GraphQuery instance = null;
[Tooltip("The url of the node endpoint of the graphQL server being queried")]
public static string url;
public delegate void QueryComplete();
public static event QueryComplete onQueryComplete;
public enum Status { Neutral, Loading, Complete, Error };
public static Status queryStatus;
public static string queryReturn;
string authURL;
public static string LoginToken;
public class Query
{
public string query;
}
public void Awake()
{
if (instance == null)
{
instance = this;
}
else if (instance != this)
{
Destroy(gameObject);
}
DontDestroyOnLoad(gameObject);
StartCoroutine("PostLogin");
}
private void Start()
{
LoginToken = "";
}
public static Dictionary<string, string> variable = new Dictionary<string, string>();
public static Dictionary<string, string[]> array = new Dictionary<string, string[]>();
// Use this for initialization
// SMT Authentication to ELP
// SMT Needed for Authentication--- if (token != null) request.SetRequestHeader("Authorization", "Bearer " + token);
//In the request body: {email: string, password: string}
//You’ll either get a 401 with an empty response or 201 with {token: string, user: object}
IEnumerator PostLogin()
{
Debug.Log("Logging in...");
string authURL = "https://myapp.com/login";
string loginBody = "{\"email\":\"myemail.com\",\"password\":\"mypassowrd\"}";
var request = new UnityWebRequest(authURL, "POST");
byte[] bodyRaw = new System.Text.UTF8Encoding().GetBytes(loginBody);
request.uploadHandler = (UploadHandler)new UploadHandlerRaw(bodyRaw);
request.downloadHandler = (DownloadHandler)new DownloadHandlerBuffer();
request.SetRequestHeader("Content-Type", "application/json");
yield return request.SendWebRequest();
if (request.isNetworkError || request.isHttpError)
{
Debug.Log("Login error!");
foreach (KeyValuePair<string, string> entry in request.GetResponseHeaders())
{
Debug.Log(entry.Value + "=" + entry.Key);
}
Debug.Log(request.error);
}
else
{
Debug.Log("Login complete!");
//Debug.Log(request.downloadHandler.text);
LoginToken = request.downloadHandler.text;
Debug.Log(LoginToken);
}
}
public static WWW POST(string details)
{
//var request = new UnityWebRequest(url, "POST");
details = QuerySorter(details);
Query query = new Query();
string jsonData = "";
WWWForm form = new WWWForm();
query = new Query { query = details };
jsonData = JsonUtility.ToJson(query);
byte[] postData = Encoding.ASCII.GetBytes(jsonData);
Dictionary<string, string> postHeader = form.headers;
//postHeader["Authorization"] = "Bearer " + downloadHandler.Token;
if (postHeader.ContainsKey("Content-Type"))
//postHeader["Content-Type"] = "application/json";
postHeader.Add("Authorization", "Bearer " + LoginToken);
else
//postHeader.Add("Content-Type", "application/json");
postHeader.Add("Authorization", "Bearer " + LoginToken);
WWW www = new WWW(url, postData, postHeader);
instance.StartCoroutine(WaitForRequest(www));
queryStatus = Status.Loading;
return www;
}
static IEnumerator WaitForRequest(WWW data)
{
yield return data; // Wait until the download is done
if (data.error != null)
{
Debug.Log("There was an error sending request: " + data.error);
queryStatus = Status.Error;
}
else
{
queryReturn = data.text;
queryStatus = Status.Complete;
}
onQueryComplete();
}
public static string QuerySorter(string query)
{
string finalString;
string[] splitString;
string[] separators = { "$", "^" };
splitString = query.Split(separators, StringSplitOptions.RemoveEmptyEntries);
finalString = splitString[0];
for (int i = 1; i < splitString.Length; i++)
{
if (i % 2 == 0)
{
finalString += splitString[i];
}
else
{
if (!splitString[i].Contains("[]"))
{
finalString += variable[splitString[i]];
}
else
{
finalString += ArraySorter(splitString[i]);
}
}
}
return finalString;
}
public static string ArraySorter(string theArray)
{
string[] anArray;
string solution;
anArray = array[theArray];
solution = "[";
foreach (string a in anArray)
{
}
for (int i = 0; i < anArray.Length; i++)
{
solution += anArray[i].Trim(new Char[] { '"' });
if (i < anArray.Length - 1)
solution += ",";
}
solution += "]";
Debug.Log("This is solution " + solution);
return solution;
}
}
}

How to use cursors to iterate through the result set of GET followers/list in Twitter API using c#?

I have been trying a way to get the data in the next page of the result set of GET followers/list API call. I can get the default data set with the data of first 20 followers and not the others. To get the data of other followers i have to access the next page using the next_cursor but it's not working. I tried using the pseudo-code mentioned in this link. https://dev.twitter.com/docs/misc/cursoring
Is it a must to use this(this is mentioned in the dev. site):
var api-path = "https://api.twitter.com/1.1/endpoint.json?screen_name=targetUser"
Because I have been using the resource URL as,
var resource_url = "https://api.twitter.com/1.1/followers/list.json";
and I tried appending the next_cursor to the same resource URL.
var url_with_cursor = resource_url + "&cursor=" + 1463101490306580067;
and then created the request.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url_with_cursor);
but I'm getting an exception in this line when getting the response.
WebResponse response = request.GetResponse();
The error I'm getting is
The Remote Server returned an Error 401 Unauthorized
Can someone tell the exact way to do cursor-ing, or the exact way to include the cursor in the request. I'm using a asp.net C# web application.
Here's my code, The oauth_token, oauth_token_secret, oauth_consumer_key, oauth_consumer_secret, oauth_version and oauth_signature_method are defined in my application
var resource_url = "https://api.twitter.com/1.1/followers/list.json";
var cursor = "-1";
do
{
var url_with_cursor = resource_url + "&cursor=" + cursor;
// unique request details
var oauth_nonce = Convert.ToBase64String(
new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
var timeSpan = DateTime.UtcNow
- new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);
var oauth_timestamp = Convert.ToInt64(timeSpan.TotalSeconds).ToString();
// create oauth signature
var baseFormat = "oauth_consumer_key={0}&oauth_nonce={1}&oauth_signature_method={2}" +
"&oauth_timestamp={3}&oauth_token={4}&oauth_version={5}";
var baseString = string.Format(baseFormat,
oauth_consumer_key,
oauth_nonce,
oauth_signature_method,
oauth_timestamp,
oauth_token,
oauth_version
//,Uri.EscapeDataString(status)
);
baseString = string.Concat("GET&", Uri.EscapeDataString(resource_url), "&", Uri.EscapeDataString(baseString));
var compositeKey = string.Concat(Uri.EscapeDataString(oauth_consumer_secret),
"&", Uri.EscapeDataString(oauth_token_secret));
string oauth_signature;
using (HMACSHA1 hasher = new HMACSHA1(ASCIIEncoding.ASCII.GetBytes(compositeKey)))
{
oauth_signature = Convert.ToBase64String(
hasher.ComputeHash(ASCIIEncoding.ASCII.GetBytes(baseString)));
}
// create the request header
var headerFormat = "OAuth oauth_nonce=\"{0}\", oauth_signature_method=\"{1}\", " +
"oauth_timestamp=\"{2}\", oauth_consumer_key=\"{3}\", " +
"oauth_token=\"{4}\", oauth_signature=\"{5}\", " +
"oauth_version=\"{6}\"";
var authHeader = string.Format(headerFormat,
Uri.EscapeDataString(oauth_nonce),
Uri.EscapeDataString(oauth_signature_method),
Uri.EscapeDataString(oauth_timestamp),
Uri.EscapeDataString(oauth_consumer_key),
Uri.EscapeDataString(oauth_token),
Uri.EscapeDataString(oauth_signature),
Uri.EscapeDataString(oauth_version)
);
// make the request
ServicePointManager.Expect100Continue = false;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url_with_cursor);
request.Headers.Add("Authorization", authHeader);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
WebResponse response = request.GetResponse();
string result = new StreamReader(response.GetResponseStream()).ReadToEnd();
JObject j = JObject.Parse(result);
JArray data = (JArray)j["users"];
cursor = (String)j["next_cursor_str"];
} while (!cursor.Equals("0"));
Thanks.
Tweetinvi would make it somewhat easier for you.
Please visit https://github.com/linvi/tweetinvi/wiki/Get-All-Followers-Code to have an idea on how to do it including RateLimit handling.
Without considering the RateLimits, you could simply use the following code.
long nextCursor = -1;
do
{
var query = string.Format("https://api.twitter.com/1.1/followers/ids.json?screen_name={0}", username);
var results = TwitterAccessor.ExecuteCursorGETCursorQueryResult<IIdsCursorQueryResultDTO>(query, cursor: cursor).ToArray();
if (results.Any())
{
nextCursor = results.Last().NextCursor;
}
else
{
nextCursor = -1;
}
}
while (nextCursor != -1 && nextCursor != 0);
you should make a different call to authenticate, by an authorization request. Once that has been granted, you can call the webresponse with the cursor. See my sample code below (Take special note to the StartCreateCall method, where the authentication is done. The data from Twitter is then retrieved by the CallData method):
public partial class twitter_followers : System.Web.UI.Page
{
public string strTwiterFollowers { get; set; }
private List<TwiterFollowers> listFollowers = new List<TwiterFollowers>();
private string screen_name = string.Empty;
// oauth application keys
private string oauth_consumer_key = string.Empty;
private string oauth_consumer_secret = string.Empty;
// oauth implementation details
private string resource_urlFormat = "https://api.twitter.com/1.1/followers/list.json?screen_name={0}&cursor={1}";
// unique request details
private string oauth_nonce = Convert.ToBase64String(new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
protected void Page_Load(object sender, EventArgs e)
{
//just get your request parameters from the config file.
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => screen_name)])) {
screen_name = ConfigurationManager.AppSettings[GetVariableName(() => screen_name)];
}
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_key)]))
{
oauth_consumer_key = ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_key)];
}
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_secret)]))
{
oauth_consumer_secret = ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_secret)];
}
StartCreateCall();
}
// Do the authenticate by an authorization request
private void StartCreateCall() {
// You need to set your own keys and screen name
var oAuthUrl = "https://api.twitter.com/oauth2/token";
// Do the Authenticate
var authHeaderFormat = "Basic {0}";
var authHeader = string.Format(authHeaderFormat,
Convert.ToBase64String(Encoding.UTF8.GetBytes(Uri.EscapeDataString(oauth_consumer_key) + ":" +
Uri.EscapeDataString((oauth_consumer_secret)))
));
var postBody = "grant_type=client_credentials";
HttpWebRequest authRequest = (HttpWebRequest)WebRequest.Create(oAuthUrl);
authRequest.Headers.Add("Authorization", authHeader);
authRequest.Method = "POST";
authRequest.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
authRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
using (Stream stream = authRequest.GetRequestStream())
{
byte[] content = ASCIIEncoding.ASCII.GetBytes(postBody);
stream.Write(content, 0, content.Length);
}
authRequest.Headers.Add("Accept-Encoding", "gzip");
WebResponse authResponse = authRequest.GetResponse();
// deserialize into an object
TwitAuthenticateResponse twitAuthResponse;
using (authResponse)
{
using (var reader = new StreamReader(authResponse.GetResponseStream()))
{
JavaScriptSerializer js = new JavaScriptSerializer();
var objectText = reader.ReadToEnd();
twitAuthResponse = JsonConvert.DeserializeObject<TwitAuthenticateResponse>(objectText);
}
}
//now we have been granted access and got a token type with authorization token from Twitter
//in the form of a TwitAuthenticateResponse object, we can retrieve the data recursively with a cursor
CallData(twitAuthResponse, authHeader, cursor);
int totalFollowers = listFollowers.Count;
lblTotalFollowers.Text = screen_name + " has " + listFollowers.Count + " Followers";
Random objRnd = new Random();
List<TwiterFollowers> randomFollowers = listFollowers.OrderBy(item => objRnd.Next()).ToList<TwiterFollowers>();
foreach (TwiterFollowers tw in randomFollowers)
{
strTwiterFollowers = strTwiterFollowers + "<li><a target='_blank' title='" + tw.ScreenName + "' href=https://twitter.com/" + tw.ScreenName + "><img src='" + tw.ProfileImage + "'/><span>" + tw.ScreenName + "</span></a></li>";
}
}
//Retrieve the data from Twitter recursively with a cursor
private void CallData(TwitAuthenticateResponse twitAuthResponse, string authHeader, string cursor)
{
try {
JObject j = GetJSonObject(twitAuthResponse, authHeader, cursor);
JArray data = (JArray)j["users"];
if (data != null)
{
foreach (var item in data)
{
TwiterFollowers objTwiterFollowers = new TwiterFollowers();
objTwiterFollowers.ScreenName = item["screen_name"].ToString().Replace("\"", "");
objTwiterFollowers.ProfileImage = item["profile_image_url"].ToString().Replace("\"", "");
objTwiterFollowers.UserId = item["id"].ToString().Replace("\"", "");
listFollowers.Add(objTwiterFollowers);
}
JValue next_cursor = (JValue)j["next_cursor"];
if (long.Parse(next_cursor.Value.ToString()) > 0)
{
//Get the following data from Twitter with the next cursor
CallData(twitAuthResponse, authHeader, next_cursor.Value.ToString());
}
}
} catch (Exception ex)
{
//do nothing
}
}
private JObject GetJSonObject(TwitAuthenticateResponse twitAuthResponse, string authHeader, string cursor)
{
string resource_url = string.Format(resource_urlFormat, screen_name, cursor);
if (string.IsNullOrEmpty(cursor))
{
resource_url = resource_url.Substring(0, resource_url.IndexOf("&cursor"));
}
HttpWebRequest fRequest = (HttpWebRequest)WebRequest.Create(resource_url);
var timelineHeaderFormat = "{0} {1}";
fRequest.Headers.Add("Authorization", string.Format(timelineHeaderFormat, twitAuthResponse.token_type, twitAuthResponse.access_token));
fRequest.Method = "Get";
WebResponse response = fRequest.GetResponse();
string result = new StreamReader(response.GetResponseStream()).ReadToEnd();
return JObject.Parse(result);
}
private string GetVariableName<T>(Expression<Func<T>> expr)
{
var body = (MemberExpression)expr.Body;
return body.Member.Name;
}
private class TwitAuthenticateResponse
{
public string token_type { get; set; }
public string access_token { get; set; }
}
private class TwiterFollowers
{
public string ScreenName { get; set; }
public string ProfileImage { get; set; }
public string UserId { get; set; }
}
}
You are getting "401 Unauthorized"
Did you check you are setting everything right?
Credentials? Check both queries on fiddler.

HtmlAgilityPack obtain Title and meta

I try to practice "HtmlAgilityPack ", but I am having some issues regarding this. here's what I coded, but I can not get correctly the title and the description of a web page ...
If someone can enlighten me on my mistake :)
...
public static void Main(string[] args)
{
string link = null;
string str;
string answer;
int curloc; // holds current location in response
string url = "http://stackoverflow.com/";
try
{
do
{
HttpWebRequest HttpWReq = (HttpWebRequest)WebRequest.Create(url);
HttpWReq.UserAgent = #"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5";
HttpWebResponse HttpWResp = (HttpWebResponse)HttpWReq.GetResponse();
//url = null; // disallow further use of this URI
Stream istrm = HttpWResp.GetResponseStream();
// Wrap the input stream in a StreamReader.
StreamReader rdr = new StreamReader(istrm);
// Read in the entire page.
str = rdr.ReadToEnd();
curloc = 0;
//WebPage result;
do
{
// Find the next URI to link to.
link = FindLink(str, ref curloc); //return the good link
Console.WriteLine("Title found: " + curloc);
//title = Title(str, ref curloc);
if (link != null)
{
Console.WriteLine("Link found: " + link);
using (System.Net.WebClient client = new System.Net.WebClient())
{
HtmlDocument htmlDoc = new HtmlDocument();
var html = client.DownloadString(url);
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
var htmlElement = htmlDoc.DocumentNode.Element("html");
HtmlNode node = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (node != null)
{
string desc = node.GetAttributeValue("content", "");
Console.Write("DESCRIPTION: " + desc);
}
else
{
Console.WriteLine("No description");
}
var titleElement =
htmlDoc.DocumentNode
.Element("html")
.Element("head")
.Element("title");
if (titleElement != null)
{
string title = titleElement.InnerText;
Console.WriteLine("Titre: {0}", title);
}
else
{
Console.WriteLine("no Title");
}
Console.Write("Done");
}
Console.Write("Link, More, Quit?");
answer = Console.ReadLine();
}
else
{
Console.WriteLine("No link found.");
break;
}
} while (link.Length > 0);
// Close the Response.
HttpWResp.Close();
} while (url != null);
}
catch{ ...}
Thanks in advance :)
Go about it this way:
HtmlNode mdnode = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (mdnode != null)
{
HtmlAttribute desc;
desc = mdnode.Attributes["content"];
string fulldescription = desc.Value;
Console.Write("DESCRIPTION: " + fulldescription);
}
I think your problem is here:
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
It should be:
htmlDoc.LoadHtml(html); //chargement de HTMLAgilityPack
LoadHtml expects a string with the HTML source, not the url.
And probably you want to change:
var html = client.DownloadString(url);
to
var html = client.DownloadString(link);
Have you used a breakpoint and gone line for line to see where the error might be occurring?
If you have, then Try something like this:
string result = string.Empty;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.google.com");
request.Method = "GET";
try
{
using (var stream = request.GetResponse().GetResponseStream())
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
result = reader.ReadToEnd();
}
}
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(result);
Then carry over the rest of your code below the htmlDoc.LoadHtml
[HttpPost]
public ActionResult Create(WebSite website)
{
string desc = HtmlAgi(website.Url, "description");
string keyword = HtmlAgi(website.Url, "Keywords");
if (ModelState.IsValid)
{
var userId = ((CustomPrincipal)User).UserId;
r.Create(new WebSite
{
Description = desc,
Tags = keyword,
Url = website.Url,
UserId = userId,
Category = website.Category
});
return RedirectToAction("Index");
}
return View(website);
}
public string HtmlAgi(string url, string key)
{
//string.Format
var Webget = new HtmlWeb();
var doc = Webget.Load(url);
HtmlNode ourNode = doc.DocumentNode.SelectSingleNode(string.Format("//meta[#name='{0}']", key));
if (ourNode != null)
{
return ourNode.GetAttributeValue("content", "");
}
else
{
return "not fount";
}
}

Categories