We Keep Coding

Call method inside multiple tasks C#

I am having a problem calling a method in a list of Tasks. I have a method that creates N number of tasks. In each task I perform some operations that in the end results an fetching data via HttpWebRequest and writing that data into a file. I use lock objects to lock the access to shared resources like variables. Everything performs great except the call for a method that creates a executes an HttpWebRequest (method GetData). Whenever I don't lock that call for the method (GetData) it seems that some data/files are skipped. For example:
With the lock object I get file 1,2,3 and 4
Without the lock object I get file 2,4 and 3
Here's the code for the method that creates the tasks
private object lockObjectWebRequest= new object();
private object lockObjectTransactions = new object();
public List<Task> ExtractLoanTransactionsData(string URLReceived, string Headers, string Body)
{
List<Task> Tasks = new List<Task>();
try
{
int Limit = 0;
int OffsetItemsTotal = 0;
int NumberOftasks = 4;
// Create the task to run in parallel
for (int i = 0; i <= NumberOftasks; i++)
{
int OffsetCalculated = 0;
if (i > 0)
{
OffsetCalculated = Limit * i;
}
Tasks.Add(Task.Factory.StartNew(() =>
{
string URL = URLReceived+ "&offset=" + OffsetCalculated .ToString() + "&limit=" + Limit.ToString();
string Output = string.Empty;
lock (lockObjectWebRequest)
{
Output = GetData(URL, Headers,Body);
}
if (Output != "[]")
{
lock (lockObjectTransactions)
{
Identifier++;
Job.Identifier = Identifier;
// write to file
string json = JValue.Parse(Output).ToString(Formatting.Indented);
string FileName = OffSet.ToString() + Identifier;
string Path = #"C:\FileFolder\" + FileName + ".json";
File.WriteAllText(Path, json);
}
}
}));
}
}
catch (Exception ex)
{
Tasks = new List<Task>();
}
return Tasks;
}
Here's the code that performs the HttpWebRequest:
public string GetData(string URL, string Headers, string Body)
{
string Data = string.Empty;
Headers = Headers.Trim('{').Trim('}');
string[] HeadersSplit = Headers.Split(new char[] { ',', ':' });
HttpWebRequest WebRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
WebRequest.Credentials = new NetworkCredential();
WebRequest.Method = "POST";
HttpWebResponse WebResponse;
// Set necessary Request Headers
for (int i = 0; i < HeadersSplit.Length; i = i + 2)
{
string HeaderPart1 = HeadersSplit[i].Replace("\"", "").Trim();
string HeaderPart2 = HeadersSplit[i + 1].Replace("\"", "").Trim();
if (HeaderPart1 == "Content-Type")
{
WebRequest.ContentType = HeaderPart2;
}
else if (HeaderPart1 == "Accept")
{
WebRequest.Accept = HeaderPart2;
}
else if (HeaderPart1 == "Authorization")
{
WebRequest.Headers["Authorization"] = HeaderPart2;
}
}
WebRequest.Headers.Add("cache-control", "no-cache");
// Add body to Request
using (var streamWriter = new StreamWriter(WebRequest.GetRequestStream()))
{
streamWriter.Write(Body);
streamWriter.Flush();
streamWriter.Close();
}
// Execute Request
WebResponse = (HttpWebResponse)WebRequest.GetResponse();
// Validate Response
if (WebResponse.StatusCode == HttpStatusCode.OK)
{
using (var streamReader = new StreamReader(WebResponse.GetResponseStream()))
{
Data = streamReader.ReadToEnd();
}
}
return Data;
}
What am I doing wrong here? The method doesn't have global data that is shared between tasks.

But you do have data that is shared between the tasks: the local varibleIdentifier and the method argument Job.
You are writing to a file using the Identifier in the file-name. If the lock is not in place, that piece of code will be running simultaniously.
The implications for Job can't be deduced from your question.
I think you can solve the identifier problem by doing this:
var identifier = Interlocked.Increment(ref Identifier);
Job.Identifier = identifier; // Use 'identifier', not 'Identifier'
// write to file
string json = ...;
string FileName = OffSet.ToString() + "_" +
"MAMBU_LT_" + DateTime.Now.ToString("yyyyMMddHHmmss") + "_" +
identifier; // Use 'identifier', not 'Identifier'
...

How can I make this c# method multithreaded? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have this c# class that I am trying to make multi-threaded, or able to run 100 threads (requests?) at once.
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
string[] lines = File.ReadAllLines("C:\\checker/in.txt");
var accCount = File.ReadLines(#"C:\checker/in.txt").Count();
Console.Write("Accounts loaded: " + accCount);
Console.WriteLine();
foreach (string line in lines)
{
string[] account = line.Split(new char[] { ':' });
string user = account[0];
string pass = account[1];
addThreads(user, pass);
Threads.ForEach(t => t.Start());
Console.WriteLine();
}
// Suspend the screen.
Console.ReadLine();
}
public static List<Thread> Threads = new List<Thread>();
public static void addThreads(string user, string pass)
{
var checker = new Checker();
Threads.Clear();
Threads.Add(new Thread(() => { checker.checkAccount(user, pass); }));
Threads.Add(new Thread(() => { checker.checkAccount(user, pass); }));
Threads.Add(new Thread(() => { checker.checkAccount(user, pass); }));
Threads.Add(new Thread(() => { checker.checkAccount(user, pass); }));
Threads.Add(new Thread(() => { checker.checkAccount(user, pass); }));
Threads.Add(new Thread(() => { checker.checkAccount(user, pass); }));
Threads.Add(new Thread(() => { checker.checkAccount(user, pass); }));
}
}
public class Checker
{
//declare vars
string getUsername;
string getMember;
string getAuth;
string check;
public void checkAccount(string username, string password)
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
byte[] data = Encoding.ASCII.GetBytes(
$"username={username}&password={password}&mod=www&ssl=1&dest=account_settings.ws");
WebRequest request = WebRequest.Create("https://secure.runescape.com/m=weblogin/login.ws");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
using (Stream stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
string responseContent = null;
using (WebResponse response = request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
using (StreamReader sr99 = new StreamReader(stream))
{
responseContent = sr99.ReadToEnd();
}
}
}
//parse captcha
string patternCaptcha = #"Please\s*complete\s*the\s*reCAPTCHA\s*box";
string inputCaptcha = responseContent;
Match matchCaptcha = Regex.Match(inputCaptcha, patternCaptcha);
string captcha = matchCaptcha.Value;
if (captcha == "Please complete the reCAPTCHA box")
{
captcha = "true";
Console.Write("captcha,captcha,captcha,captcha");
Console.WriteLine();
//return "captcha,captcha,captcha,captcha";
}
else
{
//parse valid/invalid
string patternCheck = #"Your\s*login\s*or\s*password\s*was\s*incorrect";
string inputCheck = responseContent;
Match matchCheck = Regex.Match(inputCheck, patternCheck);
check = matchCheck.Value;
if (check == "Your login or password was incorrect")
{
check = "Invalid";
}
else
{
check = "Valid";
//parse display name
string pattern = #"(<span.*class=.header-top__name.>(.*?)</span>)";
string input = responseContent;
Match match = Regex.Match(input, pattern);
getUsername = match.Groups[2].Value;
byte[] bytes = Encoding.Default.GetBytes(getUsername);
getUsername = Encoding.UTF8.GetString(bytes);
getUsername = getUsername.Replace("?", " ");
//parse member status
string patternMember = #"(Currently\s*Not\s*a\s*Member)";
string inputMember = responseContent;
Match matchMember = Regex.Match(inputMember, patternMember);
getMember = matchMember.Value;
if (getMember == "Currently Not a Member")
{
getMember = "Non Member";
}
else
{
getMember = "Member";
}
//parse auth status
string patternAuthUrl = #"iframe src=\""(.*?)""";
string inputAuthUrl = responseContent;
Match matchAuthUrl = Regex.Match(inputAuthUrl, patternAuthUrl);
string getAuthUrl = matchAuthUrl.Groups[1].Value;
using (WebClient client = new WebClient())
{
string authSource = client.DownloadString(getAuthUrl);
string patternAuth = #"RuneScape\s*Authenticator\s*is\s*disabled";
string inputAuth = authSource;
Match matchAuth = Regex.Match(inputAuth, patternAuth);
getAuth = matchAuth.Value;
if (getAuth == "RuneScape Authenticator is disabled")
{
getAuth = "Auth Disabled";
}
else
{
getAuth = "Authed";
}
}
}
captcha = "false";
string curldata = getUsername + "," + getMember + "," + getAuth + "," + check;
Console.Write(curldata);
Console.WriteLine();
}
}
}
}
Instead of making my program check once per few seconds per post webrequest, how can I make this happen 50-100 times at the same time? Is this possible? Or do I need to do this a different way?

You need to avoid using threads as each thread uses in excess of 1MB of RAM and they are slow to create. You really want to use tasks (TPL) or observables (Rx).
In this case it is quite straight forward to use tasks.
Try this code:
string[] lines = File.ReadAllLines("C:\\checker/in.txt");
var accCount = lines.Count();
Console.Write("Accounts loaded: " + accCount);
Console.WriteLine();
var checker = new Checker();
var tasks =
from line in lines
let account = line.Split(new char[] { ':' })
let user = account[0]
let pass = account[0]
select Task.Factory.StartNew(() => checker.checkAccount(user, pass));
Task.WaitAll(tasks.ToArray());
Console.ReadLine();
That will read the text file and queue up a set of tasks to be run to check each line. The Task.WaitAll pauses the code until all of the tasks are completed.
This make efficient use of the thread-pool so that you're not wasting valuable resources starting up threads.
Your checkAccount is also not thread-safe at the moment. You need to move the field-level variables to be inside your method. It should look something like this:
public void checkAccount(string username, string password)
{
string getUsername;
string getMember;
string getAuth;
string check;

It's pretty simple first you need to do a method to call the threads
public void CallingThreadsMethod()
{
ThreadStart ts = new ThreadStart(SomeFunction);
Thread t = Thread(ts);
t.IsBackground = true;
t.Start();
}
void Somefunction(){}
or if you want many threads you can make a thread list
public static List<Thread> Threads = new List<Thread>();
public static void addThreads()
{
Threads.Clear();
Threads.Add(new Thread(Method1));
Threads.Add(new Thread(Method2));
}
And start it in you'r main function
Vars.addThreads();
Vars.Threads.ForEach(t => t.Start());
but if you're using windows forms or wpf i recommend using BackgroundWorkers

How to use cursors to iterate through the result set of GET followers/list in Twitter API using c#?

I have been trying a way to get the data in the next page of the result set of GET followers/list API call. I can get the default data set with the data of first 20 followers and not the others. To get the data of other followers i have to access the next page using the next_cursor but it's not working. I tried using the pseudo-code mentioned in this link. https://dev.twitter.com/docs/misc/cursoring
Is it a must to use this(this is mentioned in the dev. site):
var api-path = "https://api.twitter.com/1.1/endpoint.json?screen_name=targetUser"
Because I have been using the resource URL as,
var resource_url = "https://api.twitter.com/1.1/followers/list.json";
and I tried appending the next_cursor to the same resource URL.
var url_with_cursor = resource_url + "&cursor=" + 1463101490306580067;
and then created the request.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url_with_cursor);
but I'm getting an exception in this line when getting the response.
WebResponse response = request.GetResponse();
The error I'm getting is
The Remote Server returned an Error 401 Unauthorized
Can someone tell the exact way to do cursor-ing, or the exact way to include the cursor in the request. I'm using a asp.net C# web application.
Here's my code, The oauth_token, oauth_token_secret, oauth_consumer_key, oauth_consumer_secret, oauth_version and oauth_signature_method are defined in my application
var resource_url = "https://api.twitter.com/1.1/followers/list.json";
var cursor = "-1";
do
{
var url_with_cursor = resource_url + "&cursor=" + cursor;
// unique request details
var oauth_nonce = Convert.ToBase64String(
new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
var timeSpan = DateTime.UtcNow
- new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);
var oauth_timestamp = Convert.ToInt64(timeSpan.TotalSeconds).ToString();
// create oauth signature
var baseFormat = "oauth_consumer_key={0}&oauth_nonce={1}&oauth_signature_method={2}" +
"&oauth_timestamp={3}&oauth_token={4}&oauth_version={5}";
var baseString = string.Format(baseFormat,
oauth_consumer_key,
oauth_nonce,
oauth_signature_method,
oauth_timestamp,
oauth_token,
oauth_version
//,Uri.EscapeDataString(status)
);
baseString = string.Concat("GET&", Uri.EscapeDataString(resource_url), "&", Uri.EscapeDataString(baseString));
var compositeKey = string.Concat(Uri.EscapeDataString(oauth_consumer_secret),
"&", Uri.EscapeDataString(oauth_token_secret));
string oauth_signature;
using (HMACSHA1 hasher = new HMACSHA1(ASCIIEncoding.ASCII.GetBytes(compositeKey)))
{
oauth_signature = Convert.ToBase64String(
hasher.ComputeHash(ASCIIEncoding.ASCII.GetBytes(baseString)));
}
// create the request header
var headerFormat = "OAuth oauth_nonce=\"{0}\", oauth_signature_method=\"{1}\", " +
"oauth_timestamp=\"{2}\", oauth_consumer_key=\"{3}\", " +
"oauth_token=\"{4}\", oauth_signature=\"{5}\", " +
"oauth_version=\"{6}\"";
var authHeader = string.Format(headerFormat,
Uri.EscapeDataString(oauth_nonce),
Uri.EscapeDataString(oauth_signature_method),
Uri.EscapeDataString(oauth_timestamp),
Uri.EscapeDataString(oauth_consumer_key),
Uri.EscapeDataString(oauth_token),
Uri.EscapeDataString(oauth_signature),
Uri.EscapeDataString(oauth_version)
);
// make the request
ServicePointManager.Expect100Continue = false;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url_with_cursor);
request.Headers.Add("Authorization", authHeader);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
WebResponse response = request.GetResponse();
string result = new StreamReader(response.GetResponseStream()).ReadToEnd();
JObject j = JObject.Parse(result);
JArray data = (JArray)j["users"];
cursor = (String)j["next_cursor_str"];
} while (!cursor.Equals("0"));
Thanks.

Tweetinvi would make it somewhat easier for you.
Please visit https://github.com/linvi/tweetinvi/wiki/Get-All-Followers-Code to have an idea on how to do it including RateLimit handling.
Without considering the RateLimits, you could simply use the following code.
long nextCursor = -1;
do
{
var query = string.Format("https://api.twitter.com/1.1/followers/ids.json?screen_name={0}", username);
var results = TwitterAccessor.ExecuteCursorGETCursorQueryResult<IIdsCursorQueryResultDTO>(query, cursor: cursor).ToArray();
if (results.Any())
{
nextCursor = results.Last().NextCursor;
}
else
{
nextCursor = -1;
}
}
while (nextCursor != -1 && nextCursor != 0);

you should make a different call to authenticate, by an authorization request. Once that has been granted, you can call the webresponse with the cursor. See my sample code below (Take special note to the StartCreateCall method, where the authentication is done. The data from Twitter is then retrieved by the CallData method):
public partial class twitter_followers : System.Web.UI.Page
{
public string strTwiterFollowers { get; set; }
private List<TwiterFollowers> listFollowers = new List<TwiterFollowers>();
private string screen_name = string.Empty;
// oauth application keys
private string oauth_consumer_key = string.Empty;
private string oauth_consumer_secret = string.Empty;
// oauth implementation details
private string resource_urlFormat = "https://api.twitter.com/1.1/followers/list.json?screen_name={0}&cursor={1}";
// unique request details
private string oauth_nonce = Convert.ToBase64String(new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
protected void Page_Load(object sender, EventArgs e)
{
//just get your request parameters from the config file.
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => screen_name)])) {
screen_name = ConfigurationManager.AppSettings[GetVariableName(() => screen_name)];
}
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_key)]))
{
oauth_consumer_key = ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_key)];
}
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_secret)]))
{
oauth_consumer_secret = ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_secret)];
}
StartCreateCall();
}
// Do the authenticate by an authorization request
private void StartCreateCall() {
// You need to set your own keys and screen name
var oAuthUrl = "https://api.twitter.com/oauth2/token";
// Do the Authenticate
var authHeaderFormat = "Basic {0}";
var authHeader = string.Format(authHeaderFormat,
Convert.ToBase64String(Encoding.UTF8.GetBytes(Uri.EscapeDataString(oauth_consumer_key) + ":" +
Uri.EscapeDataString((oauth_consumer_secret)))
));
var postBody = "grant_type=client_credentials";
HttpWebRequest authRequest = (HttpWebRequest)WebRequest.Create(oAuthUrl);
authRequest.Headers.Add("Authorization", authHeader);
authRequest.Method = "POST";
authRequest.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
authRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
using (Stream stream = authRequest.GetRequestStream())
{
byte[] content = ASCIIEncoding.ASCII.GetBytes(postBody);
stream.Write(content, 0, content.Length);
}
authRequest.Headers.Add("Accept-Encoding", "gzip");
WebResponse authResponse = authRequest.GetResponse();
// deserialize into an object
TwitAuthenticateResponse twitAuthResponse;
using (authResponse)
{
using (var reader = new StreamReader(authResponse.GetResponseStream()))
{
JavaScriptSerializer js = new JavaScriptSerializer();
var objectText = reader.ReadToEnd();
twitAuthResponse = JsonConvert.DeserializeObject<TwitAuthenticateResponse>(objectText);
}
}
//now we have been granted access and got a token type with authorization token from Twitter
//in the form of a TwitAuthenticateResponse object, we can retrieve the data recursively with a cursor
CallData(twitAuthResponse, authHeader, cursor);
int totalFollowers = listFollowers.Count;
lblTotalFollowers.Text = screen_name + " has " + listFollowers.Count + " Followers";
Random objRnd = new Random();
List<TwiterFollowers> randomFollowers = listFollowers.OrderBy(item => objRnd.Next()).ToList<TwiterFollowers>();
foreach (TwiterFollowers tw in randomFollowers)
{
strTwiterFollowers = strTwiterFollowers + "<li><a target='_blank' title='" + tw.ScreenName + "' href=https://twitter.com/" + tw.ScreenName + "><img src='" + tw.ProfileImage + "'/><span>" + tw.ScreenName + "</span></a></li>";
}
}
//Retrieve the data from Twitter recursively with a cursor
private void CallData(TwitAuthenticateResponse twitAuthResponse, string authHeader, string cursor)
{
try {
JObject j = GetJSonObject(twitAuthResponse, authHeader, cursor);
JArray data = (JArray)j["users"];
if (data != null)
{
foreach (var item in data)
{
TwiterFollowers objTwiterFollowers = new TwiterFollowers();
objTwiterFollowers.ScreenName = item["screen_name"].ToString().Replace("\"", "");
objTwiterFollowers.ProfileImage = item["profile_image_url"].ToString().Replace("\"", "");
objTwiterFollowers.UserId = item["id"].ToString().Replace("\"", "");
listFollowers.Add(objTwiterFollowers);
}
JValue next_cursor = (JValue)j["next_cursor"];
if (long.Parse(next_cursor.Value.ToString()) > 0)
{
//Get the following data from Twitter with the next cursor
CallData(twitAuthResponse, authHeader, next_cursor.Value.ToString());
}
}
} catch (Exception ex)
{
//do nothing
}
}
private JObject GetJSonObject(TwitAuthenticateResponse twitAuthResponse, string authHeader, string cursor)
{
string resource_url = string.Format(resource_urlFormat, screen_name, cursor);
if (string.IsNullOrEmpty(cursor))
{
resource_url = resource_url.Substring(0, resource_url.IndexOf("&cursor"));
}
HttpWebRequest fRequest = (HttpWebRequest)WebRequest.Create(resource_url);
var timelineHeaderFormat = "{0} {1}";
fRequest.Headers.Add("Authorization", string.Format(timelineHeaderFormat, twitAuthResponse.token_type, twitAuthResponse.access_token));
fRequest.Method = "Get";
WebResponse response = fRequest.GetResponse();
string result = new StreamReader(response.GetResponseStream()).ReadToEnd();
return JObject.Parse(result);
}
private string GetVariableName<T>(Expression<Func<T>> expr)
{
var body = (MemberExpression)expr.Body;
return body.Member.Name;
}
private class TwitAuthenticateResponse
{
public string token_type { get; set; }
public string access_token { get; set; }
}
private class TwiterFollowers
{
public string ScreenName { get; set; }
public string ProfileImage { get; set; }
public string UserId { get; set; }
}
}

You are getting "401 Unauthorized"
Did you check you are setting everything right?
Credentials? Check both queries on fiddler.

how to send a http basic auth post?

I have been given the task to create a http post using basic auth.
I am developing in C# in an asp.net MVC application.
I have also been given this example.
{
POST /v2/token_endpoint HTTP/1.1
Authorization: Basic Y2xpZW50X2lkOmNsaWVudF9zZWNyZXQ=
Accept: application/json
Content-Type: application/x-www-form-urlencoded
User-Agent: Java/1.6.0_33
Host: api.freeagent.com
Connection: close
Content-Length: 127
grant_type=authorization_code&code=12P3AsFZXwXjd7SLOE1dsaX8oCgix&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foauth
}
My question is how do i code this in C#?
If more information is need just ask, thanks in advance
edit: I have made some progress but I have not added the grant_type
public void AccessToken(string code)
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(ApiKey + ":" + ApiSecret));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
client.Headers[HttpRequestHeader.UserAgent] = "Java/1.6.0_33";
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.Connection] = "close";
client.Headers["grant_type"] = "authorization_code";
var result = client.DownloadString(url);
}
So how do i add: grant_type=authorization_code&code=12P3AsFZXwXjd7SLOE1dsaX8oCgix&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foauth to the post?

You can find here two samples how to make basic auth request with WebRequest and WebClient classes:
http://grahamrhay.wordpress.com/2011/08/22/making-a-post-request-in-c-with-basic-authentication/
http://anishshenoy57.wordpress.com/2013/01/22/basic-http-authentication-using-c/
Basically basic auth it's just Base64(username:password), so it's easy to implement it.
UPDATE1
Here is a sample based on your method:
public void AccessToken(string code)
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(ApiKey + ":" + ApiSecret));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
client.Headers[HttpRequestHeader.UserAgent] = "Java/1.6.0_33";
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.Connection] = "close";
client.Headers["grant_type"] = "authorization_code";
string data = string.Format(
"grant_type=authorization_code&code={0}&redirect_uri=http%3A%2F%2Flocalhost%3A8080",
code);
var result = client.UploadString(url, data);
}
The only different in calling method in WebClient. DownloadString will do GET request, but for POST you need to use UploadString method

I too have struggled with this, but I have now made it to the other side!
The one that held me up the most was that if you specify a redirect URL in the Auth Token request, you must also specify it in the Access Token request with the same URL, even though it's not used.
I started off with someones partial wrapper, and have evolved it almost beyond recognition. Hopefully this wrapper (complete with auth and access token code) will help others. I've only used it to create invoices and invoice items so far, and the other functions are untested, but it should get you your tokens and to the point where you can debug the requests.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;
using System.Web.Script.Serialization;
namespace ClientZone.External
{
public class FreeAgent
{
public enum Methods
{
POST = 1,
PUT
}
public enum Status
{
Draft = 1,
Sent,
Cancelled
}
private string _subDomain;
private string _identifier;
private string _secret;
private string _redirectURI;
public static string AuthToken = "";
private static string _accessToken = "";
private static string _refreshToken = "";
private static DateTime _refreshTime;
public FreeAgent(string identifier, string secret, string subdomain, string redirectURI)
{
_subDomain = subdomain;
_identifier = identifier;
_secret = secret;
_redirectURI = redirectURI; // If this was specified in the Auth Token call, you must specify it here, and it must be the same
}
public bool GetAccessToken()
{
try
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(_identifier + ":" + _secret));
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.KeepAlive] = "true";
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded;charset=UTF-8";
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
System.Net.ServicePointManager.Expect100Continue = false;
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
client.Headers[HttpRequestHeader.AcceptLanguage] = "en-US,en;q=0.8";
var result = "";
if (!(_accessToken == "" && _refreshToken != ""))
{
string data = string.Format("grant_type=authorization_code&code={0}&redirect_uri={1}", AuthToken, _redirectURI);
result = client.UploadString(url, data);
}
else
{
// Marking the access token as blank and the refresh token as not blank
// is a sign we need to get a refresh token
string data = string.Format("grant_type=refresh_token&refresh_token={0}", _refreshToken);
result = client.UploadString(url, data);
}
JavaScriptSerializer json_serializer = new JavaScriptSerializer();
var results_list = (IDictionary<string, object>)json_serializer.DeserializeObject(result);
_accessToken = results_list["access_token"].ToString();
int secondsUntilRefresh = Int32.Parse(results_list["expires_in"].ToString());
_refreshTime = DateTime.Now.AddSeconds(secondsUntilRefresh);
if (results_list.Any(x => x.Key == "refresh_token"))
{
_refreshToken = results_list["refresh_token"].ToString();
}
if (_accessToken == "" || _refreshToken == "" || _refreshTime == new DateTime())
{
return false;
}
}
catch
{
return false;
}
return true;
}
private HttpStatusCode SendWebRequest(Methods method, string URN, string request, out string ResponseData)
{
if (_accessToken == "")
{
// The access token has not been retrieved yet
GetAccessToken();
}
else
{
if (_refreshTime != new DateTime() && _refreshTime < DateTime.Now) {
// The token has expired and we need to refresh it
_accessToken = "";
GetAccessToken();
}
}
if (_accessToken != "")
{
try
{
WebClient client = new WebClient();
string url = "https://api.freeagentcentral.com/v2/" + URN;
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36";
client.Headers[HttpRequestHeader.Authorization] = "Bearer " + _accessToken;
client.Headers[HttpRequestHeader.ContentType] = "application/json";
client.Headers[HttpRequestHeader.Accept] = "application/json";
if (method == Methods.POST || method == Methods.PUT)
{
string data = request;
var result = client.UploadString(url, method.ToString(), data);
ResponseData = result;
return HttpStatusCode.Created;
}
else
{
var result = client.DownloadString(url);
ResponseData = result;
return HttpStatusCode.OK;
}
}
catch (WebException e)
{
if (e.GetType().Name == "WebException")
{
WebException we = (WebException)e;
HttpWebResponse response = (System.Net.HttpWebResponse)we.Response;
ResponseData = response.StatusDescription;
return response.StatusCode;
}
else
{
ResponseData = e.Message;
if (e.InnerException != null)
{
ResponseData = ResponseData + " - " + e.InnerException.ToString();
}
return HttpStatusCode.SeeOther;
}
}
}
else
{
ResponseData = "Access Token could not be retrieved";
return HttpStatusCode.SeeOther;
}
}
private int ExtractNewID(string resp, string URN)
{
if (resp != null && resp.Trim() != "")
{
JavaScriptSerializer json_serializer = new JavaScriptSerializer();
var results_list = (IDictionary<string, object>)json_serializer.DeserializeObject(resp);
if (results_list.Any(x => x.Key == "invoice"))
{
var returnInvoice = (IDictionary<string, object>)results_list["invoice"];
if (returnInvoice["created_at"].ToString() != "")
{
string returnURL = returnInvoice["url"].ToString();
if (returnURL.Contains("http"))
{
return int.Parse(returnURL.Remove(0, ("https://api.freeagentcentral.com/v2/" + URN).Length + 1));
}
else
{
return int.Parse(returnURL.Remove(0, URN.Length + 2));
}
}
}
}
return -1;
}
public int CreateContact(string firstName, string lastName, string emailAddress, string street, string city, string state, string postcode, string country)
{
StringBuilder request = new StringBuilder();
request.Append("{\"contact\":{");
request.Append("\"first-name\":");
request.Append(firstName);
request.Append("\",");
request.Append("\"last-name\":");
request.Append(lastName);
request.Append("\",");
request.Append("\"email\":");
request.Append(emailAddress);
request.Append("\",");
request.Append("\"address1\":");
request.Append(street);
request.Append("\",");
request.Append("\"town\":");
request.Append(city);
request.Append("\",");
request.Append("\"region\":");
request.Append(state);
request.Append("\",");
request.Append("\"postcode\":");
request.Append(postcode);
request.Append("\",");
request.Append("\"country\":");
request.Append(country);
request.Append("\"");
request.Append("}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.POST, "contacts", request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
return ExtractNewID(returnData, "contacts");
}
else
{
return -1;
}
}
public int CreateInvoice(int contactID, DateTime invoiceDate, int terms, string reference, string comments, string currency = "GBP")
{
StringBuilder request = new StringBuilder();
request.Append("{\"invoice\":{");
request.Append("\"dated_on\": \"");
request.Append(invoiceDate.ToString("yyyy-MM-ddTHH:mm:00Z"));
request.Append("\",");
if (!string.IsNullOrEmpty(reference))
{
request.Append("\"reference\": \"");
request.Append(reference);
request.Append("\",");
}
if (!string.IsNullOrEmpty(comments))
{
request.Append("\"comments\": \"");
request.Append(comments);
request.Append("\",");
}
request.Append("\"payment_terms_in_days\": \"");
request.Append(terms);
request.Append("\",");
request.Append("\"contact\": \"");
request.Append(contactID);
request.Append("\",");
if (currency == "EUR")
{
request.Append("\"ec_status\": \"");
request.Append("EC Services");
request.Append("\",");
}
request.Append("\"currency\": \"");
request.Append(currency);
request.Append("\"");
request.Append("}}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.POST, "invoices", request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
return ExtractNewID(returnData, "invoices");
}
else
{
return -1;
}
}
public bool ChangeInvoiceStatus(int invoiceID, Status status)
{
string returnData = string.Empty;
HttpStatusCode resp = SendWebRequest(Methods.PUT, "invoices/" + invoiceID.ToString() + "/transitions/mark_as_" + status.ToString().ToLower(), string.Empty, out returnData);
return false;
}
public int CreateInvoiceItem(int invoiceID, string itemType, float price, int quantity, float taxRate, string description)
{
StringBuilder request = new StringBuilder();
request.Append("{\"invoice\":{");
request.Append("\"invoice_items\":[{");
request.Append("\"item_type\": \"");
request.Append(itemType);
request.Append("\",");
request.Append("\"price\": \"");
request.Append(price.ToString("0.00"));
request.Append("\",");
request.Append("\"quantity\": \"");
request.Append(quantity);
request.Append("\",");
request.Append("\"sales_tax_rate\": \"");
request.Append(taxRate.ToString("0.00"));
request.Append("\",");
request.Append("\"description\": \"");
request.Append(description);
request.Append("\"");
request.Append("}]");
request.Append("}");
request.Append("}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.PUT, "invoices/" + invoiceID.ToString(), request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
// Invoice items is an update call to an invoice, so we just get a 200 OK with no response body to extract an ID from
return 0;
}
else
{
return -1;
}
}
}
}
To use this in an MVC setup you need to detect in the Index whether the login is necessary or not. If it is, you tell FreeAgent to come back to another ActionResult (I've used Auth) to catch the redirect and store the code.
public ActionResult Index()
{
if (MyNamespace.External.FreeAgent.AuthToken == "")
{
return Redirect("https://api.freeagent.com/v2/approve_app?redirect_uri=" + Request.Url.Scheme + "://" + Request.Url.Authority + "/Invoice/Auth" + "&response_type=code&client_id=1234567890123456789012");
}
else
{
return View();
}
}
public ActionResult Auth()
{
if (Request.Params.Get("code").ToString() != "")
{
ClientZone.External.FreeAgent.AuthToken = Request.Params.Get("code").ToString();
}
return View("Index");
}
Then to use the functions all you need is
MyNameSpace.External.FreeAgent FA = new ClientZone.External.FreeAgent("abcdefghhijklmnopqrstu", "1234567890123456789012", "acme", "http://localhost:52404/Invoice/Auth");
int newID = FA.CreateInvoice(54321, InvoiceDate, 30, "", "", "GBP");
if (newID != -1)
{
FA.CreateInvoiceItem(newID, unit, rate, quantity, number, description);
}
There is some UK/EU specific coding in the CreateInvoice. I'm not sure how custom or standard this is, but I've left it in as it's easier to delete it than recreate it.

Server Hosting Error in C#

I developed a eCommerce Website using ASP.Net 3.5 and C#. It is working good with Visual Stdio 2010. I uploaded my website to my domain under the public folder.
It show the error like below:
Click for full-size, or here's the stack trace fragment from the local error page.
NullReferenceException: Object reference not set to an instance of an object
SageFrame.Framework.PageBase.OptimizeJs(List`1 lstJsColl, Int32 Mode) +7940
SageFrame.Framework.PageBase.LoadModuleJs() +944
SageFrame.Framework.PageBase.OnPreRender(EventArgs e) +233
System.Web.UI.PreRenderRecursiveInternal() +107
It was working fine before!
The code of the page_Load() is:
protected void Page_Load(object sender, EventArgs e)
{
string selectedCurrency = string.Empty;
string MainCurrency = string.Empty;
try
{
StoreSettingConfig ssc = new StoreSettingConfig();
MainCurrency = ssc.GetStoreSettingsByKey(StoreSetting.MainCurrency, GetStoreID, GetPortalID, GetCurrentCultureName);
if (Session["SelectedCurrency"] != null && Session["SelectedCurrency"] != "")
{
selectedCurrency = Session["SelectedCurrency"].ToString();
}
else
{
selectedCurrency = MainCurrency;
}
string islive = Request.Form["custom"];
string test = string.Empty;
const string strSandbox = "https://www.sandbox.paypal.com/cgi-bin/webscr";
const string strLive = "https://www.paypal.com/cgi-bin/webscr";
test = bool.Parse(islive.Split('#')[6]) ? strSandbox : strLive;
var req = (HttpWebRequest)WebRequest.Create(test);
//Set values for the request back
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
byte[] param = Request.BinaryRead(HttpContext.Current.Request.ContentLength);
string strRequest = Encoding.ASCII.GetString(param);
strRequest += "&cmd=_notify-validate";
req.ContentLength = strRequest.Length;
var streamOut = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(strRequest);
streamOut.Close();
var streamIn = new StreamReader(req.GetResponse().GetResponseStream());
string strResponse = streamIn.ReadToEnd();
streamIn.Close();
if (strResponse == "VERIFIED")
{
string payerEmail = Request.Form["payer_email"];
string paymentStatus = Request.Form["payment_status"];
string receiverEmail = Request.Form["receiver_email"];
string amount = Request.Form["mc_gross"];
string invoice = Request.Form["invoice"];
string addressName = Request.Form["address_name"];
string addressStreet = Request.Form["address_street"];
string addressCity = Request.Form["address_city"];
string addressZip = Request.Form["address_zip"];
string addressCountry = Request.Form["address_country"];
string transID = Request.Form["txn_id"];
string custom = Request.Form["custom"];
string[] ids = custom.Split('#');
int orderID = int.Parse(ids[0]);
int storeID = int.Parse(ids[1]);
int portalID = int.Parse(ids[2]);
string userName = ids[3];
int customerID = int.Parse(ids[4]);
string sessionCode = ids[5];
string pgid = ids[7];
var tinfo = new TransactionLogInfo();
var tlog = new TransactionLog();
tinfo.TransactionID = transID;
tinfo.AuthCode = "";
tinfo.TotalAmount = decimal.Parse(amount);
tinfo.ResponseCode = "1";
tinfo.ResponseReasonText = "";
tinfo.OrderID = orderID;
tinfo.StoreID = storeID;
tinfo.PortalID = portalID;
tinfo.AddedBy = userName;
tinfo.CustomerID = customerID;
tinfo.SessionCode = sessionCode;
tinfo.PaymentGatewayID = int.Parse(pgid);
tinfo.PaymentStatus = paymentStatus;
tinfo.PayerEmail = payerEmail;
tinfo.CreditCard = "";
tinfo.RecieverEmail = receiverEmail;
tinfo.CurrencyCode = selectedCurrency;
tlog.SaveTransactionLog(tinfo);
if (paymentStatus.Equals("Completed"))
{
var paypalobj = new PayPalHandler();
paypalobj.ParseIPN(orderID, transID, paymentStatus, storeID, portalID, userName, customerID, sessionCode);
}
}
else if (strResponse == "INVALID")
{
//log for manual investigation
}
else
{
//log response/ipn data for manual investigation
}
// }
}
catch (Exception ex)
{
ProcessException(ex);
// throw new Exception("This Page is not accessible!");
}
}

From the error message, it appears that your store library cannot be loaded. This could be because it isn't installed on the server or the trust level is too low.
Either way, your hosting provider could assist with the setup.