HtmlAgilityPack obtain Title and meta - c#

I try to practice "HtmlAgilityPack ", but I am having some issues regarding this. here's what I coded, but I can not get correctly the title and the description of a web page ...
If someone can enlighten me on my mistake :)
...
public static void Main(string[] args)
{
string link = null;
string str;
string answer;
int curloc; // holds current location in response
string url = "http://stackoverflow.com/";
try
{
do
{
HttpWebRequest HttpWReq = (HttpWebRequest)WebRequest.Create(url);
HttpWReq.UserAgent = #"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5";
HttpWebResponse HttpWResp = (HttpWebResponse)HttpWReq.GetResponse();
//url = null; // disallow further use of this URI
Stream istrm = HttpWResp.GetResponseStream();
// Wrap the input stream in a StreamReader.
StreamReader rdr = new StreamReader(istrm);
// Read in the entire page.
str = rdr.ReadToEnd();
curloc = 0;
//WebPage result;
do
{
// Find the next URI to link to.
link = FindLink(str, ref curloc); //return the good link
Console.WriteLine("Title found: " + curloc);
//title = Title(str, ref curloc);
if (link != null)
{
Console.WriteLine("Link found: " + link);
using (System.Net.WebClient client = new System.Net.WebClient())
{
HtmlDocument htmlDoc = new HtmlDocument();
var html = client.DownloadString(url);
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
var htmlElement = htmlDoc.DocumentNode.Element("html");
HtmlNode node = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (node != null)
{
string desc = node.GetAttributeValue("content", "");
Console.Write("DESCRIPTION: " + desc);
}
else
{
Console.WriteLine("No description");
}
var titleElement =
htmlDoc.DocumentNode
.Element("html")
.Element("head")
.Element("title");
if (titleElement != null)
{
string title = titleElement.InnerText;
Console.WriteLine("Titre: {0}", title);
}
else
{
Console.WriteLine("no Title");
}
Console.Write("Done");
}
Console.Write("Link, More, Quit?");
answer = Console.ReadLine();
}
else
{
Console.WriteLine("No link found.");
break;
}
} while (link.Length > 0);
// Close the Response.
HttpWResp.Close();
} while (url != null);
}
catch{ ...}
Thanks in advance :)


Go about it this way:
HtmlNode mdnode = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (mdnode != null)
{
HtmlAttribute desc;
desc = mdnode.Attributes["content"];
string fulldescription = desc.Value;
Console.Write("DESCRIPTION: " + fulldescription);
}


I think your problem is here:
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
It should be:
htmlDoc.LoadHtml(html); //chargement de HTMLAgilityPack
LoadHtml expects a string with the HTML source, not the url.
And probably you want to change:
var html = client.DownloadString(url);
to
var html = client.DownloadString(link);


Have you used a breakpoint and gone line for line to see where the error might be occurring?
If you have, then Try something like this:
string result = string.Empty;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.google.com");
request.Method = "GET";
try
{
using (var stream = request.GetResponse().GetResponseStream())
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
result = reader.ReadToEnd();
}
}
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(result);
Then carry over the rest of your code below the htmlDoc.LoadHtml


[HttpPost]
public ActionResult Create(WebSite website)
{
string desc = HtmlAgi(website.Url, "description");
string keyword = HtmlAgi(website.Url, "Keywords");
if (ModelState.IsValid)
{
var userId = ((CustomPrincipal)User).UserId;
r.Create(new WebSite
{
Description = desc,
Tags = keyword,
Url = website.Url,
UserId = userId,
Category = website.Category
});
return RedirectToAction("Index");
}
return View(website);
}


public string HtmlAgi(string url, string key)
{
//string.Format
var Webget = new HtmlWeb();
var doc = Webget.Load(url);
HtmlNode ourNode = doc.DocumentNode.SelectSingleNode(string.Format("//meta[#name='{0}']", key));
if (ourNode != null)
{
return ourNode.GetAttributeValue("content", "");
}
else
{
return "not fount";
}
}

Related

Is there a function to skip repeating values being written into text file?

I'm trying to get all hyperlinks from an array that match the word "contacts" into .csv text file. The problem is if it finds another contact hyperlink on the same website it will print it again. How do I fix this? Also how to web scrape multiple websites for specific div that contains keywords: ("Phone number", "Address", "E-mail" and etc.)?
private void contactbutton_Click(object sender, EventArgs e)
{
ArrayList domainlist = new ArrayList();
const Int32 BufferSize = 128;
// -- Location of domain list file --
using (var fileStream = File.OpenRead("C:/Users/Username/Desktop/domains.txt"))
using (var streamReader = new StreamReader(fileStream, Encoding.UTF8, true, BufferSize))
{
String line;
while ((line = streamReader.ReadLine()) != null)
domainlist.Add(line);
}
foreach (string s in domainlist)
{
SearchHyperlinks("https://" + s);
}
}
public static void SearchHyperlinks(string address4)
{
HtmlWeb hw = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = hw.Load(address4);
String GetAbsoluteUrlString(string baseUrl, string url)
{
var uri = new Uri(url, UriKind.RelativeOrAbsolute);
if (!uri.IsAbsoluteUri)
uri = new Uri(new Uri(baseUrl), uri);
return uri.ToString();
}
try
{
using (var w = new StreamWriter("C:/Users/Username/Desktop/hyperlink.csv"))
foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[starts-with(., 'Contacts') or starts-with(., 'contacts') or starts-with(., 'CONTACTS') or starts-with (., 'Shop Contacts')]"))
{
String hrefValue = link.Attributes["href"].Value;
if (hrefValue != null)
{
String fullhref = GetAbsoluteUrlString(address4, hrefValue);
Console.WriteLine(fullhref);
using (var textWriter = new StreamWriter("C:/Users/Username/Desktop/hyperlinks.csv", true))
{
var writer = new CsvWriter(textWriter, CultureInfo.InvariantCulture);
writer.Configuration.Delimiter = ",";
writer.WriteField(fullhref);
writer.NextRecord();
}
}
}
}
catch (System.NullReferenceException)
{
Console.WriteLine("Hyperlinks not found");
}

Rewrite your method to actually Search, and not Write:
public static IEnumerable<string> SearchHyperlinks(string address4)
{
var hw = new HtmlWeb();
var doc = hw.Load(address4);
String GetAbsoluteUrlString(string baseUrl, string url)
{
var uri = new Uri(url, UriKind.RelativeOrAbsolute);
if (!uri.IsAbsoluteUri)
uri = new Uri(new Uri(baseUrl), uri);
return uri.ToString();
}
var links = doc.DocumentNode.SelectNodes("//a[starts-with(., 'Contacts') or starts-with(., 'contacts') or starts-with(., 'CONTACTS') or starts-with (., 'Shop Contacts')]");
if(links == null)
yield return break;
foreach (var link in links)
{
var hrefValue = link.Attributes["href"].Value;
if (hrefValue != null)
{
var fullhref = GetAbsoluteUrlString(address4, hrefValue);
yield return fullhref;
}
}
}
Then you distinct values returned:
var distinct = SearchHyperlinks(input).Distinct();
Then you write them all wherever you want.

OVER_QUERY_LIMIT status found from Geocode

I have to update my database from geocode to get longitude and latitude of address stored in database.
For that I created one console application. Daily I can update 25000 records from google per user, And I am executing my console from different 3 machines everyday since last 4 to 5 days.
But I am getting OVER_QUERY_LIMIT status from today and I am not able to update database.
What should i do for this?
Here is my code to get Longitude-latitude of console:
protected static void FindLocation(string strLocation, out string lat, out string lng, out string Output)
{
StreamWriter myWriter;
System.Net.HttpWebRequest geoRequest;
string googResponse;
googResponse = "";
lat = "";
lng = "";
Output = "";
string abc = "";
string strRequestURL;
strRequestURL = "https://maps.googleapis.com/maps/api/geocode/xml?address=" + strLocation + "&sensor=false";
try
{
geoRequest = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(strRequestURL);
geoRequest.Method = "POST";
geoRequest.KeepAlive = true;
UTF8Encoding encoding = new UTF8Encoding();
byte[] bytes = encoding.GetBytes(strRequestURL);
geoRequest.ContentLength = bytes.Length;
using (Stream writeStream = geoRequest.GetRequestStream())
{
writeStream.Write(bytes, 0, bytes.Length);
writeStream.Close();
}
System.Net.HttpWebResponse geoResponse;
geoResponse = (System.Net.HttpWebResponse)geoRequest.GetResponse();
if (geoResponse != null)
{
XPathDocument document = new XPathDocument(geoResponse.GetResponseStream());
XPathNavigator navigator = document.CreateNavigator();
// get response status
XPathNodeIterator statusIterator = navigator.Select("/GeocodeResponse/status");
while (statusIterator.MoveNext())
{
if (statusIterator.Current.Value != "OK")
{
if (statusIterator.Current.Value == "OVER_QUERY_LIMIT")
{
}
Output = "Failed";
}
else
{
Output = statusIterator.Current.Value;
}
}
// get results
XPathNodeIterator resultIterator = navigator.Select("/GeocodeResponse/result");
while (resultIterator.MoveNext())
{
XPathNodeIterator geometryIterator = resultIterator.Current.Select("geometry");
while (geometryIterator.MoveNext())
{
XPathNodeIterator locationIterator = geometryIterator.Current.Select("location");
while (locationIterator.MoveNext())
{
XPathNodeIterator latIterator = locationIterator.Current.Select("lat");
while (latIterator.MoveNext())
{
lat = latIterator.Current.Value;
}
XPathNodeIterator lngIterator = locationIterator.Current.Select("lng");
while (lngIterator.MoveNext())
{
lng = lngIterator.Current.Value;
}
}
}
}
}
else
{
Output = "Failed";
}
}
catch (Exception ex)
{
Output = "Failed";
}
}

Problems with Xml parsing using C#

I am developing my first windows App and I am facing some Problems while parsing an Xml,the code is as shown below
public void TimeParsing(string lat, string lon)
{
string urlbeg = "http://api.geonames.org/timezone?lat=";
string urlmid = "&lng=";
string urlend = "&username=dheeraj_kumar";
WebClient downloader = new WebClient();
Uri uri = new Uri(urlbeg + lat + urlmid + lon + urlend, UriKind.Absolute);
downloader.DownloadStringCompleted += new DownloadStringCompletedEventHandler(TimeDownloaded);
//downloader.DownloadStringCompleted += new DownloadStringCompletedEventHandler(TimeDownloaded);
downloader.DownloadStringAsync(uri);
}
private void TimeDownloaded(object sender, DownloadStringCompletedEventArgs e)
{
if (e.Result == null || e.Error != null)
{
MessageBox.Show("Invalid");
}
else
{
XDocument document = XDocument.Parse(e.Result);
var data1 = from query in document.Descendants("geoname")
select new Country
{
CurrentTime = (string)query.Element("time"),
};
foreach (var d in data1)
{
time = d.CurrentTime;
MessageBox.Show(d.CurrentTime);
// country = d.CountryName;
}
}
}
The problem is that the Delegate TimeDownloaded is not being called. I used the same technique is parse a different URL and it was done easily but its not working in this case.Kindly Help me as I am pretty new to this field.
Thanks in advance.

Theres a few misses regarding fetching the nodes
The output is geonames/timezone/time, it's corrected below, also testable using the method DownloadStringTaskAsync instead
[TestClass]
public class UnitTest1
{
[TestMethod]
public async Task TestMethod1()
{
await TimeParsing("-33.8674869", "151.20699020000006");
}
public async Task TimeParsing(string lat, string lon)
{
var urlbeg = "http://api.geonames.org/timezone?lat=";
var urlmid = "&lng=";
var urlend = "&username=dheeraj_kumar";
var downloader = new WebClient();
var uri = new Uri(urlbeg + lat + urlmid + lon + urlend, UriKind.Absolute);
downloader.DownloadStringCompleted += TimeDownloaded;
var test = await downloader.DownloadStringTaskAsync(uri);
Console.WriteLine(test);
}
private void TimeDownloaded(object sender, DownloadStringCompletedEventArgs e)
{
if (e.Result == null || e.Error != null)
{
Console.WriteLine("Invalid");
}
else
{
var document = XDocument.Parse(e.Result);
var data1 = from query in document.Descendants("timezone")
select new Country
{
CurrentTime = (string)query.Element("time"),
};
foreach (var d in data1)
{
Console.WriteLine(d.CurrentTime);
}
}
}
}
internal class Country
{
public string CurrentTime { get; set; }
}
}

you can use the below mentioned code.
Uri uri = new Uri(urlbeg + lat + urlmid + lon + urlend, UriKind.Absolute);
HttpWebRequest WebReq = (HttpWebRequest)WebRequest.Create(uri);
//This time, our method is GET.
WebReq.Method = "GET";
//From here on, it's all the same as above.
HttpWebResponse WebResp = (HttpWebResponse)WebReq.GetResponse();
//Now, we read the response (the string), and output it.
Stream Answer = WebResp.GetResponseStream();
StreamReader _Answer = new StreamReader(Answer);
string s = _Answer.ReadToEnd();
XDocument document = XDocument.Parse(s);
var data1 = from query in document.Descendants("geoname")
select new Country
{
CurrentTime = (string)query.Element("time"),
};
foreach (var d in data1)
{
time = d.CurrentTime;
MessageBox.Show(d.CurrentTime);
// country = d.CountryName;
}
for Windows Phone 8 you have to implement the getResponse Method.
public static System.Threading.Tasks.Task<System.Net.WebResponse> GetResponseAsync(this System.Net.WebRequest wr)
{
return Task<System.Net.WebResponse>.Factory.FromAsync(wr.BeginGetResponse, wr.EndGetResponse, null);
}

how to send a http basic auth post?

I have been given the task to create a http post using basic auth.
I am developing in C# in an asp.net MVC application.
I have also been given this example.
{
POST /v2/token_endpoint HTTP/1.1
Authorization: Basic Y2xpZW50X2lkOmNsaWVudF9zZWNyZXQ=
Accept: application/json
Content-Type: application/x-www-form-urlencoded
User-Agent: Java/1.6.0_33
Host: api.freeagent.com
Connection: close
Content-Length: 127
grant_type=authorization_code&code=12P3AsFZXwXjd7SLOE1dsaX8oCgix&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foauth
}
My question is how do i code this in C#?
If more information is need just ask, thanks in advance
edit: I have made some progress but I have not added the grant_type
public void AccessToken(string code)
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(ApiKey + ":" + ApiSecret));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
client.Headers[HttpRequestHeader.UserAgent] = "Java/1.6.0_33";
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.Connection] = "close";
client.Headers["grant_type"] = "authorization_code";
var result = client.DownloadString(url);
}
So how do i add: grant_type=authorization_code&code=12P3AsFZXwXjd7SLOE1dsaX8oCgix&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Foauth to the post?

You can find here two samples how to make basic auth request with WebRequest and WebClient classes:
http://grahamrhay.wordpress.com/2011/08/22/making-a-post-request-in-c-with-basic-authentication/
http://anishshenoy57.wordpress.com/2013/01/22/basic-http-authentication-using-c/
Basically basic auth it's just Base64(username:password), so it's easy to implement it.
UPDATE1
Here is a sample based on your method:
public void AccessToken(string code)
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(ApiKey + ":" + ApiSecret));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
client.Headers[HttpRequestHeader.UserAgent] = "Java/1.6.0_33";
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.Connection] = "close";
client.Headers["grant_type"] = "authorization_code";
string data = string.Format(
"grant_type=authorization_code&code={0}&redirect_uri=http%3A%2F%2Flocalhost%3A8080",
code);
var result = client.UploadString(url, data);
}
The only different in calling method in WebClient. DownloadString will do GET request, but for POST you need to use UploadString method

I too have struggled with this, but I have now made it to the other side!
The one that held me up the most was that if you specify a redirect URL in the Auth Token request, you must also specify it in the Access Token request with the same URL, even though it's not used.
I started off with someones partial wrapper, and have evolved it almost beyond recognition. Hopefully this wrapper (complete with auth and access token code) will help others. I've only used it to create invoices and invoice items so far, and the other functions are untested, but it should get you your tokens and to the point where you can debug the requests.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;
using System.Web.Script.Serialization;
namespace ClientZone.External
{
public class FreeAgent
{
public enum Methods
{
POST = 1,
PUT
}
public enum Status
{
Draft = 1,
Sent,
Cancelled
}
private string _subDomain;
private string _identifier;
private string _secret;
private string _redirectURI;
public static string AuthToken = "";
private static string _accessToken = "";
private static string _refreshToken = "";
private static DateTime _refreshTime;
public FreeAgent(string identifier, string secret, string subdomain, string redirectURI)
{
_subDomain = subdomain;
_identifier = identifier;
_secret = secret;
_redirectURI = redirectURI; // If this was specified in the Auth Token call, you must specify it here, and it must be the same
}
public bool GetAccessToken()
{
try
{
string url = #"https://api.freeagent.com/v2/token_endpoint";
WebClient client = new WebClient();
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(_identifier + ":" + _secret));
client.Headers[HttpRequestHeader.Host] = "api.freeagent.com";
client.Headers[HttpRequestHeader.KeepAlive] = "true";
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36";
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded;charset=UTF-8";
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;
System.Net.ServicePointManager.Expect100Continue = false;
client.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
client.Headers[HttpRequestHeader.AcceptLanguage] = "en-US,en;q=0.8";
var result = "";
if (!(_accessToken == "" && _refreshToken != ""))
{
string data = string.Format("grant_type=authorization_code&code={0}&redirect_uri={1}", AuthToken, _redirectURI);
result = client.UploadString(url, data);
}
else
{
// Marking the access token as blank and the refresh token as not blank
// is a sign we need to get a refresh token
string data = string.Format("grant_type=refresh_token&refresh_token={0}", _refreshToken);
result = client.UploadString(url, data);
}
JavaScriptSerializer json_serializer = new JavaScriptSerializer();
var results_list = (IDictionary<string, object>)json_serializer.DeserializeObject(result);
_accessToken = results_list["access_token"].ToString();
int secondsUntilRefresh = Int32.Parse(results_list["expires_in"].ToString());
_refreshTime = DateTime.Now.AddSeconds(secondsUntilRefresh);
if (results_list.Any(x => x.Key == "refresh_token"))
{
_refreshToken = results_list["refresh_token"].ToString();
}
if (_accessToken == "" || _refreshToken == "" || _refreshTime == new DateTime())
{
return false;
}
}
catch
{
return false;
}
return true;
}
private HttpStatusCode SendWebRequest(Methods method, string URN, string request, out string ResponseData)
{
if (_accessToken == "")
{
// The access token has not been retrieved yet
GetAccessToken();
}
else
{
if (_refreshTime != new DateTime() && _refreshTime < DateTime.Now) {
// The token has expired and we need to refresh it
_accessToken = "";
GetAccessToken();
}
}
if (_accessToken != "")
{
try
{
WebClient client = new WebClient();
string url = "https://api.freeagentcentral.com/v2/" + URN;
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36";
client.Headers[HttpRequestHeader.Authorization] = "Bearer " + _accessToken;
client.Headers[HttpRequestHeader.ContentType] = "application/json";
client.Headers[HttpRequestHeader.Accept] = "application/json";
if (method == Methods.POST || method == Methods.PUT)
{
string data = request;
var result = client.UploadString(url, method.ToString(), data);
ResponseData = result;
return HttpStatusCode.Created;
}
else
{
var result = client.DownloadString(url);
ResponseData = result;
return HttpStatusCode.OK;
}
}
catch (WebException e)
{
if (e.GetType().Name == "WebException")
{
WebException we = (WebException)e;
HttpWebResponse response = (System.Net.HttpWebResponse)we.Response;
ResponseData = response.StatusDescription;
return response.StatusCode;
}
else
{
ResponseData = e.Message;
if (e.InnerException != null)
{
ResponseData = ResponseData + " - " + e.InnerException.ToString();
}
return HttpStatusCode.SeeOther;
}
}
}
else
{
ResponseData = "Access Token could not be retrieved";
return HttpStatusCode.SeeOther;
}
}
private int ExtractNewID(string resp, string URN)
{
if (resp != null && resp.Trim() != "")
{
JavaScriptSerializer json_serializer = new JavaScriptSerializer();
var results_list = (IDictionary<string, object>)json_serializer.DeserializeObject(resp);
if (results_list.Any(x => x.Key == "invoice"))
{
var returnInvoice = (IDictionary<string, object>)results_list["invoice"];
if (returnInvoice["created_at"].ToString() != "")
{
string returnURL = returnInvoice["url"].ToString();
if (returnURL.Contains("http"))
{
return int.Parse(returnURL.Remove(0, ("https://api.freeagentcentral.com/v2/" + URN).Length + 1));
}
else
{
return int.Parse(returnURL.Remove(0, URN.Length + 2));
}
}
}
}
return -1;
}
public int CreateContact(string firstName, string lastName, string emailAddress, string street, string city, string state, string postcode, string country)
{
StringBuilder request = new StringBuilder();
request.Append("{\"contact\":{");
request.Append("\"first-name\":");
request.Append(firstName);
request.Append("\",");
request.Append("\"last-name\":");
request.Append(lastName);
request.Append("\",");
request.Append("\"email\":");
request.Append(emailAddress);
request.Append("\",");
request.Append("\"address1\":");
request.Append(street);
request.Append("\",");
request.Append("\"town\":");
request.Append(city);
request.Append("\",");
request.Append("\"region\":");
request.Append(state);
request.Append("\",");
request.Append("\"postcode\":");
request.Append(postcode);
request.Append("\",");
request.Append("\"country\":");
request.Append(country);
request.Append("\"");
request.Append("}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.POST, "contacts", request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
return ExtractNewID(returnData, "contacts");
}
else
{
return -1;
}
}
public int CreateInvoice(int contactID, DateTime invoiceDate, int terms, string reference, string comments, string currency = "GBP")
{
StringBuilder request = new StringBuilder();
request.Append("{\"invoice\":{");
request.Append("\"dated_on\": \"");
request.Append(invoiceDate.ToString("yyyy-MM-ddTHH:mm:00Z"));
request.Append("\",");
if (!string.IsNullOrEmpty(reference))
{
request.Append("\"reference\": \"");
request.Append(reference);
request.Append("\",");
}
if (!string.IsNullOrEmpty(comments))
{
request.Append("\"comments\": \"");
request.Append(comments);
request.Append("\",");
}
request.Append("\"payment_terms_in_days\": \"");
request.Append(terms);
request.Append("\",");
request.Append("\"contact\": \"");
request.Append(contactID);
request.Append("\",");
if (currency == "EUR")
{
request.Append("\"ec_status\": \"");
request.Append("EC Services");
request.Append("\",");
}
request.Append("\"currency\": \"");
request.Append(currency);
request.Append("\"");
request.Append("}}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.POST, "invoices", request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
return ExtractNewID(returnData, "invoices");
}
else
{
return -1;
}
}
public bool ChangeInvoiceStatus(int invoiceID, Status status)
{
string returnData = string.Empty;
HttpStatusCode resp = SendWebRequest(Methods.PUT, "invoices/" + invoiceID.ToString() + "/transitions/mark_as_" + status.ToString().ToLower(), string.Empty, out returnData);
return false;
}
public int CreateInvoiceItem(int invoiceID, string itemType, float price, int quantity, float taxRate, string description)
{
StringBuilder request = new StringBuilder();
request.Append("{\"invoice\":{");
request.Append("\"invoice_items\":[{");
request.Append("\"item_type\": \"");
request.Append(itemType);
request.Append("\",");
request.Append("\"price\": \"");
request.Append(price.ToString("0.00"));
request.Append("\",");
request.Append("\"quantity\": \"");
request.Append(quantity);
request.Append("\",");
request.Append("\"sales_tax_rate\": \"");
request.Append(taxRate.ToString("0.00"));
request.Append("\",");
request.Append("\"description\": \"");
request.Append(description);
request.Append("\"");
request.Append("}]");
request.Append("}");
request.Append("}");
string returnData = string.Empty;
HttpStatusCode responseCode = SendWebRequest(Methods.PUT, "invoices/" + invoiceID.ToString(), request.ToString(), out returnData);
if (responseCode == HttpStatusCode.OK || responseCode == HttpStatusCode.Created)
{
// Invoice items is an update call to an invoice, so we just get a 200 OK with no response body to extract an ID from
return 0;
}
else
{
return -1;
}
}
}
}
To use this in an MVC setup you need to detect in the Index whether the login is necessary or not. If it is, you tell FreeAgent to come back to another ActionResult (I've used Auth) to catch the redirect and store the code.
public ActionResult Index()
{
if (MyNamespace.External.FreeAgent.AuthToken == "")
{
return Redirect("https://api.freeagent.com/v2/approve_app?redirect_uri=" + Request.Url.Scheme + "://" + Request.Url.Authority + "/Invoice/Auth" + "&response_type=code&client_id=1234567890123456789012");
}
else
{
return View();
}
}
public ActionResult Auth()
{
if (Request.Params.Get("code").ToString() != "")
{
ClientZone.External.FreeAgent.AuthToken = Request.Params.Get("code").ToString();
}
return View("Index");
}
Then to use the functions all you need is
MyNameSpace.External.FreeAgent FA = new ClientZone.External.FreeAgent("abcdefghhijklmnopqrstu", "1234567890123456789012", "acme", "http://localhost:52404/Invoice/Auth");
int newID = FA.CreateInvoice(54321, InvoiceDate, 30, "", "", "GBP");
if (newID != -1)
{
FA.CreateInvoiceItem(newID, unit, rate, quantity, number, description);
}
There is some UK/EU specific coding in the CreateInvoice. I'm not sure how custom or standard this is, but I've left it in as it's easier to delete it than recreate it.

Try Catch - URL link not exist

I need to read some Data from URLs - but depending on the string (ICAO) - the URL does not sometimes exist (it is not valid). In this case - I should get "N/A" - but that does not work... only when all three URLs are readable - it is working.
[Invoke]
public List<Category> getWeather(string ICAO)
{
try
{
List<Category> lstcat = new List<Category>();
Category cat = new Category();
string fileString;
bool isexists = FtpDirectoryExists("ftp://tgftp.nws.noaa.gov/data/observations/metar/stations/" + ICAO);
if (isexists == true)
{
WebClient request = new WebClient();
string url = "http://weather.noaa.gov/pub/data/observations/metar/stations/" + ICAO;
byte[] newFileData = request.DownloadData(url);
fileString = System.Text.Encoding.UTF8.GetString(newFileData);
cat.Cat = "METAR";
lstcat.Add(cat);
cat = new Category();
cat.Cat = fileString;
lstcat.Add(cat);
url = "http://weather.noaa.gov/pub/data/forecasts/shorttaf/stations/" + ICAO;
newFileData = request.DownloadData(url);
fileString = System.Text.Encoding.UTF8.GetString(newFileData);
cat = new Category();
cat.Cat = "Short TAF";
lstcat.Add(cat);
cat = new Category();
cat.Cat = fileString;
lstcat.Add(cat);
url = "http://weather.noaa.gov/pub/data/forecasts/taf/stations/" + ICAO;
newFileData = request.DownloadData(url);
fileString = System.Text.Encoding.UTF8.GetString(newFileData);
cat = new Category();
cat.Cat = "Long TAF";
lstcat.Add(cat);
cat = new Category();
cat.Cat = fileString;
lstcat.Add(cat);
}
else
{
fileString = "N/A;N/A";
}
return null;
}
catch (Exception)
{
throw;
}
}

Create a method to check the remote file exists or not.
make a Heder request to the URl, if the sattus code is 200 or 302 then return true
otherwise false;
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(/* url */);
request.Method = "HEAD";
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
/* A WebException will be thrown if the status of the response is not `200 OK` */
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close()
}
}

Okay, I think I see what you want to do. I re-worked your code so that it fits what's actually needed.
Right now, it seems like the first Category in lstcat has the description (such as "METAR") of the first URL, the second Category in lstcat has the fileString that matches it, and so on. This is very vague and cumbersome. Instead, make it so that one Category object contains everything you need to know about a URL:
public class Category
{
public string description;
public string fileString;
//Other fields you might use somewhere else...
public Category(string description, string fileString /*, other fields, if any...*/)
{
this.description = description;
this.fileString = fileString;
//Initialize others...
}
}
I then eliminated all the duplication in the original code, by putting all the URL download code into a separate function.
[Invoke]
public List<Category> getWeather(string ICAO)
{
bool isexists = FtpDirectoryExists("ftp://tgftp.nws.noaa.gov/data/observations/metar/stations/" + ICAO);
if (isexists)
{
List<Category> lstcat = new List<Category>();
addCategoriesToList(
lstcat,
"http://weather.noaa.gov/pub/data/observations/metar/stations/" + ICAO,
"METAR"
);
addCategoriesToList(
lstcat,
"http://weather.noaa.gov/pub/data/forecasts/shorttaf/stations/" + ICAO,
"Short TAF"
);
addCategoriesToList(
lstcat,
"http://weather.noaa.gov/pub/data/forecasts/taf/stations/" + ICAO,
"Long TAF"
);
return lstcat;
}
else
{
return null;
}
}
private static void addCategoriesToList(List<Category> lstcat, string url, string description)
{
string fileString;
//Use "using" so that `request` always gets cleaned-up:
using (WebClient request = new WebClient())
{
try
{
byte[] newFileData = request.DownloadData(url);
fileString = System.Text.Encoding.UTF8.GetString(newFileData);
}
catch
{
fileString = "N/A";
}
}
lstcat.Add(new Category(description, fileString));
}
I think this accomplishes what you want, in a much cleaner and straightforward way. Please let me know if that is indeed the case!

Categories