We Keep Coding

Web request CSharp

I have a C# code that consumes a web service data from an external site. it was working and displaying the weather data on my page but now it throws "An exception occurred during a WebClient request" error. I was trying to research this but no luck.
protected void GetWeatherInfo(string url)
{
string url1 = string.Format(url);
using (WebClient client = new WebClient())
{
WebProxy proxyObj = new WebProxy(url1);
proxyObj.Credentials = CredentialCache.DefaultCredentials;
client.Proxy = proxyObj;
try
{
string json = client.DownloadString(url1);
WeatherInfo weatherInfo = (new JavaScriptSerializer()).Deserialize<WeatherInfo>(json);
lblCity_Country.Text = weatherInfo.city.name + "," + weatherInfo.city.country;
imgCountryFlag.ImageUrl = string.Format("http://openweathermap.org/images/flags/{0}.png", weatherInfo.city.country.ToLower());
lblDescription.Text = weatherInfo.list[0].weather[0].description;
imgWeatherIcon.ImageUrl = string.Format("http://openweathermap.org/img/w/{0}.png", weatherInfo.list[0].weather[0].icon);
lblTempMin.Text = string.Format("{0}°С", Math.Round(weatherInfo.list[0].temp.min, 1));
lblTempMax.Text = string.Format("{0}°С", Math.Round(weatherInfo.list[0].temp.max, 1));
lblTempDay.Text = string.Format("{0}°С", Math.Round(weatherInfo.list[0].temp.day, 1));
lblTempNight.Text = string.Format("{0}°С", Math.Round(weatherInfo.list[0].temp.night, 1));
lblHumidity.Text = weatherInfo.list[0].humidity.ToString();
tblWeather.Visible = true;
}
catch (Exception ex)
{
string excep = ex.Message;
}
}
}

Xamarin Forms: IOS adhoc crashes but debug works

I have a very strange problem, I programmed an app that triggers a webservice and gives me a response back. This also works on android and IOS (in debug mode) but when I install the app via adhoc it still works but when I press the button that triggers the webservice the app freezes and you have to restart it but it always stops at the same place.
I have found out with a lot of commenting and commenting out that the error happens in the marked place. In debug mode everything works as it should.
Here is the code that will be executed.
IsMYSite(strLink);
if (gotResponse)
{
gotResponse = false;
DetailPage.MYPages.Add(new MYPage { Title = portal, Link = strLink, Username = "", Password = "", IsVisible = false, CompanyImage = ImageLink });
App.Current.Properties["MYPages"] = JsonConvert.SerializeObject(DetailPage.MYPages);
App.Current.Properties["LastPortal"] = strLink;
Navigation.PopAsync(true);
LinkURL.TextColor = Color.Black;
}
else
{
LinkURL.TextColor = Color.Red;
}
public async void IsMYSite(String url)
{
try
{
String linkonly = url;
url += "/api/" + CultureInfo.CurrentUICulture.ThreeLetterWindowsLanguageName;
string strpost = "";
StringContent str = new StringContent(strpost, Encoding.UTF8, "application/x-www-form-urlencoded");
var client = new System.Net.Http.HttpClient();
client.Timeout = new TimeSpan(0, 0, 6);
client.BaseAddress = new Uri(url);
var response = await client.GetAsync(new Uri(url));
Dictionary<string, string> result = JsonConvert.DeserializeObject<Dictionary<string, string>>(response.Content.ReadAsStringAsync().Result);
portal = result["PortalTitle"];
if (result["LoginLogo_Path"].StartsWith("~"))
{
StringBuilder str2 = new StringBuilder(result["LoginLogo_Path"]);
str2.Remove(0, 1);
ImageLink = linkonly + str2.ToString();
gotResponse = true;
}
}
catch
{
}
}

How to use cursors to iterate through the result set of GET followers/list in Twitter API using c#?

I have been trying a way to get the data in the next page of the result set of GET followers/list API call. I can get the default data set with the data of first 20 followers and not the others. To get the data of other followers i have to access the next page using the next_cursor but it's not working. I tried using the pseudo-code mentioned in this link. https://dev.twitter.com/docs/misc/cursoring
Is it a must to use this(this is mentioned in the dev. site):
var api-path = "https://api.twitter.com/1.1/endpoint.json?screen_name=targetUser"
Because I have been using the resource URL as,
var resource_url = "https://api.twitter.com/1.1/followers/list.json";
and I tried appending the next_cursor to the same resource URL.
var url_with_cursor = resource_url + "&cursor=" + 1463101490306580067;
and then created the request.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url_with_cursor);
but I'm getting an exception in this line when getting the response.
WebResponse response = request.GetResponse();
The error I'm getting is
The Remote Server returned an Error 401 Unauthorized
Can someone tell the exact way to do cursor-ing, or the exact way to include the cursor in the request. I'm using a asp.net C# web application.
Here's my code, The oauth_token, oauth_token_secret, oauth_consumer_key, oauth_consumer_secret, oauth_version and oauth_signature_method are defined in my application
var resource_url = "https://api.twitter.com/1.1/followers/list.json";
var cursor = "-1";
do
{
var url_with_cursor = resource_url + "&cursor=" + cursor;
// unique request details
var oauth_nonce = Convert.ToBase64String(
new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
var timeSpan = DateTime.UtcNow
- new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);
var oauth_timestamp = Convert.ToInt64(timeSpan.TotalSeconds).ToString();
// create oauth signature
var baseFormat = "oauth_consumer_key={0}&oauth_nonce={1}&oauth_signature_method={2}" +
"&oauth_timestamp={3}&oauth_token={4}&oauth_version={5}";
var baseString = string.Format(baseFormat,
oauth_consumer_key,
oauth_nonce,
oauth_signature_method,
oauth_timestamp,
oauth_token,
oauth_version
//,Uri.EscapeDataString(status)
);
baseString = string.Concat("GET&", Uri.EscapeDataString(resource_url), "&", Uri.EscapeDataString(baseString));
var compositeKey = string.Concat(Uri.EscapeDataString(oauth_consumer_secret),
"&", Uri.EscapeDataString(oauth_token_secret));
string oauth_signature;
using (HMACSHA1 hasher = new HMACSHA1(ASCIIEncoding.ASCII.GetBytes(compositeKey)))
{
oauth_signature = Convert.ToBase64String(
hasher.ComputeHash(ASCIIEncoding.ASCII.GetBytes(baseString)));
}
// create the request header
var headerFormat = "OAuth oauth_nonce=\"{0}\", oauth_signature_method=\"{1}\", " +
"oauth_timestamp=\"{2}\", oauth_consumer_key=\"{3}\", " +
"oauth_token=\"{4}\", oauth_signature=\"{5}\", " +
"oauth_version=\"{6}\"";
var authHeader = string.Format(headerFormat,
Uri.EscapeDataString(oauth_nonce),
Uri.EscapeDataString(oauth_signature_method),
Uri.EscapeDataString(oauth_timestamp),
Uri.EscapeDataString(oauth_consumer_key),
Uri.EscapeDataString(oauth_token),
Uri.EscapeDataString(oauth_signature),
Uri.EscapeDataString(oauth_version)
);
// make the request
ServicePointManager.Expect100Continue = false;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url_with_cursor);
request.Headers.Add("Authorization", authHeader);
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
WebResponse response = request.GetResponse();
string result = new StreamReader(response.GetResponseStream()).ReadToEnd();
JObject j = JObject.Parse(result);
JArray data = (JArray)j["users"];
cursor = (String)j["next_cursor_str"];
} while (!cursor.Equals("0"));
Thanks.

Tweetinvi would make it somewhat easier for you.
Please visit https://github.com/linvi/tweetinvi/wiki/Get-All-Followers-Code to have an idea on how to do it including RateLimit handling.
Without considering the RateLimits, you could simply use the following code.
long nextCursor = -1;
do
{
var query = string.Format("https://api.twitter.com/1.1/followers/ids.json?screen_name={0}", username);
var results = TwitterAccessor.ExecuteCursorGETCursorQueryResult<IIdsCursorQueryResultDTO>(query, cursor: cursor).ToArray();
if (results.Any())
{
nextCursor = results.Last().NextCursor;
}
else
{
nextCursor = -1;
}
}
while (nextCursor != -1 && nextCursor != 0);

you should make a different call to authenticate, by an authorization request. Once that has been granted, you can call the webresponse with the cursor. See my sample code below (Take special note to the StartCreateCall method, where the authentication is done. The data from Twitter is then retrieved by the CallData method):
public partial class twitter_followers : System.Web.UI.Page
{
public string strTwiterFollowers { get; set; }
private List<TwiterFollowers> listFollowers = new List<TwiterFollowers>();
private string screen_name = string.Empty;
// oauth application keys
private string oauth_consumer_key = string.Empty;
private string oauth_consumer_secret = string.Empty;
// oauth implementation details
private string resource_urlFormat = "https://api.twitter.com/1.1/followers/list.json?screen_name={0}&cursor={1}";
// unique request details
private string oauth_nonce = Convert.ToBase64String(new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
protected void Page_Load(object sender, EventArgs e)
{
//just get your request parameters from the config file.
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => screen_name)])) {
screen_name = ConfigurationManager.AppSettings[GetVariableName(() => screen_name)];
}
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_key)]))
{
oauth_consumer_key = ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_key)];
}
if (!string.IsNullOrEmpty(ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_secret)]))
{
oauth_consumer_secret = ConfigurationManager.AppSettings[GetVariableName(() => oauth_consumer_secret)];
}
StartCreateCall();
}
// Do the authenticate by an authorization request
private void StartCreateCall() {
// You need to set your own keys and screen name
var oAuthUrl = "https://api.twitter.com/oauth2/token";
// Do the Authenticate
var authHeaderFormat = "Basic {0}";
var authHeader = string.Format(authHeaderFormat,
Convert.ToBase64String(Encoding.UTF8.GetBytes(Uri.EscapeDataString(oauth_consumer_key) + ":" +
Uri.EscapeDataString((oauth_consumer_secret)))
));
var postBody = "grant_type=client_credentials";
HttpWebRequest authRequest = (HttpWebRequest)WebRequest.Create(oAuthUrl);
authRequest.Headers.Add("Authorization", authHeader);
authRequest.Method = "POST";
authRequest.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
authRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
using (Stream stream = authRequest.GetRequestStream())
{
byte[] content = ASCIIEncoding.ASCII.GetBytes(postBody);
stream.Write(content, 0, content.Length);
}
authRequest.Headers.Add("Accept-Encoding", "gzip");
WebResponse authResponse = authRequest.GetResponse();
// deserialize into an object
TwitAuthenticateResponse twitAuthResponse;
using (authResponse)
{
using (var reader = new StreamReader(authResponse.GetResponseStream()))
{
JavaScriptSerializer js = new JavaScriptSerializer();
var objectText = reader.ReadToEnd();
twitAuthResponse = JsonConvert.DeserializeObject<TwitAuthenticateResponse>(objectText);
}
}
//now we have been granted access and got a token type with authorization token from Twitter
//in the form of a TwitAuthenticateResponse object, we can retrieve the data recursively with a cursor
CallData(twitAuthResponse, authHeader, cursor);
int totalFollowers = listFollowers.Count;
lblTotalFollowers.Text = screen_name + " has " + listFollowers.Count + " Followers";
Random objRnd = new Random();
List<TwiterFollowers> randomFollowers = listFollowers.OrderBy(item => objRnd.Next()).ToList<TwiterFollowers>();
foreach (TwiterFollowers tw in randomFollowers)
{
strTwiterFollowers = strTwiterFollowers + "<li><a target='_blank' title='" + tw.ScreenName + "' href=https://twitter.com/" + tw.ScreenName + "><img src='" + tw.ProfileImage + "'/><span>" + tw.ScreenName + "</span></a></li>";
}
}
//Retrieve the data from Twitter recursively with a cursor
private void CallData(TwitAuthenticateResponse twitAuthResponse, string authHeader, string cursor)
{
try {
JObject j = GetJSonObject(twitAuthResponse, authHeader, cursor);
JArray data = (JArray)j["users"];
if (data != null)
{
foreach (var item in data)
{
TwiterFollowers objTwiterFollowers = new TwiterFollowers();
objTwiterFollowers.ScreenName = item["screen_name"].ToString().Replace("\"", "");
objTwiterFollowers.ProfileImage = item["profile_image_url"].ToString().Replace("\"", "");
objTwiterFollowers.UserId = item["id"].ToString().Replace("\"", "");
listFollowers.Add(objTwiterFollowers);
}
JValue next_cursor = (JValue)j["next_cursor"];
if (long.Parse(next_cursor.Value.ToString()) > 0)
{
//Get the following data from Twitter with the next cursor
CallData(twitAuthResponse, authHeader, next_cursor.Value.ToString());
}
}
} catch (Exception ex)
{
//do nothing
}
}
private JObject GetJSonObject(TwitAuthenticateResponse twitAuthResponse, string authHeader, string cursor)
{
string resource_url = string.Format(resource_urlFormat, screen_name, cursor);
if (string.IsNullOrEmpty(cursor))
{
resource_url = resource_url.Substring(0, resource_url.IndexOf("&cursor"));
}
HttpWebRequest fRequest = (HttpWebRequest)WebRequest.Create(resource_url);
var timelineHeaderFormat = "{0} {1}";
fRequest.Headers.Add("Authorization", string.Format(timelineHeaderFormat, twitAuthResponse.token_type, twitAuthResponse.access_token));
fRequest.Method = "Get";
WebResponse response = fRequest.GetResponse();
string result = new StreamReader(response.GetResponseStream()).ReadToEnd();
return JObject.Parse(result);
}
private string GetVariableName<T>(Expression<Func<T>> expr)
{
var body = (MemberExpression)expr.Body;
return body.Member.Name;
}
private class TwitAuthenticateResponse
{
public string token_type { get; set; }
public string access_token { get; set; }
}
private class TwiterFollowers
{
public string ScreenName { get; set; }
public string ProfileImage { get; set; }
public string UserId { get; set; }
}
}

You are getting "401 Unauthorized"
Did you check you are setting everything right?
Credentials? Check both queries on fiddler.

HtmlAgilityPack obtain Title and meta

I try to practice "HtmlAgilityPack ", but I am having some issues regarding this. here's what I coded, but I can not get correctly the title and the description of a web page ...
If someone can enlighten me on my mistake :)
...
public static void Main(string[] args)
{
string link = null;
string str;
string answer;
int curloc; // holds current location in response
string url = "http://stackoverflow.com/";
try
{
do
{
HttpWebRequest HttpWReq = (HttpWebRequest)WebRequest.Create(url);
HttpWReq.UserAgent = #"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5";
HttpWebResponse HttpWResp = (HttpWebResponse)HttpWReq.GetResponse();
//url = null; // disallow further use of this URI
Stream istrm = HttpWResp.GetResponseStream();
// Wrap the input stream in a StreamReader.
StreamReader rdr = new StreamReader(istrm);
// Read in the entire page.
str = rdr.ReadToEnd();
curloc = 0;
//WebPage result;
do
{
// Find the next URI to link to.
link = FindLink(str, ref curloc); //return the good link
Console.WriteLine("Title found: " + curloc);
//title = Title(str, ref curloc);
if (link != null)
{
Console.WriteLine("Link found: " + link);
using (System.Net.WebClient client = new System.Net.WebClient())
{
HtmlDocument htmlDoc = new HtmlDocument();
var html = client.DownloadString(url);
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
var htmlElement = htmlDoc.DocumentNode.Element("html");
HtmlNode node = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (node != null)
{
string desc = node.GetAttributeValue("content", "");
Console.Write("DESCRIPTION: " + desc);
}
else
{
Console.WriteLine("No description");
}
var titleElement =
htmlDoc.DocumentNode
.Element("html")
.Element("head")
.Element("title");
if (titleElement != null)
{
string title = titleElement.InnerText;
Console.WriteLine("Titre: {0}", title);
}
else
{
Console.WriteLine("no Title");
}
Console.Write("Done");
}
Console.Write("Link, More, Quit?");
answer = Console.ReadLine();
}
else
{
Console.WriteLine("No link found.");
break;
}
} while (link.Length > 0);
// Close the Response.
HttpWResp.Close();
} while (url != null);
}
catch{ ...}
Thanks in advance :)

Go about it this way:
HtmlNode mdnode = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (mdnode != null)
{
HtmlAttribute desc;
desc = mdnode.Attributes["content"];
string fulldescription = desc.Value;
Console.Write("DESCRIPTION: " + fulldescription);
}

I think your problem is here:
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
It should be:
htmlDoc.LoadHtml(html); //chargement de HTMLAgilityPack
LoadHtml expects a string with the HTML source, not the url.
And probably you want to change:
var html = client.DownloadString(url);
to
var html = client.DownloadString(link);

Have you used a breakpoint and gone line for line to see where the error might be occurring?
If you have, then Try something like this:
string result = string.Empty;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.google.com");
request.Method = "GET";
try
{
using (var stream = request.GetResponse().GetResponseStream())
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
result = reader.ReadToEnd();
}
}
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(result);
Then carry over the rest of your code below the htmlDoc.LoadHtml

[HttpPost]
public ActionResult Create(WebSite website)
{
string desc = HtmlAgi(website.Url, "description");
string keyword = HtmlAgi(website.Url, "Keywords");
if (ModelState.IsValid)
{
var userId = ((CustomPrincipal)User).UserId;
r.Create(new WebSite
{
Description = desc,
Tags = keyword,
Url = website.Url,
UserId = userId,
Category = website.Category
});
return RedirectToAction("Index");
}
return View(website);
}

public string HtmlAgi(string url, string key)
{
//string.Format
var Webget = new HtmlWeb();
var doc = Webget.Load(url);
HtmlNode ourNode = doc.DocumentNode.SelectSingleNode(string.Format("//meta[#name='{0}']", key));
if (ourNode != null)
{
return ourNode.GetAttributeValue("content", "");
}
else
{
return "not fount";
}
}

Server Hosting Error in C#

I developed a eCommerce Website using ASP.Net 3.5 and C#. It is working good with Visual Stdio 2010. I uploaded my website to my domain under the public folder.
It show the error like below:
Click for full-size, or here's the stack trace fragment from the local error page.
NullReferenceException: Object reference not set to an instance of an object
SageFrame.Framework.PageBase.OptimizeJs(List`1 lstJsColl, Int32 Mode) +7940
SageFrame.Framework.PageBase.LoadModuleJs() +944
SageFrame.Framework.PageBase.OnPreRender(EventArgs e) +233
System.Web.UI.PreRenderRecursiveInternal() +107
It was working fine before!
The code of the page_Load() is:
protected void Page_Load(object sender, EventArgs e)
{
string selectedCurrency = string.Empty;
string MainCurrency = string.Empty;
try
{
StoreSettingConfig ssc = new StoreSettingConfig();
MainCurrency = ssc.GetStoreSettingsByKey(StoreSetting.MainCurrency, GetStoreID, GetPortalID, GetCurrentCultureName);
if (Session["SelectedCurrency"] != null && Session["SelectedCurrency"] != "")
{
selectedCurrency = Session["SelectedCurrency"].ToString();
}
else
{
selectedCurrency = MainCurrency;
}
string islive = Request.Form["custom"];
string test = string.Empty;
const string strSandbox = "https://www.sandbox.paypal.com/cgi-bin/webscr";
const string strLive = "https://www.paypal.com/cgi-bin/webscr";
test = bool.Parse(islive.Split('#')[6]) ? strSandbox : strLive;
var req = (HttpWebRequest)WebRequest.Create(test);
//Set values for the request back
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
byte[] param = Request.BinaryRead(HttpContext.Current.Request.ContentLength);
string strRequest = Encoding.ASCII.GetString(param);
strRequest += "&cmd=_notify-validate";
req.ContentLength = strRequest.Length;
var streamOut = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(strRequest);
streamOut.Close();
var streamIn = new StreamReader(req.GetResponse().GetResponseStream());
string strResponse = streamIn.ReadToEnd();
streamIn.Close();
if (strResponse == "VERIFIED")
{
string payerEmail = Request.Form["payer_email"];
string paymentStatus = Request.Form["payment_status"];
string receiverEmail = Request.Form["receiver_email"];
string amount = Request.Form["mc_gross"];
string invoice = Request.Form["invoice"];
string addressName = Request.Form["address_name"];
string addressStreet = Request.Form["address_street"];
string addressCity = Request.Form["address_city"];
string addressZip = Request.Form["address_zip"];
string addressCountry = Request.Form["address_country"];
string transID = Request.Form["txn_id"];
string custom = Request.Form["custom"];
string[] ids = custom.Split('#');
int orderID = int.Parse(ids[0]);
int storeID = int.Parse(ids[1]);
int portalID = int.Parse(ids[2]);
string userName = ids[3];
int customerID = int.Parse(ids[4]);
string sessionCode = ids[5];
string pgid = ids[7];
var tinfo = new TransactionLogInfo();
var tlog = new TransactionLog();
tinfo.TransactionID = transID;
tinfo.AuthCode = "";
tinfo.TotalAmount = decimal.Parse(amount);
tinfo.ResponseCode = "1";
tinfo.ResponseReasonText = "";
tinfo.OrderID = orderID;
tinfo.StoreID = storeID;
tinfo.PortalID = portalID;
tinfo.AddedBy = userName;
tinfo.CustomerID = customerID;
tinfo.SessionCode = sessionCode;
tinfo.PaymentGatewayID = int.Parse(pgid);
tinfo.PaymentStatus = paymentStatus;
tinfo.PayerEmail = payerEmail;
tinfo.CreditCard = "";
tinfo.RecieverEmail = receiverEmail;
tinfo.CurrencyCode = selectedCurrency;
tlog.SaveTransactionLog(tinfo);
if (paymentStatus.Equals("Completed"))
{
var paypalobj = new PayPalHandler();
paypalobj.ParseIPN(orderID, transID, paymentStatus, storeID, portalID, userName, customerID, sessionCode);
}
}
else if (strResponse == "INVALID")
{
//log for manual investigation
}
else
{
//log response/ipn data for manual investigation
}
// }
}
catch (Exception ex)
{
ProcessException(ex);
// throw new Exception("This Page is not accessible!");
}
}

From the error message, it appears that your store library cannot be loaded. This could be because it isn't installed on the server or the trust level is too low.
Either way, your hosting provider could assist with the setup.