We have a web application that consists of several pages. We registered our web app domain to Google Analytics and page views tracking works as expected (In the Analytics panel we can see page views for each page). Now we want this page views info to be stored in the back-end inside our DB. So we want to create a back-end process that will run once each day, and fetch the page views from Analytics API.
This is of course need to be done in code. From initial research it seems that in order to access Analytics API an authentication process must take place, meaning a human user must type in an id and password.
The question is, can it be done with code only ?
//-------------- Get Auth Token -------------------
WebClient webClient = new WebClient();
NameValueCollection data = new NameValueCollection();
data.Add("accountType", "GOOGLE");
data.Add("Email", "xxxx#gmail.com");
data.Add("Passwd", "xxxx");//Passwd, not a misspell.
data.Add("service", "analytics");
data.Add("source", "xxxx-xxxx-xx");//Could be anything.
byte[] bytes = webClient.UploadValues("https://www.google.com/accounts/ClientLogin", "POST", data);
string tokens = Encoding.UTF8.GetString(bytes);
string authToken = extractAuthToken(tokens);
//-------------- Get page views -------------------
string feed = "https://www.google.com/analytics/feeds/data";
//Required:
string ids = "ga:xxxx";
string metrics = "ga:pageviews";
string startDate = "2011-06-25";
string endDate = "2011-07-25";
//Optional:
string dimensions = "ga:pagePath";
string sort = "-ga:pageviews";
string feedUrl = string.Format("{0}?ids={1}&dimensions={2}&metrics={3}&sort={4}&start-date={5}&end-date={6}",
feed, ids, dimensions, metrics, sort, startDate, endDate);
webClient.Headers.Add("Authorization", "GoogleLogin " + authToken);
string result = webClient.DownloadString(feedUrl);
//-------------- Extract data from xml -------------------
XDocument xml = XDocument.Parse(result);
var ns1 = "{http://www.w3.org/2005/Atom}";
var ns2 = "{http://schemas.google.com/analytics/2009}";
var q = from entry in xml.Descendants()
where entry.Name == ns1 + "entry"
select new
{
PagePath = entry.Element(ns2 + "dimension").Attribute("value").Value,
Views = entry.Element(ns2 + "metric").Attribute("value").Value
};
//-------------- Do something with data -------------------
foreach (var page in q)
{
Debug.WriteLine(page.PagePath + " " + page.Views);
}
//-------------- Help Method -------------------
private string extractAuthToken(string data)
{
var tokens = data.Split(new string[] { "\n" }, StringSplitOptions.RemoveEmptyEntries);
return tokens.Where(token => token.StartsWith("Auth=")).Single();
}
Related
I want to see the shared link information from the OneDrive shared link URL.
If you use the shared link URL created by yourself (test001) as a parameter, you will succeed in getting the information.
string sharingUrl = "https://test001-my.sharepoint.com/:t:/g/personal/test001_...";
string base64Value = System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(sharingUrl));
string encodedUrl = "u!" + base64Value.TrimEnd('=').Replace('/', '_').Replace('+', '-');
var item = await graphClient.Shares[encodedUrl].Request().GetAsync();
When trying to get information from a shared link URL created by another user (test002)
string sharingUrl = "https://test001-my.sharepoint.com/:t:/g/personal/test002_...";
string base64Value = System.Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(sharingUrl));
string encodedUrl = "u!" + base64Value.TrimEnd('=').Replace('/', '_').Replace('+', '-');
var item = await graphClient.Shares[encodedUrl].Request().GetAsync();
System.NullReferenceException occurs in the above process.
What am I doing wrong?
I'm attempting to integrate version 3 of Mailchimp's API to add a subscriber to one of my mailing lists. The code below is what I have thus far, and my intention is to call it in my contact form method when the user fills out their email to subscribe. In theory it should get this email stored in emailAddress and POST it to MailChimp, but theorys are not practical. Below is my current method of POST data:
private string InsertIntoMailChimpGeneralList(string emailAddress)
{
var apiKey = ConfigurationManager.AppSettings["MailChimpAPIKeyGeneral"];
var dataCenter = ConfigurationManager.AppSettings["MailChimpDataCenterIDGeneral"];
var listId = ConfigurationManager.AppSettings["mailChimpListIDGeneral"];
var email_address = emailAddress;
var status = "subscribed";
using (var wc = new System.Net.WebClient())
{
// Data to be posted to add email address to list
var data = new { email_address, status };
// Serialize to JSON using Json.Net
var json = JsonConvert.SerializeObject(data);
// Base URL to MailChimp API
string apiUrl = "https://" + dataCenter + ".api.mailchimp.com/3.0/";
// Construct URL to API endpoint being used
var url = string.Concat(apiUrl, "lists/", listId, "/members?");
// Set content type
wc.Headers.Add("Content-Type", "application/json");
// Generate authorization header
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(":" + apiKey));
// Set authorization header
wc.Headers[HttpRequestHeader.Authorization] = string.Format("Basic {0}", credentials);
// Post and get JSON response
string sendUrl = wc.UploadString(url, json);
return sendUrl;
}
}
In my contact form I have a check where I want the email address added in the contact form (emailAddress is the variable used here) to post that data to the list, when the form is submitted:
if (ConfigurationManager.AppSettings["UseMailChimpIntegration"].ToString().ToLower().Trim() == "true")
{
InsertIntoMailChimpGeneralList(emailAddress);
}
I feel I've implemented this wrong. I was able to get it working on v2 but upgrading to v3 has left me clueless at this point. My contact form runs fine and stored my values in my local database, but does not POST that data through to mailchimp.
I've triple checked my API/datacenter values, and would appreciate some assistance.
Browserstack.com have a REST API that stores the results of Automated test sessions in a JSON file.
By default it sets the status token to "done" but you can update using the REST API e.g. "passed" or "failed".
It requires Basic authorization using your username & password.
They give an example of how to do this but it's a bit messy. So I wanted a simpler way of updating the file and only the tokens I was interested in.
I was able to do this using DalSoft's RestClient application:
public static async Task DalSoft(string SessionID, string TestStatus)
{
string Uri = "https://www.browserstack.com/automate/sessions/" + SessionID +
".json";
string AuthToken = "Basic " +
Convert.ToBase64String(Encoding.Default.GetBytes("username:password"));
dynamic client = new DalSoft.RestClient.RestClient(Uri);
var status = new { status=TestStatus };
var result = await client
.Headers(new { Authorization = AuthToken })
.Patch(status);
string Myresults = result.ToString();
}
https://github.com/DalSoft/DalSoft.RestClient/issues/40#issuecomment-334219145
I would like to execute a query on google images to fetch images using htmlagilitypack in c#.
For this I used an xpath request to the image
//*[#id="rg_s"]/div[1]/a/img
But it fails to fetch the image that way. What could be the correct way of doing this?
you can try this too : Here its possible to get the links of images by following
var links = HtmlDocument.DocumentNode.SelectNodes("//a").Where(a => a.InnerHtml.Contains("<img")).Select(b => b.Attributes["href"].Value).ToList();
foreach(var link in links)
{
// you can save the link or do your process here
}
Google keeps found images in div tags with class rg_di. Here is a query to get all links to images:
var links = hdoc.DocumentNode.SelectNodes(#"//div[#class='rg_di']/a")
.Select(a => a.GetAttributeValue("href", ""));
Searching google programmatically outside of their API's is against the TOS. Consider Google Custom Search or Bing Search API, both of which have established JSON and SOAP interfaces.
Both are free for a couple thousand queries per month and comply with the service's TOS.
Edit: Examples of using Bing API with C# below:
const string bingKey = "[your key here]";
var bing = new BingSearchContainer(new Uri("https://api.datamarket.azure.com/Bing/Search/"))
{
Credentials = new NetworkCredential(bingKey, bingKey)
};
var query = bing.Web("Jon Gallant blog", null, null, null, null, null, null, null);
var results = query.Execute();
foreach(var result in results)
{
Console.WriteLine(result.Url);
}
Console.ReadKey();
Google custom search API:
string apiKey = "Your api key";
string cx = "Your custom search engine id";
string query = "Your query";
var svc = new Google.Apis.Customsearch.v1.CustomsearchService(new BaseClientService.Initializer { ApiKey = apiKey });
var listRequest = svc.Cse.List(query);
listRequest.Cx = cx;
var search = listRequest.Fetch();
foreach (var result in search.Items)
{
Response.Output.WriteLine("Title: {0}", result.Title);
Response.Output.WriteLine("Link: {0}", result.Link);
}
I'm using the code in the following post:
Google Analytics API - Programmatically fetch page views in server side
but getting a 403 forbidden error on the highlighted line below. I don't think it's a credential issue, becuase my credentials are correct, as I have checked and double checked, and also I log in to the analytics account with these credentials. So maybe it is somekind of folder permissions issue ?
//-------------- Get Auth Token -------------------
WebClient webClient = new WebClient();
NameValueCollection data = new NameValueCollection();
data.Add("accountType", "GOOGLE");
data.Add("Email", "xxxx#gmail.com");
data.Add("Passwd", "xxxx");//Passwd, not a misspell.
data.Add("service", "analytics");
data.Add("source", "xxxx-xxxx-xx");//Could be anything.
byte[] bytes = webClient.UploadValues("https://www.google.com/accounts/ClientLogin", "POST", data);
string tokens = Encoding.UTF8.GetString(bytes);
string authToken = extractAuthToken(tokens);
//-------------- Get page views -------------------
string feed = "https://www.google.com/analytics/feeds/data";
//Required:
string ids = "ga:xxxx";
string metrics = "ga:pageviews";
string startDate = "2011-06-25";
string endDate = "2011-07-25";
//Optional:
string dimensions = "ga:pagePath";
string sort = "-ga:pageviews";
string feedUrl = string.Format("{0}?ids={1}&dimensions={2}&metrics={3}&sort={4}&start-date={5}&end-date={6}",
feed, ids, dimensions, metrics, sort, startDate, endDate);
webClient.Headers.Add("Authorization", "GoogleLogin " + authToken);
// This is the line I get the 403 error on:
**string result = webClient.DownloadString(feedUrl);**
//-------------- Extract data from xml -------------------
XDocument xml = XDocument.Parse(result);
var ns1 = "{http://www.w3.org/2005/Atom}";
var ns2 = "{http://schemas.google.com/analytics/2009}";
var q = from entry in xml.Descendants()
where entry.Name == ns1 + "entry"
select new
{
PagePath = entry.Element(ns2 + "dimension").Attribute("value").Value,
Views = entry.Element(ns2 + "metric").Attribute("value").Value
};
//-------------- Do something with data -------------------
foreach (var page in q)
{
Debug.WriteLine(page.PagePath + " " + page.Views);
}
//-------------- Help Method -------------------
private string extractAuthToken(string data)
{
var tokens = data.Split(new string[] { "\n" }, StringSplitOptions.RemoveEmptyEntries);
return tokens.Where(token => token.StartsWith("Auth=")).Single();
}
If you call the Google Analytics API too frequently, you could get 403 Forbidden errors. From that link:
General Analytics API quotas. These apply to both the Analytics APIs, i.e., Management API and Core Reporting API:
- 50,000 requests per project per day
- 10 queries per second (QPS) per IP
I've seen 403 errors returned from the AdWords API when my applications have made too many consecutive calls, so that potentially could be the cause of your problem.
EDIT
If you're not able to make any calls at all, then review the steps listed here under "Before You Begin". According to the documentation, you'll need to register your application through the Google API console before you can use the API.