OVER_QUERY_LIMIT status found from Geocode - c#

I have to update my database from geocode to get longitude and latitude of address stored in database.
For that I created one console application. Daily I can update 25000 records from google per user, And I am executing my console from different 3 machines everyday since last 4 to 5 days.
But I am getting OVER_QUERY_LIMIT status from today and I am not able to update database.
What should i do for this?
Here is my code to get Longitude-latitude of console:
protected static void FindLocation(string strLocation, out string lat, out string lng, out string Output)
{
StreamWriter myWriter;
System.Net.HttpWebRequest geoRequest;
string googResponse;
googResponse = "";
lat = "";
lng = "";
Output = "";
string abc = "";
string strRequestURL;
strRequestURL = "https://maps.googleapis.com/maps/api/geocode/xml?address=" + strLocation + "&sensor=false";
try
{
geoRequest = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(strRequestURL);
geoRequest.Method = "POST";
geoRequest.KeepAlive = true;
UTF8Encoding encoding = new UTF8Encoding();
byte[] bytes = encoding.GetBytes(strRequestURL);
geoRequest.ContentLength = bytes.Length;
using (Stream writeStream = geoRequest.GetRequestStream())
{
writeStream.Write(bytes, 0, bytes.Length);
writeStream.Close();
}
System.Net.HttpWebResponse geoResponse;
geoResponse = (System.Net.HttpWebResponse)geoRequest.GetResponse();
if (geoResponse != null)
{
XPathDocument document = new XPathDocument(geoResponse.GetResponseStream());
XPathNavigator navigator = document.CreateNavigator();
// get response status
XPathNodeIterator statusIterator = navigator.Select("/GeocodeResponse/status");
while (statusIterator.MoveNext())
{
if (statusIterator.Current.Value != "OK")
{
if (statusIterator.Current.Value == "OVER_QUERY_LIMIT")
{
}
Output = "Failed";
}
else
{
Output = statusIterator.Current.Value;
}
}
// get results
XPathNodeIterator resultIterator = navigator.Select("/GeocodeResponse/result");
while (resultIterator.MoveNext())
{
XPathNodeIterator geometryIterator = resultIterator.Current.Select("geometry");
while (geometryIterator.MoveNext())
{
XPathNodeIterator locationIterator = geometryIterator.Current.Select("location");
while (locationIterator.MoveNext())
{
XPathNodeIterator latIterator = locationIterator.Current.Select("lat");
while (latIterator.MoveNext())
{
lat = latIterator.Current.Value;
}
XPathNodeIterator lngIterator = locationIterator.Current.Select("lng");
while (lngIterator.MoveNext())
{
lng = lngIterator.Current.Value;
}
}
}
}
}
else
{
Output = "Failed";
}
}
catch (Exception ex)
{
Output = "Failed";
}
}

Related

create a file and upload the chunks to the in team drive from C#

I Have to Upload the file to my teamdrive, the file has been created to team drive but I can not upload the file chunks to it. So, please help to solve it.
On Writing a Chunk I am facing the Error of "The remote server returned an error: (404) Not Found."
I am getting the Teamdrive ID and the File ID which has been created.
/*** Creation of a File to Team Drive ***/
f_ObjFile.TeamDriveId = "/*TeamDrive ID*/";
try
{
f_ObjNewFile.Parents = f_ObjFile.Parents; // f_ObjFile = <Team Driv ID>
f_ObjNewFile.Name = f_ObjFile.Name;
f_ObjNewFile.MimeType = f_ObjFile.MimeType;
f_ObjNewFile.TeamDriveId = f_ObjFile.TeamDriveId;
f_CreateRequest = GoogleHelper.InvokeApiCall(() => { return this.DriveServiceObj.Files.Create(f_ObjNewFile); }, this);
if (f_CreateRequest != null)
{
f_CreateRequest.SupportsTeamDrives = true;
f_CreateRequest.Fields = "*";
f_ObjNewFile = GoogleHelper.InvokeApiCall(() => { return f_CreateRequest.Execute(); }, this);
}
f_ObjDocumentItem = new DocumentItem(UserEmailID, f_ObjNewFile);
f_ObjDocumentItem.ItemID = f_ObjNewFile.Id;
string f_Url = GoogleHelper.CreateChunkURL("https://www.googleapis.com/upload/drive/v3/files/{0}?uploadType=resumable", f_ObjNewFile.Id);
f_ObjDocumentItem.ChunkUploadURL = InitiateResumeRequest(f_Url, f_ObjNewFile.Id);
}
catch(Exception ex) { }
finally
{
f_ObjNewFile = null;
f_CreateRequest = null;
}
/* Writing the chunks to the file in TeamDrive */
try
{
httpRequest = GoogleHelper.CreateHttpWebRequestObj(f_ObjChunkData.ChunkUploadURL,true);
httpRequest.Method = GoogleConstant.PATCH;
httpRequest.ContentLength = f_ObjChunkData.FileData.Length;
httpRequest.SendChunked = true;
httpRequest.Headers["Content-Range"] = "bytes " + f_ObjChunkData.StartOffset +
"-" +
f_ObjChunkData.EndOffset + "/" +
f_ObjChunkData.FileSize.ToString();
using (System.IO.Stream f_ObjHttpStream = GoogleHelper.InvokeApiCall(() => { return httpRequest.GetRequestStream(); }, this))
{
if (f_ObjHttpStream != null)
{
System.IO.MemoryStream f_ChunkStream = null;
f_ChunkStream = new System.IO.MemoryStream(f_ObjChunkData.FileData);
f_ChunkStream.CopyTo(f_ObjHttpStream);
f_ObjHttpStream.Flush();
f_ObjHttpStream.Close();
f_ChunkStream.Close();
f_ChunkStream = null;
}
}
using (HttpWebResponse httpResponse = GoogleHelper.InvokeApiCall(() => { return (HttpWebResponse)(httpRequest.GetResponse()); }, this))
{
if (httpResponse != null)
{
if (httpResponse.StatusCode == HttpStatusCode.OK)
{
httpResponse.Close();
}
}
}
}
catch (Exception ex) { }
In Followin Line :
string f_Url = GoogleHelper.CreateChunkURL("https://www.googleapis.com/upload/drive/v3/files/{0}?uploadType=resumable", f_ObjNewFile.Id);
Insted Of
"https://www.googleapis.com/upload/drive/v3/files/{0}?uploadType=resumable"
Use following URL :
https://www.googleapis.com/upload/drive/v3/files/{0}?uploadType=resumable&supportsTeamDrives=true
and its Done...
Now you can upload the chunks...

request.getResponse() is getting error

This is my code which is returning the expected output on my friend's PC, but not on mine.
We are both working with Visual Studio 2017 Community:
enter image description here
This is the code that will return latitude and longitude of the entered address:
[enter image description here][2]
The first time it works fine but after that its throwing (403 forbidden error !!! / mainly problem is on the request.getResponse())
private static String[] x = new String[3];
public static String[] GetFirstLastName(string address)
{
try {
string url = "http://maps.google.com/maps/api/geocode/xml?address=" + address + "&sensor=false";
WebRequest request = WebRequest.Create(url);
// request.UseDefaultCredentials = true;
// request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;
// request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;
using (WebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
var ds = new DataSet("Employee");
ds.ReadXml(reader);
DataRow dr = null;
var dt = new DataTable("Employee");
dt.Columns.AddRange(new DataColumn[2]
{
new DataColumn("Latitude", typeof (string)),
new DataColumn("Longitude", typeof (string))
});
int i = 0;
try
{
foreach (DataRow row in ds.Tables["result"].Rows)
{
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
return x;
}
foreach (DataRow row in ds.Tables["result"].Rows)
{
if (i == 0)
{
string geometry_id = ds.Tables["geometry"].Select("result_id = " + row["result_id"])[0]["geometry_id"].ToString();
dr = ds.Tables["location"].Select("geometry_id = " + geometry_id)[0];
dt.Rows.Add(dr["lat"], dr["lng"]);
// Console.WriteLine(dr["lat"].ToString() + " " + dr["lng"].ToString());
i = 1;
break;
}
}
x[0] = dr["lat"].ToString();
x[1] = dr["lng"].ToString();
reader.Close();
}
// request.Timeout = 0;
// request.Abort();
response.Close();
return x;
}
}
catch(Exception e)
{
Console.WriteLine(e);
x[0] = "";
x[1] = "";
return x;
}
}
public static String[] GetFirstLastName1(string address)
{
try
{
string url = "http://maps.google.com/maps/api/geocode/xml?address=" + address + "&sensor=false";
WebRequest request = WebRequest.Create(url);
// request.UseDefaultCredentials = true;
// request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;
// request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;
using (WebResponse response = request.GetResponse())
{
using (var reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
var ds = new DataSet("Employee");
ds.ReadXml(reader);
DataRow dr = null;
var dt = new DataTable("Employee");
dt.Columns.AddRange(new DataColumn[2]
{
new DataColumn("Latitude", typeof (string)),
new DataColumn("Longitude", typeof (string))
});
int i = 0;
try
{
foreach (DataRow row in ds.Tables["result"].Rows)
{
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
return x;
}
foreach (DataRow row in ds.Tables["result"].Rows)
{
if (i == 0)
{
string geometry_id = ds.Tables["geometry"].Select("result_id = " + row["result_id"])[0]["geometry_id"].ToString();
dr = ds.Tables["location"].Select("geometry_id = " + geometry_id)[0];
dt.Rows.Add(dr["lat"], dr["lng"]);
// Console.WriteLine(dr["lat"].ToString() + " " + dr["lng"].ToString());
i = 1;
break;
}
}
x[0] = dr["lat"].ToString();
x[1] = dr["lng"].ToString();
reader.Close();
}
//// request.Timeout = 0;
/// request.Abort();
response.Close();
return x;
}
}
catch (Exception e)
{
Console.WriteLine(e);
x[0] = "";
x[1] = "";
return x;
}
}
static void Main(string[] args)
{
int i = 0;
for (;;)
{
String x = Console.ReadLine();
if (i == 0)
{
String[] y = GetFirstLastName(x);
Console.WriteLine(y[0] + " " + y[1]);
}
else
{
String[] y = GetFirstLastName1(x);
Console.WriteLine(y[0] + " " + y[1]);
}
i++;
}
//Console.ReadKey();
}
}
}
/*(Same Code above)
enter code here
///My Friends Output
/// My Output
[2]: https://i.stack.imgur.com/qeDcz.png */
Glad to see you've joined StackOverflow!
Now a 403 error occurs usually not in relation to a syntax error in your code but in relation to the response received from Google's servers.
Now Google in particular is very restrictive on how many API calls you can make a day (Google makes a lot of money off developers who pay for lots of API calls). This page contains the limits. If you've made more than the numbers in here, that's why you're getting the error and you'll have to wait until tomorrow. Do keep in mind not to send too many http requests and accidentally DOS them, as they'll blacklist you for this.
Make sure you are not caching their page or storing the js script locally as this will also cause a blacklist.
Make sure you use https: and not http: here.

Problems with Xml parsing using C#

I am developing my first windows App and I am facing some Problems while parsing an Xml,the code is as shown below
public void TimeParsing(string lat, string lon)
{
string urlbeg = "http://api.geonames.org/timezone?lat=";
string urlmid = "&lng=";
string urlend = "&username=dheeraj_kumar";
WebClient downloader = new WebClient();
Uri uri = new Uri(urlbeg + lat + urlmid + lon + urlend, UriKind.Absolute);
downloader.DownloadStringCompleted += new DownloadStringCompletedEventHandler(TimeDownloaded);
//downloader.DownloadStringCompleted += new DownloadStringCompletedEventHandler(TimeDownloaded);
downloader.DownloadStringAsync(uri);
}
private void TimeDownloaded(object sender, DownloadStringCompletedEventArgs e)
{
if (e.Result == null || e.Error != null)
{
MessageBox.Show("Invalid");
}
else
{
XDocument document = XDocument.Parse(e.Result);
var data1 = from query in document.Descendants("geoname")
select new Country
{
CurrentTime = (string)query.Element("time"),
};
foreach (var d in data1)
{
time = d.CurrentTime;
MessageBox.Show(d.CurrentTime);
// country = d.CountryName;
}
}
}
The problem is that the Delegate TimeDownloaded is not being called. I used the same technique is parse a different URL and it was done easily but its not working in this case.Kindly Help me as I am pretty new to this field.
Thanks in advance.
Theres a few misses regarding fetching the nodes
The output is geonames/timezone/time, it's corrected below, also testable using the method DownloadStringTaskAsync instead
[TestClass]
public class UnitTest1
{
[TestMethod]
public async Task TestMethod1()
{
await TimeParsing("-33.8674869", "151.20699020000006");
}
public async Task TimeParsing(string lat, string lon)
{
var urlbeg = "http://api.geonames.org/timezone?lat=";
var urlmid = "&lng=";
var urlend = "&username=dheeraj_kumar";
var downloader = new WebClient();
var uri = new Uri(urlbeg + lat + urlmid + lon + urlend, UriKind.Absolute);
downloader.DownloadStringCompleted += TimeDownloaded;
var test = await downloader.DownloadStringTaskAsync(uri);
Console.WriteLine(test);
}
private void TimeDownloaded(object sender, DownloadStringCompletedEventArgs e)
{
if (e.Result == null || e.Error != null)
{
Console.WriteLine("Invalid");
}
else
{
var document = XDocument.Parse(e.Result);
var data1 = from query in document.Descendants("timezone")
select new Country
{
CurrentTime = (string)query.Element("time"),
};
foreach (var d in data1)
{
Console.WriteLine(d.CurrentTime);
}
}
}
}
internal class Country
{
public string CurrentTime { get; set; }
}
}
you can use the below mentioned code.
Uri uri = new Uri(urlbeg + lat + urlmid + lon + urlend, UriKind.Absolute);
HttpWebRequest WebReq = (HttpWebRequest)WebRequest.Create(uri);
//This time, our method is GET.
WebReq.Method = "GET";
//From here on, it's all the same as above.
HttpWebResponse WebResp = (HttpWebResponse)WebReq.GetResponse();
//Now, we read the response (the string), and output it.
Stream Answer = WebResp.GetResponseStream();
StreamReader _Answer = new StreamReader(Answer);
string s = _Answer.ReadToEnd();
XDocument document = XDocument.Parse(s);
var data1 = from query in document.Descendants("geoname")
select new Country
{
CurrentTime = (string)query.Element("time"),
};
foreach (var d in data1)
{
time = d.CurrentTime;
MessageBox.Show(d.CurrentTime);
// country = d.CountryName;
}
for Windows Phone 8 you have to implement the getResponse Method.
public static System.Threading.Tasks.Task<System.Net.WebResponse> GetResponseAsync(this System.Net.WebRequest wr)
{
return Task<System.Net.WebResponse>.Factory.FromAsync(wr.BeginGetResponse, wr.EndGetResponse, null);
}

HtmlAgilityPack obtain Title and meta

I try to practice "HtmlAgilityPack ", but I am having some issues regarding this. here's what I coded, but I can not get correctly the title and the description of a web page ...
If someone can enlighten me on my mistake :)
...
public static void Main(string[] args)
{
string link = null;
string str;
string answer;
int curloc; // holds current location in response
string url = "http://stackoverflow.com/";
try
{
do
{
HttpWebRequest HttpWReq = (HttpWebRequest)WebRequest.Create(url);
HttpWReq.UserAgent = #"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5";
HttpWebResponse HttpWResp = (HttpWebResponse)HttpWReq.GetResponse();
//url = null; // disallow further use of this URI
Stream istrm = HttpWResp.GetResponseStream();
// Wrap the input stream in a StreamReader.
StreamReader rdr = new StreamReader(istrm);
// Read in the entire page.
str = rdr.ReadToEnd();
curloc = 0;
//WebPage result;
do
{
// Find the next URI to link to.
link = FindLink(str, ref curloc); //return the good link
Console.WriteLine("Title found: " + curloc);
//title = Title(str, ref curloc);
if (link != null)
{
Console.WriteLine("Link found: " + link);
using (System.Net.WebClient client = new System.Net.WebClient())
{
HtmlDocument htmlDoc = new HtmlDocument();
var html = client.DownloadString(url);
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
var htmlElement = htmlDoc.DocumentNode.Element("html");
HtmlNode node = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (node != null)
{
string desc = node.GetAttributeValue("content", "");
Console.Write("DESCRIPTION: " + desc);
}
else
{
Console.WriteLine("No description");
}
var titleElement =
htmlDoc.DocumentNode
.Element("html")
.Element("head")
.Element("title");
if (titleElement != null)
{
string title = titleElement.InnerText;
Console.WriteLine("Titre: {0}", title);
}
else
{
Console.WriteLine("no Title");
}
Console.Write("Done");
}
Console.Write("Link, More, Quit?");
answer = Console.ReadLine();
}
else
{
Console.WriteLine("No link found.");
break;
}
} while (link.Length > 0);
// Close the Response.
HttpWResp.Close();
} while (url != null);
}
catch{ ...}
Thanks in advance :)
Go about it this way:
HtmlNode mdnode = htmlDoc.DocumentNode.SelectSingleNode("//meta[#name='description']");
if (mdnode != null)
{
HtmlAttribute desc;
desc = mdnode.Attributes["content"];
string fulldescription = desc.Value;
Console.Write("DESCRIPTION: " + fulldescription);
}
I think your problem is here:
htmlDoc.LoadHtml(link); //chargement de HTMLAgilityPack
It should be:
htmlDoc.LoadHtml(html); //chargement de HTMLAgilityPack
LoadHtml expects a string with the HTML source, not the url.
And probably you want to change:
var html = client.DownloadString(url);
to
var html = client.DownloadString(link);
Have you used a breakpoint and gone line for line to see where the error might be occurring?
If you have, then Try something like this:
string result = string.Empty;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.google.com");
request.Method = "GET";
try
{
using (var stream = request.GetResponse().GetResponseStream())
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
result = reader.ReadToEnd();
}
}
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(result);
Then carry over the rest of your code below the htmlDoc.LoadHtml
[HttpPost]
public ActionResult Create(WebSite website)
{
string desc = HtmlAgi(website.Url, "description");
string keyword = HtmlAgi(website.Url, "Keywords");
if (ModelState.IsValid)
{
var userId = ((CustomPrincipal)User).UserId;
r.Create(new WebSite
{
Description = desc,
Tags = keyword,
Url = website.Url,
UserId = userId,
Category = website.Category
});
return RedirectToAction("Index");
}
return View(website);
}
public string HtmlAgi(string url, string key)
{
//string.Format
var Webget = new HtmlWeb();
var doc = Webget.Load(url);
HtmlNode ourNode = doc.DocumentNode.SelectSingleNode(string.Format("//meta[#name='{0}']", key));
if (ourNode != null)
{
return ourNode.GetAttributeValue("content", "");
}
else
{
return "not fount";
}
}

Server Hosting Error in C#

I developed a eCommerce Website using ASP.Net 3.5 and C#. It is working good with Visual Stdio 2010. I uploaded my website to my domain under the public folder.
It show the error like below:
Click for full-size, or here's the stack trace fragment from the local error page.
NullReferenceException: Object reference not set to an instance of an object
SageFrame.Framework.PageBase.OptimizeJs(List`1 lstJsColl, Int32 Mode) +7940
SageFrame.Framework.PageBase.LoadModuleJs() +944
SageFrame.Framework.PageBase.OnPreRender(EventArgs e) +233
System.Web.UI.PreRenderRecursiveInternal() +107
It was working fine before!
The code of the page_Load() is:
protected void Page_Load(object sender, EventArgs e)
{
string selectedCurrency = string.Empty;
string MainCurrency = string.Empty;
try
{
StoreSettingConfig ssc = new StoreSettingConfig();
MainCurrency = ssc.GetStoreSettingsByKey(StoreSetting.MainCurrency, GetStoreID, GetPortalID, GetCurrentCultureName);
if (Session["SelectedCurrency"] != null && Session["SelectedCurrency"] != "")
{
selectedCurrency = Session["SelectedCurrency"].ToString();
}
else
{
selectedCurrency = MainCurrency;
}
string islive = Request.Form["custom"];
string test = string.Empty;
const string strSandbox = "https://www.sandbox.paypal.com/cgi-bin/webscr";
const string strLive = "https://www.paypal.com/cgi-bin/webscr";
test = bool.Parse(islive.Split('#')[6]) ? strSandbox : strLive;
var req = (HttpWebRequest)WebRequest.Create(test);
//Set values for the request back
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
byte[] param = Request.BinaryRead(HttpContext.Current.Request.ContentLength);
string strRequest = Encoding.ASCII.GetString(param);
strRequest += "&cmd=_notify-validate";
req.ContentLength = strRequest.Length;
var streamOut = new StreamWriter(req.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(strRequest);
streamOut.Close();
var streamIn = new StreamReader(req.GetResponse().GetResponseStream());
string strResponse = streamIn.ReadToEnd();
streamIn.Close();
if (strResponse == "VERIFIED")
{
string payerEmail = Request.Form["payer_email"];
string paymentStatus = Request.Form["payment_status"];
string receiverEmail = Request.Form["receiver_email"];
string amount = Request.Form["mc_gross"];
string invoice = Request.Form["invoice"];
string addressName = Request.Form["address_name"];
string addressStreet = Request.Form["address_street"];
string addressCity = Request.Form["address_city"];
string addressZip = Request.Form["address_zip"];
string addressCountry = Request.Form["address_country"];
string transID = Request.Form["txn_id"];
string custom = Request.Form["custom"];
string[] ids = custom.Split('#');
int orderID = int.Parse(ids[0]);
int storeID = int.Parse(ids[1]);
int portalID = int.Parse(ids[2]);
string userName = ids[3];
int customerID = int.Parse(ids[4]);
string sessionCode = ids[5];
string pgid = ids[7];
var tinfo = new TransactionLogInfo();
var tlog = new TransactionLog();
tinfo.TransactionID = transID;
tinfo.AuthCode = "";
tinfo.TotalAmount = decimal.Parse(amount);
tinfo.ResponseCode = "1";
tinfo.ResponseReasonText = "";
tinfo.OrderID = orderID;
tinfo.StoreID = storeID;
tinfo.PortalID = portalID;
tinfo.AddedBy = userName;
tinfo.CustomerID = customerID;
tinfo.SessionCode = sessionCode;
tinfo.PaymentGatewayID = int.Parse(pgid);
tinfo.PaymentStatus = paymentStatus;
tinfo.PayerEmail = payerEmail;
tinfo.CreditCard = "";
tinfo.RecieverEmail = receiverEmail;
tinfo.CurrencyCode = selectedCurrency;
tlog.SaveTransactionLog(tinfo);
if (paymentStatus.Equals("Completed"))
{
var paypalobj = new PayPalHandler();
paypalobj.ParseIPN(orderID, transID, paymentStatus, storeID, portalID, userName, customerID, sessionCode);
}
}
else if (strResponse == "INVALID")
{
//log for manual investigation
}
else
{
//log response/ipn data for manual investigation
}
// }
}
catch (Exception ex)
{
ProcessException(ex);
// throw new Exception("This Page is not accessible!");
}
}
From the error message, it appears that your store library cannot be loaded. This could be because it isn't installed on the server or the trust level is too low.
Either way, your hosting provider could assist with the setup.

Categories