We Keep Coding

Why does my call to the Google Custom Search API fail with a Request Error (Invalid Argument)?

I need to get the results of google searches in order to loop through and parse them. With that aim in view, I followed (as best I could) the tutorial on how to do that here
This is my code, based on the sample/example code in the article referenced above:
private void btnRentFlick_Click(object sender, EventArgs e)
{
OpenBestPageForSearchString("rent amazon movie Will Penny");
}
private void OpenBestPageForSearchString(string searchStr)
{
try
{
const string apiKey = "blaBlaBla"; // "blaBlaBla" stands for my API key
const string searchEngineId = "bla"; // "bla" stands for various things I tried: my client_id
(also called UniqueId), private_key_id (also called KeyId), and project_id. Not having
the correct value may be the problem. If so, how do I get it?
const string query = "rent amazon movie Will Penny";
var customSearchService = new CustomsearchService(new BaseClientService.Initializer { ApiKey
= apiKey });
//CseResource.ListRequest listRequest = customSearchService.Cse.List(query); // This is the
code in the article, but it won't compile - "no overload for "List" takes one argument"
// So how is the value in "query" assigned, then?
CseResource.ListRequest listRequest = customSearchService.Cse.List();
listRequest.Cx = searchEngineId;
List<string> linksReturned = new List<string>();
IList<Result> paging = new List<Result>();
var count = 0; // I don't know what the purpose of the counting is, but I'll leave as-is
until I get it working at least
while (paging != null)
{
listRequest.Start = count * 10 + 1;
paging = listRequest.Execute().Items; // this takes several seconds, then it throws an
exception
if (paging != null)
{
foreach (var item in paging)
{
linksReturned.Add("Title : " + item.Title + Environment.NewLine + "Link : " +
item.Link +
Environment.NewLine + Environment.NewLine);
}
}
count++;
}
MessageBox.Show("Done with google amazon query");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
As the comment at the end of that line says, this line of code:
paging = listRequest.Execute().Items;
...works for several seconds, then throws an exception, namely this:
So what is causing this exception? Is it because the searchEngineId value I assigned is bad? Or is it because the search string (assigned to the query variable) has not been provided to the call?
The info about my Ids is contained in a .json file provided by google, and there is no "searchEngineId" value in it. This is what it does contain:
"type": "service_account", "project_id": "flix4famsasinlocator",
"private_key_id": "[my private key id]", "private_key": "-----BEGIN
PRIVATE KEY-----. . . PRIVATE KEY-----\n", "client_email":
"[bla].gserviceaccount.com", "client_id": "[my client Id]",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url":
"https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url":
"https://www.googleapis.com/robot/v1/metadata/x509/[bla]gserviceaccount.com"
So though the article previously mentioned purported to be, and at first appeared to be, just what the doctor ordered, I have ran into a wall of considerable dimensions. Does anybody know how to scale this wall - perhaps primarily by providing the search string to the CseResource.ListRequest object?
UPDATE
Trying DalmTo's code first, I used this (not showing his GetService() method, which I copied verbatim):
var query = "rent amazon movie Will Penny";
var service = GetService("theRainInSpainFallsMainlyOnTheDirt");
var request = service.Cse.List();
// add option values to the request here.
request.ExactTerms = query;
request.Q = query;
var response = request.ExecuteAsync();
// my contribution:
List<string> linksReturned = new List<string>();
foreach (var item in response.Result.Items)
{
//Console.WriteLine(item.Title);
// next two lines also mine
MessageBox.Show(string.Format("Title: {0}; Link: {1}; ETag: {2}", item.Title, item.Link, item.ETag));
linksReturned.Add(item.Link);
}
...but this exception was thrown while in the foreach loop:
UPDATE 2
Yes, this works (adapted from Trekco's answer):
const string apiKey = "gr8GooglyMoogly";
const string searchEngineId = "theRainInSpainFallsMainOnTheDirt";
const string query = "rent amazon movie Will Penny";
var customSearchService = new CustomsearchService(new BaseClientService.Initializer { ApiKey = apiKey });
CseResource.ListRequest listRequest = customSearchService.Cse.List();
listRequest.Cx = searchEngineId;
listRequest.Q = query;
List<string> linksReturned = new List<string>();
IList<Result> paging = new List<Result>();
var count = 0;
while (paging != null)
{
listRequest.Start = count * 10 + 1;
paging = listRequest.Execute().Items;
if (paging != null)
{
foreach (var item in paging)
{
linksReturned.Add(item.Link);
}
}
count++;
}

The query is not being send to google. To fix your code you need to tell the api what query to use. After listRequest.Cx = searchEngineId; add listRequest.Q = query;
var count = 0;
string apiKey = "THE API KEY";
string searchEngineId = "THE SEARCH ENGIN ID";
string query = "rent amazon movie Will Penny";
var customSearchService = new CustomsearchService(new BaseClientService.Initializer
{
ApiKey = apiKey
});
CseResource.ListRequest listRequest = customSearchService.Cse.List();
listRequest.Cx = searchEngineId;
listRequest.Q = query; // <---- Add this line
List<string> linksReturned = new List<string>();
while (count < 10) // Google limit you to 100 records
{
listRequest.Start = count * 10;
var paging = listRequest.Execute().Items;
foreach (var item in paging)
{
linksReturned.Add("Title : " + item.Title + Environment.NewLine + "Link : " +
item.Link +
Environment.NewLine + Environment.NewLine);
}
count++;
}
In your code you have a comment that you don't know what var count = 0; is for. It is to keep track on how many items you have requested.
If you look at google's documentation you will see that they will only return 100 results max. After that they will give you a error. That error will also be the same generic message: "INVALID_ARGUMENT"
You can review the custom search api requirements here: https://developers.google.com/custom-search/v1/reference/rest/v1/cse/list
The searchEngineId variable is the search Engine id that you generate on the site https://www.google.com/cse/all. The documentation you followed is a bit out of date. you will find the id here:

If you check the doucmntation cse.list I think you will find that the list method has no required fields which means that you need to add the option values in the following manner.
I think ExactTerms may be the one you are looking for. But it could also be Q i think you should read though the option values and decide which one is best for your purpose.
var query = "rent amazon movie Will Penny";
var service = GetService("MYKEY");
var request = service.Cse.List();
// add option values to the request here.
request.ExactTerms = query;
request.Q = query;
var response = request.ExecuteAsync();
foreach (var item in response.Result.Items)
{
Console.WriteLine(item.Title);
}
My get service method
public static CustomsearchService GetService(string apiKey)
{
try
{
if (string.IsNullOrEmpty(apiKey))
throw new ArgumentNullException("api Key");
return new CustomsearchService(new BaseClientService.Initializer()
{
ApiKey = apiKey,
ApplicationName = string.Format("{0} API key example", System.Diagnostics.Process.GetCurrentProcess().ProcessName),
});
}
catch (Exception ex)
{
throw new Exception("Failed to create new Customsearch Service", ex);
}
}

API is mixing up data from different devices

I have an API that has devices firing data to it at the same time or within a few milliseconds. What I am finding is that the data is getting mixed up. The data is sent every five minutes (on the clock 05, 10, 15 etc.) I have an execution filter that traps the URL data coming in so I always have a real source, then it goes to the endpoint and then onto processing. For example, there will a be random five minute period missing. When I debug step by step with the missing URL from the execution filter it works fine. By that I mean I take the URL and debug, then it inserts.
In summary, I have device id 1 and device id 2.I will get missing intervals even though, I can see the data has hit the execution filter.
I am assuming that the API is not handling these as separate transactions, but somehow mixing them up together, hence the data missing and the serial numbers appearing in the wrong place, such that data from id 1 is appearing in id 2 vice versa etc.
API End Point:
public class SomeController : ApiController
{
[HttpGet]
[ExecutionFilter]
public async Task<HttpResponseMessage> Get([FromUri] FixedDataModel fdm)
{
var reply = new HttpResponseMessage();
string url = HttpUtility.UrlDecode(HttpContext.Current.Request.QueryString.ToString());
if (url.Contains("timestamp"))
{
reply = TimeSyncValidation.TimeSync;
return reply;
}
else if (!url.Contains("timestamp"))
{
reply = await Task.Run(() => DeviceClass.DeviceApiAsync(fdm, url));
}
return reply;
}
}
Processing class:
namespace API.Services
{
public class DeviceClass
{
private static string serialNumber;
private static byte chk;
private static string channelName, channelReadingNumber, channelValue, queryString, readingDate;
private static int colonPosition, chanCountFrom, equalsPosition;
private static bool checkSumCorrect;
public static HttpResponseMessage DeviceApiAsync(FixedDataModel fdm, string urlQqueryString)
{
Guid guid = Guid.NewGuid();
//ExecutionTrackerHandler.Guid = guid;
//Remove question mark
var q = urlQqueryString;
queryString = q.Substring(0);
var items = HttpUtility.ParseQueryString(queryString);
serialNumber = items["se"];
//Store raw uri for fault finding
var rawUri = new List<RawUriModel>
{
new RawUriModel
{
UniqueId = guid,
RawUri = q,
TimeStamp = DateTime.Now
}
};
//Checksum validation
chk = Convert.ToByte(fdm.chk);
checkSumCorrect = CheckSumValidator.XorCheckSum(queryString, chk);
if (!checkSumCorrect)
{
return ValidationResponseMessage.ResponseHeaders("Checksum");
}
//Create list of items that exist in URL
var urldata = new UrlDataList
{
UrlData = queryString.Split('&').ToList(),
};
var data = new List<UriDataModel>();
//Split the URL string into its parts
foreach (var item in urldata.UrlData)
{
colonPosition = item.IndexOf(":");
chanCountFrom = colonPosition + 1;
equalsPosition = item.LastIndexOf("=");
if (colonPosition == -1)
{
channelName = item.Substring(0, equalsPosition);
channelReadingNumber = "";
channelValue = item.Substring(item.LastIndexOf("=") + 1);
}
else
{
channelName = item.Substring(0, colonPosition);
channelReadingNumber = item.Substring(chanCountFrom, equalsPosition - chanCountFrom);
channelValue = item.Substring(item.LastIndexOf("=") + 1);
if (channelName == "atime" || channelName == "adate")
{
readingDate = DateValidator.CreateDate(channelValue);
}
};
bool nullFlag = false;
if (channelValue == null)
nullFlag = true;
bool missingFlag = false;
if (channelValue == "x") {
missingFlag = true;
channelValue = "0";
}
//Add data to model ready for DB insert.
data.Add(new UriDataModel
{
uid = guid,
SerialNumber = serialNumber,
ChannelName = channelName,
ChannelReadingNumber = channelReadingNumber,
ChannelValue = channelValue.Replace(",", "."),
ReadingDate = readingDate,
TimeStamp = DateTime.Now.ToString("yyyy-MM-dd HH:mm"),
Processed = false,
NullFlag = nullFlag,
MissingFlag = missingFlag
});
};
//Validate dates
var allDates = (from x in data where x.ChannelName.Contains("atime") || x.ChannelName.Contains("adate") select x.ChannelValue).ToList();
bool dateValidation = DateValidator.IsValid(allDates);
if (!dateValidation)
{
return ValidationResponseMessage.ResponseHeaders("Date");
};
//Validate values
var channels = Enum.GetNames(typeof(Channels)).ToList();
List<string> allChannelValues = data.Where(d => channels.Contains(d.ChannelName)).Select(d => d.ChannelValue).ToList();
bool valueValidation = ValueValidator.IsValid(allChannelValues);
if (!valueValidation)
{
return ValidationResponseMessage.ResponseHeaders("Values");
};
//Insert live data
var insertData = DataInsert<UriDataModel>.InsertData(data, "Staging.UriData");
if (!insertData)
{
return ValidationResponseMessage.ResponseHeaders("Sql");
}
var content = "\r\nSUCCESS\r\n";
var reply = new HttpResponseMessage(System.Net.HttpStatusCode.OK)
{
Content = new StringContent(content)
};
return reply;
}
}
}
TIA

You are using global variables and static method to process your data.
Change your method to non-static.
Each DeviceClass worker must update only its own isolated data then push that off back to controller.

call back query data from inline keyboard

I'm trying to get the data from an inline keyboard, I've searched a lot but unfortunately didn't get my answer and non of codes worked for me. Here's my code please help me
static void Main(string[] args){
InlineKeyboardButton[][] buttons =
new InlineKeyboardButton[][]{
new InlineKeyboardButton[]{newInlineKeyboardButton() { Text = "show Channel", CallbackData = "Data1" }},
new InlineKeyboardButton[]{new InlineKeyboardButton() { Text = "show Website", CallbackData = "Data2" }}};
inlineKeyboardMarkup = new InlineKeyboardMarkup() { InlineKeyboard = buttons };
Task.Run(() => RunBot());
Console.ReadLine();
} // End of main method
public static async Task RunBot(){
while (true){
var u = await bot.MakeRequestAsync(new GetUpdates() { Offset
= offset });
foreach (var update in u)
{
offset = update.UpdateId + 1;
var text = update.Message.Text;
// here I want to get the data like this, but it doesn't work
if (update.ChosenInlineResult != null){
Console.WriteLine("Chosen Inline Result: " +
update.ChosenInlineResult.ToString());
}
switch(text){
case "Something":{
var req = new SendMessage(update.Message.Chat.Id, "راهنما") { ReplyMarkup = inlineKeyboardMarkup };
await bot.MakeRequestAsync(req);
break;
}
}
}
}
}

you must replace this
if (update.ChosenInlineResult != null){
Console.WriteLine("Chosen Inline Result: " +
update.ChosenInlineResult.ToString());
}
with something like This :
if (update.CallbackQuery != null)
{
Console.WriteLine(val.CallbackQuery.Message+"-"+val.CallbackQuery.Data);
}

multiple ajax calls to async Action breaks code

I'm writing a web application in .Net Core Framework and certain elements of the front end use Ajax calls to API that returns certain data.
If I'm on a page that requires 1 API call, it works fine, however as soon as I get on a page that makes more than 1 calls, debugging (well, going line by line in the code) becomes hell with the debugger jumping up and down the code) and in the end the results all end up as errors.
Rarely will I get the results if a website makes more calls, sometimes it happens but odds of that are.. 1 / 50 and this is unacceptable.
Weirdly enough, after I visit such page and then go back to a page that makes 1 call, even that 1 call ends with an error.
I'm not really sure what I need to include here as a code so I'll add both jquery and the action.
Before I put any code here, I'd like to explain my question..
I'd like to know what exactly is causing this, and why are the ajax calls causing parallel executions of the Action and how could I force that ajax calls are ran one by one or do anything else to fix this issue. I believe at most 30,40 calls should be made by some Views and if even 2 cause such issues, there's obviously a problem here, but unfortunately I'm not skilled enough yet to see and fix it.
Any help will be greatly appreciated.
jquery (I've removed pieces of code simply because they aren't relevant. it's just some class changes):
$(window).ready(function () {
$('.price').each(function (i, e) {
var id = $(e).data('skinid');
var exterior = $(e).data('exterior');
var type = $(e).data('type');
$.ajax({
type: "GET",
url: "/Skin/GetPrice",
data: { Id: id, Exterior : exterior, Type : type },
cache: false,
dataType: "json",
}).complete(function (data) {
var price = data.responseJSON.price;
if (data.responseJSON.querySuccessful)
{
if (data.responseJSON.listingExists)
{
//do stuff here
}
else
{
//other stuff here
}
}
else
{
//print error..
}
});
});
});
Action is quite long so I'll short it up as well:
[HttpGet]
public async Task<ActionResult> GetPrice(int Id = 0, string Exterior = null, string Type = null)
{
PriceViewModel model = null;
if (_context.Skins.Any(x => x.Id == Id))
{
if (Type == "Skin")
{
Skin Query = _context.Skins.Where(x => x.Id == Id).Include(x => x.Weapon).Include(x => x.Quality).Select(x => x).Single();
using (var client = new HttpClient())
{
try
{
string uri = "/market/priceoverview/?currency=3&appid=730&market_hash_name=" + Query.Weapon.Name.Replace(" ", "%20") + "%20|%20" + Query.Name.Replace(" ", "%20") + "%20(" + Exterior.Replace(" ", "%20") + ")";
client.BaseAddress = new Uri("http://steamcommunity.com");
var response = await client.GetAsync(uri);
response.EnsureSuccessStatusCode(); // Throw in not success
var stringResponse = await response.Content.ReadAsStringAsync();
JObject data = JObject.Parse(stringResponse);
if (data["success"].Value<bool>())
{
if (data["lowest_price"] != null)
{
// return good model
}
//return no result model
}
}
catch
{
// return error
}
}
}
if (Type == "Skin-List")
{
Skin Query = _context.Skins.Where(x => x.Id == Id).Include(x => x.Weapon).Include(x => x.Quality).Select(x => x).Single();
Dictionary<Exterior, PriceViewModel> PossibleExteriors = new Dictionary<Exterior, PriceViewModel>();
bool possible = false;
foreach (Exterior e in _context.Exteriors)
{
if (e.Name == Query.BestExterior)
possible = true;
if (possible)
PossibleExteriors.Add(e, null);
if (e.Name == Query.WorstExterior)
possible = false;
}
List<Exterior> exteriors = PossibleExteriors.Keys.ToList();
foreach (Exterior e in exteriors)
{
using (var client = new HttpClient())
{
try
{
client.BaseAddress = new Uri("http://steamcommunity.com");
string uri = "/market/priceoverview/?currency=3&appid=730&market_hash_name=" + Query.Weapon.Name.Replace(" ", "%20") + "%20|%20" + Query.Name.Replace(" ", "%20") + "%20(" + e.Name.Replace(" ", "%20") + ")";
var response = await client.GetAsync(uri);
response.EnsureSuccessStatusCode(); // Throw in not success
var stringResponse = await response.Content.ReadAsStringAsync();
JObject data = JObject.Parse(stringResponse);
if (data["success"].Value<bool>())
{
if (data["lowest_price"] != null)
{
model = new PriceViewModel
{
QuerySuccessful = true,
ListingExists = true,
ListingUrl = "http://steamcommunity.com/market/listings/730/" + Query.Weapon.Name + " | " + Query.Name + " (" + e.Name.Replace(" ", "%20") + ")",
Price = data["lowest_price"].ToString()
};
PossibleExteriors[e] = model;
}
else
{
model = new PriceViewModel
{
QuerySuccessful = true,
ListingExists = false,
ListingUrl = "http://steamcommunity.com/market/listings/730/" + Query.Weapon.Name + " | " + Query.Name + " (" + e.Name.Replace(" ", "%20") + ")",
Price = ""
};
PossibleExteriors[e] = model;
}
}
}
catch
{
model = new PriceViewModel
{
QuerySuccessful = false,
ListingExists = false,
Price = ""
};
PossibleExteriors[e] = null;
}
}
}
if (PossibleExteriors.Values.Any(x => x != null))
{
//return good model
}
else
{
//return empty
}
}
}
//return error
}

I'd like to know what exactly is causing this, and why are the ajax calls causing parallel executions of the Action and how could I force that ajax calls are ran one by one or do anything else to fix this issue.
You've got a fundamental misunderstanding about asynchronous calls. This JavaScript will fire off a request, then continue into the next iteration of the loop before getting a response. The complete function will be called whenever an asynchronous call is completed, whenever that happens and in whatever order it happens in. This means that new calls are arriving faster than the application can return results, so it's working on more than one request at a time. Perfectly normal!
If these calls need to be made in a specific order, why not lump them all into one request? Pass a list of items into the data property and do a loop in C#.
If this has to be a loop in JavaScript, you'll need to wait for a response from the asynchronous call (basically make it synchronous) If you want to do this calls synchronously, as in call GetPrice, get a response, do something with the response, then go to the next iteration and repeat, you'll need to change the approach.
var $prices = $('.prices').map(function() {
return new { Id: id, Exterior : exterior, Type : type }
});
getPrice($prices, 0);
function getPrice(prices, index) {
$.ajax({
type: "GET",
url: "/Skin/GetPrice",
data: prices[index] // if this were the whole prices list, you could do the loop in C# instead
cache: false,
dataType: "json",
}).complete(function (data) {
// do all that processing stuff
index++;
// if there are more prices to compute, call the function again
if (prices.length - 1 >= index) {
getPrice(prices, index);
}
});
}

.NET C# - Google AdWords API Authorization Issue "USER_PERMISSION_DENIED"

I've been trying to connect to the Google AdWords API, and I succeeded
with receiving information from only one client.
Everytime I use a different ClientCustomerId, it throws an exception with
an inner detail - "USER_PERMISSION_DENIED".
(Written in Web.config <add key="ClientCustomerId" value="val-val-val"/>)
Only one client works though. I have no idea with differences they have, they're all same. the only difference is that the working client is the first in my client list.
Here is my code for receiving campaign information:
AdWordsUser user = new AdWordsUser();
CampaignService campaignService =
(CampaignService)user.GetService(AdWordsService.v201506.CampaignService);
Selector selector = new Selector();
selector.fields = new string[] { "Id", "Name", "Status"};
selector.paging = new Paging();
int offset = 0;
int pageSize = 500;
CampaignPage page = new CampaignPage();
try
{
do
{
selector.paging.startIndex = offset;
selector.paging.numberResults = pageSize;
string result = "";
// Get the campaigns.
page = campaignService.get(selector);
// Display the results.
if (page != null && page.entries != null)
{
int i = offset;
foreach (Google.Api.Ads.AdWords.v201506.Campaign campaign in page.entries)
{
result += string.Format("{0}) Campaign with id = '{1}', name = '{2}' and status = '{3}'" +
" was found.", i + 1, campaign.id, campaign.name, campaign.status);
i++;
}
}
Response.Write(result);
offset += pageSize;
} while (offset < page.totalNumEntries);
int a = page.totalNumEntries;
}
catch (Exception e)
{
throw new System.ApplicationException("Failed to retrieve campaigns", e);
}
Thanks in advance!