We Keep Coding

Loop API till null

I need some assistance with creating a loop until null is returned on a piece of software I've written.
The software basically takes information from an API call and deserializes it to a readable format for our online service. The difficulty I am facing is that when I make an API call it is only returning 100 records of employees when the client has far more than that.
foreach (var bankRecord in bankDetailDto.Value) //
{
var deducRecords = deductions.Where(d => d.Ee_Number == bankRecord.EmployeeNumber).ToList();
if (deducRecords.Any())
{
foreach(var deducRecord in deducRecords)
{
deducRecord.Bank_Account_Number = bankRecord.BankAccountNo;
deducRecord.Bank_Account_Type = bankRecord.AccountType;
}
}
}
This is just an example of the loop I've tried to create but does not seem to work. I am under the impression i need to create a class to perhaps run a loop on the backround worker?
Apologize I have not been developing for very long.

I guess the API call has pagination with a limit is 100 per call, I think you need to use the while-loop and check if the API response still returns any object or not.
since you didn't include your API call code, I guess it's something like this
parameter.page = 1;
List<BankData> response = yourApi.yourApiAction(parameter);
while (response != null && response.Count > 0)
{
... do your logic to process the data here ...
// Increase the pagination number
parameter.page += 1;
// Call the API again to get next page data
response = yourApi.yourApiAction(parameter)
}
This is just an example code about how it should have been done.
Check your API documentation if it has pagination, and how to increase it.

Handling Firebase "Tasks" and implement Manager Classes (for Firebase) in Unity

I'm writing the code for a game whose server-side is totally based on Firebase. I expect to use Auth, Database, InstanceID, Messaging and Cloud Functions in the game.
Being a novice C# Programmer, I encountered with C# "Tasks" first time with Firebase.
I'm going to use Database for a lot of times (like Score Update, Friends Requests, Chat, Friend Did this, Friend Did that).
I mostly feel comfortable with Singleton Pattern (GameManagers, EnemyManagers, SoundManagers etc..).
But with Firebase, since most of its calls are asynchronous and implemented via Tasks. I think I need to workaround differently to implement Managers.
For example, I need to send a Friend Request to a specific friend. The UIManager is a script that deals with UI events etc. I'd like to call Method from this script to another Manager (say FriendsManager). But I need to first check if this friend is already friend of mine from Database or Not? So, what I would do is
class UIManager
{
void OnFriendRequestClicked(string friendId)
{
bool userExists = FriendsManager.instance.UserExists(friendId);
if(userExists)
// Proceed with Sending Request
FriendsManager.instance.SendRequest(friendId);
else
// Show a Dialogue that User ID is invalid
ShowError("User Id is invalid");
// NOTE: The above code block of "condition" is executed before
// the UserID is validated from FriendsManager
// I know its because of Task. But how can I alter this code
// to do something in the similar pattern?
}
}
class FriendsManager
{
bool UserExists(string userIdToCheck)
{
reference.Child("users").Child(userIdToCheck).GetValueAsync().ContinueWith(
task=>
{
if(task.IsCompleted)
{
if(task.Result == null)
return false; // (expected) Return false to Method "UserExists"
else
return true; //(expected) Return true to Method "UserExists"
// But this won't actually return "bool" to the method,
// it actually returns to its own "Task"
//NOTE: -> How to Return from here to the Method?
)};
}

Data is loaded from Firebase asynchronously. Instead of waiting/blocking while the data is being loaded, the app continues. And then when the data is available, it calls your callback.
You can most easily see this with some logging statements:
Debug.Log("Before starting to load data");
reference.Child("users").Child(userIdToCheck).GetValueAsync().ContinueWith(task=> {
Debug.Log("Data loaded");
});
Debug.Log("After starting to load data");
When you run this code it logs:
Before starting to load data
After starting to load data
Data loaded
That is probably not what you expected, but explains perfectly why you can't return a value from within the callback: the UserExists has already finished at that point.
This means that any code that needs access to the data from the database, must be inside the ContinueWith block (or be called from there).
The simplest approach is to move the code from your OnFriendRequestClicked into UserExists:
bool UserExists(string userIdToCheck) {
reference.Child("users").Child(userIdToCheck).GetValueAsync().ContinueWith(task=>
{
if(task.IsCompleted)
{
if(task.Result == null)
ShowError("User Id is invalid");
else
FriendsManager.instance.SendRequest(friendId);
)};
}
You can then call this function without the if after it.
The above approach works great, but means that your UserExists method is no longer reusable in different cases. To make it reusable again, you can pass your own callback interface into UserExists.
For example, using Task:
bool UserExists(string userIdToCheck, Action<bool> callback) {
reference.Child("users").Child(userIdToCheck).GetValueAsync().ContinueWith(task=>
{
if(task.IsCompleted)
{
if(task.Result == null)
callback(false);
else
callback(true);
)};
}
And then to invoke it:
FriendsManager.instance.UserExists(friendId, userExists => {
if(userExists)
FriendsManager.instance.SendRequest(friendId);
else
ShowError("User Id is invalid");
})

Waiting for a frame to load using Selenium

I have seen many posts on handling switching between frames in Selenium but they all seem to reference the Java 'ExpectedConditions' library for the below method.
ExpectedConditions.frameToBeAvailableAndSwitchToIt
I was wondering if there is any C# implementation anywhere or if anyone has any such work around?
Cheers

There isn't a direct equivalent in the C# bindings but it's very easy to do this yourself.
Remember that Selenium is open source so let's dig out the source code. Here is the Java ExpectedConditions and here is the C# set.
So what's the Java version doing? Well, not a lot I tell you.
try {
return driver.switchTo().frame(frameLocator);
} catch (NoSuchFrameException e) {
return null;
}
All it's doing is attempting to switch to the frame you tell it to, and providing it was successful (as in, there was no exception in attempting to do that), then it's assumed it can carry on.
So, all you'll need to do is do the same thing in C#, so something like (not compiled):
public static Func<IWebDriver, bool> WaitUntilFrameLoadedAndSwitchToIt(By byToFindFrame)
{
return (driver) =>
{
try
{
return driver.SwitchTo().Frame(driver.FindElement(byToFindFrame));
}
catch (Exception)
{
return null;
}
return true;
};
}
As in, keep the same concept: try to find the frame and switch to it, any exceptions then we return null and force the caller (usually a WebDriverWait instance) to iterate through again. Returning true will tell the caller that we are happy we can move on.
All the waiting & expected conditions classes live in the OpenQA.Selenium.Support.UI namespace which lives in the WebDriver.Support.dll assembly.

These answers are old and I had the same issue. I was able to use SeleniumExtras.WaitHelpers.ExpectedConditions from nuget to achieve this easily.
//wait for 10 seconds max for the frame
WebDriverWaitwait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
wait.Until(SeleniumExtras.WaitHelpers.ExpectedConditions.FrameToBeAvailableAndSwitchToIt(By.Id("FRAMEID")));

I have just committed such a ugly code. Dropping here for the future!
protected void SwitchToFrame(int iframe = 1)
{
var driver = GetWebDriver();
driver.SwitchTo().DefaultContent();
bool done = false, timeout = false;
int counter = 0;
do
{
counter++;
try
{
driver.SwitchTo().Frame(iframe);
done = true;
}
catch (OpenQA.Selenium.NoSuchFrameException)
{
if (counter <= Constants.GLOBAL_MAX_WAIT_SEC)
{
Wait(1);
continue;
}
else timeout = true;
}
} while (!done && !timeout);
if (timeout) throw new OpenQA.Selenium.NoSuchFrameException(iframe.ToString());
}

Restrict number of API calls per second with Thread.Sleep?

As always, im quite the noob, as im sure you will see from both my code and question. For practice im currently writing an Xamarin.Android app for a game called Eve Online. People there mine resources from planets to make cash. These mines have to be reset at different intervals, and the real pros can have up to 30 characters doing it. Each character can have 5 planets, usually there are at least 2 mines (extractors) on each. So there could be 300 timers going on.
In my app you save your characters in an sqlite db, and every hour a intentservice runs through the API and checks your times and if their expired or not. This is how i do that:
public async Task PullPlanets(long KeyID, long CharacterID, string VCode, string CharName)
{
XmlReader lesern = XmlReader.Create("https://api.eveonline.com/char/PlanetaryColonies.xml.aspx?keyID=" + KeyID + "&vCode=" + VCode + "&characterID=" + CharacterID);
while (lesern.Read())
{
long planet = 0;
string planetName;
planet = Convert.ToInt64(lesern.GetAttribute("planetID"));
planetName = lesern.GetAttribute("planetName");
if ((planet != 0) && (planetName != null))
{
planets.Add(planet);
planetNames.Add(planetName);
await GetExpirationTimes(CharName, planet, planetName, KeyID, CharacterID, VCode);
}
}
lesern.Close ();
}
public async Task GetExpirationTimes(string CharName, long planetID, string planetName, long KeyID, long CharacterID, string VCode)
{
string planet = planetID.ToString();
XmlReader lesern = XmlReader.Create("https://api.eveonline.com/char/PlanetaryPins.xml.aspx?keyID=" + KeyID + "&vCode=" + VCode + "&characterID=" + CharacterID + "&planetID=" + planet);
while (lesern.Read())
{
string expTime;
expTime = lesern.GetAttribute("expiryTime");
if ((expTime != null) && (expTime != "0001-01-01 00:00:00"))
{
allInfo.Add (new AllInfo (CharName, planetName, Convert.ToDateTime (expTime)));
}
}
lesern.Close ();
SendOrderedBroadcast (stocksIntent, null);
}
}
After this, it sends the times back to my Activity, where they get added to an extractor. It seems to work pretty fine, although ive only been able to test with 2 characters with a total of 14 extractors so far. An alarmmanger in activity calls the service every hour, and it sends a notification. When user opens the activity, it pulls the list from service, sorts it, and displays it. I would welcome input on if this is the way to do it.
I do see a problem in the horizon, though. The Eve API blocks if an app surpases 30 API-calls per second. Im pretty sure someone with 30 characters would do that. So, im wondering if i should add something to delay each call if a certain number is passed? This is how i call the first XML call.
var table = db.Table<CharsList> ();
foreach (var e in table) {
long KeyIDOut = Convert.ToInt64(e.KeyID);
long CharIDOut = Convert.ToInt64(e.CharacterID);
string VCodeOut = e.VCode.ToString();
string navnOut = e.Name.ToString();
PullPlanets(KeyIDOut, CharIDOut, VCodeOut, navnOut);
}
CheckTimes ();
}
Is it viable to add a
if (table.Count > 10) {
foreach (var e in table) {
//start the first characters call
Thread.Sleep(100)
}
The service is intentservice and not on UI thread. I guess this would bring the calls under 30 a sec, but i have never used Thread.Sleep and fear what else could happen in my code. Are there other things that could help me not blow the limit? Can this code handle 300 extractors?

I believe you are generally right in your approach. I had to do a similar thing for a reddit client I was writing, except their limits is once a second or so.
The only problem I see with your setup is that assume that Thread.Sleep does sleep for the amount of time you give it. Spurious wakeups are possible in some cases, so what I would suggest is that you give it a smaller value, save the last time you accessed the service and then put a loop around the sleep call that terminates once enough time has passed.
Finally if you are going to be firing up a lot of intent services for a relatively short amount of work, you might want to have a normal service with a thread to handle the work - that way it will only have to be created once but it is still of the UI thread.

Handle a function taking a lot of time Threading

I have a function which is taking a lot of time to execute in a web application.
I have tested this with a profiler and by my logging.
I have other functions running in the same pageload.
What is a best way to display the rest of the values from those functions and keep this function in a thread and display it in a label when it finishes?
This function is used to get events in application which takes time.
private void getEventErrors()
{
EventLog eventLog = new EventLog("Application", ".");
getEvents(eventLog.Entries);
}
private void getEvents(EventLogEntryCollection eventLogEntryCollection)
{
int errorEvents = 0;
foreach (EventLogEntry logEntry in eventLogEntryCollection)
{
if (logEntry.Source.Equals("XYZ"))
{
DateTime variable = Convert.ToDateTime(logEntry.TimeWritten);
long eventTimeTicks = (variable.Ticks);
long eventTimeUTC = (eventTimeTicks - 621355968000000000) / 10000000;
long presentDayTicks = DateTime.Now.Ticks;
long daysBackSeconds = ((presentDayTicks - 864000000000) - 621355968000000000) / 10000000;
if (eventTimeUTC > daysBackSeconds)
{
if (logEntry.EntryType.ToString() == "Error")
{
errorEvents = errorEvents + 1;
}
}
}
}
btn_Link_Event_Errors_Val.Text = errorEvents.ToString(GUIUtility.TWO_DECIMAL_PT_FORMAT);
if (errorEvents == 0)
{
lbl_EventErrorColor.Attributes.Clear();
lbl_EventErrorColor.Attributes.Add("class", "green");
}
else
{
lbl_EventErrorColor.Attributes.Clear();
lbl_EventErrorColor.Attributes.Add("class", "red");
}
}
I have 3 functions in the pageload event, two to get the values from the DB and the other one is shown above.
Should both these functions be service calls?
What i wanted was, the page should load fast and if there is a function taking a lot of time it should run in the background and display when done and in the process if the user want to navigate to a new page it should kill it and move on.

If you have a function that is running in a separate thread in ASP.NET, you may want to consider moving it to a service. There are many reason for this
See this answer (one of many on SO) for why running long running tasks in ASP.NET is not always a good idea.
One option for the service is to use WCF. You can get started here. Your service could implement a method, say GetEvents() which you could use to pull your events. That way you won't tie up your page waiting for this process to complete (using AJAX of course). Also, this allows you to change your implementation of GetEvents() without touching your code on your website.