I would like to compile and execute code from a string. Calling the method ExecuteCode() successfully works and returns a string value. However, calling ExecuteEndlessCode() hangs the Console application. How can I detect that CSharpScript.EvaluateAsync is potentially executing code that is running forever?
public static string ExecuteEndlessCode()
{
var script = #"string Run() { while(true){} return ""1"";} Run()";
var result = CSharpScript.EvaluateAsync<string>(script, null).Result;
return result;
}
public static string ExecuteCode()
{
var script = #"string Run() { return ""1"";} Run()";
var result = CSharpScript.EvaluateAsync<string>(script, null).Result;
return result;
}
You can set a timeout for the script, and execute the script execution inside a task, like this:
var script = #" int Run() { while(true){} return 1;} Run();";
var task = CSharpScript.EvaluateAsync<string>(script, null)
if (await Task.WhenAny(task, Task.Delay(timeout) != task)
{ // handle timeout
}
Related
I wrote a web crawler and I want to know if my approach is correct. The only issue I'm facing is that it stops after some hours of crawling. No exception, it just stops.
1 - the private members and the constructor:
private const int CONCURRENT_CONNECTIONS = 5;
private readonly HttpClient _client;
private readonly string[] _services = new string[2] {
"https://example.com/items?id=ID_HERE",
"https://another_example.com/items?id=ID_HERE"
}
private readonly List<SemaphoreSlim> _semaphores;
public Crawler() {
ServicePointManager.DefaultConnectionLimit = CONCURRENT_CONNECTIONS;
_client = new HttpClient();
_semaphores = new List<SemaphoreSlim>();
foreach (var _ in _services) {
_semaphores.Add(new SemaphoreSlim(CONCURRENT_CONNECTIONS));
}
}
Single HttpClient instance.
The _services is just a string array that contains the URL, they are not the same domain.
I'm using semaphores (one per domain) since I read that it's not a good idea to use the network queue (I don't remember how it calls).
2 - The Run method, which is the one I will call to start crawling.
public async Run(List<int> ids) {
const int BATCH_COUNT = 1000;
var svcIndex = 0;
var tasks = new List<Task<string>>(BATCH_COUNT);
foreach (var itemId in ids) {
tasks.Add(DownloadItem(svcIndex, _services[svcIndex].Replace("ID_HERE", $"{itemId}")));
if (++svcIndex >= _services.Length) {
svcIndex = 0;
}
if (tasks.Count >= BATCH_COUNT) {
var results = await Task.WhenAll(tasks);
await SaveDownloadedData(results);
tasks.Clear();
}
}
if (tasks.Count > 0) {
var results = await Task.WhenAll(tasks);
await SaveDownloadedData(results);
tasks.Clear();
}
}
DownloadItem is an async function that actually makes the GET request, note that I'm not awaiting it here.
If the number of tasks reaches the BATCH_COUNT, I will await all to complete and save the results to file.
3 - The DownloadItem function.
private async Task<string> DownloadItem(int serviceIndex, string link) {
var needReleaseSemaphore = true;
var result = string.Empty;
try {
await _semaphores[serviceIndex].WaitAsync();
var r = await _client.GetStringAsync(link);
_semaphores[serviceIndex].Release();
needReleaseSemaphore = false;
// DUE TO JSON SIZE, I NEED TO REMOVE A VALUE (IT'S USELESS FOR ME)
var obj = JObject.Parse(r);
if (obj.ContainsKey("blah")) {
obj.Remove("blah");
}
result = obj.ToString(Formatting.None);
} catch {
result = string.Empty;
// SINCE I GOT AN EXCEPTION, I WILL 'LOCK' THIS SERVICE FOR 1 MINUTE.
// IF I RELEASED THIS SEMAPHORE, I WILL LOCK IT AGAIN FIRST.
if (!needReleaseSemaphore) {
await _semaphores[serviceIndex].WaitAsync();
needReleaseSemaphore = true;
}
await Task.Delay(60_000);
} finally {
// RELEASE THE SEMAPHORE, IF NEEDED.
if (needReleaseSemaphore) {
_semaphores[serviceIndex].Release();
}
}
return result;
}
4- The function that saves the result.
private async Task SaveDownloadedData(List<string> myData) {
using var fs = new FileStream("./output.dat", FileMode.Append);
foreach (var res in myData) {
var blob = Encoding.UTF8.GetBytes(res);
await fs.WriteAsync(BitConverter.GetBytes((uint)blob.Length));
await fs.WriteAsync(blob);
}
await fs.DisposeAsync();
}
5- Finally, the Main function.
static async Task Main(string[] args) {
var crawler = new Crawler();
var items = LoadItemIds();
await crawler.Run(items);
}
After all this, is my approach correct? I need to make millions of requests, will take some weeks/months to gather all data I need (due to the connection limit).
After 12 - 14 hours, it just stops and I need to manually restart the app (memory usage is ok, my VPS has 1 GB and it never used more than 60%).
Edit:
here is my main method
static void Main(string[] args)
{
LogNewOrders();
DataTable initialData = ControllerSqlAgent.SelectQuery(Resources.Controller, Resources.qryGetInitalData);
Console.WriteLine($"|There are {initialData.Rows.Count} orders to check|");
Task.WhenAll(UpdateItemCountsField(initialData));
}
I have a method called UpdateItemCountsField(datatable)
The purpose of this method is to get both:
total cancelled items
total shipped items
private static async Task UpdateItemCountsField(DataTable initialData)
{
try
{
foreach (DataRow row in initialData.Rows)
{
string narvarId = row["NarvarID"].ToString();
int orderedItemCount = (int)row["ItemsOrdered"];
int totalShippedItems = (int)row["ItemsShipped"]; ;
int totalCancelledItems = (int)row["ItemsCancelled"];
string locateConstraint = GetLocateInConstraint(row["OrderNumber"].ToString(), row["CompanyNumber"].ToString());
Task<int> totalShippedItemsTask = CheckShipmentCountsAsync(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalShippedItems, narvarId);
Task<int> totalCancelledItemsTask = CheckCancellationCountsAsync(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalCancelledItems, narvarId); ;
int[] result = await Task.WhenAll(totalShippedItemsTask, totalCancelledItemsTask);
//totalShippedItems = CheckShipmentCounts(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalShippedItems, narvarId);
//totalCancelledItems = CheckCancellationCounts(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalCancelledItems, narvarId);
Console.WriteLine($"|ID:{narvarId}|Ordered: {orderedItemCount}|Shipped: {result[0]}|Cancelled: {result[1]}|");
Console.WriteLine("|___________________________________________|");
}
}
catch
{
throw;
}
}
I have two Task objects that I obtain from async methods:
Task<int> totalShippedItemsTask = CheckShipmentCountsAsync(params);
Task<int> totalCancelledItemsTask = CheckCancellationCountsAsync(params);
Then i get my results like so (this is where the program crashes)
int[] result = await Task.WhenAll(totalShippedItemsTask,totalCancelledItemsTask);
within CheckShipmentCoutnsAsync(params) and CheckCancellationCountsAsync(params) are two more Task<int> objects . This is due to there being two data sources that i have to pull from
private static async Task<int> CheckShipmentCountsAsync(string locateConstraint, string orderNumber, string companyNumber, int currentShippedItemCount, string narvarId)
{
Task<int> wmsShippedCountTask = WmsSqlAgent.GetCountOfShippedItemsAsync(orderNumber, companyNumber);
Task<int> locateShippedCountTask = LocateSqlAgent.GetCountOfShippedItemsAsync(locateConstraint);
int[] result = await Task.WhenAll(wmsShippedCountTask, locateShippedCountTask);
int newShippedItemCount = result.Sum();
if (newShippedItemCount > currentShippedItemCount)
ControllerSqlAgent.UpdateShippedItemCount(narvarId, newShippedItemCount);
return newShippedItemCount;
}
The method CheckCancelCountsAsync(PARAMS) is the same as the above but for cancellations.
When my program runs, It hits the task and then crashes without any exception and never reaches the next datarow to perform the asynchronous methods on. Am I missing a piece here? Why would my application crash without an exception. the reason I believe it is crashing due to the fact that it never makes it to the Console.Writeline(); right after i get the result array from Task.WhenAll
And here is the data source method that retrieves the actual count
internal static async Task<int> GetCountOfCancelledItemsAsync(string orderNumber, string companyNumber)
{
int itemCount = 0;
await Task.Run(() =>
{
try
{
using (var connection = new SqlConnection(Resources.WMS))
{
connection.Open();
using (var command = new SqlCommand(Resources.qryWmsGetCancelledItemCount, connection))
{
command.Parameters.AddWithValue("#orderNumber", orderNumber);
command.Parameters.AddWithValue("#companyNumber", companyNumber);
itemCount = int.Parse(command.ExecuteScalar().ToString());
}
connection.Close();
}
}
catch (SqlException E)
{
string message = E.Message;
throw;
}
return itemCount;
});
return itemCount;
While stepping through the code I can see that my program crashes when i get to nt[] result = await Task.WhenAll(totalShippedItemsTask, totalCancelledItemsTask); but still no exception.
You have not awaited the results of Task.WhenAll(UpdateItemCountsField(initialData)) in your main method. This means that the program will exit without waiting for that code to finish executing.
Change your method to this:
static async Task Main(string[] args)
{
LogNewOrders();
DataTable initialData = ControllerSqlAgent.SelectQuery(Resources.Controller, Resources.qryGetInitalData);
Console.WriteLine($"|There are {initialData.Rows.Count} orders to check|");
await UpdateItemCountsField(initialData);
}
Since UpdateItemCountsField returns a single Task, the Task.WhenAll wrapping it was redundant, so I removed that from the code in my answer.
Note that you must be using C# 7.1 or later in order to have an async main method.
Make sure on any method that returns a Task that you await the result.
Something is definitely flawed in my understanding of async/await. I want a piece of code named SaveSearchCase to run asynchronously in background.
I want it to be fired and forget about it and continue with the current method's return statement.
public IList<Entities.Case.CreateCaseOutput> createCase(ARC.Donor.Data.Entities.Case.CreateCaseInput CreateCaseInput, ARC.Donor.Data.Entities.Case.SaveCaseSearchInput SaveCaseSearchInput)
{
..........
..........
..........
var AcctLst = rep.ExecuteStoredProcedure<Entities.Case.CreateCaseOutput>(strSPQuery, listParam).ToList();
if (!string.IsNullOrEmpty(AcctLst.ElementAt(0).o_case_seq.ToString()))
{
Task<IList<Entities.Case.SaveCaseSearchOutput>> task = saveCaseSearch(SaveCaseSearchInput, AcctLst.ElementAt(0).o_case_seq);
Task t = task.ContinueWith(
r => { Console.WriteLine(r.Result); }
);
}
Console.WriteLine("After the async call");
return AcctLst;
}
And the SaveCaseSearch looks like
public async Task<IList<Entities.Case.SaveCaseSearchOutput>> saveCaseSearch(ARC.Donor.Data.Entities.Case.SaveCaseSearchInput SaveCaseSearchInput,Int64? case_key)
{
Repository rep = new Repository();
string strSPQuery = string.Empty;
List<object> listParam = new List<object>();
SQL.CaseSQL.getSaveCaseSearchParameters(SaveCaseSearchInput, case_key,out strSPQuery, out listParam);
var AcctLst = await rep.ExecuteStoredProcedureAsync<Entities.Case.SaveCaseSearchOutput>(strSPQuery, listParam);
return (System.Collections.Generic.IList<ARC.Donor.Data.Entities.Case.SaveCaseSearchOutput>)AcctLst;
}
But when I see the debugger createCase method waits for SaveCaseSearch to complete first and then only
it prints "After Async Call "
and then returns . Which I do not want definitely .
So which way is my understanding flawed ? Please help to make it run async and continue with current method's print and return statement .
UPDATE
I updated the SaveCaseSearch method to reflect like :
public async Task<IList<Entities.Case.SaveCaseSearchOutput>> saveCaseSearch(ARC.Donor.Data.Entities.Case.SaveCaseSearchInput SaveCaseSearchInput,Int64? case_key)
{
return Task.Run<IList<Entities.Case.SaveCaseSearchOutput>>(async (SaveCaseSearchInput, case_key) =>
{
Repository rep = new Repository();
string strSPQuery = string.Empty;
List<object> listParam = new List<object>();
SQL.CaseSQL.getSaveCaseSearchParameters(SaveCaseSearchInput, case_key, out strSPQuery, out listParam);
var AcctLst = await rep.ExecuteStoredProcedureAsync<Entities.Case.SaveCaseSearchOutput>(strSPQuery, listParam);
return (System.Collections.Generic.IList<ARC.Donor.Data.Entities.Case.SaveCaseSearchOutput>)AcctLst;
});
}
But there is something wrong with the params. It says
Error 4 A local variable named 'SaveCaseSearchInput' cannot be declared in this scope because it would give a different meaning to 'SaveCaseSearchInput', which is already used in a 'parent or current' scope to denote something else C:\Users\m1034699\Desktop\Stuart_V2_12042016\Stuart Web Service\ARC.Donor.Data\Case\Search.cs 43 79 ARC.Donor.Data
Well this saveCaseSearch() method runs synchronously in main thread and this is the main problem here. Instead of returning result with a task you should return Task with operation itself. Here is some simplified example :
Runs synchronously and waits 5 seconds
public IList<int> A()
{
var AcctLst = new List<int> { 0, 2, 5, 8 };
if (true)
{
Task<IList<int>> task = saveCaseSearch();
Task t = task.ContinueWith(
r => { Console.WriteLine(r.Result[0]); }
);
}
Console.WriteLine("After the async call");
return AcctLst;
}
// runs sync and in the end returns Task that is never actually fired
public async Task<IList<int>> saveCaseSearch()
{
Thread.Sleep(5000);
return new List<int>() { 10, 12, 16 };
}
Runs asynchronously - fires task & forgets :
public IList<int> A()
{
... same code as above
}
// notice that we removed `async` keyword here because we just return task.
public Task<IList<int>> saveCaseSearch()
{
return Task.Run<IList<int>>(() =>
{
Thread.Sleep(5000);
return new List<int>() { 10, 12, 16 };
});
}
Here is full code for this example
Against all that I believe in pertaining to "fire-and-forget" you can do this by writing your code this way:
public Task<SaveCaseSearchOutput> SaveCaseSearch(
SaveCaseSearchInput saveCaseSearchInput,
long? caseKey)
{
var rep = new Repository();
var query = string.Empty;
var listParam = new List<object>();
SQL.CaseSQL
.getSaveCaseSearchParameters(
saveCaseSearchInput,
caseKey,
out query,
out listParam);
return rep.ExecuteStoredProcedureAsync<SaveCaseSearchOutput>(
strSPQuery,
istParam);
}
And then if the place where you would like to fire it and log when it returns (which is really what you have -- so you're not forgetting about it), do this:
public IList<CreateCaseOutput> CreateCase(
CreateCaseInput createCaseInput,
SaveCaseSearchInput saveCaseSearchInput)
{
// Omitted for brevity...
var AcctLst =
rep.ExecuteStoredProcedure<CreateCaseOutput>(
strSPQuery,
listParam)
.ToList();
if (!string.IsNullOrEmpty(AcctLst.ElementAt(0).o_case_seq.ToString()))
{
SaveCaseSearch(saveCaseSearchInput,
AcctLst.ElementAt(0).o_case_seq)
.ContinueWith(r => Console.WriteLine(r.Result));
}
Console.WriteLine("After the async call");
return AcctLst;
}
The issue was that you were using async and await in the SaveSearchCase function, and this basically means that your code is the opposite of "fire-and-forget".
As a side note, you should really just use async and await, and avoid the "fire-and-forget" idea! Make your DB calls asynchronous and leverage this paradigm for what it's worth!
Consider the following:
The SaveCaseSearch call can stay as I have defined it above.
public Task<SaveCaseSearchOutput> SaveCaseSearch(
SaveCaseSearchInput saveCaseSearchInput,
long? caseKey)
{
var rep = new Repository();
var query = string.Empty;
var listParam = new List<object>();
SQL.CaseSQL
.getSaveCaseSearchParameters(
saveCaseSearchInput,
caseKey,
out query,
out listParam);
return rep.ExecuteStoredProcedureAsync<SaveCaseSearchOutput>(
strSPQuery,
istParam);
}
Then in your call to it, do this instead:
public async Task<IList<CreateCaseOutput>> CreateCase(
CreateCaseInput createCaseInput,
SaveCaseSearchInput saveCaseSearchInput)
{
// Omitted for brevity...
var AcctLst =
await rep.ExecuteStoredProcedureAsync<CreateCaseOutput>(
strSPQuery,
listParam)
.ToList();
if (!string.IsNullOrEmpty(AcctLst.ElementAt(0).o_case_seq.ToString()))
{
await SaveCaseSearch(saveCaseSearchInput,
AcctLst.ElementAt(0).o_case_seq)
.ContinueWith(r => Console.WriteLine(r.Result));
}
Console.WriteLine("After the async call");
return AcctLst;
}
This makes for a much better solution!
I have this code:
private async Task<DataSharedTheatres.TheatresPayload> GetTheatres()
{
var callMgr = new ApiCallsManager();
var fileMgr = new FileSystemManager();
string cachedTheatres = fileMgr.ReadFile(Path.Combine(System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal), "TheatreTemp.txt"));
if (string.IsNullOrEmpty(cachedTheatres))
{
**string generalModelPull = await callMgr.GetData(new Uri("somecrazyapi.com/api" + apiAccessKey));**
bool saveResult = fileMgr.WriteToFile(Path.Combine(System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal), "TheatreTemp.txt"), generalModelPull);
if (!saveResult)
{
testText.Text = "Failed to load Theatre Data";
return null;
}
cachedTheatres = fileMgr.ReadFile(Path.Combine(System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal), "TheatreTemp.txt"));
}
return Newtonsoft.Json.JsonConvert.DeserializeObject<DataSharedTheatres.TheatresPayload>(cachedTheatres);
**}**
I set the breakpoint on the first highlighted line (which it hits), then I press F10, and debugger jumps to the last bracket! I am not understanding why.
GetData method:
public async Task<string> GetData(Uri source)
{
if (client.IsBusy)
client.CancelAsync ();
string result = await client.DownloadStringTaskAsync (source);
return result;
}
Because that's what "await" does. When you use "await" in an async method, it tells the compiler that you want the method to return at that point, and then to re-enter the method later only when the "awaited" task has completed.
For my app I make calls to a web service to get customer data. The problem I am encountering is when I make this particular call it gets the the asynchronous await call and loops back without finishing the call and then storing the results.
private void DatabaseTest()
{
cNum = Convert.ToString(db.selectCustomerNumber());
callC = "SELECT * FROM dashboardCustomer WHERE period = 'C' AND customerNumber = " + cNum;
callB = "SELECT * FROM dashboardCustomer WHERE period = 'B' AND customerNumber = " + cNum;
callF = "SELECT * FROM dashboardCustomer WHERE period = 'F' AND customerNumber = " + cNum;
if (db.selectDashboard(callC).Count == 0)
{
GetDataSummary(passC);
}
if (db.selectDashboard(callB).Count == 0)
{
GetDataSummary(passB);
}
if (db.selectDashboard(callF).Count == 0)
{
GetDataSummary(passF);
}
}
private async void GetDataSummary(string r)
{
long customerNum = db.selectCustomerNumber();
pin = db.selectPinByCustomerNumber(customerNum);
string cType = r;
try
{
Windows.Security.Credentials.PasswordVault vault = new Windows.Security.Credentials.PasswordVault();
IReadOnlyList<PasswordCredential> userCredential = vault.FindAllByResource(pin);
userCredential[0].RetrievePassword();
try
{
getCustomerBillSummaryResponse billSum = await
UBPclient.getCustomerBillSummaryAsync(userCredential[0].UserName, userCredential[0].Password, customerNum, cType);
invoiceSummaryBean[] summaryList = billSum.#return;
rh.DashboardHandler(summaryList, customerNum);
}
catch
{
}
}
catch
{
}
}
it runs to the following part
getCustomerBillSummaryResponse billSum = await
UBPclient.getCustomerBillSummaryAsync(userCredential[0].UserName, userCredential[0].Password, customerNum, cType);
and then loops back to the try and runs again until it has ran three times.
How do I make it return the data it is suppose to for each call and store it in my database?
Also I have tested the web service in SoapUI and the call is returning results, so the problem is not with the web service.
You need to do this:
private async Task GetDataSummary(string r)
You need to return Task instead of void because your caller needs to have something to wait for. When you return void, the caller must treat method as "fire-and-forget". When you return Task, the caller can create the necessary code to await for async method to finish.
And don't forget to add the await keyword when you call it: await GetDataSummaryAsync(...);
You should avoid async void. Change GetDataSummary to return Task and then await it from DatabaseTest:
private async Task DatabaseTestAsync()
{
...
if (db.selectDashboard(callC).Count == 0)
{
await GetDataSummaryAsync(passC);
}
if (db.selectDashboard(callB).Count == 0)
{
await GetDataSummaryAsync(passB);
}
if (db.selectDashboard(callF).Count == 0)
{
await GetDataSummaryAsync(passF);
}
}
private async Task GetDataSummaryAsync(string r)