We Keep Coding

Variable cannot be accessed below a certain point in the code

I have a small problem with a piece of code I cannot seem to work out what the issue is, I never wrote this initial part of the code which is why I don't understand the problem.
Code:
public async Task GetEnumerableLinksAsync(string entryPointUrl)
{
Uri baseUrl = new Uri(entryPointUrl);
await foreach (Uri url in GetEnumerableLinksAsync(baseUrl, 1000))
ListBoxMain.Items.Add(url);
// TEST
string[] threads = File.ReadAllLines(#"logic\markers.txt");
if (threads.Any(url.Contains)) {
}
// TEST
}
The problem is here if (threads.Any(url.Contains)) { I cannot access the url variable.
ListBoxMain.Items.Add(url); the listbox can access it fine, but any code below this and I get the name `url` does not exist in the current context
Can anyone see the issue at all? any help is appreciated.

The code you have written looks correct because of indentation, but C# ignores any formatting.
This is your code formatted properly:
await foreach (Uri url in GetEnumerableLinksAsync(baseUrl, 1000))
ListBoxMain.Items.Add(url);
// TEST
string[] threads = File.ReadAllLines(#"logic\markers.txt");
if (threads.Any(url.Contains)) {
}
// TEST
Can you see the why url doesn't exist after the foreach now?
You need to add a 'scope' to the foreach to allow url to exist beyond the first line:
await foreach (Uri url in GetEnumerableLinksAsync(baseUrl, 1000))
{
ListBoxMain.Items.Add(url);
// TEST
string[] threads = File.ReadAllLines(#"logic\markers.txt");
if (threads.Any(url.Contains)) {
}
// TEST
}
Two hints:
start using Ctrl+KD more often. This formats your code and would have shown the issue.
Always use braces when using compound functions, like if,else,foreach etc.

Why a new thread executed without creating a new one?

Once in a while, there is another thread which starts to execute my code.
// the main method:
// ...
MatchResultsForRules(rules, results);
ApplyRules(rules);
//...
public void MatchResultsForRules(List<Rule> Rules, List<SearchResult> Results)
{
foreach (Rule rule in Rules)
{
foreach (SearchResult res in Results)
{
if (isResultMatchRule(rule, res))
{
rule.searchResults.Add(res);
}
}
}
}
public void ApplyRules (List<Rule> Rules)
{
foreach (Rule rule in Rules)
{
foreach(SearchResult res in rule.searchResults)
{
ApplyRule(rule, res);
}
}
}
I know there is another thread because once I saw the issue occurs (calculation mismatches), I printed a detailed log including the thread Id for each action and I saw a mess in the order of actions and of course, two different thread ids.
I fixed the issue by manipulating the search results itself instead of manipulating each rule's search results list (explained beneath).
The fix:
public void ApplyRules (List<Rule> rules, List<SearchResult> searchResults)
{
foreach (Rule rule in rules)
{
foreach(SearchResult resFromRule in rule.searchResults)
{
SearchResult res = searchResults.First(
r => r.Id.Equals(resFromRule.Id)
);
ApplyRule(rule, res);
}
}
}
I just want to understand this issue better in order to not repeat this mistake in the future.

This answer is based on the assumption that rules and results are shared or singleton instances and that the following code is called in parallel:
MatchResultsForRules(rules, results);
ApplyRules(rules);
MatchResultsForRules() edits your searchResults via rule.searchResults.Add(res). Then you might have a short time gap before ApplyRules() iterates throuh the results via foreach(SearchResult res in rule.searchResults). During that time gap another thread might be executing MatchResultsForRules() and edit searchResults. Even more: if one thread attempts to edit searchResults at the same time as another iterats through it, you are going to end up with an Exception.
To avoid these kind of error
Ask yourself if it is required to store the searchResults only to iterate it later. Instead of rule.searchResults.Add(res); you could directly call ApplyRule(rule, res); I am aware that there might be other reasons or code portions why you might need to store the result.
Consider whether you could not use shared instances, but provide every thread with it's own instances.
Use the lock keyword to make sure that this portion of code is not executed in parallel. This approach might drastically decrease your runtime performance.
lock(yourSharedReadonlyLockObject)
{
MatchResultsForRules(rules, results);
ApplyRules(rules);
}

Unexpected behavior when using Parallel.Foreach

I have this code:
// positions is a List<Position>
Parallel.ForEach(positions, (position) =>
{
DeterminePostPieceIsVisited(position, postPieces);
});
private void DeterminePostPieceIsVisited(Position position, IEnumerable<Postpieces> postPieces)
{
foreach (var postPiece in postPieces)
{
if (postPiece.Deliverd)
continue;
var distanceToClosestPosition = postPiece.GPS.Distance(position.GPS);
postPiece.Deliverd = distanceToClosestPosition.HasValue && IsInRadius(distanceToClosestPosition.Value);
}
}
}
I know that 50 post pieces must have the property Deliverd set to true. But, when running this code, I get changing results. Sometimes I get 44, when I run it another time I get 47. The results are per execution different.
When I run this code using a plain foreach-loop I get the expected result. So I know my implementation of the method DeterminePostPieceIsVisited is correct.
Could someone explain to me why using the Parallel foreach gives me different results each time I execute this code?

You've already, I think, tried to avoid a race, but there is still one - if two threads are examining the same postPiece at the same time, they may both observe that Deliverd (sic) is false, and then both assess whether it's been delivered to position (a distinct value for each thread) and both attempt to set a value for Deliverd - and often, I would guess, one of them will be trying to set it to false. Simple fix:
private void DeterminePostPieceIsVisited(Position position, IEnumerable<Postpieces> postPieces)
{
foreach (var postPiece in postPieces)
{
if (postPiece.Deliverd)
continue;
var distanceToClosestPosition = postPiece.GPS.Distance(position.GPS);
var delivered = distanceToClosestPosition.HasValue && IsInRadius(distanceToClosestPosition.Value);
if(delivered)
postPiece.Deliverd = true;
}
}
Also, by the way:
When I run this code using a plain foreach-loop I get the expected result. So I know my implementation of the method DeterminePostPieceIsVisited is correct.
The correct thing to state is would be "I know my implementation is correct for single threaded access" - what you hadn't established is that the method was safe for calling from multiple threads.

I have solved my issue with ConcurrentBag<T>. Here's what I use now:
var concurrentPostPiecesList = new ConcurrentBag<Postpiece>(postPieces);
Parallel.ForEach(positions, (position) =>
{
DeterminePostPieceIsVisited(position, concurrentPostPiecesList);
});
private void DeterminePostPieceIsVisited(Position position, ConcurrentBag<Postpieces> postPieces)
{
foreach (var postPiece in postPieces)
{
if (postPiece.Deliverd)
continue;
var distanceToClosestPosition = postPiece.GPS.Distance(position.GPS);
postPiece.Deliverd = distanceToClosestPosition.HasValue && IsInRadius(distanceToClosestPosition.Value);
}
}

Foreach loop and error that not all code paths return a value: async-await issue?

I am not sure why the "not all code paths return a value" error appears (see commented code below). Is it because the foreach loop is considered as not one path but many, and it's possible that an iteration might not return a value and yet the loop would continue?
[BTW, my goal is to process 100K urls, creating many web requests so they are active simultaneously, and be able to capture the status of each url (alive, moved, timed out) while updating the UI as the set of urls is being processed with real-time information about the number of bad urls found so far and the number or urls processed so far. I've tried Parallel.ForEach approach with synchronous web requests but the UI became unresponsive.]
EDITED to include a test if Rows.Count == 0.
public async Task<UrlInfo> ProcessUrls(DataTable urls)
{
if (urls.Rows.Count == 0)
{
return new UrlInfo();
}
else
{
foreach (DataRow r in urls.Rows)
{
UrlInfo info = new UrlInfo()
{
Url = (string)r["url"],
status = UrlStatusCode.untested,
articleid = (int)r["articleid"]
};
return await Foo(info);
}
}
//return new UrlInfo(); // error unless this line is uncommented
}
public async Task<UrlInfo> Foo(UrlInfo info) {
<snip>
}

The compiler is complaining about the situation where urls.Rows does not contain any elements. In that case, the method will never encounter a return statement.
On a side note, I do not think that method means what you think it means. A method only returns once, so it would only process the first url.

Stuck on implementation of WebBrowser Control and working with AJAX responses

Here we go...
I am trying to create a bot to walk through various functions of a website I do not have control of. Initially, I thought it would be best to just connect to the database (MySql) that the site is tied to and make my associations there...The database is built so extensively that I can't make out where to point what to where how etc...It's beyond my (DBA)programmer casting... ;)
So, my next idea was, create a bot...simple enough right? First hurdle, not everything on the page carries an ID...bring on the loops...
Got it.
Now I'm stuck with working with my data and page response.
I'm trying to fill out part of a form and perform an AJAX search. The problem is, there is no DocumentCompleted for this. And honestly, that isn't where my problem lies. I've tried using Thread.Sleep, Timers, etc...no avail.
// The app reads categories from a csv file,
// then performs a search for the category
// Search results are only displayed if I break the foreach loop
foreach (var item in bar)
{
var document = wbBrowser.Document;
if (document != null)
{
var name = document.GetElementById("product_filter_name");
if (name != null)
{
name.SetAttribute("value", item.Key.ToString());
var buttons = document.GetElementsByTagName("button");
foreach (HtmlElement button in buttons)
{
var findSearch = button.InnerHtml.IndexOf("Search");
if (findSearch > -1)
{
button.InvokeMember("click");
}
}
}
// This where the problem starts...
// I want the AJAX to run, then perform Step two,
// but the WebBrowser doesn't return the search
// results until the end (break;)
// Step Two
var elems = document.GetElementsByTagName("tr");
foreach (HtmlElement elem in elems)
{
// find particular item in result table
}
break;
// Now the search results display!!!!
// I tried implementing a timer, Thread.Sleep,
// everything I could find via Google before
//starting Step Two, but it hasn't worked!
}
}

The actual browser control has a WebBrowser.OnDocumentCompleted event which you might need to hook into so that you can be alerted when the ajax call has returned back from the server.