We Keep Coding

Attempting to load AssetReference that has already been loaded

When I start the main scene and test a new character it shows this error why?
Attempting to load AssetReference that has already been loaded. Handle is exposed through getter OperationHandle
UnityEngine.AddressableAssets.AssetReference:LoadAssetAsync<UnityEngine.GameObject> ()
TrackManager/<SpawnFromAssetReference>d__104:MoveNext () (at Assets/Scripts/Tracks/TrackManager.cs:565)
UnityEngine.MonoBehaviour:StartCoroutine (System.Collections.IEnumerator)
TrackManager:SpawnObstacle (TrackSegment) (at Assets/Scripts/Tracks/TrackManager.cs:556)
TrackManager/<SpawnNewSegment>d__102:MoveNext () (at Assets/Scripts/Tracks/TrackManager.cs:538)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)
my code :
if (m_SafeSegementLeft <= 0)
{
SpawnObstacle(newSegment);
}
else
m_SafeSegementLeft -= 1;
m_Segments.Add(newSegment);
if (newSegmentCreated != null) newSegmentCreated.Invoke(newSegment);
}
public void SpawnObstacle(TrackSegment segment)
{
if (segment.possibleObstacles.Length != 0)
{
for (int i = 0; i < segment.obstaclePositions.Length; ++i)
{
AssetReference assetRef = segment.possibleObstacles[Random.Range(0, segment.possibleObstacles.Length)];
StartCoroutine(SpawnFromAssetReference(assetRef, segment, i));
}
}
StartCoroutine(SpawnCoinAndPowerup(segment));
}
private IEnumerator SpawnFromAssetReference(AssetReference reference, TrackSegment segment, int posIndex)
{
AsyncOperationHandle op = reference.LoadAssetAsync<GameObject>();
yield return op;
GameObject obj = op.Result as GameObject;
if (obj != null)
{
Obstacle obstacle = obj.GetComponent<Obstacle>();
if (obstacle != null)
yield return obstacle.Spawn(segment, segment.obstaclePositions[posIndex]);
}
}
It says i have error in line 565 which is AsyncOperationHandle op = reference.LoadAssetAsync<GameObject>();
What is the error here?

The error message sounds quite self-explanatory: You try to load the same addressable twice.
As said in
AssetReference assetRef = segment.possibleObstacles[Random.Range(0, segment.possibleObstacles.Length)];
you pick a random entry from available addressables. However, nothing here prevents that you get casually the same element twice.
I would rather either keep track of which ones already are loaded
Dictionary<AssetReference, GameObject> loadedAssets = new Dictionary<AssetReference, GameObject>();
and then do
private IEnumerator SpawnFromAssetReference(AssetReference reference, TrackSegment segment, int posIndex)
{
if(!loadedAssets.TryGetValue(reference, out var obj || !obj)
{
loadedAssets.Add(reference, null);
AsyncOperationHandle op = reference.LoadAssetAsync<GameObject>();
yield return op;
obj = op.Result;
loadedAssets[reference] = obj;
}
if(!obj.TryGetComponent<Obstacle>(out var obstacle))
{
Debug.LogError($"No {nameof(Obstacle)} component on loaded object!");
yield break;
}
yield return obstacle.Spawn(segment, segment.obstaclePositions[posIndex]);
}
And then of course whenever you Release one of the loaded assets you also want to
loadedAssets.Remove(reference);
Or depending on your use case and needs load them all and then start your app if you are going to spawn them more often anyway.

Following derHugo approach of storing values I came up with this to always have the addressable result ready to be returned as a gameobject but prevent it to be loaded more than once, addressables also has a 'address.LoadAssetAsync().WaitForCompletion()' method but in my case when loading many things it gets too laggy.
private async Task<T> GetNewInstance<T>(AssetReferenceGameObject address) where T : MonoBehaviour
{
if (!loadedAssetsTask.ContainsKey(address))
{
LoadNewAddress<T>(address);
}
var asset = await loadedAssetsTask[address];
var newGameObject = Instantiate(asset);
var component = newGameObject.GetComponent<T>();
return component;
}
private void LoadNewAddress<T>(AssetReferenceGameObject address) where T : MonoBehaviour
{
var loadedAssetTask = address.LoadAssetAsync().Task;
loadedAssetsTask.Add(address, loadedAssetTask);
}

A little bit late, but after some dig up I found the explaination from Unity_Bill:
AssetReference.LoadAssetAsync() is a helper we've provided that you in
no way need to use when loading an AssetReference.
Addressables.LoadAssetAsync(AssetReference) will work just as well. If
you do the latter, the async operation handle is returned to you and
you are in charge of it. You can call that as many times as you want,
and get as many handles as you want. Each will be ref-counted.
If you choose to use the AssetReference.LoadAssetAsync() helper, the
asset reference itself will hold on to the handle. This enabled the
helper method AssetReference.ReleaseAsset(). Prior to 1.15.1, if you
called load twice, the first handle would simply get stomped. If you
happened to keep up with it, great, but if not, it was lost forever.
So, in short, AssetReference.LoadAssetAsync() is a convenience helper
that really only works in the most simple case. If you are doing
anything beyond the simple, and are keeping up with the handles
yourself, just use Addr.Load... If I were starting addressables over I
likely wouldn't have the helpers at all, requiring the
Addressables.Load..(AssetRef) be used instead.
TLDR: Use Addressables.LoadAssetAsync(AssetReference) instead of AssetReference.LoadAssetAsync
You can read more here: https://forum.unity.com/threads/1-15-1-assetreference-not-allow-loadassetasync-twice.959910/

Foreach loop and error that not all code paths return a value: async-await issue?

I am not sure why the "not all code paths return a value" error appears (see commented code below). Is it because the foreach loop is considered as not one path but many, and it's possible that an iteration might not return a value and yet the loop would continue?
[BTW, my goal is to process 100K urls, creating many web requests so they are active simultaneously, and be able to capture the status of each url (alive, moved, timed out) while updating the UI as the set of urls is being processed with real-time information about the number of bad urls found so far and the number or urls processed so far. I've tried Parallel.ForEach approach with synchronous web requests but the UI became unresponsive.]
EDITED to include a test if Rows.Count == 0.
public async Task<UrlInfo> ProcessUrls(DataTable urls)
{
if (urls.Rows.Count == 0)
{
return new UrlInfo();
}
else
{
foreach (DataRow r in urls.Rows)
{
UrlInfo info = new UrlInfo()
{
Url = (string)r["url"],
status = UrlStatusCode.untested,
articleid = (int)r["articleid"]
};
return await Foo(info);
}
}
//return new UrlInfo(); // error unless this line is uncommented
}
public async Task<UrlInfo> Foo(UrlInfo info) {
<snip>
}

The compiler is complaining about the situation where urls.Rows does not contain any elements. In that case, the method will never encounter a return statement.
On a side note, I do not think that method means what you think it means. A method only returns once, so it would only process the first url.

Windows 8 Store App and Linq

The snippet below is from a Windows 8 store app in c# and xaml.
I have put this code together from variou samples on the web so this may not be the neatest way of doing this. Most of it is from the Grid template supplied in VS2012 and I have hooked up my web api as the source of the data
Please explain the following
When i call the Get method all works fine and i get data back into the xaml view
When i uncomment the Take(10) in the same method i get no data back.
It seems any attempt to put an extension method of a LINQ variety just stops the data being returned and also gives no indication why, it complies fine!
Any help appreciated
Thanks
Mark
public class TeamDataSource
{
private static TeamDataSource _sampleDataSource = new TeamDataSource();
private ObservableCollection<TeamDataItem> _items = new ObservableCollection<TeamDataItem>();
public ObservableCollection<TeamDataItem> Items
{
get { return this._items; }
}
public TeamDataSource()
{
this.Initialize();
}
public static IEnumerable<TeamDataItem> Get()
{
var thisdata = _sampleDataSource.Items;
return thisdata;//.Take(10);
}
private async void Initialize()
{
using (var client = new DataServiceClient())
{
List<TeamDataItem> list = await client.Download<List<TeamDataItem>>("/teams");
foreach (var i in list.OrderByDescending(t => t.Points).ThenByDescending(t => t.GoalDiff))
{
TeamDataItem team = i;
_items.Add(team);
}
}
}
}

Your problem is that Take doesn't immediately enumerate the items. It defers enumeration until either foreach is called on it or GetEnumerator is called on it. In this case the collection it is enumerating is disposed (as soon as the Get content ends) and so when it finally enumerates the items, there are no items anymore. Try adding thisdata.GetEnumerator(); as a line before your return statement.
From here:
This method is implemented by using deferred execution. The immediate
return value is an object that stores all the information that is
required to perform the action. The query represented by this method
is not executed until the object is enumerated either by calling its
GetEnumerator method directly or by using foreach in Visual C# or For
Each in Visual Basic.

Seems it was quite obvious in the end. As I was using aync and await, the call was immediately returning before the data had arrived. Therefore nothing for the Take(4) to work on.
Only problem now is when can i tell the task has completed?

How to unit test Thread Safe Generic List in C# using NUnit?

I asked a question about building custom Thread Safe Generic List now I am trying to unit test it and I absolutely have no idea how to do that. Since the lock happens inside the ThreadSafeList class I am not sure how to make the list to lock for a period of time while I am try to mimic the multiple add call. Thanks.
Can_add_one_item_at_a_time
[Test]
public void Can_add_one_item_at_a_time() //this test won't pass
{
//I am not sure how to do this test...
var list = new ThreadSafeList<string>();
//some how need to call lock and sleep inside list instance
//say somehow list locks for 1 sec
var ta = new Thread(x => list.Add("a"));
ta.Start(); //does it need to aboard say before 1 sec if locked
var tb = new Thread(x => list.Add("b"));
tb.Start(); //does it need to aboard say before 1 sec if locked
//it involves using GetSnapshot()
//which is bad idea for unit testing I think
var snapshot = list.GetSnapshot();
Assert.IsFalse(snapshot.Contains("a"), "Should not contain a.");
Assert.IsFalse(snapshot.Contains("b"), "Should not contain b.");
}
Snapshot_should_be_point_of_time_only
[Test]
public void Snapshot_should_be_point_of_time_only()
{
var list = new ThreadSafeList<string>();
var ta = new Thread(x => list.Add("a"));
ta.Start();
ta.Join();
var snapshot = list.GetSnapshot();
var tb = new Thread(x => list.Add("b"));
tb.Start();
var tc = new Thread(x => list.Add("c"));
tc.Start();
tb.Join();
tc.Join();
Assert.IsTrue(snapshot.Count == 1, "Snapshot should only contain 1 item.");
Assert.IsFalse(snapshot.Contains("b"), "Should not contain a.");
Assert.IsFalse(snapshot.Contains("c"), "Should not contain b.");
}
Instance method
public ThreadSafeList<T> Instance<T>()
{
return new ThreadSafeList<T>();
}

Let's look at your first test, Can_add_one_item_at_a_time.
First of all, your exit conditions don't make sense. Both items should be added, just one at a time. So of course your test will fail.
You also don't need to make a snapshot; remember, this is a test, nothing else is going to be touching the list while your test is running.
Last but not least, you need to make sure that you aren't trying to evaluate your exit conditions until all of the threads have actually finished. Simplest way is to use a counter and a wait event. Here's an example:
[Test]
public void Can_add_from_multiple_threads()
{
const int MaxWorkers = 10;
var list = new ThreadSafeList<int>(MaxWorkers);
int remainingWorkers = MaxWorkers;
var workCompletedEvent = new ManualResetEvent(false);
for (int i = 0; i < MaxWorkers; i++)
{
int workerNum = i; // Make a copy of local variable for next thread
ThreadPool.QueueUserWorkItem(s =>
{
list.Add(workerNum);
if (Interlocked.Decrement(ref remainingWorkers) == 0)
workCompletedEvent.Set();
});
}
workCompletedEvent.WaitOne();
workCompletedEvent.Close();
for (int i = 0; i < MaxWorkers; i++)
{
Assert.IsTrue(list.Contains(i), "Element was not added");
}
Assert.AreEqual(MaxWorkers, list.Count,
"List count does not match worker count.");
}
Now this does carry the possibility that the Add happens so quickly that no two threads will ever attempt to do it at the same time. No Refunds No Returns partially explained how to insert a conditional delay. I would actually define a special testing flag, instead of DEBUG. In your build configuration, add a flag called TEST, then add this to your ThreadSafeList class:
public class ThreadSafeList<T>
{
// snip fields
public void Add(T item)
{
lock (sync)
{
TestUtil.WaitStandardThreadDelay();
innerList.Add(item);
}
}
// snip other methods/properties
}
static class TestUtil
{
[Conditional("TEST")]
public static void WaitStandardThreadDelay()
{
Thread.Sleep(1000);
}
}
This will cause the Add method to wait 1 second before actually adding the item as long as the build configuration defines the TEST flag. The entire test should take at least 10 seconds; if it finishes any faster than that, something's wrong.
With that in mind, I'll leave the second test up to you. It's similar.

You will need to insert some TESTONLY code that adds a delay in your lock. You can create a function like this:
[Conditional("DEBUG")]
void SleepForABit(int delay) { thread.current.sleep(delay); }
and then call it in your class. The Conditional attribute ensure it is only called in DEBUG builds and you can leave it in your compiled code.
Write something which consistently delays 100Ms or so and something that never waits and let'em slug it out.

You might want to take a look at Chess. It's a program specifically designed to find race conditions in multi-threaded code.

Dual-queue producer-consumer in .NET (forcing member variable flush)

I have a thread which produces data in the form of simple object (record). The thread may produce a thousand records for each one that successfully passes a filter and is actually enqueued. Once the object is enqueued it is read-only.
I have one lock, which I acquire once the record has passed the filter, and I add the item to the back of the producer_queue.
On the consumer thread, I acquire the lock, confirm that the producer_queue is not empty,
set consumer_queue to equal producer_queue, create a new (empty) queue, and set it on producer_queue. Without any further locking I process consumer_queue until it's empty and repeat.
Everything works beautifully on most machines, but on one particular dual-quad server I see in ~1/500k iterations an object that is not fully initialized when I read it out of consumer_queue. The condition is so fleeting that when I dump the object after detecting the condition the fields are correct 90% of the time.
So my question is this: how can I assure that the writes to the object are flushed to main memory when the queue is swapped?
Edit:
On the producer thread:
(producer_queue above is m_fillingQueue; consumer_queue above is m_drainingQueue)
private void FillRecordQueue() {
while (!m_done) {
int count;
lock (m_swapLock) {
count = m_fillingQueue.Count;
}
if (count > 5000) {
Thread.Sleep(60);
} else {
DataRecord rec = GetNextRecord();
if (rec == null) break;
lock (m_swapLock) {
m_fillingQueue.AddLast(rec);
}
}
}
}
In the consumer thread:
private DataRecord Next(bool remove) {
bool drained = false;
while (!drained) {
if (m_drainingQueue.Count > 0) {
DataRecord rec = m_drainingQueue.First.Value;
if (remove) m_drainingQueue.RemoveFirst();
if (rec.Time < FIRST_VALID_TIME) {
throw new InvalidOperationException("Detected invalid timestamp in Next(): " + rec.Time + " from record " + rec);
}
return rec;
} else {
lock (m_swapLock) {
m_drainingQueue = m_fillingQueue;
m_fillingQueue = new LinkedList<DataRecord>();
if (m_drainingQueue.Count == 0) drained = true;
}
}
}
return null;
}
The consumer is rate-limited, so it can't get ahead of the consumer.
The behavior I see is that sometimes the Time field is reading as DateTime.MinValue; by the time I construct the string to throw the exception, however, it's perfectly fine.

Have you tried the obvious: is microcode update applied on the fancy 8-core box(via BIOS update)? Did you run Windows Updates to get the latest processor driver?
At the first glance, it looks like you're locking your containers. So I am recommending the systems approach, as it sound like you're not seeing this issue on a good-ol' dual core box.

Assuming these are in fact the only methods that interact with the m_fillingQueue variable, and that DataRecord cannot be changed after GetNextRecord() creates it (read-only properties hopefully?), then the code at least on the face of it appears to be correct.
In which case I suggest that GregC's answer would be the first thing to check; make sure the failing machine is fully updated (OS / drivers / .NET Framework), becasue the lock statement should involve all the required memory barriers to ensure that the rec variable is fully flushed out of any caches before the object is added to the list.