We Keep Coding

Game not printing or updating when AI is thinking

I have a game I am developing in Unity where AI is doing large calculations when it is its turn. It searches the position to depth 1, then 2, then 3 etc. Between each depth I want to instantiate a Gameobject with info about the depth to UI. The problem is that nothing happens until the AI is completely finished, then all items are added at once. Here is some code to explain better:
private void AIMakeMove()
{
for (int currentDepth = 1; currentDepth < maxDepth + 1; currentDepth++)
{
SearchPosition(currentDepth);
}
}
private void SearchPosition(int _currentDepth)
{
// Search the position to the given depth
score = Negamax(_currentDepth);
// Print things PROBLEM HERE
GameObject printItem = Instantiate(printItemPrefab, printItemParent.transform);
Debug.Log(_currentDepth);
}
I also tried just a simple Debug.Log instead of Instantiate but same thing happens then, all prints to console happens after the AI is done with its thinking process.
Why is my UI not updating with information? I tell it to create some things after it run the first iteration with depth 0 but it skips this step and goes on depth 2 instead. Can someone please let me know how to get information out between each depth?

The problem is that nothing happens until the AI is completely finished
well the UI is only updated if the Unity main-thread is allowed to finish a frame.
You, however, block the main thread until all iterations are finished.
If it is okey for you to block between each instantiation then you could simply use a Coroutine and do something like
private void AIMakeMove()
{
StartCoroutine(AIMakeMoveRoutine());
}
private IEnuemrator AIMakeMoveRoutine()
{
for (int currentDepth = 1; currentDepth < maxDepth + 1; currentDepth++)
{
SearchPosition(currentDepth);
// This now tells Unity to "interrupt" this routine here
// render the current frame and continue from here in the next frame
yield return null;
}
}
private void SearchPosition(int _currentDepth)
{
score = Negamax(_currentDepth);
GameObject printItem = Instantiate(printItemPrefab, printItemParent.transform);
Debug.Log(_currentDepth);
}
This will finish a frame and start a new one (thus refresh the UI) after each finished iteration.
However, if this still blocks the rest of your application too much you should additionally actually run the calculation async e.g. using a Task like
private void AIMakeMove()
{
StartCoroutine(AIMakeMoveRoutine());
}
private IEnuemrator AIMakeMoveRoutine()
{
for (int currentDepth = 1; currentDepth < maxDepth + 1; currentDepth++)
{
// you can yield another IEnuemrator -> executes this and waits for it to finish
yield return SearchPosition(currentDepth);
// This now tells Unity to "interrupt" this routine here
// render the current frame and continue from here in the next frame
yield return null;
}
}
private IEnumerator SearchPosition(int _currentDepth)
{
// run the NegamaxTask asynchronously in the background
var task = Task.Run(() => Negamax(_currentDepth));
// wait for the task to finish
while(!task.IsCompleted)
{
// do nothing but skip frames to allow the rest of the application to run smoothly
yield return null;
}
// If you do nothing else inside the loop this could also be written as
//yield return new WaitWhile(() => !task.IsComoleted);
// or
//yield return new WaitUntil(() => task.IsCompleted);
// since the task is already finished it is save / non-blocking to access the result now
score = task.Result;
var printItem = Instantiate(printItemPrefab, printItemParent.transform);
Debug.Log(_currentDepth);
}
Now this allows your application to continue with a normal frame-rate while in the background you do the heavy calculations and once in a while get a result back when an iteration is finished.

Try using a thread:
private void AIMakeMove()
{
new System.Threading.Thread(() =>
{
for (int currentDepth = 1; currentDepth < maxDepth + 1; currentDepth++)
{
SearchPosition(currentDepth);
}
}).Start();
}
private void SearchPosition(int _currentDepth)
{
// Search the position to the given depth
score = Negamax(_currentDepth);
// Print things PROBLEM HERE
GameObject printItem = Instantiate(printItemPrefab, printItemParent.transform);
Debug.Log(_currentDepth);
}

How to disable Gameobjects for Screenshot capturing in Unity on Android?

I have a simple goal for my Unity application running on Android: create a function, which disables my GUI, creates a screenshot, stores the image and enables the GUI again.
This is my current code:
The whole UI is stored in a GameObject which my handler class holds as property:
private bool makeScreenshotFlag;
public GameObject UserInterface;
makeScreenshotFlag above is used in the Update function:
// Update is called once per frame
void Update()
{
if (makeScreenshotFlag)
{
// Working with this flag is necessary,
// since we need to wait for the next frame,
// where the gui is not visible;
MakeScreenShot();
}
}
If the corresponding button gets pressed I fire the following Method:
public void OnCreateScreenshotButtonPressed()
{
UserInterface.SetActive(false);
makeScreenshotFlag = true;
}
And at last the method for the screenshot itself:
private void MakeScreenShot()
{
if (UserInterface.activeSelf) return; // wait for the next frame if the gui is still active
try
{
string filename = "screenshot_" + DateTime.Now.ToString("HH-mm-ss--dd-MM-yyyy") + ".png";
ScreenCapture.CaptureScreenshot(filename);
}
catch (Exception e)
{
DebugHelper.NewLog = "Error capturing screenshot: " + e.Message;
}
finally
{
makeScreenshotFlag = false;
UserInterface.SetActive(true);
}
}
In theory this should work, however the documentation for ScreenCapture.CaptureScreenshot() states the following:
The CaptureScreenshot returns immediately on Android. The screen capture continues in the background. The resulting screen shot is saved in the file system after a few seconds.
In practice this means, when my screenshot is made, the GUI is already enabled again. So far I couldn't find any event or similar to get notified as soon as the screenshot is made. The only thing I can think of is to store the screenshot filename and check if the file exists each frame after the screenshot is started.
Is there any other, more practical way?

Most probably, you are capturing the frame with UI itself. It's better to wait till the end of the frame using Coroutine.
public IEnumerator MakeScreenShot()
{
// Wait till the last possible moment before screen rendering to hide the UI
yield return null;
// Disable UI
UserInterface.SetActive(false);
// Wait for screen rendering to complete
yield return new WaitForEndOfFrame();
// Take screenshot
Application.CaptureScreenshot("screenshot.png");
// Show UI after we're done
UserInterface.SetActive(true);
}

StartCoroutine is being called so many times (C# Unity)

I'm creating a Pop up menu Option in Unity. Now my Problem here is that the coroutine i made in void update is being called so many times. What i mean by that is on my Unity Console the Debug.Logs are incrementing . It should not right because its already coroutine. Could some help me understand more coroutine and help me solve my little problem .
Here is my code:
[SerializeField]
GameObject Option;
[SerializeField]
Button btn,btn2;
[SerializeField]
GameObject open, close;
[SerializeField]
GameObject[] opt;
bool startFinding = false;
void Start()
{
Option.SetActive(false);
Button popUp = btn.GetComponent<Button>();
Button popUp2 = btn2.GetComponent<Button>();
popUp.onClick.AddListener(PopUpOption);
popUp2.onClick.AddListener(ClosePopUp);
}
void Update()
{
if (startFinding)
{
StartCoroutine(GameOptions());
}
}
IEnumerator GameOptions()
{
//Get All the tags
opt = GameObject.FindGameObjectsWithTag("MobileOptions");
if (opt[0].GetComponent<Toggle>().isOn == true && opt[1].GetComponent<Toggle>().isOn == true)
{
Debug.Log("Disable first the check box then choose only 1 option between" + "'rendering'"+ "and" + "'livestreaming'");
}
//Livestreaming
if (opt[0].GetComponent<Toggle>().isOn == true)
{
Debug.Log("Livestreaming Activate");
} else
{
Debug.Log("Livestreaming Deactivate");
}
//Rendering
if (opt[1].GetComponent<Toggle>().isOn == true)
{
Debug.Log("Rendering Activate");
} else
{
Debug.Log("Rendering Deactivate");
}
//Fog
if (opt[2].GetComponent<Toggle>().isOn == true)
{
Debug.Log("Fog Activated");
} else
{
Debug.Log("Fog Deactivated");
}
//Camera Effect
if (opt[3].GetComponent<Toggle>().isOn == true)
{
Debug.Log("Camera Effect Activated");
} else {
Debug.Log("Camera Effect Deactivated");
}
yield return null;
}
void PopUpOption()
{
startFinding = true;
//Disable The Mobile Option Button
open.SetActive(false);
//Enable the Close Option Button
close.SetActive(true);
//activate the Mobile Options
Option.SetActive(true);
}
void ClosePopUp()
{
startFinding = false;
//eanble the mobile option button
open.SetActive(true);
//disable the close option button
close.SetActive(false);
//deactivate the Mobile Option
Option.SetActive(false);
}

Here is how coroutines work:
Let's say I have a couroutine function called MyRoutine (in your case, you called it GameOptions)
private IEnumerator MyRoutine()
Then, anywhere in my code, calling
StartCoroutine(MyRoutine));
Is going to simply call MyRoutine like any usual method. So if you call it in update, it will be called all the time, as any method would. This is not what you want. What make coroutines special is that you can use the yield keyword in them. There are many ways to use it but the most used (and simple) one is to do yield return null
yield return null means "Stop this coroutine, but resume the execution on next frame". You don't need to call any other function (certainly not StartCoroutine). The execution will resume next frame.
To go back to what you posted in your question, you wrote yield return null at the end. So your method is executing, and just at the end, stops and resumes next frame, but since there is nothing left to do, it exits on the next frame.
A typical way to use coroutines is to have the yield return null in a while loop, so when it resumes, it continues the loop. Here is an example that do it
private IEnumerator MyRoutine()
{
while(running) //running is a member bool that you could set to false to exit
{
// Do all the stuff you want to do in ONE frame
// ...
yield return null;
}
}
Typically, the StartCoroutine would be called in the Start() function, or later when an event is triggered.
If you want to know more about coroutine, or check that you understood them properly, check out this page: https://docs.unity3d.com/Manual/Coroutines.html
or this video https://unity3d.com/learn/tutorials/topics/scripting/coroutines
// Edit: quickly present one useful option
In the snippet above, the while loop is very similar to the Update function (the inside of the loop is executed each frame). One nice option is to replace
yield return null
by
yield return new WaitForSeconds(waitTime)
where waitTime is a the time you want to wait before resuming, in seconds
// End of edit

Do not use StartCoroutine() in the Update method. Call it in another method and use a while loop inside your coroutine function if needed. Just control your StartCoroutine() outside of Update method

Update is called every frame, if your condition is ever true, you launch your coroutine every frame.
Just set down your flag to only join 1 time.
void Update()
{
if (startFinding)
{
startFinding = false;
StartCoroutine(GameOptions());
}
}

Unity C# Yield WaitForSeconds within For Loop Only Works Once

I am trying to use two yields within a coroutine loop (because I need to iterate out arrays with pauses between each loop).
The first loop works correctly, with all the yields working for the right amount of time. By the second loop, the yield return new WaitForSeconds() begins counting down right away, not waiting for the yield and code before it to complete (it seems). By the time of the third loop, the timing is all off.
I tried using a while loop instead of a for but got the same result.
TLDR: I need to loop out my arrays with pauses between each one. How can I use more than one yield past the first loop through in a coroutine?
public IEnumerator doPathfinding()
{
for (int i = 0; i < waypoint.Length; i++)
{
// get first waypoint from array
var _waypoint = waypoint[i];
// get node on A* of cloest waypoint
closestPointOnNavmesh = AstarPath.active.GetNearest(_waypoint.transform.position, NNConstraint.Default).clampedPosition;
// Move towards destination
ai.destination = closestPointOnNavmesh;
// Wait until within X range of waypoint
yield return new WaitUntil(() => distanceReached == true);
// Agent is now at waypoint and automatically stops. Wait 5 seconds before looping to next waypoint.
yield return new WaitForSeconds(5f);
}
Debug.Log("Loop End");
}
public override void OnUpdate()
{
// Get current distance to the target. If distance is less than offset, then sent event.
currentDistance = Vector3.Distance(go.transform.position, closestPointOnNavmesh);
if(currentDistance <= arrivalOffset.Value)
{
distanceReached = true;
}
else
{
distanceReached = false;
}
}

The code inside the couroutine is fine, it works as intended.
Most probably, based on the issue you reported, you're calling more than once the coroutine at the same time.
Use a bool to check if the coroutine has already started or not, like this:
bool isDoPathFindingRunning = false;
IEnumerator = doPathFinding();
private void Awake() {
pathFinding = doPathfinding();
}
private void WhereeverYouStartCoroutine() {
if (!isDoPathFindingRunning)
StartCoroutine(pathFinding);
}
public IEnumerator doPathfinding() {
isDoPathFindingRunning = true;
// Do your stuff
isDoPathFindingRunning = false;
}

How to extract timestamps from each frame obtained by USB camera?

This scene is common in real time video proccessing. And I need timestamps to synchronize with other devices.
I have tried cv::VideoCapture, but it can not extract the timestamps frome video stream.
So I have two questions here:
Does video stream provided by USB camera indeed contains the timestamp information ?
If it has. What should I do to extract it ? A C# solution is best, while C++ is OK.
Addition:
Using these two properties doesn't work:
secCounter = (long) cap.get(CAP_PROP_POS_MSEC);
frameNumber = (long) cap.get(CAP_PROP_POS_FRAMES);
It always gives the following result:
VIDEOIO ERROR: V4L2: getting property #1 is not supported
msecCounter = 0
frameNumber = -1

OpenCV's VideoCapture class is a very high level interface to retrieve frames from a camera, so it "hides" a lot of the details that are necessary to connect to the camera, retrieve frames from the camera, and decode those frames in to a useful color space like BGR. This is nice because you don't have to worry about the details of grabbing frames, but the downside is that you don't have direct access to other data you might want, like the frame number or frame timestamp. That doesn't mean it's impossible to get the data you want, though!
Here's a sample frame grabbing loop that will get you what you want, loosely based on the example code from here. This is in C++.
#include "opencv2/opencv.hpp"
using namespace cv;
int main(int, char**)
{
VideoCapture cap(0); // open the default camera
if(!cap.isOpened()) // check if we succeeded
return -1;
// TODO: change the width, height, and capture FPS to your desired
// settings.
cap.set(CAP_PROP_FRAME_WIDTH, 1920);
cap.set(CAP_PROP_FRAME_HEIGHT, 1080);
cap.set(CAP_PROP_FPS, 30);
Mat frame;
long msecCounter = 0;
long frameNumber = 0;
for(;;)
{
// Instead of cap >> frame; we'll do something different.
//
// VideoCapture::grab() tells OpenCV to grab a frame from
// the camera, but to not worry about all the color conversion
// and processing to convert that frame into BGR.
//
// This means there's less processing overhead, so the time
// stamp will be more accurate because we are fetching it
// immediately after.
//
// grab() should also wait for the next frame to be available
// based on the capture FPS that is set, so it's okay to loop
// continuously over it.
if(cap.grab())
{
msecCounter = (long) cap.get(CAP_PROP_POS_MSEC);
frameNumber = (long) cap.get(CAP_PROP_POS_FRAMES);
// VideoCapture::retrieve color converts the image and places
// it in the Mat that you provide.
if(cap.retrieve(&frame))
{
// Pass the frame and parameters to your processing
// method.
ProcessFrame(&frame, msecCounter, frameNumber);
}
}
// TODO: Handle your loop termination condition here
}
// the camera will be deinitialized automatically in VideoCapture destructor
return 0;
}
void ProcessFrame(Mat& frame, long msecCounter, long frameNumber)
{
// TODO: Make a copy of frame if you are going to process it
// asynchronously or put it in a buffer or queue and then return
// control from this function. This is because the reference Mat
// being passed in is "owned" by the processing loop, and on each
// iteration it will be destructed, so any references to it will be
// invalid. Hence, if you do any work async, you need to copy frame.
//
// If all your processing happens synchronously in this function,
// you don't need to make a copy first because the loop is waiting
// for this function to return.
// TODO: Your processing logic goes here.
}
If you're using C# and Emgu CV it will look a bit different. I haven't tested this code, but it should work or be very close to the solution.
using System;
using Emgu.CV;
using Emgu.CV.CvEnum;
static class Program
{
[STAThread]
static void Main()
{
VideoCapture cap = new VideoCapture(0);
if(!cap.IsOpened)
{
return;
}
cap.SetCaptureProperty(CapProp.FrameWidth, 1920);
cap.SetCaptureProperty(CapProp.FrameHeight, 1080);
cap.SetCaptureProperty(CapProp.Fps, 30);
Mat frame = new Mat();
long msecCounter = 0;
long frameNumber = 0;
for(;;)
{
if(cap.Grab())
{
msecCounter = (long) cap.GetCaptureProperty(CapProp.PosMsec);
frameNumber = (long) cap.GetCaptureProperty(CapProp.PosFrames);
if(cap.Retrieve(frame))
{
ProcessFrame(frame, msecCounter, frameNumber);
}
}
// TODO: Determine when to quit the processing loop
}
}
private static void ProcessFrame(Mat frame, long msecCounter, long frameNumber)
{
// Again, copy frame here if you're going to queue the frame or
// do any async processing on it.
// TODO: Your processing code goes here.
}
}
Emgu's VideoCapture implementation also allows for asynchronous Grab operations to be done for you, and notifications when a grabbed frame is ready to be used with Retrieve. That looks like this:
using System;
using Emgu.CV;
using Emgu.CV.CvEnum;
static class Program
{
private static Mat s_frame;
private static VideoCapture s_cap;
private static object s_retrieveLock = new object();
[STAThread]
static void Main()
{
s_cap = new VideoCapture(0);
if(!s_cap.IsOpened)
{
return;
}
s_frame = new Mat();
s_cap.SetCaptureProperty(CapProp.FrameWidth, 1920);
s_cap.SetCaptureProperty(CapProp.FrameHeight, 1080);
s_cap.SetCaptureProperty(CapProp.Fps, 30);
s_cap.ImageGrabbed += FrameIsReady;
s_cap.Start();
// TODO: Wait here until you're done with the capture process,
// the same way you'd determine when to exit the for loop in the
// above example.
s_cap.Stop();
s_cap.ImageGrabbed -= FrameIsReady;
}
private static void FrameIsReady(object sender, EventArgs e)
{
// This function is being called from VideoCapture's thread,
// so if you rework this code to run with a UI, be very careful
// about updating Controls here because that needs to be Invoke'd
// back to the UI thread.
// I used a lock here to be extra careful and protect against
// re-entrancy, but this may not be necessary if Emgu's
// VideoCapture thread blocks for completion of this event
// handler.
lock(s_retrieveLock)
{
msecCounter = (long) s_cap.GetCaptureProperty(CapProp.PosMsec);
frameNumber = (long) s_cap.GetCaptureProperty(CapProp.PosFrames);
if(s_cap.Retrieve(s_frame))
{
ProcessFrame(s_frame, msecCounter, frameNumber);
}
}
}
private static void ProcessFrame(Mat frame, long msecCounter, long frameNumber)
{
// Again, copy frame here if you're going to queue the frame or
// do any async processing on it.
// TODO: Your processing code goes here.
}
}