We Keep Coding

Unity function stepping out of execution and debugging when I search for gameobject

Ok, so this is something that never happened to me.
I'm working on a audio-based game, and need things to happen whenever a given midi note is played. No problem getting the note event and details, but once I call the actual function that's supposed to do something with it, something weird happens. (I'm using DryWetMidi to listen for midi NoteOn events, if that is relevant to the issue)
I'm trying to find a specific gameobject and change something within it. As soon as I search for it, the function simply stops execution without any error, exception or feedback of any kind. If I debug the code, it reaches that line fine, stops if I set a breakpoint on it, but if I click "step over" to go through the code, it simply continues execution and "steps out of the debug break" (not sure how to call this, hope it makes sense).
Here's the code
public void PlayNote(int note, int velocity)
{
Debug.Log("Playing note " + note + " at velocity " + velocity); // this logs the message fine
GameObject key = GameObject.Find(note.toString()); // execution reaches this line
// nothing is executed from this point on
if (key != null)
key.GetComponent<ObjectSetup>().SetPressed(true);
Debug.Log("Done"); // never logs anything
}
The gameobject in question is already created, as it is visible in-game, and the function works if I call it from somewhere else (let's say: from Start function of the current script).
I'm stumped, as I can't even understand why it's "jumping ship" mid-execution... If I try to search for another gameobject, or simply create a new one, it does exactly the same thing.

The first thing I see with your code is toString() instead of ‘ToString()`.
I also hazard a guess that you’re not seeing any error message because you might have unselected the error category in the console. That’s the red icon in the top right of the Console window. If that’s not selected, you won’t see any errors in the console.
You can read up about the switches at the Unity documentation.

Since you are listening to events from some external library: Are you sure these are involved on the Unity main thread?
Most of the Unity API can only be used on the main thread! I suspect this to be the issue here as it reaches the line containing GameObject.Find but nothing beyond.
You would basically need to dispatch the calls back into the Unity main thread. A pattern often used for this could look like
public class MainThreadDispatcher : MonoBehaviour
{
private static MainThreadDispatcher _instance;
public static MainThreadDispatcher Instance => _instance;
private readonly ConcurrentQueue<Action> _actions = new ();
private void Awake ()
{
if(_instance && _instance != this)
{
Destroy(gameObject);
return;
}
_instance = this;
// optionally
DontDestroyOnLoad(gameObject);
}
private void Update()
{
while(_actions.TryDequeue(out var action))
{
action?.Invoke();
}
}
public void DoInNextUpdate(Action action)
{
_actions.Enqueue(action);
}
}
public void PlayNote(int note, int velocity)
{
Debug.Log("Playing note " + note + " at velocity " + velocity); // this logs the message fine
GameObject key = GameObject.Find(note.toString()); // execution reaches this line
// nothing is executed from this point on
if (key != null)
key.GetComponent<ObjectSetup>().SetPressed(true);
Debug.Log("Done"); // never logs anything
}
And then you do
public void PlayNote(int note, int velocity)
{
MainThreadDispatcher.Instance.DoInNextUpdate(() =>
{
Debug.Log("Playing note " + note + " at velocity " + velocity); // this logs the message fine
GameObject key = GameObject.Find(note.ToString());
if (key && key.TryGetComponent<ObjectSetup>(out var setup))
setup.SetPressed(true);
Debug.Log("Done");
});
}

Know when a function is finished proccessing

I tried to make a screen capture but I want to not include certain canvas. i tried the code below but canvasses are still included in the capture. how to know when the screencapture function has finish before executing the next line of code?
void Update()
{
if (Input.GetKeyDown(KeyCode.Space))
{
GameManager.Instance.memeButton.SetActive(false);
ScreenCapture.CaptureScreenshot("C:/Users/jamjam/dwhelper/Desktop/test/jamjam.png");
GameManager.Instance.memeButton.SetActive(true);
}
}

See ScreenCapture.CaptureScreenshot
The CaptureScreenshot returns immediately on Android. The screen capture continues in the background. The resulting screen shot is saved in the file system after a few seconds.
=> There is no information really how it behaves on a PC. So basically there is no build-in way to know when it is finished which is valid on all platforms ... you could however build a Coroutine for checking that like e.g.
private bool alreadyTakingScreenshot;
void Update()
{
if (!alreadyTakingScreenshot && Input.GetKeyDown(KeyCode.Space))
{
// Start a new routine
StartCoroutine(ScreenshotRoutine);
}
}
// or in case you do NOT want the GUI to appear in your screenshot (see comment in ScreenshotRoutine)
//void LateUpdate()
//{
// if (!alreadyTakingScreenshot && Input.GetKeyDown(KeyCode.Space))
// {
// // Start a new routine
// StartCoroutine(ScreenshotRoutine);
// }
//}
private void IEnumerator ScreenshotRoutine()
{
if(alreadyTakingScreenshot) yield break;
alreadyTakingScreenshot = true;
GameManager.Instance.memeButton.SetActive(false);
// It is convenient to wait until the end of the frame which is right
// before the frame is rendered by the camera
// If you don't want the GUI to appear in your screenshot then
// remove this and rather start the routine from LateUpdate
yield return new WaitForEndOfFrame();
// Use a timestamp so everytime a new image is created
var dateStamp = DateTime.Now.ToString("yyyy-dd-M_HH-mm-ss");
var path = #"C:/Users/jamjam/dwhelper/Desktop/test/jamjam_" + dateStamp + ".png";
// Or alternatively simply delete the exsting file as you will overwrite it
//var path = #"C:/Users/jamjam/dwhelper/Desktop/test/jamjam.png"
//if(!File.Exists(path))
//{
// File.Delete(path);
//}
// start the capture
ScreenCapture.CaptureScreenshot(path);
// Wait until the according file is actually created
while(!File.Exists(path))
{
yield return null;
}
GameManager.Instance.memeButton.SetActive(true);
alreadyTakingScreenshot = false;
}

AnimationPlayer animations do not play in Godot

Forewords
Firstly, I know posting graphical resources for codes is not encouraged in this platform. I will also post the code but, in this particular case, I think posting a video about it is much more helpful than just posting some arbitrary code because the structuring of game projects really vary depending on their requirements. However, I still respect the platform's rules so if a mod asks me to format my question according to the community rules, I can do that or they also can simply delete my question. I respect that.
The Issue
It's actually a simple issue but it's driving me crazy because of its simplicity. I just want to fade in when I load a scene and then fade out whenever I click a button. As to how I do that, this is the video about it.
To sum up, I load another scene called "Fader" which contains a ColorRect with a black color and AnimationPlayer to change ColorRect's alpha value.
The code is below with extra comments on relevant parts:
using Godot;
using System;
public class TitleScreen : Control
{
private Button[] buttons;
private Control fader; // the scene that I inject
public override void _Ready() // when title screen gets ready
{
GD.Print("Preparing TitleScreen...");
InitButtons();
InitFader(); // initialize fader
FadeIn(); // do fade in animation
}
private void InitFader() // initializing fader
{
GD.Print("Initializing fader...");
var faderScene = (PackedScene)ResourceLoader.Load("res://components/Fader.tscn"); // load external fader scene
fader = (Control)faderScene.Instance(); // instantiate the scene
fader.SetSize(OS.WindowSize); // set the size of fader scene to the game window, just in case
var rect = (ColorRect)fader.GetNode("rect"); // get "rect" child from fader scene
rect.SetSize(OS.WindowSize); // set "rect" size to the game window as well, just in case
fader.Visible = false; // set the visibility to false
AddChild(fader); // add initialized fader scene as a child of title screen
}
private void InitButtons()
{
GD.Print("Initializing buttons...");
buttons = new Button[3]{
(Button)GetNode("menu_container/leftmenu_container/menu/start_button"),
(Button)GetNode("menu_container/leftmenu_container/menu/continue_button"),
(Button)GetNode("menu_container/leftmenu_container/menu/exit_button"),
};
GD.Print("Adding events to buttons...");
buttons[0].Connect("pressed", this, "_StartGame");
buttons[2].Connect("pressed", this, "_QuitGame");
}
private void FadeIn()
{
GD.Print("Fading in...");
fader.Visible = true; // set visibility of fader to true
var player = (AnimationPlayer)fader.GetNode("player"); // get animation player
player.Play("FadeIn"); // play FadeIn animation
fader.Visible = false; // set visibility of fader to false
}
private void FadeOut()
{
// similar to FadeIn
GD.Print("Fading out...");
fader.Visible = true;
var player = (AnimationPlayer)fader.GetNode("player");
player.Play("FadeOut");
fader.Visible = false;
}
public void _StartGame() // whenever I click start game button
{
FadeOut(); // fade out
GetTree().ChangeScene("res://stages/Demo01.tscn");
}
public void _QuitGame() // whenever I click quit game button
{
FadeOut(); // fade out
GetTree().Quit();
}
}
Seems like I can't see something. Why does it not fade in and out?
Environment
Manjaro 19.0.2
Mono JIT Compiler 6.4.0 (if it is relevant)
Godot 3.2

So, the issue was Play method on AnimationPlayer object kinda runs like async (dunno if this is the correct term for it).
Luckily, there is a feature called signals in Godot. There are animation_started and animation_finished signals on AnimationPlayer objects. Basically, I created a C# script for Fader scene, hooked the signals from player to fader as in:
animation_started to _FaderAnimationStart
animation_finished to _FaderAnimationEnd
At the end, my script looks like below:
using Godot;
using System;
public class Fader : Control
{
private ColorRect rect;
private AnimationPlayer player;
public override void _Ready()
{
GD.Print("Initializing Fader...");
rect = (ColorRect)GetNode("rect");
player = (AnimationPlayer)GetNode("player");
SetSize(OS.WindowSize);
rect.SetSize(OS.WindowSize);
Visible = false;
}
private void _FaderAnimationStart(String anim_name)
{
Visible = true;
}
private void _FaderAnimationEnd(String anim_name)
{
Visible = false;
}
}
I solved it thanks to njamster's answer and Hans Passant's comment.
However, this only solves half of the problem. Yes, the scene now fades in when it loads but it does not fade out. Given that it executes kinda-async (again, I'm not sure if this is the correct term), changing scene interrupts while running the animation. I will update the answer when I solve that problem as well.
Update
Well, I cannot seem to solve the fade out part because it requires to access parent node from initialized child scene. There are some methods I can think of.
First one is to somehow parameterize "Fader" scene. This can be done in many ways but at the end, when you initialize it from another scene, you need to cast it to Fader and I don't know if this is a valid way to do it. Another concern is standardizing this in the codebase. A similar method is discussed in here.
Second one is to write it as a plugin which has it benefits and drawbacks. C# is not really battle-tested in this particular area.
Third one is to use a state management system. Here is a redux implementation for Godot. And you need to somehow integrate it for signals, which seems like a hassle.
So, overall, I still do not know how to fade out.

How to repeat function while button is held down (New unity input system)

Trying to repeat the function function OnAttack() continuously while a button is being held down.
Basically I'm looking for an equivalent to Update() { GetKeyDown() {//code }} But with the input system.
Edit: using a joystick, cant tell what button is being pressed.

Okay I solved it by using "press" in the interactions and giving that The trigger behavior "Press and release", then did
bool held = false;
Update()
{
if(held)
{
//animation
}
else if(!held)
{
//idle animation
}
}
OnAttack() {
held = !held;
}
This way if I press the button held goes to true so it repeats the animation every frame, letting go makes "held" untrue and does the idle animation

Essentially, the function you assign to the button will be triggered twice per button press, once when it is pressed (performed), and once when it is released (canceled). You can pass in this context at the beginning of your function, just make sure you are using the library seen at the top of this script.
Now you can toggle a bool on and off stating whether or not the button is pressed, then perform actions during update dependent on the state of the bool
using static UnityEngine.InputSystem.InputAction;
bool held = false;
Update()
{
if(held)
{
//Do hold action like shooting or whatever
}
else if(!held)
{
//do alternatice action. Not Else if required if no alternative action
}
}
//switch the status of held based on whether the button is being pressed or released. OnAttack is called every time the button is pressed and every time it is released, the if statements are what determine which of those two is currently happening.
OnAttack(CallbackContext ctx) {
if (ctx.performed)
held= true;
if (ctx.canceled)
held= false;
}

This is paraphrasing a solution I created for a click to move arpg mechanic.
using System.Threading.Tasks;
using UnityEngine;
[SerializeField] private InputAction pointerClickAction;
private bool pointerHeld;
void Start()
{
pointerClickAction.canceled += ClickMouseMove;
pointerClickAction.started += PointerHoldBegin;
pointerClickAction.performed += ClickMouseMove;
pointerClickAction.canceled += PointerHoldEnd;
}
private void OnEnable()
{
pointerClickAction.Enable();
pointerPositionAction.Enable();
}
private void OnDisable()
{
pointerClickAction.Disable();
pointerPositionAction.Disable();
}
public async void ClickMouseMove(InputAction.CallbackContext context)
{
while (pointerHeld)
{
DoSomething();
await Task.Delay(500);
}
}
public void PointerHoldBegin(InputAction.CallbackContext context)
{
pointerHeld = true;
}
public void PointerHoldEnd(InputAction.CallbackContext context)
{
pointerHeld = false;
}
public void DoSomething()
{
//Your Code
}
In Task.Delay() you can insert your own polling rate in milliseconds, using Task.Yield() seems to be faster than Update so I don't recommend that, you should poll at the minimum the same delay as your physics/fixed update, having a higher delay gives a performance boost if you don't need a high amount of repetitions per loop. I set mine to 500 since I don't need my character to plot its navigation that often. In regards to TC, you would set the delay to something sensible e.g the attack's animation length, or whatever the delay rate would be for how many attacks can be performed per second.

If you want your project to scale you might want to avoid as much as possible assertions(i.e if functions) in your Update/FixedUpdate/LateUpdate functions as they are executed constantly. I recommand you to read this article about coroutines https://gamedevbeginner.com/coroutines-in-unity-when-and-how-to-use-them/
You can build coroutines which act as local update() functions which are executed only when needed. This will lead you to a better organization of your code and might boost performance in some cases.
For exemple in your case you could use something like this.
bool held = false;
Update()
{
/* Whatever you want but the least assertion possible */
}
IEnumerator RenderHeldAnimation()
{
while (held)
{
// held animation
yield return new WaitForFixedUpdate(); /* Will block until next fixed frame right after FixedUpdate() function */
// yield return null /* Will block until next next frame right after Update() function */
}
}
IEnumerator RenderIdleAnimation()
{
while (!held)
{
// idle animation
yield return new WaitForFixedUpdate(); /* Will block until next fixed frame right after FixedUpdate() function */
// yield return null /* Will block until next next frame right after Update() function */
}
}
OnAttack() {
held = !held;
if (held) {
StartCoroutine(RenderHeldAnimation());
} else {
StartCoroutine(RenderIdleAnimation());
}
}

As mentioned in another answer, the context.canceled is not called when using the Press And Release interaction. As a follow up for documentation purposes as this is a top Google result, to correctly use a bool held without doing a blind toggle (held = !held) which may end up with drift, you can access context.control.IsPressed() like the following:
void OnAttack(CallbackContext context)
{
held = context.control.IsPressed();
}

I encountered the same issue and this was the method that seemed to work for me
private float _moveSpeed = 3f;
private float _moveDirection;
private void Update()
{
transform.Translate(_moveSpeed * _moveDirection * Time.deltaTime * transform.forward);
}
public void Move(InputAction.CallbackContext ctx)
{
_moveDirection = ctx.ReadValue<float>();
}
For some odd reason,the hold interaction works properly in reading the input, but I still need the update function to implement the actual logic.
Can't really complain though, it works. Although I'd love to know why it happens this way.
Hopefully this can be of help to someone.

You can use a timer for that purpose, in combination with events KeyUp and KeyDown.
Please look at the following link. It is pretty much similar to your problem.
Link

OpenCV dll calls from Unity3D lead to FPS drop

I want to recognize ArUco Marker in Unity3D and attach a GameObject to their position. I know there are packages in the Asset Store, but as other people got it working I was looking for an free existing solution or try it by myself.
ArucoUnity The detection worked like a charm, but Unity crashes when accessing the data of the rvecs, tvecs. The error somewhere occur in the Get(int i) in the Vec3d class, when the c++ method au_cv_Vec3d_get(CppPtr, i, CppPtr); is called
OpenCv plus Unity This implementation seems not to be complete as there exist no function to EstimatePoseSingleMarker or similar to get the rvecs and tvecs. Also the last updated was Jan 2019.
As Unity (C#) provides the possibility to access unmanaged code, I followed this great tutorial and for the begging I was able to forward the cv::VideoCaputre stream to Unity. The only problem that occured was a FPS drop to around 35-40 whereas normally I get around 90-100.
The C# code:
void Update()
{
MatToTexture2D();
image.texture = tex;
}
void MatToTexture2D()
{
OpenCVInterop.GetRawImageBytes(pixelPtr);
//Update the Texture2D with array updated in C++
tex.SetPixels32(pixel32);
tex.Apply();
}
The C++ code:
extern "C" void __declspec(dllexport) __stdcall GetRawImageBytes(unsigned char* data)
{
_capture >> _currentFrame;
cv::Mat resizedMat(camHeight, camWidth, _currentFrame.type());
cv::resize(_currentFrame, resizedMat, resizedMat.size(), cv::INTER_CUBIC);
//Convert from RGB to ARGB
cv::Mat argb_img;
cv::cvtColor(resizedMat, argb_img, cv::COLOR_BGR2BGRA);
std::vector<cv::Mat> bgra;
cv::split(argb_img, bgra);
std::swap(bgra[0], bgra[3]);
std::swap(bgra[1], bgra[2]);
std::memcpy(data, argb_img.data, argb_img.total() * argb_img.elemSize());
}
The cause seems to be the first line _capture >> _currentFrame;, but as the others projects have to do the same (at least I guess so), I wonder if there is another reason.
If I don't manage to fix this issue I have to look for alternative approaches.

Just adding to / building on Mars' answer:
For the threading problem I would actually use a thread-save ConcurrentStack<Color32[]>. A stack is "last-in | first-out" so that the first item returned is always the last image data added by the thread.
The thread uses Push(pixel32) to add a new entry with image data
in Update you only use the latest entry (TryPop) for updating the texture.
The rest you ignore (Clear).
So something like
// the OpenCV thread will add(push) entries
// the Unity main thread will work on the entries
private ConcurrentStack<Color32[]> stack = new ConcurrentStack<Color32[]>();
public RawImage image;
public Texture2D tex;
private Thread thread;
void Start()
{
// Wherever you get your tex from
tex = new Texture2D(...);
// it should be enough to do this only once
// the texture stays the same, you only update its content
image.texture = tex;
}
// do things in OnEnable so everytime the object gets enabled start the thread
void OnEnable()
{
stack.Clear();
if(thread != null)
{
thread.Abort();
}
thread = new Thread(MatToTexture2D);
thread.Start();
}
void Update()
{
// here in the main thread work the stack
if (stack.TryPop(out var pixels32))
{
// Only use SetPixels and Apply when really needed
tex.SetPixels32(pixels32);
tex.Apply();
}
// Erase older data
stack.Clear();
}
// Make sure to terminate the thread everytime this object gets disabled
private void OnDisable()
{
if(thread == null) return;
thread.Abort();
thread = null;
}
// Runs in a thread!
void MatToTexture2D()
{
while(true)
{
try
{
// Do what you already have
OpenCVInterop.GetRawImageBytes(pixelPtr);
// However you convert the pixelPtr into Color32
Color32[] pixel32 = GetColorArrayFromPtr(pixelPtr);
// Now add this data to the stack
stack.Push(pixel32);
}
catch (ThreadAbortException ex)
{
// This exception is thrown when calling Abort on the thread
// -> ignore the exception since it is produced on purpose
}
}
}

If I recall correctly, the C++ call to get an image (_capture >> _currentFrame;) is blocking/synchronous, meaning your code won't continue until it actually retrieves the image. You probably want to run your MatToTexture2D code asynchronously.
※This will mean that your frame rate will be higher than your image retrieval rate.
Have your MatToTexture2D function run continuously as needed, updating tex. Then just continue to set your texture to the latest tex, which may be the same value 2-3 frames in a row.
Edit:
#derHugo's answer is much more solid for the programming side, so I'll hide that part. The basic issue is explained above, and derHugo's work-around is much better than my pseudo-code :)