We Keep Coding

Simple 2d physics with polygons C#

I have been working on a 2D physics engine using polygons.
And i am having trouble implementing the actual physics part. For a bit of background, i am not experienced at all when it comes to physics and therefor even if a found how to do the entire physics thing online, i would not be able to implement it into my project.
My goal is:
To have polygons fall with gravity.
Have weight drag etc.
Collision between multiple polygons.
What i have already made:
A way of displaying and creating multiple polygons.
Moving and rotating specified object(polygon).
Coeffients for drag, gravity and weight.
Hit boxes and visual boxes. (Visual boxes are what gets displayed and hit boxes are for physics)
A center point for every object. (So far is used for rotation)
A tick for when everything gets calculated. (Gametick/tickrate or whatever you wanna call it)
What i was not able to add / looking for:
Actual gravity.
Collision detection
Velocity for each object.
Collision between object.
Code snippets / how stuff works so far:
Beware that my code is janky and could be made better or more efficient.
Efficiency is not what im looking for!
Function for creating object:
public Object CreateNew(PointF[] hb, PointF[] vb, float rt, Color cl, bool gr, PointF ps)
{
Object obj = new Object
{
pos = ps,
rotation = rt,
offsets = vb,
hitBox = hb,
visBox = vb,
gravity = gr,
clr = cl,
};
#region center
List<Vector2> v2Points = new List<Vector2>();
foreach (PointF p in obj.offsets)
{
v2Points.Add(new Vector2(p.X, p.Y));
}
PointF point = ToPoint(Centroid(v2Points));
obj.center = new PointF(point.X, point.Y);
#endregion
return obj;
}
Function for changing position of object:
public Object ChangePosition(PointF pos, double rot, Object obj)
{
//////////////
int i = 0;
foreach (PointF p in obj.visBox)
{
float minPosX = (float)Math.Sqrt((Math.Pow(obj.center.X - pos.X, 2) + Math.Pow(0 - 0, 2)));
float minPosY = (float)Math.Sqrt((Math.Pow(obj.center.Y - pos.Y, 2) + Math.Pow(0 - 0, 2)));
obj.visBox[i] = new PointF(obj.offsets[i].X + pos.X, obj.offsets[i].Y + pos.Y);
i++;
}
i = 0;
foreach (PointF p in obj.hitBox)
{
float minPosX = (float)Math.Sqrt((Math.Pow(obj.center.X - pos.X, 2) + Math.Pow(0 - 0, 2)));
float minPosY = (float)Math.Sqrt((Math.Pow(obj.center.Y - pos.Y, 2) + Math.Pow(0 - 0, 2)));
obj.hitBox[i] = new PointF(obj.offsets[i].X + pos.X, obj.offsets[i].Y + pos.Y);
i++;
}
obj.pos = pos;
List<Vector2> v2Points = new List<Vector2>();
foreach (PointF p in obj.offsets)
{
v2Points.Add(new Vector2(p.X, p.Y));
}
PointF point = ToPoint(Centroid(v2Points));
obj.center = point;
List<Vector2> v2Points2 = new List<Vector2>();
foreach (PointF p in obj.hitBox)
{
v2Points2.Add(new Vector2(p.X, p.Y));
}
PointF point2 = ToPoint(Centroid(v2Points2));
obj.centerHitBox = point2;
obj.hitBox = RotatePolygon(obj.hitBox, obj.center, rotation * -1);
obj.visBox = RotatePolygon(obj.visBox, obj.center, rotation * -1);
obj.offsets = RotatePolygon(obj.offsets, obj.center, rotation * -1);
obj.hitBox = RotatePolygon(obj.hitBox, obj.center, rot);
obj.visBox = RotatePolygon(obj.visBox, obj.center, rot);
obj.offsets = RotatePolygon(obj.offsets, obj.center, rot);
rotation = rot;
return obj;
}
Pastebin link to object script:
https://pastebin.com/9SnG4vyj
I will provide more information or scripts if anybody needs it!

Why would frames drop considerably the longer a Unity Game runs?

I am using Unity 2020.3.4f1 to create a 2D game for mobile.
I created a map builder for the game.
When I hit play, it makes a bunch of 'maps' that are saved as json files. The longer the game plays seems to result in extremely slow frame rates (200fps -> 2 fps) based on the stats menu while the game is running.
The strange thing is if I go to the "Scene" tab and left click on a sprite the fps instantly jumps back up again.
Screenshots
The problem seems related to taking screenshots within Unity.
The big bulge happens & only resets when I un-pause the game.
Questions
Why would the frame rates drop considerably over time the longer the game is running?
Why would the frame rate jump back up after selecting a sprite in the "Scene" tab?
What happens in Unity when selecting a sprite in the "Scene" tab? Is there a garbage collection method?
Script:
private void Awake()
{
myCamera = gameObject.GetComponent<Camera>();
instance = this;
width = 500;
height = 500;
}
private void OnPostRender()
{
if(takeScreenShotOnNextFrame)
{
takeScreenShotOnNextFrame = false;
RenderTexture renderTexture = myCamera.targetTexture;
Texture2D renderResult = new Texture2D(renderTexture.width, renderTexture.height, TextureFormat.ARGB32, false);
Rect rect = new Rect(0, 0, renderTexture.width, renderTexture.height);
renderResult.ReadPixels(rect, 0, 0);
float myValue = 0;
float totalPixels = renderResult.width * renderResult.height;
for (int i = 0; i < renderResult.width; i++)
{
for (int j = 0; j < renderResult.height; j++)
{
Color myColor = renderResult.GetPixel(i, j);
myValue += myColor.r;
//Debug.Log("Pixel (" + i + "," + j + "): " + myColor.r);
}
}
occlusion = ((myValue / totalPixels) * 100);
byte[] byteArray = renderResult.EncodeToPNG();
System.IO.File.WriteAllBytes(Application.dataPath + "/Resources/ScreenShots/CameraScreenshot.png", byteArray);
// Cleanup
RenderTexture.ReleaseTemporary(renderTexture);
myCamera.targetTexture = null;
renderResult = null;
}
}
private void TakeScreenshot(int screenWidth, int screenHeight)
{
width = screenWidth;
height = screenHeight;
if(myCamera.targetTexture != null)
{
RenderTexture.ReleaseTemporary(myCamera.targetTexture);
//Debug.Log("Camera target texture null: " + myCamera.targetTexture == null);
}
myCamera.targetTexture = RenderTexture.GetTemporary(width, height, 16);
takeScreenShotOnNextFrame = true;
}
public static void TakeScreenshot_Static(int screenWidth, int screenHeight)
{
instance.TakeScreenshot(screenWidth, screenHeight);
}
}

Unity iOS memory usage rises and wont go down again

My game has a baseline memory usage of around 315 MB. However calling the following functions leads to a sharp rise in memory usage, leveling out at around 480 MB while reaching spikes of 580 MB and more, accompanied by memory warnings and even crashes.
What happens: First the TakeScreenshot IEnum is called three times in a row, wich is the max. count for screenshots in one session. Second the function SendEmailTask is called showing all three pictures for the user to choose one. By choosing picture "#1" the function SendImage1 is triggered.
Maybe someone can point me to where and how I can get some of that memory back, that would be really great!
All relevant code should be here:
public class Picture : MonoBehaviour {
private int ssCount = 0;
private Sprite cachedImage1sprite;
private Sprite cachedImage2sprite;
private Sprite cachedImage3sprite;
private Texture2D cachedImage1;
private Texture2D cachedImage2;
private Texture2D cachedImage3;
private Texture2D JPGtex1;
private Texture2D JPGtex2;
private Texture2D JPGtex3;
private Texture2D tex;
void Awake () {
}
// Use this for initialization
void Start () {
JPGtex1 = new Texture2D (2, 2, TextureFormat.RGB24, false );
JPGtex2 = new Texture2D (2, 2, TextureFormat.RGB24, false );
JPGtex3 = new Texture2D (2, 2, TextureFormat.RGB24, false );
// Create a texture the size of the screen, RGB24 format
int width = Screen.width;
int height = Screen.height;
tex = new Texture2D( width, height, TextureFormat.RGB24, false );
}
// Update is called once per frame
void Update () {
if (ssCount == 0) {
SendEmail.interactable = false;
TakePhoto.interactable = true;
} else if (ssCount == 1) {
SendEmail.interactable = true;
} else if (ssCount == 3) {
TakePhoto.interactable = false;
}
//Debug.Log (ssCount);
}
void SendEmailTask(){
if (ssCount == 3) {
cachedImage1 = SA.IOSNative.Storage.AppCache.GetTexture ("IMAGE_1");
cachedImage2 = SA.IOSNative.Storage.AppCache.GetTexture ("IMAGE_2");
cachedImage3 = SA.IOSNative.Storage.AppCache.GetTexture ("IMAGE_3");
ImagePicker.SetActive (true);
//Image1
Rect rec1 = new Rect(0, 0, cachedImage1.width, cachedImage1.height);
cachedImage1sprite = Sprite.Create(cachedImage1, rec1, new Vector2(0,0),1);
Image1.image.sprite = cachedImage1sprite;
//Image2
Rect rec2 = new Rect(0, 0, cachedImage2.width, cachedImage2.height);
cachedImage2sprite = Sprite.Create(cachedImage2, rec2, new Vector2(0,0),1);
Image2.image.sprite = cachedImage2sprite;
//Image3
Rect rec3 = new Rect(0, 0, cachedImage3.width, cachedImage3.height);
cachedImage3sprite = Sprite.Create(cachedImage3, rec3, new Vector2(0,0),1);
Image3.image.sprite = cachedImage3sprite;
SA.IOSNative.Storage.AppCache.Remove ("IMAGE_1");
SA.IOSNative.Storage.AppCache.Remove ("IMAGE_2");
SA.IOSNative.Storage.AppCache.Remove ("IMAGE_3");
}
}
IEnumerator TakeScreenshot() {
// Wait till the last possible moment before screen rendering to hide the UI
yield return null;
GameObject.Find("Buttons").GetComponent<Canvas>().enabled = false;
FlashImage();
// Wait for screen rendering to complete
yield return new WaitForEndOfFrame();
// Create a texture the size of the screen, RGB24 format
int width = Screen.width;
int height = Screen.height;
// Read screen contents into the texture
tex.ReadPixels( new Rect(0, 0, width, height), 0, 0 );
tex.Apply();
//byte[] screenshot = tex.EncodeToPNG();
print("Size is " + tex.width + " by " + tex.height);
if (ssCount == 0) {
SA.IOSNative.Storage.AppCache.Save ("IMAGE_1", tex);
ssCount++;
} else if (ssCount == 1) {
SA.IOSNative.Storage.AppCache.Save ("IMAGE_2", tex);
ssCount++;
} else if (ssCount == 2) {
SA.IOSNative.Storage.AppCache.Save ("IMAGE_3", tex);
ssCount++;
}
IOSCamera.Instance.SaveTextureToCameraRoll(tex); //Save to Cameraroll
// Show UI after we're done
GameObject.Find("Buttons").GetComponent<Canvas>().enabled = true;
}
public void SendImage1() {
byte[] screenshot1;
screenshot1 = cachedImage1.EncodeToJPG ();
if (Facebook == false) {
JPGtex1.LoadImage (screenshot1);
TextureScale.Bilinear (JPGtex1, 1200, 900);
IOSSocialManager.Instance.SendMail (SubjectText, EmailText, "", JPGtex1);
} else {
StartCoroutine(UploadToPage(screenshot1));
}
backToGame ();
}
public void backToGame() {
Destroy (cachedImage1sprite);
Destroy (cachedImage2sprite);
Destroy (cachedImage3sprite);
SA.IOSNative.Storage.AppCache.Remove ("IMAGE_1");
SA.IOSNative.Storage.AppCache.Remove ("IMAGE_2");
SA.IOSNative.Storage.AppCache.Remove ("IMAGE_3");
Destroy(cachedImage1);
Destroy(cachedImage2);
Destroy(cachedImage3);
cachedImage1 = null;
cachedImage2 = null;
cachedImage3 = null;
Image3Obj.SetActive (true);
ImagePicker.SetActive (false);
}
}
EDIT
Detailed memory profiler after going thru the routine twice:
Xcode memory profiler after going thru the routine twice:

From your profiler report;
You're using a big chunk of memory on Meshes and Texture2D Assets - that'd suggest to me you're possibly drawing over them/hiding them from the user rather than actually removing them from memory. 50+ MB in Texture2D Assets is a little odd considering you've got a further 120+ MB of Meshes being loaded in. I'm assuming from that that it's a 3D game, but has a 2D UI? If so, then 50MB is quite a lot to be spending on Texture2D Assets.
"Objects" are what you think they are - they're Objects. Instantiated classes, or GameObjects containing attached components. So if you made a Player GameObject, and attached to that is a "playerStats" object with a couple of variables for health, speed, stamina etc etc, then that'd count as 2 objects.
80MB isn't too worrying for Objects. Your Texture2D and Meshes use is what would strike me as being pretty high for a game that's targeting iOS. Make sure you're using mobile-friendly models, and textures that aren't too high in resolution.

I had a similar problem with audio files in iOS, I don't know if this could be your case.
I loaded big audio files of 20mb+ on memory to process them and then release the memory, the thing is that the memory kept going up in the xcode profiler. This is caused by memory fragmentation, you can read more about it here: https://stackoverflow.com/a/3770593/4024219
My solution was to load the files in little chunks and reuse my arrays instead of creating and destroying new ones.

Low pass filter for accelerometer

I'm creating an app for windows phone using c# that uses the accelerator but its not smooth when displayed on the screen. I only need to move the Y-axis. I have seen this formula on microsofts website but I'm not sure how i should use it
O = O-1 + α(I – O-1)
where O is the output, α is the coefficient and I is the input(raw value)
how do i implement this into my code which is
private void UpdateUI(AccelerometerReading accelerometerReading)
{
statusTextBlock.Text = "getting data";
Vector3 acceleration = accelerometerReading.Acceleration;
// Show the numeric values on screen.
yTextBlock.Text = "Y: " + acceleration.Y.ToString("0.00");
//low pass filter
//????
//move ball on screen
var TopMar = (278.5*acceleration.Y)+278.5;
var BotMar = 557 - TopMar;
yDot.Margin = new Thickness(203, BotMar, 203, TopMar);
}

If Vector3 has overloaded the operators, this should do:
private Vector3 MeanAcceleration = null;
private void UpdateUI(AccelerometerReading accelerometerReading)
{
const double alpha = 0.05;
statusTextBlock.Text = "getting data";
Vector3 acceleration = accelerometerReading.Acceleration;
// Show the numeric values on screen.
yTextBlock.Text = "Y: " + acceleration.Y.ToString("0.00");
//low pass filter
if (MeanAcceleration == null)
MeanAcceleration = acceleration;
else
MeanAcceleration = (1 - alpha) * MeanAcceleration + alpha * acceleration;
//move ball on screen
var TopMar = (278.5 * MeanAcceleration.Y) + 278.5;
var BotMar = 557 - TopMar;
yDot.Margin = new Thickness(203, BotMar, 203, TopMar);
}
You need a field (or something of similar scope) and assign it the mean value. Every timestep, you update this mean value.
Alpha must be between 0 and, 1, to effectively low-pass the signal it should be 0.1 or below. Decrease this if the output is too wiggly and increase alpha if the output is too slow. If both is the case, you probably need a more sophisticated digital filter.
If the beginning is not important, you can initialize the mean with something like
private Vector3 MeanAcceleration = new Vector3(0, 0, 0);
but I'm not sure about the constructor, because I don't know exactly which Vector3 that is.

Eye detection using OpenCVSharp in Unity (fps issues)

I'm currently working on a project involving integrating OpenCVSharp into Unity, to allow eye tracking within a game environment. I've managed to get OpenCVSharp integrated into the Unity editor and currently have eye-detection (not tracking) working within a game. It can find your eyes within a webcam image, then display where its currently detected them on a texture, which I display within the scene.
However its causing a HUGE fps drop, mainly because every frame its converting a webcam texture into an IPLimage so that OpenCV can handle it. It then has to convert it back to a 2Dtexture to be displayed within the scene, after its done all the eye detection. So understandably its too much for the CPU to handle. (As far as I can tell its only using 1 core on my CPU).
Is there a way to do all the eye detection without converting the texture to an IPLimage? Or any other way to fix the fps drop. Some things that I've tried include:
Limiting the frames that it updates on. However this just causes it
to run smoothly, then stutter horribly on the frame that it has to
update.
Looking at threading, but as far as I'm aware Unity doesn't allow it.
As far as I can tell its only using 1 core on my CPU which seems a bit silly. If there was a way to change this it could fix the issue?
Tried different resolutions on the camera, however the resolution that the game can actually run smoothly at, is too small for the eye's to actually be detected, let alone tracked.
I've included the code below, of if you would prefer to look at it in a code editor here is a link to the C# File. Any suggestions or help would be greatly appreciated!
For reference I used code from here (eye detection using opencvsharp).
using UnityEngine;
using System.Collections;
using System;
using System.IO;
using OpenCvSharp;
//using System.Xml;
//using OpenCvSharp.Extensions;
//using System.Windows.Media;
//using System.Windows.Media.Imaging;
public class CaptureScript : MonoBehaviour
{
public GameObject planeObj;
public WebCamTexture webcamTexture; //Texture retrieved from the webcam
public Texture2D texImage; //Texture to apply to plane
public string deviceName;
private int devId = 1;
private int imWidth = 640; //camera width
private int imHeight = 360; //camera height
private string errorMsg = "No errors found!";
static IplImage matrix; //Ipl image of the converted webcam texture
CvColor[] colors = new CvColor[]
{
new CvColor(0,0,255),
new CvColor(0,128,255),
new CvColor(0,255,255),
new CvColor(0,255,0),
new CvColor(255,128,0),
new CvColor(255,255,0),
new CvColor(255,0,0),
new CvColor(255,0,255),
};
const double Scale = 1.25;
const double ScaleFactor = 2.5;
const int MinNeighbors = 2;
// Use this for initialization
void Start ()
{
//Webcam initialisation
WebCamDevice[] devices = WebCamTexture.devices;
Debug.Log ("num:" + devices.Length);
for (int i=0; i<devices.Length; i++) {
print (devices [i].name);
if (devices [i].name.CompareTo (deviceName) == 1) {
devId = i;
}
}
if (devId >= 0) {
planeObj = GameObject.Find ("Plane");
texImage = new Texture2D (imWidth, imHeight, TextureFormat.RGB24, false);
webcamTexture = new WebCamTexture (devices [devId].name, imWidth, imHeight, 30);
webcamTexture.Play ();
matrix = new IplImage (imWidth, imHeight, BitDepth.U8, 3);
}
}
void Update ()
{
if (devId >= 0)
{
//Convert webcam texture to iplimage
Texture2DtoIplImage();
/*DO IMAGE MANIPULATION HERE*/
//do eye detection on iplimage
EyeDetection();
/*END IMAGE MANIPULATION*/
if (webcamTexture.didUpdateThisFrame)
{
//convert iplimage to texture
IplImageToTexture2D();
}
}
else
{
Debug.Log ("Can't find camera!");
}
}
void EyeDetection()
{
using(IplImage smallImg = new IplImage(new CvSize(Cv.Round (imWidth/Scale), Cv.Round(imHeight/Scale)),BitDepth.U8, 1))
{
using(IplImage gray = new IplImage(matrix.Size, BitDepth.U8, 1))
{
Cv.CvtColor (matrix, gray, ColorConversion.BgrToGray);
Cv.Resize(gray, smallImg, Interpolation.Linear);
Cv.EqualizeHist(smallImg, smallImg);
}
using(CvHaarClassifierCascade cascade = CvHaarClassifierCascade.FromFile (#"C:\Users\User\Documents\opencv\sources\data\haarcascades\haarcascade_eye.xml"))
using(CvMemStorage storage = new CvMemStorage())
{
storage.Clear ();
CvSeq<CvAvgComp> eyes = Cv.HaarDetectObjects(smallImg, cascade, storage, ScaleFactor, MinNeighbors, 0, new CvSize(30, 30));
for(int i = 0; i < eyes.Total; i++)
{
CvRect r = eyes[i].Value.Rect;
CvPoint center = new CvPoint{ X = Cv.Round ((r.X + r.Width * 0.5) * Scale), Y = Cv.Round((r.Y + r.Height * 0.5) * Scale) };
int radius = Cv.Round((r.Width + r.Height) * 0.25 * Scale);
matrix.Circle (center, radius, colors[i % 8], 3, LineType.AntiAlias, 0);
}
}
}
}
void OnGUI ()
{
GUI.Label (new Rect (200, 200, 100, 90), errorMsg);
}
void IplImageToTexture2D ()
{
int jBackwards = imHeight;
for (int i = 0; i < imHeight; i++) {
for (int j = 0; j < imWidth; j++) {
float b = (float)matrix [i, j].Val0;
float g = (float)matrix [i, j].Val1;
float r = (float)matrix [i, j].Val2;
Color color = new Color (r / 255.0f, g / 255.0f, b / 255.0f);
jBackwards = imHeight - i - 1; // notice it is jBackward and i
texImage.SetPixel (j, jBackwards, color);
}
}
texImage.Apply ();
planeObj.renderer.material.mainTexture = texImage;
}
void Texture2DtoIplImage ()
{
int jBackwards = imHeight;
for (int v=0; v<imHeight; ++v) {
for (int u=0; u<imWidth; ++u) {
CvScalar col = new CvScalar ();
col.Val0 = (double)webcamTexture.GetPixel (u, v).b * 255;
col.Val1 = (double)webcamTexture.GetPixel (u, v).g * 255;
col.Val2 = (double)webcamTexture.GetPixel (u, v).r * 255;
jBackwards = imHeight - v - 1;
matrix.Set2D (jBackwards, u, col);
//matrix [jBackwards, u] = col;
}
}
}
}

You can move these out of the per frame update loop :
using(CvHaarClassifierCascade cascade = CvHaarClassifierCascade.FromFile (#"C:\Users\User\Documents\opencv\sources\data\haarcascades\haarcascade_eye.xml"))
using(CvMemStorage storage = new CvMemStorage())
No reason to be building the recognizer graph each frame.
Threading is the logical way to go moving forward if you want real speed updates, unity itself is not threaded, but you can fold in other threads if your careful.
Do the texture -> ipl image on the main thread then trigger an event to fire off your thread.
The thread can do all the CV work, probably construct the tex2d and then push back to main to render.

You should also be able to gain some performance improvements if you use:
Color32[] pixels;
pixels = new Color32[webcamTexture.width * webcamTexture.height];
webcamTexture.GetPixels32(pixels);
The Unity doco suggests that this can be quite a bit faster than calling "GetPixels" (and certainly faster than calling GetPixel for each pixel), and then you don't need to scale each RGB channel against 255 manually.