Related
As the title suggests I have a problem with the error occurring at the row
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
Error:
ReadPixels was called to read pixels from system frame buffer, while
not inside drawing frame. UnityEngine.Texture2D:ReadPixels(Rect,
Int32, Int32)
As I have understood from other posts one way to solve this issue is to make a Ienumerator method which yield return new WaitForSeconds or something, and call it like: StartCoroutine(methodname) so that the frames gets to load in time so that there will be pixels to read-ish.
What I don't get is where in the following code this method would make the most sense. Which part does not get to load in time?
PhotoCapture photoCaptureObject = null;
Texture2D targetTexture = null;
public string path = "";
CameraParameters cameraParameters = new CameraParameters();
private void Awake()
{
var cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);
// Create a PhotoCapture object
PhotoCapture.CreateAsync(false, captureObject =>
{
photoCaptureObject = captureObject;
cameraParameters.hologramOpacity = 0.0f;
cameraParameters.cameraResolutionWidth = cameraResolution.width;
cameraParameters.cameraResolutionHeight = cameraResolution.height;
cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
});
}
private void Update()
{
// if not initialized yet don't take input
if (photoCaptureObject == null) return;
if (Input.GetKey("k") || Input.GetKey("k"))
{
Debug.Log("k was pressed");
VuforiaBehaviour.Instance.gameObject.SetActive(false);
// Activate the camera
photoCaptureObject.StartPhotoModeAsync(cameraParameters, result =>
{
if (result.success)
{
// Take a picture
photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
}
else
{
Debug.LogError("Couldn't start photo mode!", this);
}
});
}
}
private static string FileName(int width, int height)
{
return $"screen_{width}x{height}_{DateTime.Now:yyyy-MM-dd_HH-mm-ss}.png";
}
private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
private void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result)
{
// Shutdown the photo capture resource
VuforiaBehaviour.Instance.gameObject.SetActive(true);
photoCaptureObject.Dispose();
photoCaptureObject = null;
}
Sorry if this counts as a duplicate to this for example.
Edit
And this one might be useful when I get to that point.
Is it so that I don't need these three lines at all?
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
As written in the comments the difference between using these three lines and not is that the photo saved has a black background + the AR-GUI. Without the second line of code above is a photo with the AR-GUI but with the background is a live stream of my computer webcam. And really I don't wanna see the computer webcam but what the HoloLens sees.
Your three lines
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
make not much sense to me. Texture2D.ReadPixels is for creating a Screenshot so you would overwrite the texture you just received from PhotoCapture with a screenshot? (Also with incorrect dimensions since camera resolution very probably != screen resolution.)
That's also the reason for
As written in the comments the difference between using these three lines and not is that the photo saved has a black background + the AR-GUI.
After doing
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
you already have the Texture2D received from the PhotoCapture in the targetTexture.
I think you probably confused it with Texture2D.GetPixels which is used to get the pixel data of a given Texture2D.
I would like to crop the captured photo from the center in the end and am thinking that maybe that is possible with this code row? Beginning the new rect at other pixels than 0, 0)
What you actually want is cropping the received Texture2D from the center as you mentioned in the comments. You can do that using GetPixels(int x, int y, int blockWidth, int blockHeight, int miplevel) which is used to cut out a certain area of a given Texture2D
public static Texture2D CropAroundCenter(Texture2D input, Vector2Int newSize)
{
if(input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!", this);
return null;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Create a new texture with newSize
var output = new Texture2D(newSize.x, newSize.y);
output.SetPixels(pixels);
output.Apply();
return output;
}
for (hopefully) better understanding this is an illustration what that GetPixels overload with the given values does here:
and than use it in
private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
// for example take only half of the textures width and height
targetTexture = CropAroundCenter(targetTexture, new Vector2Int(targetTexture.width / 2, targetTexture.height / 2);
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
Or you could make it an extension method in an apart static class like
public static class Texture2DExtensions
{
public static void CropAroundCenter(this Texture2D input, Vector2Int newSize)
{
if (input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!");
return;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Resize the texture (creating a new one didn't work)
input.Resize(newSize.x, newSize.y);
input.SetPixels(pixels);
input.Apply(true);
}
}
and use it instead like
targetTexture.CropAroundCenter(new Vector2Int(targetTexture.width / 2, targetTexture.height / 2));
Note:
UploadImageDataToTexture: You may only use this method if you specified the BGRA32 format in your CameraParameters.
Luckily you use that anyway ;)
Keep in mind that this operation will happen on the main thread and therefore be slow.
However the only alternative would be CopyRawImageDataIntoBuffer and generate the texture in another thread or external, so I'ld say it is ok to stay with UploadImageDataToTexture ;)
and
The captured image will also appear flipped on the HoloLens. You can reorient the image by using a custom shader.
by flipped they actually mean that the Y-Axis of the texture is upside down. X-Axis is correct.
For flipping the Texture vertically you can use a second extension method:
public static class Texture2DExtensions
{
public static void CropAroundCenter(){....}
public static void FlipVertically(this Texture2D texture)
{
var pixels = texture.GetPixels();
var flippedPixels = new Color[pixels.Length];
// These for loops are for running through each individual pixel and
// write them with inverted Y coordinates into the flippedPixels
for (var x = 0; x < texture.width; x++)
{
for (var y = 0; y < texture.height; y++)
{
var pixelIndex = x + y * texture.width;
var flippedIndex = x + (texture.height - 1 - y) * texture.width;
flippedPixels[flippedIndex] = pixels[pixelIndex];
}
}
texture.SetPixels(flippedPixels);
texture.Apply();
}
}
and use it like
targetTexture.FlipVertically();
Result: (I used FlipVertically and cropp to the half of size every second for this example and a given Texture but it should work the same for a taken picture.)
Image source: http://developer.vuforia.com/sites/default/files/sample-apps/targets/imagetargets_targets.pdf
Update
To your button problem:
Don't use
if (Input.GetKey("k") || Input.GetKey("k"))
First of all you are checking the exact same condition twice. And than GetKey fires every frame while the key stays pressed. Instead rather use
if (Input.GetKeyDown("k"))
which fires only a single time. I guess there was an issue with Vuforia and PhotoCapture since your original version fired so often and maybe you had some concurrent PhotoCapture processes...
I made a simple app for android devices that uses mobile's camera to take a photo and save it in internal storage folder /mnt/sdcard/DCIM/Camerizeman/
Photos are saved correctly but the problem that i am faceing is that i can't see the photos from mobile's gallery. I can see them correctly if i use a file manager or reboot me device. i am searchig 10 days now and the problem is that i have to refresh the gallery after saving the image.
I didn't found any working solution.
my code is bellow:
RenderTexture renderTex;
Texture2D screenshot;
Texture2D LoadScreenshot;
int width = Screen.width; // for Taking Picture
int height = Screen.height; // for Taking Picture
string fileName;
string myScreenshotLocation;
string screenShotName = "MyImage_AR_" + System.DateTime.Now.ToString("yyyy-MM-dd-HHmmss") + ".png";
public void Snapshot ()
{
StartCoroutine (CaptureScreen ());
}
public IEnumerator CaptureScreen ()
{
yield return null; // Wait till the last possible moment before screen rendering to hide the UI
//GameObject.FindGameObjectWithTag("Snapshoot").SetActive(false);
yield return new WaitForEndOfFrame (); // Wait for screen rendering to complete
if (Screen.orientation == ScreenOrientation.Portrait || Screen.orientation == ScreenOrientation.PortraitUpsideDown) {
mainCamera = Camera.main.GetComponent<Camera> (); // for Taking Picture
renderTex = new RenderTexture (width, height, 24);
mainCamera.targetTexture = renderTex;
RenderTexture.active = renderTex;
mainCamera.Render ();
screenshot = new Texture2D (width, height, TextureFormat.RGB24, false);
screenshot.ReadPixels (new Rect (0, 0, width, height), 0, 0);
screenshot.Apply (); //false
RenderTexture.active = null;
mainCamera.targetTexture = null;
}
if (Screen.orientation == ScreenOrientation.LandscapeLeft || Screen.orientation == ScreenOrientation.LandscapeRight) {
mainCamera = Camera.main.GetComponent<Camera> (); // for Taking Picture
renderTex = new RenderTexture (height, width, 24);
mainCamera.targetTexture = renderTex;
RenderTexture.active = renderTex;
mainCamera.Render ();
screenshot = new Texture2D (height, width, TextureFormat.RGB24, false);
screenshot.ReadPixels (new Rect (0, 0, height, width), 0, 0);
screenshot.Apply (); //false
RenderTexture.active = null;
mainCamera.targetTexture = null;
}
myScreenshotLocation = myFolderLocation + screenShotName;
File.WriteAllBytes (myFolderLocation + screenShotName, screenshot.EncodeToPNG ());
}
Please help!
The second working solution is:
using (AndroidJavaClass jcUnityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer"))
using (AndroidJavaObject joActivity = jcUnityPlayer.GetStatic<AndroidJavaObject>("currentActivity"))
using (AndroidJavaObject joContext = joActivity.Call<AndroidJavaObject>("getApplicationContext"))
using (AndroidJavaClass jcMediaScannerConnection = new AndroidJavaClass("android.media.MediaScannerConnection"))
using (AndroidJavaClass jcEnvironment = new AndroidJavaClass("android.os.Environment"))
using (AndroidJavaObject joExDir = jcEnvironment.CallStatic<AndroidJavaObject>("getExternalStorageDirectory"))
{
jcMediaScannerConnection.CallStatic("scanFile", joContext, new string[] { YOURFULL IMAGE PATH}, null, null);
}
Use MediaScannerConnection. You need the image "scanned into" the gallery.
Native Java Code goes something like:
MediaScannerConnection.scanFile(unityPlayerActivity, new String[]{externalImagePath}, null, null);
May Create plugin - Unity Android plugin tutorial (1/3) Fundamentals -, or use AndroidJavaClass.CallStatic to invoke a MediaScannerConnection.scanFile.
I am working on a game for learning purposes, I want to make it only with the .NET-Framework and a Windows Forms project in C#.
I want to get the 'screen' (Something that can be displayed on the window) as an int[]. Modify the array and reapply the altered array to the 'screen' in a buffered manner (So that it doesn't flicker).
I am currently using a Panel, which I draw a Bitmap on with Graphics. The Bitmap is converted to an int[] which I then can modify and reapply to the Bitmap and redraw. It works, but is very slow, especially because I have to scale up the image every frame because my game is only 300x160 and the screen 900x500.
Build up:
// Renders 1 frame
private void Render()
{
// Buffer setup
_bufferedContext = BufferedGraphicsManager.Current;
_buffer = _bufferedContext.Allocate(panel_canvas.CreateGraphics(), new Rectangle(0, 0, _scaledWidth, _scaledHeight));
_screen.clear();
// Get position of player on map
_xScroll = _player._xMap - _screen._width / 2;
_yScroll = _player._yMap - _screen._height / 2;
// Indirectly modifies the int[] '_pixels'
_level.render(_xScroll, _yScroll, _screen);
_player.render(_screen);
// Converts the int[] into a Bitmap (unsafe method is faster)
unsafe
{
fixed (int* intPtr = &_screen._pixels[0])
{
_screenImage = new Bitmap(_trueWidth, _trueHeight, _trueWidth * 4, PixelFormat.Format32bppRgb, new IntPtr(intPtr));
}
}
// Draw generated image on buffer
Graphics g = _buffer.Graphics;
g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.NearestNeighbor;
g.DrawImage(_screenImage, new Rectangle(0, 0, 900, 506));
// Update panel buffered
_buffer.Render();
}
Is there a faster way without external libraries to make this work?
I'm not to sure about the unsafe code , But I do know about the buffered graphics manager. I think you should create a class for it instead of creating a new one every time.As well as having all of your sprites widths and heights be determined at the load instead of scaling them. That sped up my small game engine a good bit.
class Spritebatch
{
private Graphics Gfx;
private BufferedGraphics bfgfx;
private BufferedGraphicsContext cntxt = BufferedGraphicsManager.Current;
public Spritebatch(Size clientsize, Graphics gfx)
{
cntxt.MaximumBuffer = new Size(clientsize.Width + 1, clientsize.Height + 1);
bfgfx = cntxt.Allocate(gfx, new Rectangle(Point.Empty, clientsize));
Gfx = gfx;
}
public void Begin()
{
bfgfx.Graphics.Clear(Color.Black);
}
public void Draw(Sprite s)
{
bfgfx.Graphics.DrawImageUnscaled(s.Texture, new Rectangle(s.toRec.X - s.rotationOffset.Width,s.toRec.Y - s.rotationOffset.Height,s.toRec.Width,s.toRec.Height));
}
public void drawImage(Bitmap b, Rectangle rec)
{
bfgfx.Graphics.DrawImageUnscaled(b, rec);
}
public void drawImageClipped(Bitmap b, Rectangle rec)
{
bfgfx.Graphics.DrawImageUnscaledAndClipped(b, rec);
}
public void drawRectangle(Pen p, Rectangle rec)
{
bfgfx.Graphics.DrawRectangle(p, rec);
}
public void End()
{
bfgfx.Render(Gfx);
}
}
This is a example of what I used. It's set up to mimic the Spritebatch in Xna. Drawing the images Unscaled will really increase the speed of it.Also creating one instance of the buffered graphics and Context will be faster then creating a new one every time you have to render. So I would advise you to change the line g.DrawImage(_screenImage, new Rectangle(0, 0, 900, 506)); to DrawImageUnscaled(_screenImage, new Rectangle(0, 0, 900, 506));
Edited : Example of how to scale code on sprite load
public Sprite(Bitmap texture, float x, float y, int width, int height)
{
//texture is the image you originally start with.
Bitmap b = new Bitmap(width, height);
// Create a bitmap with the desired width and height
using (Graphics g = Graphics.FromImage(b))
{
g.DrawImage(texture, 0, 0, width, height);
}
// get the graphics from the new image and draw the old image to it
//scaling it to the proper width and height
Texture = b;
//set Texture which is the final picture to the sprite.
//Uppercase Texture is different from lowercase
Scaling of the image is expensive enough, even when is done without any interpolation. To speed up the things, you should minimize memory allocations: when you create brand new Bitmap every frame, it leads to object creation and pixmap buffer allocation. This fact negates all the benefits you get from BufferedGraphics. I advise you to do the following:
Create the Bitmap instance of required size (equal to screen size) only once, outside of Render method.
Use direct access to bitmap data through LockBits method, and try to implement the scaling be hand using nearest pixel.
Of course, using some sort of hardware acceleration for scaling operation is the most preferred option (for example, in opengl all images are usually drawn using textured rectangles, and rendering such rectangles implicitly involves the process of "scaling" when texture sampling is performed).
I'm wondering why do you call this "very slow", because I did some tests and the performance doesn't seem bad. Also have you measured the performance of your rendering code into int[] '_pixels' (unfortunately you haven't provided that code) separately from the bitmap operations, because it might be the slow part.
About your concrete question. As others mentioned, using preallocated buffered graphics and bitmap objects would speed up it a bit.
But do you really need that int[] buffer? BufferedGraphics is already backed internally with a bitmap, so what really happens is:
(1) You fill the int[] buffer
(2) int[] buffer is copied to the new/preallocated Bitmap
(3) Bitmap from step 2 is copied (applying scale) to the BufferedGraphics internal bitmap (via DrawImage)
(4) BufferedGraphics internal bitmap is copied to the screen (via Render)
As you can see, there are a lot of copy operations. The intended usage of BufferedGraphics is:
(1) You fill the BufferedGraphics internal bitmap via drawing methods of the BufferedGraphics.Graphics property. If setup, the Graphics will do the scaling (as well other transformations) for you.
(2) BufferedGraphics internal bitmap is copied to the screen (via Render)
I don't know what your drawing code is doing, but if you can afford it, this definitely should provide the best performance.
Here is my quick and dirty test in case you are interested in:
using System;
using System.Diagnostics;
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Drawing.Imaging;
using System.Threading;
using System.Windows.Forms;
namespace Test
{
enum RenderMode { NewBitmap, PreallocatedBitmap, Graphics }
class Screen
{
Control canvas;
public Rectangle area;
int[,] pixels;
BitmapData info;
Bitmap bitmap;
BufferedGraphics buffer;
float scaleX, scaleY;
public RenderMode mode = RenderMode.NewBitmap;
public Screen(Control canvas, Size size)
{
this.canvas = canvas;
var bounds = canvas.DisplayRectangle;
scaleX = (float)bounds.Width / size.Width;
scaleY = (float)bounds.Height / size.Height;
area.Size = size;
info = new BitmapData { Width = size.Width, Height = size.Height, PixelFormat = PixelFormat.Format32bppRgb, Stride = size.Width * 4 };
pixels = new int[size.Height, size.Width];
bitmap = new Bitmap(size.Width, size.Height, info.PixelFormat);
buffer = BufferedGraphicsManager.Current.Allocate(canvas.CreateGraphics(), bounds);
buffer.Graphics.InterpolationMode = InterpolationMode.NearestNeighbor;
ApplyMode();
}
public void ApplyMode()
{
buffer.Graphics.ResetTransform();
if (mode == RenderMode.Graphics)
buffer.Graphics.ScaleTransform(scaleX, scaleY);
}
public void FillRectangle(Color color, Rectangle rect)
{
if (mode == RenderMode.Graphics)
{
using (var brush = new SolidBrush(color))
buffer.Graphics.FillRectangle(brush, rect);
}
else
{
rect.Intersect(area);
if (rect.IsEmpty) return;
int colorData = color.ToArgb();
var pixels = this.pixels;
for (int y = rect.Y; y < rect.Bottom; y++)
for (int x = rect.X; x < rect.Right; x++)
pixels[y, x] = colorData;
}
}
public unsafe void Render()
{
if (mode == RenderMode.NewBitmap)
{
var bounds = canvas.DisplayRectangle;
using (var buffer = BufferedGraphicsManager.Current.Allocate(canvas.CreateGraphics(), bounds))
{
Bitmap bitmap;
fixed (int* pixels = &this.pixels[0, 0])
bitmap = new Bitmap(info.Width, info.Height, info.Stride, info.PixelFormat, new IntPtr(pixels));
buffer.Graphics.InterpolationMode = InterpolationMode.NearestNeighbor;
buffer.Graphics.DrawImage(bitmap, bounds);
buffer.Render();
}
}
else
{
if (mode == RenderMode.PreallocatedBitmap)
{
fixed (int* pixels = &this.pixels[0, 0])
{
info.Scan0 = new IntPtr(pixels); info.Reserved = 0;
bitmap.LockBits(area, ImageLockMode.WriteOnly | ImageLockMode.UserInputBuffer, info.PixelFormat, info);
bitmap.UnlockBits(info);
}
buffer.Graphics.DrawImage(bitmap, canvas.DisplayRectangle);
}
buffer.Render();
}
}
}
class Game
{
[STAThread]
public static void Main()
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
var game = new Game();
game.Run();
}
Form form;
Control canvas;
Screen screen;
Level level;
Player player;
private Game()
{
form = new Form();
canvas = new Control { Parent = form, Bounds = new Rectangle(0, 0, 900, 506) };
form.ClientSize = canvas.Size;
screen = new Screen(canvas, new Size(300, 160));
level = new Level { game = this };
player = new Player { game = this };
}
private void Run()
{
bool toggleModeRequest = false;
canvas.MouseClick += (sender, e) => toggleModeRequest = true;
var worker = new Thread(() =>
{
int frameCount = 0;
Stopwatch drawT = new Stopwatch(), applyT = new Stopwatch(), advanceT = Stopwatch.StartNew(), renderT = Stopwatch.StartNew(), infoT = Stopwatch.StartNew();
while (true)
{
if (advanceT.ElapsedMilliseconds >= 3)
{
level.Advance(); player.Advance();
advanceT.Restart();
}
if (renderT.ElapsedMilliseconds >= 8)
{
frameCount++;
drawT.Start(); level.Render(); player.Render(); drawT.Stop();
applyT.Start(); screen.Render(); applyT.Stop();
renderT.Restart();
}
if (infoT.ElapsedMilliseconds >= 1000)
{
double drawS = drawT.ElapsedMilliseconds / 1000.0, applyS = applyT.ElapsedMilliseconds / 1000.0, totalS = drawS + applyS;
var info = string.Format("Render using {0} - Frames:{1:n0} FPS:{2:n0} Draw:{3:p2} Apply:{4:p2}",
screen.mode, frameCount, frameCount / totalS, drawS / totalS, applyS / totalS);
form.BeginInvoke(new Action(() => form.Text = info));
infoT.Restart();
}
if (toggleModeRequest)
{
toggleModeRequest = false;
screen.mode = (RenderMode)(((int)screen.mode + 1) % 3);
screen.ApplyMode();
frameCount = 0; drawT.Reset(); applyT.Reset();
}
}
});
worker.IsBackground = true;
worker.Start();
Application.Run(form);
}
class Level
{
public Game game;
public int pos = 0; bool right = true;
public void Advance() { Game.Advance(ref pos, ref right, 0, game.screen.area.Right - 1); }
public void Render()
{
game.screen.FillRectangle(Color.SaddleBrown, new Rectangle(0, 0, pos, game.screen.area.Height));
game.screen.FillRectangle(Color.DarkGreen, new Rectangle(pos, 0, game.screen.area.Right, game.screen.area.Height));
}
}
class Player
{
public Game game;
public int x = 0, y = 0;
public bool right = true, down = true;
public void Advance()
{
Game.Advance(ref x, ref right, game.level.pos, game.screen.area.Right - 5, 2);
Game.Advance(ref y, ref down, 0, game.screen.area.Bottom - 1, 2);
}
public void Render() { game.screen.FillRectangle(Color.Yellow, new Rectangle(x, y, 4, 4)); }
}
static void Advance(ref int pos, ref bool forward, int minPos, int maxPos, int delta = 1)
{
if (forward) { pos += delta; if (pos < minPos) pos = minPos; else if (pos > maxPos) { pos = maxPos; forward = false; } }
else { pos -= delta; if (pos > maxPos) pos = maxPos; else if (pos < minPos) { pos = minPos; forward = true; } }
}
}
}
I'm building a web based app that takes a screenshot of a play area and then posts it to a web server to be called up in a image gallery for other to view. Currently, when running in the editor I can take the screenshot and save it locally but that won't work once it's deployed. I don't know how to take that screenshot and save it to a texture (rather than to disk) to then upload to my server. How do I do this? I'm new at this and especially new at Render Texture functionality. Can someone help me sort this out?
I have found this Snippet on a forum here. But not tested by myself on WebPlayer.
using UnityEngine;
using System.Collections;
public class Main : MonoBehaviour
{
private string _data = string.Empty;
public Texture2D bg;
void OnGUI()
{
if (GUI.Button(new Rect(Screen.width*0.5f-32,32,64,32),"Save"))
StartCoroutine(ScreeAndSave());
}
IEnumerator ScreeAndSave()
{
yield return new WaitForEndOfFrame();
var newTexture = ScreenShoot(Camera.main, bg.width, bg.height);
LerpTexture(bg, ref newTexture);
_data = System.Convert.ToBase64String(newTexture.EncodeToPNG());
Application.ExternalEval("document.location.href='data:octet-stream;base64," + _data + "'");
}
private static Texture2D ScreenShoot(Camera srcCamera, int width, int height)
{
var renderTexture = new RenderTexture(width, height, 0);
var targetTexture = new Texture2D(width, height, TextureFormat.RGB24, false);
srcCamera.targetTexture = renderTexture;
srcCamera.Render();
RenderTexture.active = renderTexture;
targetTexture.ReadPixels(new Rect(0, 0, width, height), 0, 0);
targetTexture.Apply();
srcCamera.targetTexture = null;
RenderTexture.active = null;
srcCamera.ResetAspect();
return targetTexture;
}
private static void LerpTexture(Texture2D alphaTexture, ref Texture2D texture)
{
var bgColors = alphaTexture.GetPixels();
var tarCols = texture.GetPixels();
for (var i = 0; i < tarCols.Length; i++)
tarCols[i] = bgColors[i].a > 0.99f ? bgColors[i] : Color.Lerp(tarCols[i], bgColors[i], bgColors[i].a);
texture.SetPixels(tarCols);
texture.Apply();
}
}
Reference Link
I am trying to play video on Direct3D 9 device, using:
nVLC - for fetching the RGB32 frames from file
SlimDX - Actually displaying frames on video device using textures.
Here is my code to receive RGB32 frames;
_videoWrapper.SetCallback(delegate(Bitmap frame)
{
if (_mainContentSurface == null || _dead)
return;
var bmpData = frame.LockBits(new Rectangle(0, 0, frame.Width, frame.Height), ImageLockMode.ReadOnly, frame.PixelFormat);
var ptr = bmpData.Scan0;
var size = bmpData.Stride * frame.Height;
_mainContentSurface.Buffer = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(ptr, _mainContentSurface.Buffer, 0, size);
_mainContentSurface.SetTexture(_mainContentSurface.Buffer, frame.Width, frame.Height);
_secondaryContentSurface.SetTexture(_mainContentSurface.Buffer, frame.Width, frame.Height); // same buffer to second WINDOW
_mainContentSurface.VideoFrameRate.Value =_videoWrapper.ActualFrameRate;
frame.UnlockBits(bmpData);
});
And here is my actual usage of SetTexture and mapping texture to square:
public void SetTexture(byte[] image, int width, int height)
{
if (Context9 != null && Context9.Device != null)
{
if (IsFormClosed)
return;
// rendering is seperate from the "FRAME FETCH" thread, if it makes sense.
// also note that we recreate video texture if needed.
_renderWindow.BeginInvoke(new Action(() =>
{
if (_image == null || _currentVideoTextureWidth != width || _currentVideoTextureHeight != height)
{
if(_image != null)
_image.Dispose();
_image = new Texture(Context9.Device, width, height, 0, Usage.Dynamic, Format.A8R8G8B8,
Pool.Default);
_currentVideoTextureWidth = width;
_currentVideoTextureHeight = height;
if(_image == null)
throw new Exception("Video card does not support textures power of TWO or dynamic textures. Get a video card");
}
//upload data into texture.
var data = _image.LockRectangle(0, LockFlags.None);
data.Data.Write(image, 0, image.Length);
_image.UnlockRectangle(0);
}));
}
}
and finally the actual rendering:
Context9.Device.SetStreamSource(0, _videoVertices, 0, Vertex.SizeBytes);
Context9.Device.VertexFormat = Vertex.Format;
// Setup our texture. Using Textures introduces the texture stage states,
// which govern how Textures get blended together (in the case of multiple
// Textures) and lighting information.
Context9.Device.SetTexture(0, _image);
// The sampler states govern how smooth the texture is displayed.
Context9.Device.SetSamplerState(0, SamplerState.MinFilter, TextureFilter.Linear);
Context9.Device.SetSamplerState(0, SamplerState.MagFilter, TextureFilter.Linear);
Context9.Device.SetSamplerState(0, SamplerState.MipFilter, TextureFilter.Linear);
// Now drawing 2 triangles, for a quad.
Context9.Device.DrawPrimitives(PrimitiveType.TriangleList, 0, 2);
Now, it works on my machine. Without problems. With every video file and in every position. But when I checked the WinXP, picture was completely broken. Here is a screencaps for both nonworking and working;
http://www.upload.ee/image/2941734/untitled.PNG
http://www.upload.ee/image/2941762/Untitled2.png
Note that on the first picture, they are _maincontentSurface and _secondaryContentSurface. Does anyone have idea what could be the problem?
You shouldn't need to recreate your texture every time, just create it as dynamic:
this.Texture = new Texture(device, w, h, 1, Usage.Dynamic, Format.X8R8G8B8, Pool.Default);
About the copy issue could come from stride (row length might be different since it is padded):
to get Row pitch of the texture:
public int GetRowPitch()
{
if (rowpitch == -1)
{
DataRectangle dr = this.Texture.LockRectangle(0, LockFlags.Discard);
this.rowpitch = dr.Pitch;
this.Texture.UnlockRectangle(0);
}
return rowpitch;
}
If your texture row pitch is equal to your frame pitch, you can copy the way you do, otherwise you can do it this way:
public void WriteDataPitch(IntPtr ptr, int len)
{
DataRectangle dr = this.Texture.LockRectangle(0, LockFlags.Discard);
int pos = 0;
int stride = this.Width * 4;
byte* data = (byte*)ptr.ToPointer();
for (int i = 0; i < this.Height; i++)
{
dr.Data.WriteRange((IntPtr)data, this.Width * 4);
pos += dr.Pitch;
dr.Data.Position = pos;
data += stride;
}
this.Texture.UnlockRectangle(0);
}
If you want an example of fully working vlc player with slimdx let me know got that around (need to wrap it up nicely)