I'm trying to get an image in skiasharp that's left rotated by 90 degrees to be centered and fit perfectly on the canvas. I've tried 2 ways. My own custom way, and another one that seems like a popular solution but maybe I'm not understanding how it works correctly?
My own way.
Here is the code:
SKSurface surf = e.Surface;
SKCanvas canvas = surf.Canvas;
SKSize size = canvasView.CanvasSize;
canvas.Clear();
SKRect rect = SKRect.Create(0.0f, 0.0f, size.Height, size.Width);
canvas.RotateDegrees(85);
canvas.DrawBitmap(m_bm, rect);
"m_bm" is a bitmap that was retrieved in a separate function. That function is:
// Let user take a picture.
var result = await MediaPicker.CapturePhotoAsync(new MediaPickerOptions
{
Title = "Take a picture"
});
// Save stream.
var stream = await result.OpenReadAsync();
// Create the bitmap.
m_bm = SKBitmap.Decode(stream);
// Set to true because the image will be prepared soon.
m_displayedImage = true;
I only put 85 instead of 90 because I wanted to visually see it getting closer but when I do that, it goes off screen. I'm coming from a game programming background so this is normally solved with getting the width of whatever we're working with (like the player in the game) and adding that to the x position, and boom. But with Xamarin, didn't work. That's my own attempt. Then I hit the internet of course to find help, and a different implementation was given to me.
Popular solution.
See here for this popular solution and it's the FIRST answer to this users question. The code I used is SLIGHTLY different because I didn't see the point in returning an image in that users function. Here it is:
// Save stream.
var stream = await result.OpenReadAsync();
using (var bitmap = SKBitmap.Decode(stream))
{
var rotated = new SKBitmap(bitmap.Height, bitmap.Width);
using (var surface = new SKCanvas(rotated))
{
surface.Clear();
surface.Translate(rotated.Height, rotated.Width);
surface.RotateDegrees(90);
surface.DrawBitmap(bitmap, 0, 0);
}
}
I'm drawing the bitmap with the canvas and I thought that would work because testing it in other code samples it did exactly that, so I definitely am not rotating properly or something?
The link that #Cheesebaron gave me in the reply to the original post ended up working out. But a new issue arises but I'll google that myself. Here's my own code:
namespace BugApp
{
public partial class MainPage : ContentPage
{
// Save bitmaps for later use.
static SKBitmap m_bm;
static SKBitmap m_editedBm;
// Boolean for displaying the image captured with the camera.
bool m_displayedImage;
public MainPage()
{
// Set to explicit default values to always be in control of the assignments.
m_bm = null;
m_editedBm = null;
// No picture has been taken yet.
m_displayedImage = false;
InitializeComponent();
}
// Assigned to the button in the xaml page.
private async void SnapPicture(Object sender, System.EventArgs e)
{
// Let user take a picture.
var result = await MediaPicker.CapturePhotoAsync(new MediaPickerOptions
{
Title = "Take a picture"
});
// Save stream.
var stream = await result.OpenReadAsync();
// Create the bitmap.
m_bm = SKBitmap.Decode(stream);
// Get the rotated image.
m_editedBm = Rotate();
// Set to true because the image will be prepared soon.
m_displayedImage = true;
}
public static SKBitmap Rotate()
{
using (var bitmap = m_bm)
{
var rotated = new SKBitmap(bitmap.Height, bitmap.Width);
using (var surface = new SKCanvas(rotated))
{
surface.Translate(bitmap.Width, 0);
surface.RotateDegrees(90);
surface.DrawBitmap(bitmap, 0, 0);
}
return rotated;
}
}
private void OnCanvasViewPaintSurface(object sender, SKPaintSurfaceEventArgs e)
{
if(m_bm != null && m_displayedImage == true)
{
e.Surface.Canvas.Clear();
// Draw in a new rect space.
e.Surface.Canvas.DrawBitmap(m_editedBm, new SKRect(0.0f, 0.0f, 300.0f, 300.0f));
// ---Testing.
// e.Surface.Canvas.DrawBitmap(m_editedBm, new SKRect(112, 238, 184, 310), new SKRect(0, 0, 9, 9));
// Avoid having this function launch again for now.
m_displayedImage = false;
}
}
}
}
The main portion of code that matters is the rotate function which was this one here: Link.
Thanks to everyone that replied.
Related
I have a MVC C# application that includes a .Net wrapper for tesseract-ocr nuget. The current version I am using is v4.1.0-beta1. The image that I am try to scan is shown below
My aim is to extract the player name and the number just above them to the left.
I tried making the OCR scan the field/pitch area but the results are way off base. So, I decided to section off all player names and all numbers as seen in the image below. Ratings area marked in blue and player names marked in red. As you can see the name and rating are always the same distance apart.
My current code setup is shown below.
public void Get(HttpPostedFileBase file)
{
using (var engine = new TesseractEngine(Path.Combine(HttpRuntime.AppDomainAppPath, "tessdata"), "eng+deu", EngineMode.Default))
{
var bitmap = (Bitmap)Image.FromStream(file.InputStream, true, true);
using (var img = PixConverter.ToPix(bitmap))
{
SetPlayerRatings(engine, img);
}
}
}
private void SetPlayerRatings(TesseractEngine engine, Pix img)
{
var width = 285;
var height = 76;
var textPositions = Service.Get<Formation>(this.FormationId).TextPositions.ToList();
foreach (var textPosition in textPositions)
{
var playerRating = GetPlayerData(engine, img, new Rect(textPosition.X, textPosition.Y, width, height));
}
}
private static PlayerRating GetPlayerData(TesseractEngine engine, Pix img, Rect region)
{
using (var page = engine.Process(img, region, PageSegMode.Auto))
{
var playerName = page.GetText();
}
var ratingRegion = new Rect(region.X1, region.Y1 - 52, 80, 50);
using (var page = engine.Process(img, ratingRegion, PageSegMode.Auto))
{
var playerRating = page.GetText();
}
}
This code is producing the correct results for the 1st image.
Is there any way to train OCR so that I dont have to workout the X and Y co-ordinates for each player position? I would like to just specify the area of the pitch and have OCR read in the rating followed by the player name.
With specifying coordinates you solved several problems regarding image processing. So if you do not want to specify coordinates, you have to deal with them: e.g. removing graphics component from OCR area like T-shirt, lines.
Next idea: Tesseract API has option GetComponentImages (I expect C# wrapper should provide it too - I am not familiar with C#), so you can iterate over found components.
I'm using SFML for C#. I want to create a BackgroundImage Sprite and then start drawing it with an Agent, represented as a Circle, on top of it like that:
static void Main(string[] args)
{
Window = new RenderWindow(new VideoMode((uint)map.Size.X * 30, (uint)map.Size.Y * 30), map.Name + " - MAZE", Styles.Default);
while (Window.IsOpen)
{
Update();
}
}
static public RenderWindow Window { get; private set; }
static Map map = new Map(string.Format(#"C:\Users\{0}\Desktop\Maze.png", Environment.UserName));
static public void Update()
{
Window.Clear(Color.Blue);
DrawBackground();
DrawAgent();
Window.Display();
}
static void DrawAgent()
{
using (CircleShape tempCircle = new CircleShape
{
FillColor = Color.Cyan,
Radius = 15,
Position = new Vector2f(30, 30),
Origin = new Vector2f(30, 30),
Scale = new Vector2f(.5f, .5f)
})
{
Window.Draw(tempCircle);
}
}
static private Sprite BackgroundImage { get; set; }
static void DrawBackground()
{
if (BackgroundImage == null)
BackgroundImage = GetBackground();
Window.Draw(BackgroundImage);
}
static Sprite GetBackground()
{
RenderTexture render = new RenderTexture((uint)map.Size.X * 30, (uint)map.Size.Y * 30);
foreach (var point in map.Grid.Points)
{
RectangleShape pointShape = new RectangleShape(new Vector2f(30, 30));
switch (point.PointType)
{
case PointType.Walkable:
pointShape.FillColor = Color.White;
break;
case PointType.NotWalkable:
pointShape.FillColor = Color.Black;
break;
case PointType.Start:
pointShape.FillColor = Color.Red;
break;
case PointType.Exit:
pointShape.FillColor = Color.Blue;
break;
}
pointShape.Position = new Vector2f(point.Position.X * 30, point.Position.Y * 30);
render.Draw(pointShape);
}
Sprite result = new Sprite(render.Texture);
result.Origin = new Vector2f(0, result.GetLocalBounds().Height);
result.Scale = new Vector2f(1, -1);
return result;
}
Everything works as intended when I start it, but after a few seconds, around the time when process memory reaches 70MB, BackgroundImage turns into completely white sprite. If I change the type of BackgroundImage and GetBackground() to RenderTexture, return "render" object and then change DrawBackground() function like this
void RenderBackground()
{
if (BackgroundImage == null)
BackgroundImage = GetBackground();
using (Sprite result = new Sprite(BackgroundImage.Texture))
{
result.Origin = new Vector2f(0, result.GetLocalBounds().Height);
result.Scale = new Vector2f(1, -1);
Window.Draw(result);
}
}
then the background sprite doesn't turn white, but storing entire RenderTexture, instead of Sprite and then constantly creating new Sprite objects every time we call RenderBackground() function seems like a bad idea.
Is there any way for GetBackground() function to return a Sprite which won't turn white once the function's local "render" variable is destroyed?
You're not completely off with your assumptions. Simplified, SFML knows two types of resources:
Light resources are small objects that are quick to create and destroy. It's not that bad to just drop them and recreate them later. Typical examples would be Sprite, Sound, Text, and basically most SFML classes.
Heavy resourcces are often big objects or objects requiring file access to create or use. Typical examples would be Image, Texture, SoundBuffer, and Font. You shouldn't recreate these and instead keep them alive while you need them. If they're disposed too early, light resources using them will fail in some way or another.
A sprite's texture turning white is – as you've discovered – a typical sign of the assigned texture being freed/disposed.
There are many different approaches to this, but I'd suggest you create some kind of simple resource manager that will load resources just in time or just return them, if they're loaded already.
I haven't used SFML with C# and I haven't really touched C# for quite a while, but for a simple implementation you'd just have a Dictionary<string, Texture>. When you want to load a texture file like texture.png, you look whether there's a dictionary entry with that key name. If there is, just return it. If there isn't, create the new entry and load the texture, then return it.
I'm out of practice, so please consider this pseudo code!
private Dictionary<string, Texture> mTextureCache; // initialized in constructor
public Texture getTexture(file) {
Texture tex;
if (mTextureCache.TryGetValue(file, out tex))
return tex;
tex = new Texture(file);
mTextureCache.add(file, tex);
return tex;
}
// Somewhere else in your code:
Sprite character = new Sprite(getTexture("myCharacter.png"));
If your heavy resource is a RenderTexture, you just have to ensure that it stays alive as long as it's used (e.g. as a separate member).
It turned out the answer was simpler that I expected. All I had to to was create new Texture object and then make a Sprite out of it. So instead of
Sprite result = new Sprite(render.Texture);
I wrote
Sprite result = new Sprite(new Texture(render.Texture));
Now garbage collector doesn't dispose Sprite's texture
I've did a lot of research, but I can't find a suitable solution that works with Unity3d/c#. I'm using a Fove-HMD and would like to record/make a video of the integrated camera. So far I managed every update to take a snapshot of the camera, but I can't find a way to merge this snapshots into a video. Does someone know a way of converting them? Or can someone point me in the right direction, in which I could continue my research?
public class FoveCamera : SingletonBase<FoveCamera>{
private bool camAvailable;
private WebCamTexture foveCamera;
private List<Texture2D> snapshots;
void Start ()
{
//-------------just checking if webcam is available
WebCamDevice[] devices = WebCamTexture.devices;
if (devices.Length == 0)
{
Debug.LogError("FoveCamera could not be found.");
camAvailable = false;
return;
}
foreach (WebCamDevice device in devices)
{
if (device.name.Equals("FOVE Eyes"))
foveCamera = new WebCamTexture(device.name);//screen.width and screen.height
}
if (foveCamera == null)
{
Debug.LogError("FoveCamera could not be found.");
return;
}
//-------------camera found, start with the video
foveCamera.Play();
camAvailable = true;
}
void Update () {
if (!camAvailable)
{
return;
}
//loading snap from camera
Texture2D snap = new Texture2D(foveCamera.width,foveCamera.height);
snap.SetPixels(foveCamera.GetPixels());
snapshots.Add(snap);
}
}
The code works so far. The first part of the Start-Method is just for finding and enabling the camera. In the Update-Method I'm taking every update a snapshot of the video.
After I "stop" the Update-Method, I would like to convert the gathered Texture2D object into a video.
Thanks in advance
Create MediaEncoder
using UnityEditor; // VideoBitrateMode
using UnityEditor.Media; // MediaEncoder
var vidAttr = new VideoTrackAttributes
{
bitRateMode = VideoBitrateMode.Medium,
frameRate = new MediaRational(25),
width = 320,
height = 240,
includeAlpha = false
};
var audAttr = new AudioTrackAttributes
{
sampleRate = new MediaRational(48000),
channelCount = 2
};
var enc = new MediaEncoder("sample.mp4", vidAttr, audAttr);
Convert each snapshot to Texture2D
Call consequently AddFrame to add each snapshot to MediaEncoder
enc.AddFrame(tex);
Once done call Dispose to close the file
enc.Dispose();
I see two methods here, one is fast to implement, dirty and not for all platforms, second one harder but pretty. Both rely on FFMPEG.
1) Save every frame into image file (snap.EncodeToPNG()) and then call FFMPEG to create video from images (FFmpeg create video from images) - slow due to many disk operations.
2) Use FFMPEG via wrapper implemented in AForge and supply its VideoFileWriter class with images that you have.
Image sequence to video stream?
Problem here is it uses System.Bitmap, so in order to convert Texture2D to Bitmap you can use: How to create bitmap from byte array?
So you end up with something like:
Bitmap bmp;
Texture2D snap;
using (var ms = new MemoryStream(snap.EncodeToPNG()))
{
bmp = new Bitmap(ms);
}
vFWriter.WriteVideoFrame(bmp);
Both methods are not the fastest ones though, so if performance is an issue here you might want to operate on lower level data like DirectX or OpenGL textures.
I am setting the NavBar's background with this code which works great in Retina and non-Retina displays. There is a #2x and normal image. So, all good:
UINavigationBar.Appearance.SetBackgroundImage(
GetImage(ImageTheme.menubar), UIBarMetrics.Default);
Now, when I apply this ChangeHue() transformation to the image to adjust its hue, on Retina displays the image is twice the size. Non-Retina displays are fine:
UINavigationBar.Appearance.SetBackgroundImage(
ChangeHue(GetImage(ImageTheme.menubar)), UIBarMetrics.Default);
...
UIImage ChangeHue(UIImage originalImage){
var hueAdjust = new CIHueAdjust() {
Image = CIImage.FromCGImage(originalImage.CGImage),
Angle = hue * (float)Math.PI / 180f // angles to radians
};
var output = hueAdjust.OutputImage;
var context = CIContext.FromOptions(null);
var cgimage = context.CreateCGImage(output, output.Extent);
var i = UIImage.FromImage(cgimage);
return i;
}
Here is the result in Non-Retina and Retina displays after the Hue is applied:
Ignore those HACKs, and edit this line in your ChangeHue method:
var i = UIImage.FromImage(cgimage);
to do this instead:
float scale = 1f;
if (UIScreen.MainScreen.RespondsToSelector (new MonoTouch.ObjCRuntime.Selector ("scale"))) {
scale = UIScreen.MainScreen.Scale; // will be 2.0 for Retina
}
var i = new UIImage(cgimage, scale, UIImageOrientation.Up);
This should return a UIImage object that is has the correct 'scale' information to be properly displayed in the UINavigationBar.
I never tried for CoreImage but, for CoreGraphics, you need to use UIGraphics.BeginImageContextWithOptions and specify 0 for scaling (so it will be done automagically for both Retina and non-Retina displays).
So the first thing I would try is to replace your:
var context = CIContext.FromOptions(null);
with the following block:
UIGraphics.BeginImageContextWithOptions (new SizeF (size, size), false, 0);
using (var c = UIGraphics.GetCurrentContext ()) {
var context = CIContext.FromContext (c);
...
}
UIGraphics.EndImageContext ();
UPDATE: FromContext is not available in iOS (it's OSX specific) so the above code won't work.
Filed a bug with the MonoTouch team. Will post solution shortly.
I've got three 'hacks' to suggest for now:
Hack #1
To force the extent to trim the image by replacing this line:
var cgimage = context.CreateCGImage(output, output.Extent);
with this:
var extent = output.Extent;
if (UIScreen.MainScreen.RespondsToSelector (new MonoTouch.ObjCRuntime.Selector("scale"))) {
if (UIScreen.MainScreen.Scale == 2f) {
extent = new System.Drawing.RectangleF(extent.X, extent.Y, extent.Width / 2f, extent.Height / 2f);
}
}
var cgimage = context.CreateCGImage(output, extent);
HOWEVER you 'lose' the Retina resolution on the adjusted image (it uses the #2x image as the source, but only displays the bottom-left quadrant of it after applying the filter, thanks to the image origin starting at the bottom-left).
Hack #2
Along the same lines, you can scale the image returned from the ChangeHue method so that it doesn't expand beyond the navigation bar:
var hued = ChangeHue (navBarImage);
if (hued.RespondsToSelector(new MonoTouch.ObjCRuntime.Selector("scale")))
hued = hued.Scale (new System.Drawing.SizeF(320, 47));
UINavigationBar.Appearance.SetBackgroundImage (hued, UIBarMetrics.Default);
UNFORTUNATELY you 'lose' the Retina resolution again, but at least the image is displayed correctly (just downsampled to 320 wide).
Hack #3
You could save the filtered image to disk and then set the UIAppearance using the image file on 'disk'. The code would look like this:
bool retina = false;
if (UIScreen.MainScreen.RespondsToSelector (new MonoTouch.ObjCRuntime.Selector ("scale"))) {
if (UIScreen.MainScreen.Scale == 2f) {
retina = true;
}
}
if (retina) {
NSError err; // unitialized
UIImage img = ChangeHue (navBarImage);
img.AsPNG ().Save ("tempNavBar#2x.png", true, out err);
if (err != null && err.Code != 0) {
// error handling
}
UINavigationBar.Appearance.SetBackgroundImage (UIImage.FromFile ("tempNavBar.png"), UIBarMetrics.Default);
} else {
UINavigationBar.Appearance.SetBackgroundImage (ChangeHue (navBarImage), UIBarMetrics.Default);
}
The benefit of this final hack is that the image looks correct (ie. Retina resolution is preserved).
I'm still looking for the "perfect" solution, but at least these ideas 'fix' your problem one-way-or-another...
I will like to get a screen capture and save it in the format of png of the entire screen. How can I do that?
Could I use the Snipping Tool library to accomplish this? There are some tutorials on the internet that show you how to do this with windows forms and the image is in the format of bitmap.
Here's a little method to capture the contents of any screen.
private static void CaptureScreen(Screen window, string file)
{
try
{
Rectangle s_rect = window.Bounds;
using (Bitmap bmp = new Bitmap(s_rect.Width, s_rect.Height))
{
using (Graphics gScreen = Graphics.FromImage(bmp))
gScreen.CopyFromScreen(s_rect.Location, Point.Empty, s_rect.Size);
bmp.Save(file, System.Drawing.Imaging.ImageFormat.Png);
}
}
catch (Exception) { /*TODO: Any exception handling.*/ }
}
Example of usage:
CaptureScreen(Screen.PrimaryScreen, #"B:\exampleScreenshot.png");
EDIT: Coming back to this later I realized it's probably more useful to return an Image object from the function so you can choose how to use the captured bitmap.
I've also made the function a bit more robust now so that it can capture multiple screens (i.e. in a multi-monitor setup). It should accommodate screens of varying heights, but I can't test this myself.
public static Image CaptureScreens(params Screen[] screens) {
if (screens == null || screens.Length == 0)
throw new ArgumentNullException("screens");
// Order them in logical left-to-right fashion.
var orderedScreens = screens.OrderBy(s => s.Bounds.Left).ToList();
// Calculate the total width needed to fit all the screen into a single image
var totalWidth = orderedScreens.Sum(s => s.Bounds.Width);
// In order to handle screens of different sizes, make sure to make the Bitmap large enough to fit the tallest screen
var maxHeight = orderedScreens.Max(s => s.Bounds.Top + s.Bounds.Height);
var bmp = new Bitmap(totalWidth, maxHeight);
int offset = 0;
// Copy each screen to the bitmap
using (var g = Graphics.FromImage(bmp)) {
foreach (var screen in orderedScreens) {
g.CopyFromScreen(screen.Bounds.Left, screen.Bounds.Top, offset, screen.Bounds.Top, screen.Bounds.Size);
offset += screen.Bounds.Width;
}
}
return bmp;
}
New example:
// Capture all monitors and save them to file
CaptureScreens(Screen.AllScreens).Save(#"C:\Users\FooBar\screens.png");