Alright, so I'm working on a game in MonoGame which is set in a computer operating system. As expected, it does a lot of text rendering. The in-game OS allows users to customize almost every aspect of the operating system - people have made skins for the OS that make it look like Mac OS Sierra, almost every najor Windows release since 95, Xubuntu, Ubuntu, and way more.
This game used to be written in Windows Forms however there are features I want to implement that simply are not possible in WinForms. So, we decided to move from WinForms to MonoGame, and we are faced with one huge problem.
The skin format we've made allows the user to select any font installed on their computer to use for various elements like titlebar text, main UI text, terminal text etc. This was fine in WinForms because we could use System.Drawing to render text and that allows the use of any TrueType font on the system. If it can be loaded into a System.Drawing.Font, it can be rendered.
But, MonoGame uses a different technology for rendering text on-screen. SpriteFont objects. The problem is, there seems to be no way at all to dynamically generate a SpriteFont from the same data used to generate System.Drawing.Fonts (family, size, style, etc) in code.
So, since I seemingly can't create SpriteFonts dynamically, in my graphics helper class (which deals with drawing textures etc onto the current graphics device without needing copy-pasted code everywhere), I have my own DrawString and MeasureString methods which use System.Drawing.Graphics to composite text onto a bitmap and use that bitmap as a texture to draw onto the screen.
And, here's my code for doing exactly that.
public Vector2 MeasureString(string text, System.Drawing.Font font, int wrapWidth = int.MaxValue)
{
using(var gfx = System.Drawing.Graphics.FromImage(new System.Drawing.Bitmap(1, 1)))
{
var s = gfx.SmartMeasureString(text, font, wrapWidth); //SmartMeasureString is an extension method I made for System.Drawing.Graphics which applies text rendering hints and formatting rules that I need to make text rendering and measurement accurate and usable without copy-pasting the same code.
return new Vector2((float)Math.Ceiling(s.Width), (float)Math.Ceiling(s.Height)); //Better to round up the values returned by SmartMeasureString - it's just easier math-wise to deal with whole numbers
}
}
public void DrawString(string text, int x, int y, Color color, System.Drawing.Font font, int wrapWidth = 0)
{
x += _startx;
y += _starty;
//_startx and _starty are used for making sure coordinates are relative to the clip bounds of the current context
Vector2 measure;
if (wrapWidth == 0)
measure = MeasureString(text, font);
else
measure = MeasureString(text, font, wrapWidth);
using (var bmp = new System.Drawing.Bitmap((int)measure.X, (int)measure.Y))
{
using (var gfx = System.Drawing.Graphics.FromImage(bmp))
{
var textformat = new System.Drawing.StringFormat(System.Drawing.StringFormat.GenericTypographic);
textformat.FormatFlags = System.Drawing.StringFormatFlags.MeasureTrailingSpaces;
textformat.Trimming = System.Drawing.StringTrimming.None;
textformat.FormatFlags |= System.Drawing.StringFormatFlags.NoClip; //without this, text gets cut off near the right edge of the string bounds
gfx.TextRenderingHint = System.Drawing.Text.TextRenderingHint.SingleBitPerPixel; //Anything but this and performance takes a dive.
gfx.DrawString(text, font, new System.Drawing.SolidBrush(System.Drawing.Color.FromArgb(color.A, color.R, color.G, color.B)), 0, 0, textformat);
}
var lck = bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb); //Lock the bitmap in memory and give us the ability to extract data from it so we can load it into a Texture2D
var data = new byte[Math.Abs(lck.Stride) * lck.Height]; //destination array for bitmap data, source for texture data
System.Runtime.InteropServices.Marshal.Copy(lck.Scan0, data, 0, data.Length); //cool, data's in the destination array
bmp.UnlockBits(lck); //Unlock the bits. We don't need 'em.
using (var tex2 = new Texture2D(_graphicsDevice, bmp.Width, bmp.Height))
{
for (int i = 0; i < data.Length; i += 4)
{
byte r = data[i];
byte b = data[i + 2];
data[i] = b;
data[i + 2] = r;
} //This code swaps the red and blue values of each pixel in the bitmap so that they are arranged as BGRA. If we don't do this, we get weird rendering glitches where red text is blue etc.
tex2.SetData<byte>(data); //Load the data into the texture
_spritebatch.Draw(tex2, new Rectangle(x, y, bmp.Width, bmp.Height), Color.White); //...and draw it!
}
}
}
I'm already caching heaps of textures created dynamically - window buffers for in-game programs, skin textures, etc, so those don't hit the performance hard if at all, but this text rendering code hits it hard. I have trouble even getting the game above 29 FPS!
So, is there a better way of doing text rendering without SpriteFonts, and if not, is there any way at all to create a spritefont dynamically in code simply by specifying a font family, font size and style (bold, italic, strikeout etc)?
I'd say I'm intermediate with MonoGame now but I have a hard enough time getting RenderTargets to work - so if you want to answer this question please answer it as if you were talking to a kindergarten student.
Any help would be greatly appreciated, and as this is a major hot-buttin' issue in my game's development team you may see yourself mentioned in the game's credits as a major help :P
You could create a custom spritefont using System.Drawing and use this one. It is basically every character that can be used, stored in a Dictionary with the corresponding Texture2D.
When you want to draw a text, you just draw every char next to eachother.
This is still slow (because drawing text without vector graphics is always slow) but at least you do not have to parse everything every frame.
Just specify somewhere what characters can be used and import them. Dictionaries are very fast in C# when it comes to indexing, so this shouldn't be a problem at all.
Hope this helps. Good luck.
Related
I working on a live stream App that receives JPEG image as arrays of bytes and displays it on the screen with UI.Image. It works fine but I am making optimization and have few questions. Currently, the code I have below converts arrays of bytes to Texture2D then creates a Sprite from the Texture2D then assign that Sprite to UI.Iamge to display on the screen.
Texture2D camTexture;
Image screenDisplay;
public byte[] JPEG_VIDEO_STREAM;
bool updateScreen = false;
//Initializing
JPEG_VIDEO_STREAM = new byte[20000];
camTexture = new Texture2D(Screen.width, Screen.height, TextureFormat.RGB24, false);
//Main Code that runs in the Update function
if(updateScreen){
camTexture.LoadImage(JPEG_VIDEO_STREAM);
Sprite tempSprite = Sprite.Create(camTexture, new Rect(0, 0, camTexture.width, camTexture.height), Vector2.zero, 0);
screenDisplay.sprite = tempSprite;
updateScreen = false;
}
The code above currently perform 3 steps just to display image to screen.
byte array -> Texture2D -> Sprite -> UI.Image.
but I want it to look like byte array -> Texture2D-> UI.Image.
I want to write Texture2D directly to UI.Image without creating new Sprite because I believe that Sprite.Create(camTexture, new Rect(0, 0, camTexture.width, camTexture.height), Vector2.zero, 0); allocates new memory each time Sprite.Create called. I looked at the Unity Documentation and couldn't find any other way to do this.
My questions are:
How can I assign camTexture(Texture2D) to the screen screenDisplay(UI.Image) without converting camTexture(Texture2D) to Sprite first?
Does Sprite.Create allocate new memory when called?
If there is a solution to this, is that solution better than what I currently have in terms of performance and memory management?
Note: I have no plans on using OnGUI to draw Texture2D. I want to do this with the new Unity UI. Thanks.
Edit:
With Joe's answer of RawImage, the final code looks like this:
RawImage screenDisplay;
if(updateScreen){
camTexture.LoadImage(JPEG_VIDEO_STREAM);
screenDisplay.texture = camTexture;
updateScreen = false;
}
No more Sprite needed.
I think that by specifically using a RawImage rather than Image, one can do this.
I use RawImage extensively, because, we have to "display PNGs" and it's easier.
Consider the very handy trick:
just start with a trivial gray PNG which you have imported .. and then modify that .. rather than try to build from scratch?
An interesting curiosity I found is: normally to mirror an image, you just simply scale of x or y to -1. Unless it's been fixed, Unity has a problem where this won't work for RawImage.
// currently in Unity, the ONLY way to mirror a RAW image is by fooling with
// the uvRect. changing the scale is completely broken.
if ( shouldWeMirror )
rawImage.uvRect = new Rect(1,0,-1,1); // means mirror
else
rawImage.uvRect = new Rect(0,0,1,1); // means no flip
Another interesting factor. For this reason, many Unity projects still use (even 2017) the superlative 2dToolkit. It instantly solves issues such as this.
I love SO. In 8 out of 10 questions it produces a straightforward and immediately usable answer. Or at least explains why my question is somehow wrong.
So I found it strange that I couldn't find an answer to my liking for this simple and, I had asumed, rather common question.
After searching high and low I patched together the below solution. It works alright but I wonder:
Are there flaws in it? (e.g.: do I need the dispose?)
Is there a better solution, maybe with less copying going on?
I'd like to see a soultion with CopyFromScreen , which potentially uses only a 1x1 bitmap size, but needs a Graphics.. So an alternative solution would be appreciated.
Or one that accesses the control's pixels directly.
Note 1: I want to grab the colors from a panel! Not from a picturebox and not from the screen..
Note 2: For my project speed is not important, as I want to create an eyedropper tool. But speedy ways are welcome, too; who knows where I'll go next ..
My solution:
public Color getColor(Control ctl, Point location)
{
Bitmap bmp = new Bitmap(ctl.Width, ctl.Height);
ctl.DrawToBitmap(bmp, new Rectangle(0, 0, ctl.Width, ctl.Height));
Color col = bmp.GetPixel(location.X, location.Y);
bmp.Dispose();
return col;
}
I use it like this in the colorsPanel_MouseClick event:
myPen = new Pen(getColor(colorsPanel, e.Location), myStrokeWidth);
I have come up with a version that uses CopyFromScreen like this:
public Color getScrColor(Control ctl, Point location)
{
Bitmap bmp = new Bitmap(1, 1);
Graphics g = Graphics.FromImage(bmp);
Point screenP = ctl.PointToScreen(location);
g.CopyFromScreen(screenP.X, screenP.Y, 0, 0, new Size(1, 1));
Color col = bmp.GetPixel(0, 0);
bmp.Dispose();
return col;
}
It works fine, too but seems to be a lot slower (by a factor of 10) than the one that uses DrawToBitmap. I doubt that the PointToScreen is the reason and a test version, that hands over a Graphics without creating it each time was just as slow.
So I guess the CopyFromScreen call is so much slower, that the number of pixels is not important. Well, within reasonable limits, probably. My color palette Control is about 60x400 pixels.
So for the time being I guess the original GetColor solution is OK to use.
I'm writing something to create small bitmap previews from a vector format file. From the file I have a (large) list of line coordinates. What I want to do is scale it to make it fit in a fixed-width image (96x96). I had previously been drawing the bitmap at full size and then just resizing it to 96x96 but since the thumbnails needed to be created on the fly that's turned out to not be fast enough (and it was a really dumb way to do it in the first place!). Now I just want to scale all of the coordinates as if the original size was 96x96, drop all of the points that draw on top of each other and that should greatly increase performance.
I'm an absolute newbie with any and all of the .NET Graphics/GDI stuff and the first version was pretty simple (code below). I'm wondering if there is something in the Graphics library (or elsewhere) that does that without me having to loop through all of the points and do the math on each one.
Can a Graphics/GDI guru point me in the right direction (or let me know there isn't a direction)? I'm using C#, and .NET framework target is OK.
So far it's pretty simple (tmpblocks is an array of points):
Bitmap DrawArea;
Graphics xGraph;
DrawArea = new Bitmap(64, 64);
// ^- this is GetWidth() and GetHeight() when drawing the full file at full size
xGraph = Graphics.FromImage(DrawArea);
for (int i = 0; i < tmpblocks.Count; i++)
{
if (tmpblocks[i].stitches.Length > 1)
{
Pen tempPen = new Pen(tmpblocks[i].color, 1.0f);
tempPen.StartCap = System.Drawing.Drawing2D.LineCap.Round;
tempPen.EndCap = System.Drawing.Drawing2D.LineCap.Round;
tempPen.LineJoin = System.Drawing.Drawing2D.LineJoin.Round;
xGraph.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
xGraph.DrawLines(tempPen, tmpblocks[i].stitches);
}
}
For the thumbnail I'd just scale the resulting bitmap using the .GetThumbnail method. Really slow, though (obviously)...
You can use the Graphics.ScaleTransform(float sx, float sy) to accomplish this.
You can get sx from: TargetWidth / SourceWidth
and sy from: TargetHeight / SourceHeight
where the target is defined by your target image size, and source is your source image size.
I'm currently using Brendan Tompkins ImageQuantization dll.
http://codebetter.com/blogs/brendan.tompkins/archive/2007/06/14/gif-image-color-quantizer-now-with-safe-goodness.aspx
But it doesn't run in medium trust in asp.net.
Does anyone know of a Image Quantization library that does run in medium trust?
Update
I don't care if the solution is slow. I just need something that works.
You should be able to replace the code using Marshal with explicit reading of the underlying stream via something like BinaryReader. This may be slower since you must read the stream entirely into your managed memory or seek into it rather than relying on the copy already in unmanaged memory being quickly accessible but is fundamentally your only option.
You simply cannot go spelunking into unmanaged memory from a medium trust context, even if only performing read operations.
Having looked at the linked code there's a reason you're not allowed to do this sort of thing. For starters he's ignoring the 64/32bit aspect of the IntPtr!
The underlying BitMapData class he's using is utterly predicated on having unfettered read access to arbitrary memory, this is never happening under medium trust.
A significant rewrite of his base functionality will be required to either use BitMap's directly (with the slow GetPixel calls) or read the data directly via conventional stream apis, dropping it into an array(s) and then parse it out yourself. Neither of these are likely to be pleasant. The former will be much slower (I would expect order of magnitude due to the high overhead per pixel read), the later less slow (though still slower) but has much more associated effort in terms of rewriting the low level parsing of the image data.
Here's a rough guide to what you need to change based on the current code:
from Quantizer.cs
public Bitmap Quantize(Image source)
{
// Get the size of the source image
int height = source.Height;
int width = source.Width;
// And construct a rectangle from these dimensions
Rectangle bounds = new Rectangle(0, 0, width, height);
// First off take a 32bpp copy of the image
Bitmap copy = new Bitmap(width, height, PixelFormat.Format32bppArgb);
// And construct an 8bpp version
Bitmap output = new Bitmap(width, height, PixelFormat.Format8bppIndexed);
// Now lock the bitmap into memory
using (Graphics g = Graphics.FromImage(copy))
{
g.PageUnit = GraphicsUnit.Pixel;
// Draw the source image onto the copy bitmap,
// which will effect a widening as appropriate.
g.DrawImage(source, bounds);
}
//!! BEGIN CHANGES - no locking here
//!! simply use copy not a pointer to it
//!! you could also simply write directly to a buffer then make the final immage in one go but I don't bother here
// Call the FirstPass function if not a single pass algorithm.
// For something like an octree quantizer, this will run through
// all image pixels, build a data structure, and create a palette.
if (!_singlePass)
FirstPass(copy, width, height);
// Then set the color palette on the output bitmap. I'm passing in the current palette
// as there's no way to construct a new, empty palette.
output.Palette = GetPalette(output.Palette);
// Then call the second pass which actually does the conversion
SecondPass(copy, output, width, height, bounds);
//!! END CHANGES
// Last but not least, return the output bitmap
return output;
}
//!! Completely changed, note that I assume all the code is changed to just use Color rather than Color32
protected virtual void FirstPass(Bitmap source, int width, int height)
{
// Loop through each row
for (int row = 0; row < height; row++)
{
// And loop through each column
for (int col = 0; col < width; col++)
{
InitialQuantizePixel(source.GetPixel(col, row));
} // Now I have the pixel, call the FirstPassQuantize function...
}
}
you would need to do roughly the same in the other functions.
This removes any need for Color32, the Bitmap class will deal with all that for you.
Bitmap.SetPixel() will deal with the second pass. Note that this is the easiest way to port things but absolutely not the fastest way to do it within a medium trust environment.
The function below prints a color raster image to a PCL-5 printer. The function was adapted from a 2-color (1bpp) printing function we had that worked perfectly, except for the grainy 2-color printing. The problem is that the image comes out with a large black bar extending from the right of the image to the edge of the page like this:
IMAGE#########################################
IMAGE#########AREA COMPLETELY BLACK###########
IMAGE#########################################
The image itself looks perfect, otherwise.
Various PCL-to-PDF tools don't show the image at all, which leads me to believe I've forgotten do to something. Appropriate resets (\u001bE\u001b%-12345X) were sent before, and page-feeds after.
Any PCL experts out there? I've got the PCL 5 Color Technical Reference Manual, and it's gotten me this far. This last thing is driving me crazy though.
*Edit:
I now know what command is causing the problem, but I don't know why:
stream("\u001b*r0F");
This should keep the image rotated along with the page (portrait, landscape). If I remove this, the problem goes away. I can compensate by rotating the bitmap beforehand, but I really want to know what caused this!
static void PrintImage()
{
// Get an image into memory
Image original = Image.FromFile("c:\\temp\\test.jpg");
Bitmap newBitmap = new Bitmap(original, original.Width, original.Height);
stream(String.Format("\u001b*p{0:d}x*p{1:d}Y", 1000, 1000));// Set cursor.
stream("\u001b*t300R"); // 300 DPI
stream(String.Format("\u001b*r{0:d}T", original.Height)); // Height
stream(String.Format("\u001b*r{0:d}S", original.Width)); // Width
stream("\u001b*r3U"); // 8-bit color palette
stream("\u001b*r0F"); // Follow logical page layout (landscape, portrait, etc..)
// Set palette depth, 3 bytes per pixel RGB
stream("\u001b*v6W\u0000\u0003\u0000\u0008\u0008\u0008");
stream("\u001b*r1A"); // Start raster graphics
stream("\u001b*b0M"); // Compression 0 = None, 1 = Run Length Encoding
// Not fast, but fast enough.
List<byte> colors = new List<byte>();
for (int y2 = 0; y2 < original.Height; y2++)
{
colors.Clear();
for (int x2 = 0; x2 < original.Width; x2++)
{
Color c = newBitmap.GetPixel(x2, y2);
colors.Add(c.R);
colors.Add(c.G);
colors.Add(c.B);
}
stream(String.Format("\u001b*b{0}W", colors.Count)); // Length of data to send
streamBytes(colors.ToArray()); // Binary data
}
stream("\u001b*rB"); // End raster graphics (also tried *rC -- no effect)
}
There are a few problems with your code. First off your cursor position code is incorrect, it should read:
"\u001b*p{0:d}x1:d}Y", 1000, 1000
This equates to:
<esc>*p1000x1000Y
you had:
<esc>*p1000x*p1000Y
When joining PCL commands together you match up the same parameterized value and group and then simply add value + parametrized character + value + parametrized character etc. Ensure that the final parametrized character is a capital letter which signifies the end of the PCL command.
Also when defining an image I recommend you also specify the width & hight in decipoints, this should help with the scaling of the image (*r3A) on the page so add this (just after your resolution command should be an okay place for it):
Int32 deciHeight = original.Height / (int)original.HorizontalResolution * 720;
Int32 deciWidth = original.Width / (int)original.VerticalResolution * 720;
stream("\u001b*t{0:d}h{1:d}V", deciHeight, deciWidth));
The other recommendation is to write all of this to file (watch your encodings) and use one of the handful of PCL viewers to view your data vs. always printing it. Should save you some time and a forest or two! I've tried all of them and wold recommend spending the $89 and purchasing pclWorks. They also have a complete SDK if you are going to do a lot of PCL. We don't use that as we hardcode all PCL ourselves but it does look good.
As for rotation, we've had problems on some device, You could just rotate the the jpg first (original.RotateFlip) an then write it out.
I don't have much time today but hope that my comments assist. I can test your code on Monday or Tuesday and work with it and post any further comments.
Keep in mind that even though PCL is a standard its support from manufacturer to manufacturer and device to device can be a problem and vastly different. When doing basic things most devices seem okay; however, if you get into macros or complex graphics you will find difference.