I'm working on 10 megapixel images taken by a video camera.
The aim is to register in a matrix (a two-dimensional array) the grayscale values for each pixel.
I first used GetPixel but it took 25 seconds to do it. Now I use Lockbits but it sill takes 10 seconds, and 3 if I don't save the results in a text file.
My tutor said they don't need to register the results but 3 seconds is still too slow. So am I doing something wrong in my program or is there something faster than Lockbits for my application?
Here is my code:
public void ExtractMatrix()
{
Bitmap bmpPicture = new Bitmap(nameNumber + ".bmp");
int[,] GRAY = new int[3840, 2748]; //Matrix with "grayscales" in INTeger values
unsafe
{
//create an empty bitmap the same size as original
Bitmap bmp = new Bitmap(bmpPicture.Width, bmpPicture.Height);
//lock the original bitmap in memory
BitmapData originalData = bmpPicture.LockBits(
new Rectangle(0, 0, bmpPicture.Width, bmpPicture.Height),
ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
//lock the new bitmap in memory
BitmapData newData = bmp.LockBits(
new Rectangle(0, 0, bmpPicture.Width, bmpPicture.Height),
ImageLockMode.WriteOnly, PixelFormat.Format24bppRgb);
//set the number of bytes per pixel
// here is set to 3 because I use an Image with 24bpp
int pixelSize = 3;
for (int y = 0; y < bmpPicture.Height; y++)
{
//get the data from the original image
byte* oRow = (byte*)originalData.Scan0 + (y * originalData.Stride);
//get the data from the new image
byte* nRow = (byte*)newData.Scan0 + (y * newData.Stride);
for (int x = 0; x < bmpPicture.Width; x++)
{
//create the grayscale version
byte grayScale =
(byte)((oRow[x * pixelSize] * .114) + //B
(oRow[x * pixelSize + 1] * .587) + //G
(oRow[x * pixelSize + 2] * .299)); //R
//set the new image's pixel to the grayscale version
// nRow[x * pixelSize] = grayScale; //B
// nRow[x * pixelSize + 1] = grayScale; //G
// nRow[x * pixelSize + 2] = grayScale; //R
GRAY[x, y] = (int)grayScale;
}
}
Here are some more optimizations that may help:
Use jagged arrays ([][]); in .NET, accessing them is faster than multidimensional;
Cache properties that will be used inside of a loop. Though this answer states that JIT will optimize it, we don't know what's happening internally;
Multiplication is (generally) slower than addition;
As others have stated, float is faster than double, which applies to older processors (~10+ years). The only upside here is that you're using them as constants, and thus consume less memory (especially because of the many iterations);
Bitmap bmpPicture = new Bitmap(nameNumber + ".bmp");
// jagged instead of multidimensional
int[][] GRAY = new int[3840][]; //Matrix with "grayscales" in INTeger values
for (int i = 0, icnt = GRAY.Length; i < icnt; i++)
GRAY[i] = new int[2748];
unsafe
{
//create an empty bitmap the same size as original
Bitmap bmp = new Bitmap(bmpPicture.Width, bmpPicture.Height);
//lock the original bitmap in memory
BitmapData originalData = bmpPicture.LockBits(
new Rectangle(0, 0, bmpPicture.Width, bmpPicture.Height),
ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
//lock the new bitmap in memory
BitmapData newData = bmp.LockBits(
new Rectangle(0, 0, bmpPicture.Width, bmpPicture.Height),
ImageLockMode.WriteOnly, PixelFormat.Format24bppRgb);
//set the number of bytes per pixel
// here is set to 3 because I use an Image with 24bpp
const int pixelSize = 3; // const because it doesn't change
// store Scan0 value for reuse...we don't know if BitmapData caches it internally, or recalculated it every time, or whatnot
int originalScan0 = originalData.Scan0;
int newScan0 = newData.Scan0;
// incrementing variables
int originalStride = originalData.Stride;
int newStride = newData.Stride;
// store certain properties, because accessing a variable is normally faster than a property (and we don't really know if the property recalculated anything internally)
int bmpwidth = bmpPicture.Width;
int bmpheight = bmpPicture.Height;
for (int y = 0; y < bmpheight; y++)
{
//get the data from the original image
byte* oRow = (byte*)originalScan0 + originalStride++; // by doing Variable++, you're saying "give me the value, then increment one" (Tip: DON'T add parenthesis around it!)
//get the data from the new image
byte* nRow = (byte*)newScan0 + newStride++;
int pixelPosition = 0;
for (int x = 0; x < bmpwidth; x++)
{
//create the grayscale version
byte grayScale =
(byte)((oRow[pixelPosition] * .114f) + //B
(oRow[pixelPosition + 1] * .587f) + //G
(oRow[pixelPosition + 2] * .299f)); //R
//set the new image's pixel to the grayscale version
// nRow[pixelPosition] = grayScale; //B
// nRow[pixelPosition + 1] = grayScale; //G
// nRow[pixelPosition + 2] = grayScale; //R
GRAY[x][y] = (int)grayScale;
pixelPosition += pixelSize;
}
}
Your code is converting from a row-major representation into a column-major representation.
In the bitmap, pixel (x,y) is followed by (x+1,y) in memory; but in your GRAY array, pixel (x,y) is followed by (x,y+1).
This causes inefficient memory access when writing, as every write touches a different cache line; and you end up trashing the CPU cache if the image is big enough. This is especially bad if your image size is a power of two (see Why is transposing a matrix of 512x512 much slower than transposing a matrix of 513x513?).
Store your array in row-major order as well if possible to avoid the inefficient memory access (replace GRAY[x,y] with GRAY[y,x]).
If you really need it in column-major order, look at more cache-friendly algorithms for matrix transposition (e.g. A Cache Efficient Matrix Transpose Program?)
Your code may not be optimal, but a quick skim seems to show even this version should run in a fraction of a second. This suggests there's some other problem:
Are you:
Compiling in Release mode? Debug mode turns off various optimizations
Running with a debugger attached? If you run from visual studio using F5 then (with the default C# keyshortcuts) the debugger will be attached. This can dramatically slow down your program, particularly if you have any breakpoints or intellitrace enabled.
Running on some limited device? It sounds like you're running on a PC, but if you're not, then device-specific limitations might be relevant.
I/O limited? Although you talk about a video camera, your code suggests you're dealing with the filesystem. Any file-system interaction can be a bottleneck, particularly once networked disks, virus scanners, physical platters and fragmentation come into play. A 10 mp image is 30MB (if uncompressed RGB without an alpha channel), and reading/writing that could easily take 3 seconds depending on the details of the filesystem.
I'm not sure why the second part of the inner for loop is commented out, but if you don't need that, you're doing some unnecessary casting. Removing it might improve your performance.
Also, as leppie suggested, you could use single precision floats:
for (int x = 0; x < bmpPicture.Width; x++)
{
//create the grayscale version
GRAY[x, y] =
(int)((oRow[x * pixelSize] * .114f) + //B
(oRow[x * pixelSize + 1] * .587f) + //G
(oRow[x * pixelSize + 2] * .299f)); //R
}
You can try to avoid multiplies and increment setting up a pointer with the x * pixelSize starting value and changing your code to this:
for (int x = 0; x < bmpPicture.Width; x++)
{
int *p = x * pixelSize;
GRAY[x, y]=
(int)((oRow[*p] * .114) + //B
(oRow[*p++] * .587) + //G
(oRow[*p++] * .299)); //R
}
This will speed up your code, but I'm not sure it will be significantly faster.
Note: this will speed up code only if iterating through an array of value type and will not work if oRow changes to some other type.
Here's an alternative transformation that uses only integer arithmetic, it's slightly different (due to rounding of the factors) but not anything you'd notice with the naked eye: (not tested)
byte grayScale = (byte)((
(oRow[pixelPosition] * 29) +
(oRow[pixelPosition + 1] * 151) +
(oRow[pixelPosition + 2] * 105)) >> 8);
The scale factors are approximately the old ones multiplied by 256, the shift in the end divides by 256.
Huge optimation will be achieved by using a 1D array instead of 2D array.
All other will not give you a high speedup...
Related
My goal is to be able to take a row of 70 pixels, analyze all 70 pixels for a certain kind of color, and then throw to another function if criteria are met. This needs to happen at least once every 50 milliseconds, and preferably even faster than that.
My current code looks like so
public void CaptureArea()
{
using (Bitmap capture = new Bitmap(70, 35))
{
using (Graphics g = Graphics.FromImage(capture))
{
for (int i = 0; i < 10; i++)
{
g.CopyFromScreen(copyPoint, pastePoint, new Size(70, 35));
evaluteBitmap(capture);
}
}
}
}
public void evaluteBitmap(Bitmap scanArea)
{
Rectangle rect = new Rectangle(0, 0, 70, 35);
BitmapData data = scanArea.LockBits(rect, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
IntPtr ptr = data.Scan0;
int bytes = data.Stride * scanArea.Height;
byte[] rgbValues = new byte[bytes];
byte[] r = new byte[bytes / 3];
byte[] g = new byte[bytes / 3];
byte[] b = new byte[bytes / 3];
Marshal.Copy(ptr, rgbValues, 0, bytes);
int count = 0;
int stride = data.Stride;
for (int column = 0; column < 30; column++)
{
b[count] = (byte)(rgbValues[(column * stride) + (34 * 3)]);
g[count] = (byte)(rgbValues[(column * stride) + (34 * 3) + 1]);
r[count++] = (byte)(rgbValues[(column * stride) + (34 * 3) + 2]);
}
scanArea.UnlockBits(data);
}
Just CaptureArea() can drop about 60 bitmaps into memory every second, which is fine, but it currently takes about 600 milliseconds for EvaluateBitmap() to grab a pixel and split it out into RGB values 70 times.
The end result is that a single frame of data takes well over 500 milliseconds to process when it needs to be much closer to 50 milliseconds. This kind of problem-solving is beyond me and I'm not experienced enough with code of this nature to be able to eyeball solutions and know what will end up being faster or slower and by how much.
Is there a way I can get performance quicker by an order of magnitude or am I asking the impossible?
Profiling session for the given code gives an unambiguous result:
CopyFromScreen - 40.00% exclusive samples
Bitmap..ctor - 15.00% exclusive samples
clr.dll - 11.67% exclusive samples
KernelBase.dll - 8.33% exclusive samples
GdiPlus.dll - 6.67% exclusive samples
LockBits - 6.67% exclusive samples
Image.Dispose - 5.00% exclusive samples
....
....
EvaluateBitmap - 1.67% exclusive samples
....
....
CaptureArea - 0.0% exclusive samples
The biggest portion of the time is spent on .NET methods which cannot be improved.
Conclusion: there no possible significant improvement of the code.
You can probably use multithreading to process more frames during the given time period, though.
I have three array of byte stored three colors (Red, Green, Blue), how can I display this array in picture box in c#, and type of file is bitmap file for image
byte[,] R=new byte[width, height];
byte[,] G=new byte[width, height];
byte[,] B=new byte[width, height];
these three array are not empty ,there are data stored in each array.
You mean:
Bitmap bmp = new Bitmap(width,height);
for(int i=0;i<width;i++)
for(int j=0;j<height;j++) {
SetPixel(i,j,Color.FromArgb(R[i,j],G[i,j],B[i,j]));
}
picturebox.image=bmp;
You have to build a single byte array from the data, which isn't going to be very fast since you have to interleave the data. Basically, you'd do something like this:
var bytes= new byte[width * height * 4];
for (var x = 0; x < width; x++)
for (var y = 0; y < height; y ++)
{
bytes[(x + y * width) * 4 + 1] = R[x, y];
bytes[(x + y * width) * 4 + 2] = G[x, y];
bytes[(x + y * width) * 4 + 3] = B[x, y];
}
And you can then use the byte array to create a bitmap, like this:
var bmp = new Bitmap(width, height);
var data = bmp.LockBits(new Rectangle(0, 0, width, height), System.Drawing.Imaging.ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format32bppArgb)
Marshal.Copy(bytes, 0, data.Scan0, width * height * 4);
bmp.UnlockBits(data);
Note that you should ensure that bmp.UnlockBits is always called, so you should probably put it in a finally block.
This isn't necessarily the best or fastest way, but that depends on your needs anyway :)
If you're really going for the fastest way, you'd probably use unsafe code (not because it's faster by itself, but rather because the .NET Bitmap is not natively-managed - it's a managed wrapper for an unmanaged bitmap). You'd allocate the memory for the byte array on the unmanaged heap, then you'd fill in the data and create a bitmap using the constructor that takes an IntPtr scan0 as a parameter. If done correctly, it should avoid unnecessary array boundary checks, as well as unnecessary copying.
I have small function which will recolor pixels in a Bitmap from a given color to a new given color.
The problems I have with the code are as follows:
1)
The function gives results which are remapping white pixels which should not be concidered since I have a threshold... (unless I have defined this calculation wrong)
2) When certain colors are given e.g. LimeGreen wierd results are seen in the image returned from the function (I beleive this is due to overflow of the byte type in the addition or subtraction case)
The base image I am using can be found here:
http://www.freeimagehosting.net/uploads/c8745a9de1.png
Results I have obtained can be found here:
freeimagehosting.net/uploads/fa48e5a0eb.png (Called with Color.Magenta as remapColor, Color.Red as newColor, Seems like white pixels are effected and the end of the gradient is not colored correctly)
freeimagehosting.net/uploads/8faec6a569.png (Called with Color.Magenta as remapColor, Color.Yellow as newColor, Seems like white pixels are effected and the end of the gradient is not colored correctly)
freeimagehosting.net/uploads/2efd4c04aa.png (Called with Color.Magenta as remapColor, Color.Blue as newColor, Seems like gradient not colored correctly)
freeimagehosting.net/uploads/defdf04e16.png (Called with Color.Magenta as remapColor, Color.Teal as newColor, Seems like white pixels are effected and none of the gradient is calculated correctly)
The function I have for this code is below: UPDATED per suggestions
public unsafe static Bitmap RecolorImage(Bitmap original, Color remapColor, Color newColor)
{
Bitmap result = new Bitmap(original.Width, original.Height);
//lock the original bitmap in memory
BitmapData originalData = original.LockBits(
new Rectangle(0, 0, original.Width, original.Height),
ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
//lock the new bitmap in memory
BitmapData newData = result.LockBits(
new Rectangle(0, 0, original.Width, original.Height),
ImageLockMode.WriteOnly, PixelFormat.Format32bppArgb);
//set the number of bytes per pixel
int pixelSize = 4;
int rthreshold = 128;
int gthreshold = 128;
int bthreshold = 128;
for (int y = 0; y < original.Height; y++)
{
//get the data from the original image
byte* oRow = (byte*)originalData.Scan0 + (y * originalData.Stride);
//get the data from the new image
byte* nRow = (byte*)newData.Scan0 + (y * newData.Stride);
for (int x = 0; x < original.Width; x++)
{
//examine the rgb values
byte r = (byte)((oRow[x * pixelSize]));
byte g = (byte)((oRow[x * pixelSize + 1]));
byte b = (byte)((oRow[x * pixelSize + 2]));
byte a = (byte)((oRow[x * pixelSize + 3]));
if (a > 0 &&
Math.Abs(remapColor.R - r) <= rthreshold &&
Math.Abs(remapColor.B - b) <= bthreshold &&
Math.Abs(remapColor.G - g) <= gthreshold
)
{
if (newColor.R == 0)
{
r = 0;
}
else
{
if (newColor.R > remapColor.R)
r = (byte)(r - newColor.R);
else
r = (byte)(r + newColor.R);
}
if (newColor.G == 0)
{
g = 0;
}
else
{
if (newColor.G > remapColor.G)
g = (byte)(g - newColor.G);
else
g = (byte)(g + newColor.G);
}
if (newColor.B == 0)
{
b = 0;
}
else
{
if (newColor.B > remapColor.B)
b = (byte)(b - newColor.B);
else
b = (byte)(b + newColor.B);
}
}
//set the new image's pixel remaped pixel color
nRow[x * pixelSize] = b; //B
nRow[x * pixelSize + 1] = g; //G
nRow[x * pixelSize + 2] = r; //R
nRow[x * pixelSize + 3] = a; //A
}
}
original.UnlockBits(originalData);
result.UnlockBits(newData);
return result;
}
What gives....
Is what I am trying to do possible?
Is it reliable?
Is there just a bug in my code?
Is there a better way to achive this "re-mapable technique" on bitmaps using gradients?
Thank you for your time.
It looks like your threshold test is incorrect. Take the line:
remapColor.R - r <= rthreshold
If the current pixel is white, then r will be 255, and the test will always be true, no matter what remapColor.R and rthreshold are.
I think Math.Abs(remapColor.R - r) might work.
And you're likely correct about your byte values being out of bounds. Fixing the threshold test might stop that from happening. Otherwise, try putting some bounds checking in to see where it's happening.
I have decided that although this may be possible if I study the various materials regarding color spaces and their supporting theories. It seems that this will take a bit more than some quick threshold calculation and normalization to the remapColor.
I am going to propose that instead of performing this type of modification on a raster bitmap image that the graphics be modified in their vector form.
The process should be something like this:
The graphics are created in whatever imaging suite the designer is working in.
They are saved to a vector format e.g. SVG this will allow the customizable paths to be named, traversed and altered programmatically (and for more than color if needed) with SVG Rendering Engine(http://svg.codeplex.com/)
With this solution we can either output the SVG direct to the browser if supported and do the modifications directly on the client or use the server and output as PNG when needed.
I feel that this arrangement will provide us with more flexibility and a more robust solution than what I was initially going to hack together.
Thank you guys for your time!
Is there an efficient way of adjusting the contrast of an image in C#?
I've seen this article which advocates doing a per-pixel operation. Not quick.
I'm using colour matrices in places already and find them to be quick. Is there a way to adjust contrast using them? (Note: This guy gets it wrong.)
I'm also using EmguCV. I notice that OpenCV (which Emgu wraps) seems to have a contrast function - is there any way of accessing this through Emgu? At the moment all I can do in Emgu is normalise the histogram, which does change the contrast, but not with any degree of control on my part.
Anyone got any ideas?
If the code in that sample works for you, you can speed it up massively (by orders of magnitude) by using Bitmap.LockBits, which returns a BitmapData object that allows access to the Bitmap's pixel data via pointers. There are numerous samples on the web and on StackOverflow that show how to use LockBits.
Bitmap.SetPixel() and Bitmap.GetPixel() are the slowest methods known to mankind, and they both utilize the Color class, which is the slowest class known to mankind. They should have been named Bitmap.GetPixelAndByGodYoullBeSorryYouDid() and Bitmap.SetPixelWhileGettingCoffee as a warning to unwary developers.
Update: If you're going to modify the code in that sample, note that this chunk:
System.Drawing.Bitmap TempBitmap = Image;
System.Drawing.Bitmap NewBitmap = new System.Drawing.Bitmap(TempBitmap.Width,
TempBitmap.Height);
System.Drawing.Graphics NewGraphics =
System.Drawing.Graphics.FromImage(NewBitmap);
NewGraphics.DrawImage(TempBitmap, new System.Drawing.Rectangle(0, 0,
TempBitmap.Width, TempBitmap.Height),
new System.Drawing.Rectangle(0, 0, TempBitmap.Width, TempBitmap.Height),
System.Drawing.GraphicsUnit.Pixel);
NewGraphics.Dispose();
can be replaced with this:
Bitmap NewBitmap = (Bitmap)Image.Clone();
Update 2: Here is the LockBits version of the AdjustContrast method (with a few other speed improvements):
public static Bitmap AdjustContrast(Bitmap Image, float Value)
{
Value = (100.0f + Value) / 100.0f;
Value *= Value;
Bitmap NewBitmap = (Bitmap)Image.Clone();
BitmapData data = NewBitmap.LockBits(
new Rectangle(0, 0, NewBitmap.Width, NewBitmap.Height),
ImageLockMode.ReadWrite,
NewBitmap.PixelFormat);
int Height = NewBitmap.Height;
int Width = NewBitmap.Width;
unsafe
{
for (int y = 0; y < Height; ++y)
{
byte* row = (byte*)data.Scan0 + (y * data.Stride);
int columnOffset = 0;
for (int x = 0; x < Width; ++x)
{
byte B = row[columnOffset];
byte G = row[columnOffset + 1];
byte R = row[columnOffset + 2];
float Red = R / 255.0f;
float Green = G / 255.0f;
float Blue = B / 255.0f;
Red = (((Red - 0.5f) * Value) + 0.5f) * 255.0f;
Green = (((Green - 0.5f) * Value) + 0.5f) * 255.0f;
Blue = (((Blue - 0.5f) * Value) + 0.5f) * 255.0f;
int iR = (int)Red;
iR = iR > 255 ? 255 : iR;
iR = iR < 0 ? 0 : iR;
int iG = (int)Green;
iG = iG > 255 ? 255 : iG;
iG = iG < 0 ? 0 : iG;
int iB = (int)Blue;
iB = iB > 255 ? 255 : iB;
iB = iB < 0 ? 0 : iB;
row[columnOffset] = (byte)iB;
row[columnOffset + 1] = (byte)iG;
row[columnOffset + 2] = (byte)iR;
columnOffset += 4;
}
}
}
NewBitmap.UnlockBits(data);
return NewBitmap;
}
NOTE: this code requires using System.Drawing.Imaging; in your class' using statements, and it requires that the project's allow unsafe code option be checked (on the Build Properties tab for the project).
One of the reasons GetPixel and SetPixel are so slow for pixel-by-pixel operations is that the overhead of the method call itself starts to become a huge factor. Normally, my code sample here would be considered a candidate for refactoring, since you could write your own SetPixel and GetPixel methods that use an existing BitmapData object, but the processing time for the math inside the functions would be very small relative to the method overhead of each call. This is why I removed the Clamp calls in the original method as well.
One other way to speed this up would be to simply make it a "destructive" function, and modify the passed Bitmap parameter instead of making a copy and returning the modified copy.
#MusiGenesis,
Just wanted to note that I used this method for an image editor I've been writing. It works well, but sometimes this method triggers an AccessViolationException on this line:
byte B = row[columnOffset];
I realised it was because there was no standardisation of BitDepth, so if an image was 32 bit colour I was getting this error. So I changed this line:
BitmapData data = NewBitmap.LockBits(new Rectangle(0, 0, NewBitmap.Width, NewBitmap.Height), ImageLockMode.ReadWrite, NewBitmap.PixelFormat);
to:
BitmapData data = NewBitmap.LockBits(new Rectangle(0, 0, NewBitmap.Width, NewBitmap.Height), ImageLockMode.ReadWrite, PixelFormat.Format32bppRgb);
Hope this helps as it seems to have eradicated my problem.
Thanks for the post.
Jib
I'm a bit late, but use a color matrix implementation as these will be optimised for such transformations and is much easier than manipulating the pixels yourself: http://www.geekpedia.com/tutorial202_Using-the-ColorMatrix-in-Csharp.html
I am working on constructing and saving a bitmap, and i have a loop that sets the pixels in the bitmap to their proper values. However it crashes after a short period of ime with an IndexOutOfRange exception at the noted point in the code.
//data is an array of bytes of size (image width * image height) * 2;
Bitmap b = new Bitmap(width, height, PixelFormat.Format32bppArgb);
for (int i = 0; i < data.Length; i += 2)
{
int luminance = ((int)data[i] << 8) | (int)data[i + 1];
Color c = Color.FromArgb(luminance,luminance,luminance,luminance);
int x = i / 2;
int y = x / width;
x %= width;
b.SetPixel(x, y, c);//crashes here when Y is at 513, should only go to 512
}
b.Save(Path.GetFileNameWithoutExtension(fileName) + ".bmp");
I'm stumped as to why this happens.Why does this happen and how can i fix it?
(a note ot all of those that reommend unsafe code: I am going for a working program then a fast one. I'll be sure to write up 3 questions on the subject when i start! ;) )
When Length is odd, then at some point i+1 == Length will be true.
for (int i = 0; i < data.Length; i += 2)
{
int luminance = ((int)data[i] << 8) | (int)data[i + 1];
int x = (i + 1) / 2;
}
I would suggest replacing
//data is an array of bytes of size (image width * image height) * 2;
with
System.Diagnostics.Debug.Assert(data.Length == width * height * 2);
System.Diagnostics.Debug.Assert((data.Length % 2) == 0);
It's hard to tell what might be wrong without knowing what your data actually is. I suspect that it might be organised into rows like a bitmap, but sometimes bitmap format data requires that rows be a multiple of 4 bytes in length (with unused padding at the end, see BMP file format). If this is the case, your y value might become larger than you expect. You may need to take such padding into account.