We Keep Coding

DirectX 11 render BGRA32 Frame

First time trying to render something and I have big troubles... I am using DirectN library and SwapChainSurface class from KlearTouch.MediaPlayer. I am trying to render BGRA32 frame using D3D11Device.
For this I have slightly modified OnNewSurfaceAvailable:
public void OnNewSurfaceAvailable2(Action<ID3D11Device, ID3D11DeviceContext> updateSurface)
{
if (rendering)
{
return;
}
try
{
if (this.swapChain is null || swapChainComObject is null)
{
return;
}
swapChainComObject.GetDesc(out var swapChainDesc).ThrowOnError();
if (swapChainDesc.BufferDesc.Width != PanelWidth || swapChainDesc.BufferDesc.Height != PanelHeight)
{
swapChainComObject.ResizeBuffers(2, PanelWidth, PanelHeight, DXGI_FORMAT.DXGI_FORMAT_UNKNOWN, 0).ThrowOnError();
}
var device = swapChain.Object.GetDevice1().Object.As<ID3D11Device>();
device.GetImmediateContext(out var context);
// context.ClearRenderTargetView(renderTargetView.Object, new []{0f, 1f, 1f, 1f});
updateSurface(device, context);
swapChainComObject.Present(1, 0).ThrowOnError();
}
catch (ObjectDisposedException)
{
Reinitialize();
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("\nException: " + ex, nameof(SwapChainSurface) + '.' + nameof(OnNewSurfaceAvailable));
}
rendering = false;
}
OnSurfaceAvailable2 is called from:
void VideoFrameArrived(Bgra32VideoFrame frame)
{
DispatcherQueue.TryEnqueue(() =>
{
previewSurface.OnNewSurfaceAvailable2((device, context) =>
{
var size = frame.m_height * frame.m_height * 4;
D3D11_TEXTURE2D_DESC td;
td.ArraySize = 1;
td.BindFlags = (uint) D3D11_BIND_FLAG.D3D11_BIND_SHADER_RESOURCE;
td.Usage = D3D11_USAGE.D3D11_USAGE_DYNAMIC;
td.CPUAccessFlags = (uint) D3D11_CPU_ACCESS_FLAG.D3D11_CPU_ACCESS_WRITE;
td.Format = DXGI_FORMAT.DXGI_FORMAT_B8G8R8A8_UNORM;
td.Height = (uint) frame.m_height;
td.Width = (uint) frame.m_width;
td.MipLevels = 1;
td.MiscFlags = 0;
td.SampleDesc.Count = 1;
td.SampleDesc.Quality = 0;
D3D11_SUBRESOURCE_DATA srd;
srd.pSysMem = frame.m_pixelBuffer;
srd.SysMemPitch = (uint) frame.m_height;
srd.SysMemSlicePitch = 0;
var texture = device.CreateTexture2D<ID3D11Texture2D>(td, new []{srd});
var mappedResource = context.Map(texture.Object, 0, D3D11_MAP.D3D11_MAP_WRITE_DISCARD);
var mappedData = mappedResource.pData;
unsafe
{
Buffer.MemoryCopy(frame.m_pixelBuffer.ToPointer(), mappedData.ToPointer(), size, size);
}
// Just for debug
var pixelsInFrame = new byte[size];
var pixelsInResource = new byte[size];
Marshal.Copy(frame.m_pixelBuffer, pixelsInFrame, 0, size);
Marshal.Copy(mappedResource.pData, pixelsInResource, 0, size);
context.Unmap(texture.Object, 0);
});
});
}
Problem is that I can't see anything rendered and surface stay black and I assume it should not be.
Update: Project repository
Update 2:
I solved my issue. I had too little knowledge about DX11 so I had to study more how things work there. With this knowledge I updated repository which can display preview from black magic design card. It is just example with many issues so be careful and feel free to look for or inspiration there.

There's a various amount of issues here.
First on frame arrived, you have
var texture = device.CreateTexture2D<ID3D11Texture2D>(td, new []{srd});
So your create a texture, but you do not use it anywhere, it needs to be blitted to the swapchain (can do a CopyResource on device context or draw a full screen triangle/quad).
Note that CopyResource will only work if your swapchain has the same size as your incoming texture, which is rather unlikely, so you will have to draw a blit with a shader most likely.
Also you actually copying the data in the texture twice :
var texture = device.CreateTexture2D<ID3D11Texture2D>(td, new []{srd});
Since you provide initial data, the content is already there.
also, pitch is incorrect :
srd.SysMemPitch = (uint) frame.m_height;
pitch is the length (in bytes) of a line, so it should be :
srd.SysMemPitch = frame.GetRowBytes();
Please also note that in case of a non converted Decklink frame,
GetRowBytes can be different from width*4 (they can align row size to multiple of 16/32 or other values).
Next, in the case of resource map, the following is also incorrect :
unsafe
{
Buffer.MemoryCopy(frame.m_pixelBuffer.ToPointer(), mappedData.ToPointer(), size, size);
}
You are not checking the pitch/stride requirement of a texture (which can be different as well),
so you need to do :
if (mappedResource.RowPitch == frame.GetRowBytes())
{
//here you can use a direct copy as above
}
else
{
//here you need to copy data line per line
}

Xamarin MasterDetailPage memory leak with custom renderer

I am using a MasterDetailPage in Xamarin.Forms/Android with each item in the hamburger page representing a page. I change the detail page by calling:
Detail = new NavigationPage(new Pages.HomePage());
IsPresented = false;
What I have noticed is that the allocation for this page is somewhat large (it's requesting like 22mb). Not only that, but if I were to repeatedly press the sidebar's home button, I get an OutOfMemoryException after about 5 clicks. So something isn't clicking with the Garbage Collection.
The home page has a custom renderer that essentially composites an image using three png files. This works with minimal slowdown on my phone, but the emulator lags significantly when it is on screen, leading me to believe that it is causing performance issues.
public class GoalJarViewRenderer : ImageRenderer
{
Bitmap fillBmp = null;
Bitmap jarHud = null;
Paint font;
Paint fontBold;
string goalString = "";
string progString = "";
protected override void OnElementChanged(ElementChangedEventArgs<Image> e)
{
this.SetWillNotDraw(false);
base.OnElementChanged(e);
Element.PropertyChanged += SignalUpdate;
//Cache
if (fillBmp == null) fillBmp = BitmapFactory.DecodeResource(this.Context.Resources, Resource.Drawable.jar_fill);
if (jarHud == null) jarHud = BitmapFactory.DecodeResource(this.Context.Resources, Resource.Drawable.jar_hud);
if (font == null)
{
font = new Paint();
font.AntiAlias = true;
font.Color = Android.Graphics.Color.White;
font.TextSize = 50;
font.TextAlign = Paint.Align.Center;
}
if (fontBold == null)
{
fontBold = new Paint(font);
fontBold.FakeBoldText = true;
fontBold.TextSize = 80;
}
}
private void SignalUpdate(object sender, System.ComponentModel.PropertyChangedEventArgs e)
{
GoalJarView handle = (GoalJarView)this.Element;
goalString = String.Format("${0:#0.0#} / ${1:##.##}", handle.CurrentValue, handle.GoalValue);
progString = String.Format("{0:##0.00}%", (handle.CurrentValue / handle.GoalValue) * 100);
this.Invalidate();
}
public override void Draw(Canvas canvas)
{
base.Draw(canvas);
//Draw the fill
double LID_FIX = 0.91;
GoalJarView handle = (GoalJarView)this.Element;
double fillRatio = Math.Min(1,((handle.CurrentValue * LID_FIX) / handle.GoalValue));
int srcStartY = (int)(fillBmp.Height - Math.Floor(fillBmp.Height * fillRatio));
int destStartY = (int)(canvas.Height - Math.Floor(canvas.Height * fillRatio));
Rect fillSrc = new Rect(0, srcStartY, fillBmp.Width, fillBmp.Height);
RectF fillDest = new RectF(0, destStartY, canvas.Width, canvas.Height);
canvas.DrawBitmap(fillBmp, fillSrc, fillDest, null);
//Draw the text container
canvas.DrawBitmap(jarHud, null, new Rect(0,0,canvas.Width,canvas.Height), null);
//Draw the Text
canvas.DrawText(progString, canvas.Width / 2, (canvas.Height / 2) - 20, fontBold);
canvas.DrawText(goalString, canvas.Width / 2, (canvas.Height / 2) + 30, font);
}
}
I was told that BitmapFactory.DecodeResource was incredibly taxing, so I moved that into the OnElementChanged rather than the draw function. I don't see anything else immediately taxing inside of Draw(), but I am new to Xamarin/Android and assume the problem is glaringly obvious to a seasoned developer.
I have tried creating local variables on the MasterDetailPage that stores pages so they don't need to be created every time an item is pressed, but this led to crashing when the page reopened, as the actual control class that stores the data bindings was null.
What is causing this memory leak, and how can I correct it?

WIA: setting dinamic page size

I'm developing a WPF application to scan different documents with a scanner. The size of the documents won't be the same, can be variable.
I have my code working without scanner dialogs, and I would like the user not to have to preview the image and then scanning it to get the real size (resulting in two scans).
The problem is that I try to set page-size to auto before scanning
SetWIAProperty(item.Properties, "3097", 100);
but I get HRESULT: 0x80210067 System.Runtime.InteropServices.COMException.
I've googled to this and seens that my scanner is not supporting this property.
So, is there any way of achieving this? I need the resulting scanned image to be only the document, not all the scanner area (which I'm obtaning right now).
In case I couldn't tell the scanner to scan only the document, I've thought also in cropping the resulting image to obtain only the document I need, but don't know how to do this right now.
Here is my code:
DeviceManager deviceManager = new DeviceManager();
Device scanner = null;
foreach (DeviceInfo deviceInfo in deviceManager.DeviceInfos)
{
if (deviceInfo.DeviceID == scannerId)
{
scanner = deviceInfo.Connect();
break;
}
}
if (scanner == null)
{
throw new Exception("Scanner not found");
}
Item item = scanner.Items[1] as Item;
int dpi = 300;
SetWIAProperty(item.Properties, "6146", 1); // 1 Color
SetWIAProperty(item.Properties, "6147", dpi); // dpis
SetWIAProperty(item.Properties, "6148", dpi); // dpis
// This line throws the exception
//SetWIAProperty(item.Properties, "3097", 100); // page size 0=A4, 1=letter, 2=custom, 100=auto
try
{
ICommonDialog wiaCommonDialog = new CommonDialog();
ImageFile scannedImage = (ImageFile)wiaCommonDialog.ShowTransfer(item, FormatID.wiaFormatPNG, false);
if (scannedImage != null)
{
ImageProcess imgProcess = new ImageProcess();
object convertFilter = "Convert";
string convertFilterID = imgProcess.FilterInfos.get_Item(ref convertFilter).FilterID;
imgProcess.Filters.Add(convertFilterID, 0);
SetWIAProperty(imgProcess.Filters[imgProcess.Filters.Count].Properties, "FormatID", FormatID.wiaFormatPNG);
scannedImage = imgProcess.Apply(scannedImage);
if (System.IO.File.Exists(#"D:\temp\scanwia3.png"))
System.IO.File.Delete(#"D:\temp\scanwia3.png");
scannedImage.SaveFile(#"D:\temp\scanwia3.png");
}
scannedImage = null;
}
finally
{
item = null;
scanner = null;
}
And SetWIAProperty function:
private static void SetWIAProperty(IProperties properties, object propName, object propValue)
{
Property prop = properties.get_Item(ref propName);
prop.set_Value(ref propValue);
}
Any help would be appreciated.
Kind regards,
Jose.

Property Page Size belongs to device, not to item.
var WIA_IPS_PAGE_SIZE = "3097";
var WIA_PAGE_AUTO = 100;
SetWIAProperty(scanner.Properties, WIA_IPS_PAGE_SIZE, WIA_PAGE_AUTO);

C# Rendering Performance Difference

I am using Dynamic Data Display to draw some plots.
It provides a class called MarkerPointsGraph to render markers to the screen. The OnRender method is called during Render and this in turn calls the appropriate marker's (Circle, Triangle, etc.) render method.
protected override void OnRenderCore(DrawingContext dc, RenderState state)
{
if (DataSource == null) return;
if (Marker == null) return;
var transform = Plotter2D.Viewport.Transform;
DataRect bounds = DataRect.Empty;
using (IPointEnumerator enumerator = DataSource.GetEnumerator(GetContext()))
{
Point point = new Point();
while (enumerator.MoveNext())
{
enumerator.GetCurrent(ref point);
enumerator.ApplyMappings(Marker);
//Point screenPoint = point.Transform(state.Visible, state.Output);
Point screenPoint = point.DataToScreen(transform);
bounds = DataRect.Union(bounds, point);
Marker.Render(dc, screenPoint);
}
}
Viewport2D.SetContentBounds(this, bounds);
}
/// <summary>Renders circle around each point of graph</summary>
public class CirclePointMarker : ShapePointMarker {
public override void Render(DrawingContext dc, Point screenPoint) {
dc.DrawEllipse(Fill, Pen, screenPoint, Size / 2, Size / 2);
}
}
Now, my calling code is as below.
plotter.AddLineGraph(
data.Data,
new Pen
{
Brush = Brushes.Violet,
DashStyle = DashStyles.DashDot,
Thickness = 3
},
GetMarker(data),
new StandardDescription(data.Title));
The main difference I noticed in performance is in the methods below.
Referring to data from the ChartData class
private ShapePointMarker GetMarker(ChartData data)
{
ShapePointMarker marker = null;
switch (data.MarkerShape)
{
case Shapes.Circle:
marker = new CirclePointMarker();
break;
case Shapes.Triangle:
marker = new TrianglePointMarker();
break;
case Shapes.None:
default:
marker = null;
break;
}
if (marker != null)
{
//Referring to stuff here.
marker.Fill = new SolidColorBrush(data.MarkerColor);
marker.Size = data.MarkerSize;
}
return marker;
}
Hardcoding the data in place of the reference.
private ShapePointMarker GetMarker(ChartData data)
{
ShapePointMarker marker = null;
switch (data.MarkerShape)
{
case Shapes.Circle:
marker = new CirclePointMarker();
break;
case Shapes.Triangle:
marker = new TrianglePointMarker();
break;
case Shapes.None:
default:
marker = null;
break;
}
if (marker != null)
{
//Hard coding stuff here.
marker.Fill = Brushes.Red;
marker.Size = 5;
}
return marker;
}
I see an improvement of nearly 100x in the render times merely by changing the reference based data in the GetMarker method to a hard coded value.
What behaviour of C# causes this difference?
EDIT: The MarkerPointsGraph does not create an object of Marker for each point. It merely calls the render method.

You could try freezing your Brush:
if (marker != null)
{
Brush b = new SolidColorBrush(data.MarkerColor);
b.Freeze();
marker.Fill = b;
marker.Size = data.MarkerSize;
}
Brushes.Red is already Frozen that might be why you see the performance increase.

Without knowing too much more about your setup, your issue likely lies in the fact that you are creating a SolidColorBrush in a relatively tight loop (and not disposing them, but that's another issue). That object has a fair amount of overhead associated with creating a new one, whereas using Brushes.Red uses the same instance and so doesn't have the same issues. If you were to create your own brush once I don't think you'd see the same problems.

Can't Beat 'Out Of Memory' Exceptions

I am developing an app with Mono for Android.
I have been struggling with Out of memory exceptions for the last few days and am starting to lose hope!
I have a ListView displaying anything from 200 to 600 items. These items consist of a bitmap thumbnail and some text.
I am decoding the Bitmap asynchronously using an AsyncTask, here is the code:
public class BitmapWorkerTask : AsyncTask
{
private WeakReference imageViewReference;
public string thisURL = "";
private int sampleSize = 0;
private int reqHeight = 0;
private int reqWidht = 0;
public BitmapWorkerTask(ImageView imageView, int pSampleSize, int pReqWidth, int pReqHeight)
{
//_____________________________________________________________________
// Use a WeakReference to ensure the ImageView can be garbage collected
imageViewReference = new WeakReference(imageView);
reqHeight = pReqHeight;
reqWidht = pReqWidth;
sampleSize = pSampleSize;
}
protected override Java.Lang.Object DoInBackground(params Java.Lang.Object[] #params)
{
string strUrl = #params[0].ToString();
try
{
return DecodeSampleBitmapFromStream(strUrl, reqWidht, reqHeight);
}
catch (Exception ex)
{
return null;
}
}
protected override void OnPostExecute(Java.Lang.Object result)
{
base.OnPostExecute(result);
if (IsCancelled)
{
result = null;
Log.Debug("TT", "OnPostExecute - Task Cancelled");
}
else
{
Bitmap bmpResult = result as Bitmap;
if (imageViewReference != null && bmpResult != null)
{
ImageView view = imageViewReference.Target as ImageView;
if (view != null)
{
view.SetImageBitmap(bmpResult);
}
}
}
}
public static int CalculateInSampleSize( BitmapFactory.Options options, int reqWidth, int reqHeight)
{
//_____________________________
// Raw height and width of image
int height = options.OutHeight;
int width = options.OutWidth;
int inSampleSize = 1;
if (height > reqHeight || width > reqWidth)
{
if (width > height)
{
inSampleSize = (int)Math.Round((float)height / (float)reqHeight);
}
else
{
inSampleSize = (int)Math.Round((float)width / (float)reqWidth);
}
}
return inSampleSize;
}
public static Bitmap DecodeSampleBitmapFromStream(string URL, int reqWidth, int reqHeight)
{
URL url = new URL(URL);
try
{
//______________________________________________________________
// First decode with inJustDecodeBounds=true to check dimensions
BitmapFactory.Options options = new BitmapFactory.Options();
options.InJustDecodeBounds = true;
BitmapFactory.DecodeStream(url.OpenConnection().InputStream, null, options);
//______________________
// Calculate inSampleSize
options.InSampleSize = CalculateInSampleSize(options, reqWidth, reqHeight);
//____________________________________
// Decode bitmap with inSampleSize set
options.InJustDecodeBounds = false;
return BitmapFactory.DecodeStream(url.OpenConnection().InputStream, null, options);
}
catch (Exception ex)
{
return null;
}
finally
{
url.Dispose();
}
}
I am starting this AsyncTask from the Lists GetView() function using this Method:
public void loadBitmap(string url, ImageView imageView)
{
if (Common.cancelPotentialWork(url, imageView))
{
BitmapWorkerTask task = new BitmapWorkerTask(imageView, 2, 80,80);
AsyncDrawable asyncDrawable = new AsyncDrawable(null, null, task);
imageView.SetImageDrawable(asyncDrawable);
task.Execute(url);
}
}
Everything works as expected for a period of time, but if I continuously scroll up and down through my list I eventually start getting OutOfMemoryExceptions and the app crashes. My understanding of how the Android list works is it disposes of the ListItem views as they move off screen, but it feels as though this is not happening!
It feels like all those bitmaps I am decoding as I scroll through the list are for whatever reason being held in memory? What could I be missing here that is preventing those bitmaps from being disposed of? Where could I implement a call to Bitmap.Recycle() to ensure the bitmaps are cleared?
I did a test whereby I made a call to GC.Collect on every call to GetView which did seem to keep my memory usage fairly consistent, but I know this shouldn't be needed and it affects scrolling performance.
Why when I scroll through my list without the call to GC.Collect() am I not seeing those garbage collection message indicating that the system is, in fact, doing routine collections?
Any help is appreciated, I am losing the will to code!

My understanding of how the Android list works is it disposes of the List item views as they move off screen, but It feels as though this is not happening!
This isn't correct.
What Android does is it hold on to the set of item views and it tries to reuse them after they have gone off screen. This is what the convertView parameter is for.
I can't see your Adapter code posted in the question, so I'm not sure what your code is for using the convertView parameter, but I'd guess what it should do in the case of a convertView is:
it should cancel any existing async image fetch/conversion
it should start a new one
The MvvmCross code may be a little too complicated for you as a reference/example here, but you can at least see convertView in use in MvxBindableListAdapter.cs - see protected virtual View GetBindableView(View convertView, object source, int templateId)