We Keep Coding

Unity: CopyTexture to External texture2D

I need to expose a Unity texture/Rendertexture to some native plugin, which requires the "D3D11_RESOURCE_MISC_SHARED" flag on the texture.
textures created by unity doesn't have this flag, so, I created it from the plugin side, and then created a reference texture within unity using CreateExternalTexture, and copied the contents to this native texture using Graphics.CopyTexture.
the 2 textures have the same dimension, same size, same format, and same mipCount(0)
the problem is, when I show it in unity (for debugging purpose), I can see nothing and no error occurs.
btw, if I copy by ReadPixel, an error occures :
ReadPixels called on undefined image 0 (valid values are 0 - -1
if I create the texture using unity api, CopyTexture succeeds and result can be seen. but then, I lose the "D3D11_RESOURCE_MISC_SHARED" flag.
so, maybe the texture I created is not valid?
my code:
D3D11_TEXTURE2D_DESC desc = { 0 };
desc.Width = width;
desc.Height = height;
desc.MipLevels = 0;
desc.ArraySize = 1;
desc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;//这里,格式能不能调整?比如,A8是需要的吗?
desc.SampleDesc.Count = 1;
desc.Usage = D3D11_USAGE_DEFAULT;
desc.BindFlags = D3D11_BIND_SHADER_RESOURCE;//普通资源
//desc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;//应该不需要cpu访问的,不是read也不是write
desc.MiscFlags = D3D11_RESOURCE_MISC_SHARED;//for "OpenSharedHandle" d3d11 api
HRESULT hr = E_FAIL;
if (FAILED(hr = pDevice->CreateTexture2D(&desc, nullptr, &g_unityEquirectTexture)))
{
Log(" Create Shared Texture Failed!");
return NULL;
}
Log("CreateSharedTexture success");
//return g_unityEquirectTexture;
Unity CopyTexture Code:
if (output == null)
{
Debug.Log($"limit = {QualitySettings.masterTextureLimit}");
//output = new Texture2D(equirect.width, equirect.height, TextureFormat.RGBA32,false);//uncomment this line and then copyTexture below succeeds
IntPtr externalTextureData = CGDKInterOp.cgdk_c_CreateExternalTexture(equirectLeft.GetNativeTexturePtr(), equirectLeft.width * 2, equirectLeft.height);
if (externalTextureData != IntPtr.Zero)
{
output = Texture2D.CreateExternalTexture(equirectLeft.width * 2, equirectLeft.height, TextureFormat.RGBA32, false, true, externalTextureData);
}
}
if (output == null)
{
Debug.LogError("create texture from external failed!");
return;
}
//RenderTexture.active = equirect;
//output.ReadPixels(new Rect(0, 0, equirect.width, equirect.height), 0, 0);
//RenderTexture.active = null;
Graphics.CopyTexture(equirect, output);

OK, solved by myself.
the problem is: Miplevel == 0.this causes d3d11 creating a texture with 0B memory allocated!
change Miplevel to 1 solved the problem
note:
In unity inspector, we can see the memory allocated by textures. I found that my texture has 0B memory from the inspector and then, I search using this clue and found the solution

There might be a better solution for this but the last time I needed to copy over texture data from a native side to managed (Unity), I did it through marshalling of the data.
You essentially just need to expose a method in the native plugin to pass you the texture data as an array and then have a method in the C# code to fetch it data (by calling the method), releasing the pointer to native memory when you're done. You can find information about marshalling and interop in Microsoft's documentation (e.g. https://learn.microsoft.com/en-us/dotnet/framework/interop/marshalling-different-types-of-arrays).
If your texture is always guaranteed to be of the same size and format, it's easier - but if you need to know some additional parameters so that you know how the Texture should be represented in managed-land, you can always pass yourself the additional data through the same method.

Guide to use Yolo with gpu c#

I've been trying to find a tutorial or something on how to make yolo c# use gpu instead of cpu, I always find that it says that it works on both cpu and gpu but no one ever says how to use the gpu since it always uses cpu for me. Here's my code with yolo v5 c#. It doesn't really matter for me if it uses yolo v5 just that it uses gpu. Tutorial I found that tutorial but i can't even find the download for Nvidia cuDNN v7.6.3 for CUDA 10.1. It feels very unclear on how to use it with gpu please help me :D
var image = pictureBox1.Image;
var scorer = new YoloScorer<YoloCocoP5Model>("Assets/Weights/yolov5n.onnx");
List<YoloPrediction> predictions = scorer.Predict(image);
var graphics = Graphics.FromImage(image);
foreach (var prediction in predictions) // iterate predictions to draw results
{
using (MemoryStream ms = new MemoryStream())
{
pictureBox1.Image.Save(ms, ImageFormat.Png);
prediction.Label.Color = Color.FromArgb(255, 255, 0, 0);
double score = Math.Round(prediction.Score, 2);
graphics.DrawRectangles(new Pen(prediction.Label.Color, 1),
new[] { prediction.Rectangle });
var (x, y) = (prediction.Rectangle.X - 3, prediction.Rectangle.Y - 23);
graphics.DrawString($"{prediction.Label.Name} ({score})",
new Font("Consolas", 16, GraphicsUnit.Pixel), new SolidBrush(prediction.Label.Color),
new PointF(x, y));
pictureBox1.Image = image;
}
}

Before you are going to use scorer you need an option set.
//https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html
bool initResult = false;
var cudaProviderOptions = new Microsoft.ML.OnnxRuntime.OrtCUDAProviderOptions(); // Dispose this finally
var providerOptionsDict = new Dictionary<string, string>();
providerOptionsDict["device_id"] = "0";
providerOptionsDict["gpu_mem_limit"] = "2147483648";
providerOptionsDict["arena_extend_strategy"] = "kSameAsRequested";
/*
cudnn_conv_algo_search
The type of search done for cuDNN convolution algorithms.
Value Description
EXHAUSTIVE (0) expensive exhaustive benchmarking using cudnnFindConvolutionForwardAlgorithmEx
HEURISTIC (1) lightweight heuristic based search using cudnnGetConvolutionForwardAlgorithm_v7
DEFAULT (2) default algorithm using CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_PRECOMP_GEMM
Default value: EXHAUSTIVE
*/
providerOptionsDict["cudnn_conv_algo_search"] = "DEFAULT";
/*
do_copy_in_default_stream
Whether to do copies in the default stream or use separate streams. The recommended setting is true. If false, there are race conditions and possibly better performance.
Default value: true
*/
providerOptionsDict["do_copy_in_default_stream"] = "1";
/*
cudnn_conv_use_max_workspace
Check tuning performance for convolution heavy models for details on what this flag does. This flag is only supported from the V2 version of the provider options struct when used using the C API. The V2 provider options struct can be created using this and updated using this. Please take a look at the sample below for an example.
Default value: 0
*/
providerOptionsDict["cudnn_conv_use_max_workspace"] = "1";
/*
cudnn_conv1d_pad_to_nc1d
Check convolution input padding in the CUDA EP for details on what this flag does. This flag is only supported from the V2 version of the provider options struct when used using the C API. The V2 provider options struct can be created using this and updated using this. Please take a look at the sample below for an example.
Default value: 0
*/
providerOptionsDict["cudnn_conv1d_pad_to_nc1d"] = "1";
cudaProviderOptions.UpdateOptions(providerOptionsDict);
options = SessionOptions.MakeSessionOptionWithCudaProvider(cudaProviderOptions); // Dispose this finally
if (options != null)
{
// check yolo model file is accesible
if (File.Exists(yoloModelFile))
{
scorer = new YoloScorer<YoloCocoP5Model>(yoloModelFile, options);
initResult = true;
}
else
{
DebugMessage("Yolo model ONNX file (" + yoloModelFile + ") is missing!\r\n", 2);
}
}
else
DebugMessage("Yolo instance initializing error! Session options are empty!\r\n", 2);
}

Error when trying to capture desktop using DXGI and DirectX11 on intel 630 HD

I get the below error when trying to use DXGI to capture the builtin screen on my laptop that runs on an Intel 630 HD with the latest driver. The code works when I capture the external screen running on my GTX 1070.
SharpDX.SharpDXException
HResult=0x80070057
Message=HRESULT: [0x80070057], Module: [General], ApiCode: [E_INVALIDARG/Invalid Arguments], Message: The parameter is incorrect.
The code in my form:
desktopDuplicator = new DesktopDuplicatorD11(1,0, DesktopDuplicatorD11.VSyncLevel.None);
The section of the code that errors:
private bool RetrieveFrame()
{
if (desktopImageTexture == null)
desktopImageTexture = new Texture2D(mDevice, mTextureDescription);
frameInfo = new OutputDuplicateFrameInformation();
try
{
mDeskDuplication.AcquireNextFrame(500, out frameInfo, out desktopResource);
}
catch (SharpDXException ex)
{
if (ex.ResultCode.Code == SharpDX.DXGI.ResultCode.WaitTimeout.Result.Code)
{
return true;
}
if (ex.ResultCode.Failure)
{
throw new DesktopDuplicationException("Failed to acquire next frame.");
}
}
using (var tempTexture = desktopResource.QueryInterface<Texture2D>())
{
mDevice.ImmediateContext.CopyResource(tempTexture, desktopImageTexture);
}
return false;
}
It errors specifically on the line:
desktopImageTexture = new Texture2D(mDevice, mTextureDescription);
What is causing the error when using the internal display and the intel 630?
Edit #1:
mTextureDescription creation:
this.mTextureDescription = new Texture2DDescription()
{
CpuAccessFlags = CpuAccessFlags.Read,
BindFlags = BindFlags.None,
Format = Format.B8G8R8A8_UNorm,
Width = this.mOutputDescription.DesktopBounds.Right,
Height = this.mOutputDescription.DesktopBounds.Bottom,
OptionFlags = ResourceOptionFlags.None,
MipLevels = 1,
ArraySize = 1,
SampleDescription = { Count = 1, Quality = 0 },
Usage = ResourceUsage.Staging
};
The whole Desktop Duplication process is done on the same thread.
Update #2:
On the intel 630 Width = this.mOutputDescription.DesktopBounds.Right, returns 0 where as on my 1070 it returns 1920.

The most simplest reason is usually the actual problem.
Intel's final WDDM 2.6 drivers do not work properly with switchable graphics, update to the DCH WDDM 2.7 driver.

First, to get hints from API about invalid argument (this is exactly what you have) you need to enable Direct3D Debug Layer. The article explains it for C++ and it is possible to do a similar trick with C# as well.
Second, important is what are effectively the arguments in the failing call, not just the code.
The code is about right but if coordinates in mOutputDescription are zero or invalid, the mentioned API call is going to fail as well. You need to set a break point and inspect the variable.

Take screenshot and draw it on a Windows Form using SharpDX

I'm trying to capture a screenshot and show it on a Windows Form (or on a Panel) using SharpDX, but I've some trouble. I capture the screenshot using OutputDuplication.AcquireNextFrame, but when I try to draw the caputured frame using WindowRenderTarget.DrawBitmap it fail with the WrongResourceDomain exception. I added a Form and modified the code of the ScreenCapture sample to illustrate my problem:
using System;
using SharpDX;
using SharpDX.Direct2D1;
using SharpDX.Direct3D11;
using SharpDX.DXGI;
namespace MiniTri
{
/// <summary>
/// Screen capture of the desktop using DXGI OutputDuplication.
/// </summary>
internal static class Program
{
[STAThread]
private static void Main()
{
var Form1 = new ScreenCapture.Form1();
// # of graphics card adapter
const int numAdapter = 0;
// # of output device (i.e. monitor)
const int numOutput = 0;
// Create DXGI Factory1
var dxgiFactory = new SharpDX.DXGI.Factory1();
var dxgiAdapter = dxgiFactory.GetAdapter1(numAdapter);
// Create device from Adapter
var d3dDevice = new SharpDX.Direct3D11.Device(dxgiAdapter, DeviceCreationFlags.BgraSupport);
// Create Direct2D1 Factory1
var d2dFactory = new SharpDX.Direct2D1.Factory1();
// Create Direct2D device
SharpDX.Direct2D1.Device d2dDevice;
using (var dxgiDevice = d3dDevice.QueryInterface<SharpDX.DXGI.Device>())
d2dDevice = new SharpDX.Direct2D1.Device(d2dFactory, dxgiDevice);
// Get the Direct2D1 DeviceContext
var d2dContext = new SharpDX.Direct2D1.DeviceContext(d2dDevice, DeviceContextOptions.None);
// Create Direct2D1 WindowRenderTarget
var WindowRenderTarget = new WindowRenderTarget(d2dFactory,
new RenderTargetProperties
{
Type = RenderTargetType.Default
},
new HwndRenderTargetProperties
{
Hwnd = Form1.Handle,
PixelSize = new Size2(Form1.Width, Form1.Height),
PresentOptions = PresentOptions.Immediately
});
// Get DXGI.Output
var output = dxgiAdapter.GetOutput(numOutput);
var output1 = output.QueryInterface<Output1>();
// Create RawRectangleF with the Form size
var FormRectangle = new SharpDX.Mathematics.Interop.RawRectangleF(0, 0, Form1.Width, Form1.Height);
// Duplicate the output
var duplicatedOutput = output1.DuplicateOutput(d3dDevice);
bool captureDone = false;
for (int i = 0; !captureDone; i++)
{
try
{
SharpDX.DXGI.Resource screenResource;
OutputDuplicateFrameInformation duplicateFrameInformation;
// Try to get duplicated frame within given time
duplicatedOutput.AcquireNextFrame(10000, out duplicateFrameInformation, out screenResource);
if (i > 0)
{
// Create a Direct2D1 Bitmap1 from screenResource
var dxgiSurface = screenResource.QueryInterface<Surface>();
var d2dBitmap1 = new Bitmap1(d2dContext, dxgiSurface);
// Should draw the bitmap on the form, but fails with "WrongResourceDomain" exception
WindowRenderTarget.BeginDraw();
WindowRenderTarget.DrawBitmap(d2dBitmap1, FormRectangle, 1.0f, BitmapInterpolationMode.Linear);
WindowRenderTarget.EndDraw();
// Capture done
captureDone = true;
}
screenResource.Dispose();
duplicatedOutput.ReleaseFrame();
}
catch (SharpDXException e)
{
if (e.ResultCode.Code != SharpDX.DXGI.ResultCode.WaitTimeout.Result.Code)
{
throw e;
}
}
}
// TODO: We should cleanp up all allocated COM objects here
}
}
}
I know I can convert the frame to a System.Drawing.Bitmap (like in the original sample) and then use System.Drawing.Graphics.DrawImage to draw the bitmap to the Form, but I want to avoid this and draw the frame using SharpDX.
I've tried also to create a SwapChain and draw the frame using Direct2D1.RenderTarget.DrawBitmap (similar to the MiniRect sample) but it fail with the same exception. The same using Direct2D1.DeviceContext.DrawBitmap.
I read here that this exception means that I am mixing resources from different resource domains, but I don't understand exactly what it means in my code. I am not experienced in using DirectX.

I tried using the SwapChain again, instead of the WindowRenderTarget and this time I was successful.
Create a SwapChainDescription specifing Width, Height and Handle of your target control (for example a Form or a Panel).
Create a new SwapChain using the dxgiFactory, d3dDevice and SwapChainDescription.
Get the back buffer of the SwapChain.
Get a DXGI Surface from the back buffer.
(more about creation of SwapChain, back buffer and DXGI Surface in the MiniRect sample)
Use the d2dContext to create a Direct2D1.Bitmap1 from the DXGI Surface.
Use the d2dContext to create another Direct2D1.Bitmap1 from the screenResource (as illustrated in the above code).
Set the first Direct2D1.Bitmap1 as target of the d2dContext.
Use the d2dContext do draw the second Direct2D1.Bitmap1.
Present the SwapChain.
Done.

How to use function D3DXSaveSurfaceToFile() in C#

I use directx 9 to build model.
I want to save it to bmp file. I found the function D3DXSaveSurfaceToFile()
But I do not know how to use it in C#.
How can I use it?

Unfortunately, there is no such function in c#.
Try the following code instead:
try
{
// initialize D3D device
PresentParameters presentParams = new PresentParameters();
presentParams.Windowed = true;
presentParams.SwapEffect = SwapEffect.Discard;
Device myDevice = new
Device(0,DeviceType.Hardware,this,CreateFlags.SoftwareVertexProcessing,presentParams);
// create a surface the size of screen,
// format had to be A8R8G8B8, as the GetFrontBufferData returns
// only memory pool types allowed are Pool.Scratch or Pool.System memory
Surface mySurface =
myDevice.CreateOffscreenPlainSurface(SystemInformation.PrimaryMonitorSize.Width,SystemInformation.PrimaryMonitorSize.Height,Format.A8R8G8B8,Pool.SystemMemory);
//Get the front buffer.
myDevice.GetFrontBufferData(0,mySurface);
//saves surface to file
SurfaceLoader.Save("surface.bmp",ImageFileFormat.Bmp,mySurface);
}
catch
{
//whatever
}