Related
I am using this libvlc wrapper (https://github.com/ZeBobo5/Vlc.DotNet) to stream live video from IP camera. And there is a problem I need to solve with this:
I use initialized VlcControl in two WPF windows. On the main window there is live video-stream and when the snapshot is taken, it opens the image editor with the taken snapshot. In this second window it is possible to switch to the camera live view and the problem is here:
if video is paused on the main window and started in image editor, it starts exactly where it was stopped, not the live view. Moreover, it does not stop buffering the video that leads to unnecessary network load.
I have already tried to stop player at main window and to re-initialize VlcControl in the image editor, but it takes about 10 seconds to start streaming, which is not the solution.
Also it is possible to continue stream without pausing it, but it creates unnecessary network load of 4k frames, which is not the option.
I have also asked the question in the GitHub, but none of the suggestions worked for me: https://github.com/ZeBobo5/Vlc.DotNet/issues/576
Here is the code that is used to remove VlcControl from one UI and add to another.
public sealed class LiveVideoVlcPlayer : ILiveVideoPlayer
{
private readonly VlcControl _videoPlayer;
/// <param name="vlcLibraryDirectory"></param>
/// <param name="options">VLC options can be given here. Please refer to the VLC command line documentation.</param>
/// <inheritdoc />
public LiveVideoVlcPlayer(IDirectoryInfo vlcLibraryDirectory, string[] options)
{
_videoPlayer = new VlcControl();
_videoPlayer.SourceProvider.CreatePlayer(vlcLibraryDirectory.DirectoryInfo, options);
_videoPlayer.Background = Brushes.White;
}
/// <inheritdoc />
public void Dispose()
{
_videoPlayer?.Dispose();
}
public void AddToUiChildren(UIElementCollection children)
{
if (children.Contains(_videoPlayer))
return;
children.Add(_videoPlayer);
}
public void Pause()
{
_videoPlayer.SourceProvider.MediaPlayer.Pause();
}
public void Play(Uri videoSource)
{
// https://wiki.videolan.org/Documentation:Streaming_HowTo/Command_Line_Examples/
_videoPlayer.SourceProvider.MediaPlayer.SetMedia(videoSource);
_videoPlayer.SourceProvider.MediaPlayer.Play();
}
/// <summary>
/// Continue paused video.
/// </summary>
public void Play()
{
_videoPlayer.SourceProvider.MediaPlayer.Play();
}
/// <summary>
/// Removing the player from a visual thee will pause the video
/// </summary>
/// <param name="children"></param>
public void RemoveFromUiChildren(UIElementCollection children)
{
Pause();
if (!children.Contains(_videoPlayer))
return;
children.Remove(_videoPlayer);
}
}
And here is the code that initializes new VlcPlayer. This code is used in the main window and video player is passed to the image editor as argument.
_vlcLibraryDirectory = new DirectoryInfoWrap(_pathWrap.Combine(currentDirectory, "libvlc", IntPtr.Size == 4 ? "win-x86" : "win-x64"));
var options = new string[]
{
"--network-caching=50"
// VLC options can be given here. Please refer to the VLC command line documentation.
};
try
{
LogManager.Debug("Initialise VLC player.");
_liveVideoVlcPlayer = new LiveVideoVlcPlayer(_vlcLibraryDirectory, options);
}
catch (Exception exception)
{
LogManager.Error("Failed to initialise VLC player. Is libvlc package installed?", exception);
}
What I want is to find the way to stop the streaming and buffering(to reduce network load) and restart its as a live stream with the least possible time for loading the live view.
Scenario
I'm planning to learn the basics about how to develop a custom HUD for a 3rd party PC-game (free, legal), my intention is to develop an application that will draw/show additional info on the screen (by reading mem addresses).
So I've researched for profesional projects and I've found TurboHUD that uses SharpDX library to draw objects/overlays/text over the screen using that DirectX based library and the results are very good (it does not loose performancy at any moment when drawing multiple objects on the screen),
but, since the author of that project does not provide a source to understand how they did it then I'm trying to learn by myself to use the same professional technics.
Question
If I'm on the wrong way or I missed some other better alternatives than SharpDX to develop this, please let me know.
My main question is:
In C# or preferably VB.Net, how I could just draw efficiently a custom string over the screen using SharpDX?.
Note that I could set the Form's oppacity to 0 but I think it should exists a proper way and I'm asking to know that proper way to draw directlly on the "desktop" screen.
My expects are to launch that PC-Game, then launch my custom HUD that will draw over the "desktop" screen surface to add additional info on the game, I hope you understand me.
Research
I should clarify that I'm totally unexperienced whit this kind of DirectX libs, and I'm using the SharpDX samples package to try learn its usage.
Since all the samples are in C# its more difficult to learn its usage under VB.Net.
Inside the SharpDX samples package there is an 'AdvancedTextRenderingApp' C# project, but as its name says it is an advanced example and also it instances a custom Form (a SharpDX.Windows.RenderForm) to draw on that Form.
This is the an VB.Net code translation of that C# project that I mentioned:
http://pastebin.com/KG2c3v09
UPDATE:
Just with the intention to comment about a research I did:
I recentlly discovered this useful GitHub repository, however, it fails to compile in Visual Studio 2015 (because missing namespace usings, that when added they generate more compiler errors), also it is oriented for advanced SharpDX users, and after analyzed the full sample I still don't have any idea of how to write/draw over the surface of a 3rd part process window ...also the C# syntax difficults me the overall understanding of SharpDX usage, because also the author did custom implementations a big variety of SharpDX members, then... I'm more than lost with all those examples.
The official SharpDX samples are another of those things that seems very useful ...maybe for advanced users. Some samples seems that demonstrates how to render a custom window/surface (with 500 tedious and incomprehensible lines of code to do it. And their "Hello world" sample is a nightmare for me.), however, what I would like to acchieve as I explained in my question is to draw on the surface of an existing window of another process, and I'm aware that probablly for that I would need to render a "surface" from scratch with SharpDX, then positionate it in the target window, then make invisible the surface, then draw on it, but I don't know how to do those things.
It took me a while to find how to load the font in XNA to draw the text but everything works fine.
You need 4 things:
Make form topmost
Extend aero glass style to whole form (for transparency to work)
Initialize XNA, Microsoft XNA Game Studio 4.0
Draw the texture and text
One limitation
The game must not be in fullscreen mode. The same as the TurboHUD limitation
Form1.cs
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Text;
using System.IO;
using System.Windows.Forms;
using System.Threading;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Content;
using System.Runtime.InteropServices;
using System.Drawing;
namespace XNATransparentWindow
{
public partial class Form1 : Form
{
private ContentBuilder contentBuilder;
public Form1()
{
InitializeComponent();
TopMost = true;
FormBorderStyle = System.Windows.Forms.FormBorderStyle.None;
MARGINS margins = new MARGINS();
margins.leftWidth = 0;
margins.rightWidth = 0;
margins.topHeight = this.Width;
margins.bottomHeight = this.Height;
// Extend aero glass style to whole form
DwmExtendFrameIntoClientArea(this.Handle, ref margins);
//Load XNA directX
this.contentBuilder = new ContentBuilder();
graphicsDeviceService = GraphicsDeviceService.AddRef(Handle, ClientSize.Width, ClientSize.Height);
//Register the service, so components like ContentManager can find it.
services.AddService<IGraphicsDeviceService>(graphicsDeviceService);
//Get the Graphics device
dev = graphicsDeviceService.GraphicsDevice;
if(dev == null){/*error message*/}
//Load texture
int bufferSize;
System.IO.MemoryStream memoryStream;
Bitmap img;
using (img = new Bitmap(#"C:\...\.png"))
{
bufferSize = img.Height * img.Width * 4;
memoryStream = new System.IO.MemoryStream(bufferSize);
img.Save(memoryStream, System.Drawing.Imaging.ImageFormat.Png);
memoryStream.Seek(0, SeekOrigin.Begin);
texture = Texture2D.FromStream(dev, memoryStream, img.Width, img.Height, false);
memoryStream.Close();
if(texture == null){/*error message*/}
}
//Create sprite
s_Batch = new SpriteBatch(dev);
if(s_Batch == null){/*error message*/}
FontPos = new Vector2(270.0F, 110.0F);
//Load font
contentManager = new ContentManager(services, this.contentBuilder.OutputDirectory);
this.contentBuilder.Clear();
this.contentBuilder.Add(#"C:\...\my_font1.spritefont","my_font1", "FontDescriptionImporter", "FontDescriptionProcessor");
//Build spritefont to get the .xbn file
string error = this.contentBuilder.Build();
//If build fail
if (!String.IsNullOrEmpty(error))
{
MessageBox.Show(error);
return;
}
//Load the .xbn file
Font1 = contentManager.Load<SpriteFont>("my_font1");
if(Font1 == null){/*error message*/}
}
[DllImport("dwmapi.dll")]
static extern int DwmExtendFrameIntoClientArea(IntPtr hWnd, ref MARGINS margin);
[DllImport("user32.dll")]
static extern IntPtr SendMessage(IntPtr hWnd, UInt32 Msg, Int32 wParam, Int32 lParam);
[StructLayout(LayoutKind.Sequential)]
public struct MARGINS
{
public int leftWidth;
public int rightWidth;
public int topHeight;
public int bottomHeight;
}
public ServiceContainer Services
{
get { return services; }
}
ServiceContainer services = new ServiceContainer();
GraphicsDevice dev;
SpriteFont Font1;
Vector2 FontPos;
SpriteBatch s_Batch;
Texture2D texture;
ContentManager contentManager;
GraphicsDeviceService graphicsDeviceService;
private const UInt32 WM_NCLBUTTONDOWN = 0xA1;
private const Int32 HTCAPTION = 0x2;
private void Form1_MouseDown(object sender, MouseEventArgs e)
{
if (e.Button == System.Windows.Forms.MouseButtons.Right)
{
this.Close();
}
else //to move the form
{
this.Capture = false;
SendMessage(this.Handle, WM_NCLBUTTONDOWN, HTCAPTION, 0);
}
}
private void Form1_Paint(object sender, PaintEventArgs e)
{
//There are two buffers. One offscreen-backbuffer and the
//frontbuffer which is the actual form in this example. The
//drawings are done to the backbuffer and in the end the two
//buffers flip. The backbuffer becomes frontbuffer and the
//frontbuffer becomes backbuffer.
//The drawing should start when the last resource is loaded.
//Since Font1 is the last one in this example I check for this
if (Font1 == null)
{
return;
}
//clear the backbuffer with transparent color.
dev.Clear(Microsoft.Xna.Framework.Color.Transparent);
//Do all your drawings here
//draw texture and text with the sprite
s_Batch.Begin(SpriteSortMode.Deferred, BlendState.AlphaBlend);
s_Batch.Draw(texture, new Microsoft.Xna.Framework.Rectangle(0, 0, this.Width, this.Height), Microsoft.Xna.Framework.Color.White);
s_Batch.DrawString(Font1, #"XNA FRAMEWORK", FontPos, Microsoft.Xna.Framework.Color.Black);
s_Batch.End();
//here the flip is performed
dev.Present();
}
//Release resources
private void Form1_FormClosing(object sender, FormClosingEventArgs e)
{
graphicsDeviceService.GraphicsDevice.Dispose();
graphicsDeviceService.Release(true);
s_Batch.Dispose();
texture.Dispose();
}
}
}
The following classes are just copy-paste from an example XNA 4.0 Content Compiler I found(with minor adjustments). They are used just to load the font for drawing the text:
GraphicsDeviceService.cs
#region File Description
//-----------------------------------------------------------------------------
// GraphicsDeviceService.cs
//
// Microsoft XNA Community Game Platform
// Copyright (C) Microsoft Corporation. All rights reserved.
//-----------------------------------------------------------------------------
#endregion
#region Using Statements
using System;
using System.Threading;
using Microsoft.Xna.Framework.Graphics;
#endregion
// The IGraphicsDeviceService interface requires a DeviceCreated event, but we
// always just create the device inside our constructor, so we have no place to
// raise that event. The C# compiler warns us that the event is never used, but
// we don't care so we just disable this warning.
#pragma warning disable 67
namespace XNATransparentWindow
{
/// <summary>
/// Helper class responsible for creating and managing the GraphicsDevice.
/// All GraphicsDeviceControl instances share the same GraphicsDeviceService,
/// so even though there can be many controls, there will only ever be a single
/// underlying GraphicsDevice. This implements the standard IGraphicsDeviceService
/// interface, which provides notification events for when the device is reset
/// or disposed.
/// </summary>
class GraphicsDeviceService : IGraphicsDeviceService
{
#region Fields
// Singleton device service instance.
static GraphicsDeviceService singletonInstance;
// Keep track of how many controls are sharing the singletonInstance.
static int referenceCount;
#endregion
/// <summary>
/// Constructor is private, because this is a singleton class:
/// client controls should use the public AddRef method instead.
/// </summary>
GraphicsDeviceService(IntPtr windowHandle, int width, int height)
{
parameters = new PresentationParameters();
parameters.BackBufferWidth = Math.Max(width, 1);
parameters.BackBufferHeight = Math.Max(height, 1);
parameters.BackBufferFormat = SurfaceFormat.Vector4; // SurfaceFormat.Color;
parameters.DeviceWindowHandle = windowHandle;
parameters.PresentationInterval = PresentInterval.Immediate;
parameters.IsFullScreen = false;
graphicsDevice = new GraphicsDevice(GraphicsAdapter.DefaultAdapter, GraphicsProfile.Reach, parameters);
}
/// <summary>
/// Gets a reference to the singleton instance.
/// </summary>
public static GraphicsDeviceService AddRef(IntPtr windowHandle,
int width, int height)
{
// Increment the "how many controls sharing the device" reference count.
if (Interlocked.Increment(ref referenceCount) == 1)
{
// If this is the first control to start using the
// device, we must create the singleton instance.
singletonInstance = new GraphicsDeviceService(windowHandle,
width, height);
}
return singletonInstance;
}
/// <summary>
/// Releases a reference to the singleton instance.
/// </summary>
public void Release(bool disposing)
{
// Decrement the "how many controls sharing the device" reference count.
if (Interlocked.Decrement(ref referenceCount) == 0)
{
// If this is the last control to finish using the
// device, we should dispose the singleton instance.
if (disposing)
{
if (DeviceDisposing != null)
DeviceDisposing(this, EventArgs.Empty);
graphicsDevice.Dispose();
}
graphicsDevice = null;
}
}
/// <summary>
/// Resets the graphics device to whichever is bigger out of the specified
/// resolution or its current size. This behavior means the device will
/// demand-grow to the largest of all its GraphicsDeviceControl clients.
/// </summary>
public void ResetDevice(int width, int height)
{
if (DeviceResetting != null)
DeviceResetting(this, EventArgs.Empty);
parameters.BackBufferWidth = Math.Max(parameters.BackBufferWidth, width);
parameters.BackBufferHeight = Math.Max(parameters.BackBufferHeight, height);
graphicsDevice.Reset(parameters);
if (DeviceReset != null)
DeviceReset(this, EventArgs.Empty);
}
/// <summary>
/// Gets the current graphics device.
/// </summary>
public GraphicsDevice GraphicsDevice
{
get { return graphicsDevice; }
}
GraphicsDevice graphicsDevice;
// Store the current device settings.
PresentationParameters parameters;
// IGraphicsDeviceService events.
public event EventHandler<EventArgs> DeviceCreated;
public event EventHandler<EventArgs> DeviceDisposing;
public event EventHandler<EventArgs> DeviceReset;
public event EventHandler<EventArgs> DeviceResetting;
}
}
ServiceContainer.cs
#region File Description
//-----------------------------------------------------------------------------
// ServiceContainer.cs
//
// Microsoft XNA Community Game Platform
// Copyright (C) Microsoft Corporation. All rights reserved.
//-----------------------------------------------------------------------------
#endregion
#region Using Statements
using System;
using System.Collections.Generic;
#endregion
namespace XNATransparentWindow
{
/// <summary>
/// Container class implements the IServiceProvider interface. This is used
/// to pass shared services between different components, for instance the
/// ContentManager uses it to locate the IGraphicsDeviceService implementation.
/// </summary>
public class ServiceContainer : IServiceProvider
{
Dictionary<Type, object> services = new Dictionary<Type, object>();
/// <summary>
/// Adds a new service to the collection.
/// </summary>
public void AddService<T>(T service)
{
services.Add(typeof(T), service);
}
/// <summary>
/// Looks up the specified service.
/// </summary>
public object GetService(Type serviceType)
{
object service;
services.TryGetValue(serviceType, out service);
return service;
}
}
}
ContentBuilder.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Diagnostics;
using Microsoft.Build.Construction;
using Microsoft.Build.Evaluation;
using Microsoft.Build.Execution;
using Microsoft.Build.Framework;
namespace XNATransparentWindow
{
public class ContentBuilder : IDisposable
{
#region Fields
// What importers or processors should we load?
const string xnaVersion = ", Version=4.0.0.0, PublicKeyToken=842cf8be1de50553";
static string[] pipelineAssemblies =
{
"Microsoft.Xna.Framework.Content.Pipeline.FBXImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.XImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.TextureImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.EffectImporter" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.AudioImporters" + xnaVersion,
"Microsoft.Xna.Framework.Content.Pipeline.VideoImporters" + xnaVersion,
// If you want to use custom importers or processors from
// a Content Pipeline Extension Library, add them here.
//
// If your extension DLL is installed in the GAC, you should refer to it by assembly
// name, eg. "MyPipelineExtension, Version=1.0.0.0, PublicKeyToken=1234567812345678".
//
// If the extension DLL is not in the GAC, you should refer to it by
// file path, eg. "c:/MyProject/bin/MyPipelineExtension.dll".
};
// MSBuild objects used to dynamically build content.
Project buildProject;
ProjectRootElement projectRootElement;
BuildParameters buildParameters;
List<ProjectItem> projectItems = new List<ProjectItem>();
//ErrorLogger errorLogger;
// Temporary directories used by the content build.
string buildDirectory;
string processDirectory;
string baseDirectory;
// Generate unique directory names if there is more than one ContentBuilder.
static int directorySalt;
// Have we been disposed?
bool isDisposed;
#endregion
#region Properties
/// Gets the output directory, which will contain the generated .xnb files.
public string OutputDirectory
{
get { return Path.Combine(buildDirectory, "bin/Content"); }
}
#endregion
#region Initialization
/// Creates a new content builder.
public ContentBuilder()
{
CreateTempDirectory();
CreateBuildProject();
}
/// Finalizes the content builder.
~ContentBuilder()
{
Dispose(false);
}
/// Disposes the content builder when it is no longer required.
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
/// Implements the standard .NET IDisposable pattern.
protected virtual void Dispose(bool disposing)
{
if (!isDisposed)
{
isDisposed = true;
DeleteTempDirectory();
}
}
#endregion
#region MSBuild
/// Creates a temporary MSBuild content project in memory.
void CreateBuildProject()
{
string projectPath = Path.Combine(buildDirectory, "content.contentproj");
string outputPath = Path.Combine(buildDirectory, "bin");
// Create the build project.
projectRootElement = ProjectRootElement.Create(projectPath);
// Include the standard targets file that defines how to build XNA Framework content.
projectRootElement.AddImport("$(MSBuildExtensionsPath)\\Microsoft\\XNA Game Studio\\" +
"v4.0\\Microsoft.Xna.GameStudio.ContentPipeline.targets");
buildProject = new Project(projectRootElement);
buildProject.SetProperty("XnaPlatform", "Windows");
buildProject.SetProperty("XnaProfile", "Reach");
buildProject.SetProperty("XnaFrameworkVersion", "v4.0");
buildProject.SetProperty("Configuration", "Release");
buildProject.SetProperty("OutputPath", outputPath);
// Register any custom importers or processors.
foreach (string pipelineAssembly in pipelineAssemblies)
{
buildProject.AddItem("Reference", pipelineAssembly);
}
// Hook up our custom error logger.
//errorLogger = new ErrorLogger();
buildParameters = new BuildParameters(ProjectCollection.GlobalProjectCollection);
//buildParameters.Loggers = new ILogger[] { errorLogger };
}
/// Adds a new content file to the MSBuild project. The importer and
/// processor are optional: if you leave the importer null, it will
/// be autodetected based on the file extension, and if you leave the
/// processor null, data will be passed through without any processing.
public void Add(string filename, string name, string importer, string processor)
{
ProjectItem item = buildProject.AddItem("Compile", filename)[0];
item.SetMetadataValue("Link", Path.GetFileName(filename));
item.SetMetadataValue("Name", name);
if (!string.IsNullOrEmpty(importer))
item.SetMetadataValue("Importer", importer);
if (!string.IsNullOrEmpty(processor))
item.SetMetadataValue("Processor", processor);
projectItems.Add(item);
}
/// Removes all content files from the MSBuild project.
public void Clear()
{
buildProject.RemoveItems(projectItems);
projectItems.Clear();
}
/// Builds all the content files which have been added to the project,
/// dynamically creating .xnb files in the OutputDirectory.
/// Returns an error message if the build fails.
public string Build()
{
// Create and submit a new asynchronous build request.
BuildManager.DefaultBuildManager.BeginBuild(buildParameters);
BuildRequestData request = new BuildRequestData(buildProject.CreateProjectInstance(), new string[0]);
BuildSubmission submission = BuildManager.DefaultBuildManager.PendBuildRequest(request);
submission.ExecuteAsync(null, null);
// Wait for the build to finish.
submission.WaitHandle.WaitOne();
BuildManager.DefaultBuildManager.EndBuild();
// If the build failed, return an error string.
if (submission.BuildResult.OverallResult == BuildResultCode.Failure)
{
//return string.Join("\n", errorLogger.Errors.ToArray());
}
return null;
}
#endregion
#region Temp Directories
/// Creates a temporary directory in which to build content.
void CreateTempDirectory()
{
// Start with a standard base name:
//
// %temp%\WinFormsContentLoading.ContentBuilder
baseDirectory = Path.Combine(Path.GetTempPath(), GetType().FullName);
// Include our process ID, in case there is more than
// one copy of the program running at the same time:
//
// %temp%\WinFormsContentLoading.ContentBuilder\<ProcessId>
int processId = Process.GetCurrentProcess().Id;
processDirectory = Path.Combine(baseDirectory, processId.ToString());
// Include a salt value, in case the program
// creates more than one ContentBuilder instance:
//
// %temp%\WinFormsContentLoading.ContentBuilder\<ProcessId>\<Salt>
directorySalt++;
buildDirectory = Path.Combine(processDirectory, directorySalt.ToString());
// Create our temporary directory.
Directory.CreateDirectory(buildDirectory);
PurgeStaleTempDirectories();
}
/// <summary>
/// Deletes our temporary directory when we are finished with it.
/// </summary>
void DeleteTempDirectory()
{
Directory.Delete(buildDirectory, true);
// If there are no other instances of ContentBuilder still using their
// own temp directories, we can delete the process directory as well.
if (Directory.GetDirectories(processDirectory).Length == 0)
{
Directory.Delete(processDirectory);
// If there are no other copies of the program still using their
// own temp directories, we can delete the base directory as well.
if (Directory.GetDirectories(baseDirectory).Length == 0)
{
Directory.Delete(baseDirectory);
}
}
}
/// <summary>
/// Ideally, we want to delete our temp directory when we are finished using
/// it. The DeleteTempDirectory method (called by whichever happens first out
/// of Dispose or our finalizer) does exactly that. Trouble is, sometimes
/// these cleanup methods may never execute. For instance if the program
/// crashes, or is halted using the debugger, we never get a chance to do
/// our deleting. The next time we start up, this method checks for any temp
/// directories that were left over by previous runs which failed to shut
/// down cleanly. This makes sure these orphaned directories will not just
/// be left lying around forever.
/// </summary>
void PurgeStaleTempDirectories()
{
// Check all subdirectories of our base location.
foreach (string directory in Directory.GetDirectories(baseDirectory))
{
// The subdirectory name is the ID of the process which created it.
int processId;
if (int.TryParse(Path.GetFileName(directory), out processId))
{
try
{
// Is the creator process still running?
Process.GetProcessById(processId);
}
catch (ArgumentException)
{
// If the process is gone, we can delete its temp directory.
Directory.Delete(directory, true);
}
}
}
}
#endregion
}
}
The font which you are loading is an xml file: any_name.spritefont
Example:
<?xml version="1.0" encoding="utf-8"?>
<!--
This file contains an xml description of a font, and will be read by the XNA
Framework Content Pipeline. Follow the comments to customize the appearance
of the font in your game, and to change the characters which are available to draw
with.
-->
<XnaContent xmlns:Graphics="Microsoft.Xna.Framework.Content.Pipeline.Graphics">
<Asset Type="Graphics:FontDescription">
<!--
Modify this string to change the font that will be imported.
-->
<FontName>Segoe UI Mono</FontName>
<!--
Size is a float value, measured in points. Modify this value to change
the size of the font.
-->
<Size>14</Size>
<!--
Spacing is a float value, measured in pixels. Modify this value to change
the amount of spacing in between characters.
-->
<Spacing>0</Spacing>
<!--
UseKerning controls the layout of the font. If this value is true, kerning information
will be used when placing characters.
-->
<UseKerning>true</UseKerning>
<!--
Style controls the style of the font. Valid entries are "Regular", "Bold", "Italic",
and "Bold, Italic", and are case sensitive.
-->
<Style>Regular</Style>
<!--
If you uncomment this line, the default character will be substituted if you draw
or measure text that contains characters which were not included in the font.
-->
<!-- <DefaultCharacter>*</DefaultCharacter> -->
<!--
CharacterRegions control what letters are available in the font. Every
character from Start to End will be built and made available for drawing. The
default range is from 32, (ASCII space), to 126, ('~'), covering the basic Latin
character set. The characters are ordered according to the Unicode standard.
See the documentation for more information.
-->
<CharacterRegions>
<CharacterRegion>
<Start> </Start>
<End>~</End>
</CharacterRegion>
</CharacterRegions>
</Asset>
</XnaContent>
If you want the form to be click-through then you need to add a random
transparency key color to form and:
[DllImport("user32.dll", EntryPoint="GetWindowLong")]
static extern IntPtr GetWindowLongPtr(IntPtr hWnd, int nIndex);
[DllImport("user32.dll")]
static extern int SetWindowLong(IntPtr hWnd, int nIndex, IntPtr dwNewLong);
In public Form1()
IntPtr initialStyle = GetWindowLongPtr(this.Handle, -20);
SetWindowLong(this.Handle, -20, (IntPtr)((int)initialStyle | 0x20));
Feel free to correct any mistakes I have made.
I have started developing in c# for the kinect with WPF.
When I start the sample program "colorBasics" from Kinect for Windows Developer Toolkit, the camera works fine, but freezes after a couple of seconds.
I copied over the relevant code ( so only the code for viewing the camera ) and it also happens in my own program.
Anyone know what I'm doing wrong ?
I don't get any error's.
Here is the code
namespace Testapp
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
private KinectSensor sensor;
private WriteableBitmap colorBitmap;
private byte[] colorPixels;
public MainWindow()
{
InitializeComponent();
}
private void WindowLoaded(object sender, RoutedEventArgs e)
{
foreach (var potentialSensor in KinectSensor.KinectSensors)
{
if (potentialSensor.Status == KinectStatus.Connected)
{
this.sensor = potentialSensor;
break;
}
}
if (null != this.sensor)
{
// Turn on the color stream to receive color frames
this.sensor.ColorStream.Enable(ColorImageFormat.RgbResolution640x480Fps30);
// Allocate space to put the pixels we'll receive
this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength];
// This is the bitmap we'll display on-screen
this.colorBitmap = new WriteableBitmap(this.sensor.ColorStream.FrameWidth, this.sensor.ColorStream.FrameHeight, 96.0, 96.0, PixelFormats.Bgr32, null);
// Set the image we display to point to the bitmap where we'll put the image data
this.Image.Source = this.colorBitmap;
// Add an event handler to be called whenever there is new color frame data
this.sensor.ColorFrameReady += this.SensorColorFrameReady;
// Start the sensor!
try
{
this.sensor.Start();
}
catch (IOException)
{
this.sensor = null;
}
}
}
private void SensorColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
{
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (colorFrame != null)
{
// Copy the pixel data from the image to a temporary array
colorFrame.CopyPixelDataTo(this.colorPixels);
// Write the pixel data into our bitmap
this.colorBitmap.WritePixels(
new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
this.colorPixels,
this.colorBitmap.PixelWidth * sizeof(int),
0);
}
}
}
}
}
Are you using any sort of extension with the USB cable, or is it plugged directly into the machine? I've seen what you are describing happen when the latency was too long between the sensor and the machine. In that instance it was caused by the Kinect USB cable being plugged into an extension cord.
Had same problem here, camera worked untill eventually just freezed and powered off, coming back after a few seconds only to repeat the freeze loop.
After many tests, my conclusion is that the issue is caused by three things:
PC is not fast/powerful as it should be to run your code.
Kinect is getting too hot. Even if you touch it and it's "not so hot", the sensor is very sensitive to overheat.
The Kinect is being "disturbed" somehow. This refers to vibrations or motion physically and/or too many things in the image that resembles a human, so the soft is trying it to compute it at every frame at 30fps, this is a lot of calculus, and this fact might lead to the two other problems listed above.
This also also may cause the latency issue described by Michal
I'm using XAudio2 with SlimDX and I've managed to get it playing a short (~8second) wav file on loop, however as it approaches the end of the first loop, the audio begins to stutter and the stuttering continues into the next loop, getting worse and worse as time goes on.
I turned on the debug profile and in the output window I get these errors:
XAUDIO2: WARNING: Spent 5.63ms in the OnVoiceProcessingPassStart callback
XAUDIO2: WARNING: Spent 5.60ms in the OnVoiceProcessingPassStart callback
XAUDIO2: WARNING: Spent 5.59ms in the OnVoiceProcessingPassStart callback
XAUDIO2: WARNING: Spent 5.69ms in the OnVoiceProcessingPassStart callback
And these coincide with when the stuttering occurs. I am doing nothing in these callbacks (I haven't even added anything to the events), and yet it's slowing down. I've added my code below for reference:
Wave class for holding the data stream and the buffer:
public class Wave
{
public WaveStream Data { get; private set; }
public AudioBuffer Buffer { get; private set; }
public Wave(string path, bool repeating)
{
Data = new WaveStream(path);
Buffer = new AudioBuffer();
Buffer.AudioBytes = (int)Data.Length;
Buffer.AudioData = Data;
if (repeating)
{
Buffer.Flags = BufferFlags.EndOfStream;
}
else
{
Buffer.Flags = BufferFlags.None;
}
Buffer.PlayBegin = 0;
Buffer.PlayLength = 0;
Buffer.LoopBegin = 0;
Buffer.LoopCount = 100;
Buffer.LoopLength = 0;
}
}
Sound class for holding the XAudio engine and the voices, and to cover adding/removing voices:
public class Sound
{
private XAudio2 audio;
private MasteringVoice master;
private List<SourceVoice> sources;
public Sound()
{
audio = new XAudio2(XAudio2Flags.DebugEngine, ProcessorSpecifier.AnyProcessor);
master = new MasteringVoice(audio);
sources = new List<SourceVoice>();
}
public void AddSound(Wave wave)
{
SlimDX.Multimedia.WaveFormat format = wave.Data.Format;
SourceVoice source = new SourceVoice(audio, format);
source.Start();
source.SubmitSourceBuffer(wave.Buffer);
sources.Add(source);
}
}
And to run it, I use:
Wave wave = new Wave("Music2/untitled.wav", true);
Sound sound = new Sound();
sound.AddSound(wave);
Rather embarrassingly this one's my own fault. I had a D3D buffer being recreated and destroyed every frame that I'd forgotten to change to a dynamic buffer. This was causing my memory usage to balloon to several gigabytes in size, which meant there probably wasn't enough to allocate for the music.
Once I removed the memory leak, the music sounded fine.
I'm not sure if the XAudio2 exception is very well-worded though...
General purpose
I am developing a .NET user control that can be used for multipurpose graphics rendering using Direct2D. This control is based on the ID2D1HwndRenderTarget interface with a ID2D1BitmapRenderTarget back-buffer/double-buffer.
Background
I have written a wrapper using C++/CLI that exposes a set of managed wrapper objects for Direct2D, and I have a designed a Singleton resource manager for Direct2D bitmaps that code can send to the binary data to, and get a resource key back from, which code can then pass in a message to the user control to render a bitmap.
Usage so far
I have been able to load bitmaps, JPEGs, custom imaging formats and the like and send them to the Direct2D render control all with no issue. I've written a custom video decoder that could load frames from a 15-fps video (decoding in a background thread) and render them from events raised by a Win32 multimedia timer, no problem.
Issue
My issue is that when I attempted to expand out from a single multimedia format into something more flexible (specifically connecting to LibAV/ffmpeg), the Direct2D render control starts to crash display drivers. This does not happen when frames are rendered sequentially (using a button to render the next frame, rather than a timer). It also does not happen straightaway. It also does not happen when I block inside the Win32 timer callback, but does when I use a Mutex to raise a thread that will free the timer callback up and let the interim thread instruct the control to render.
Symptoms
If I start a timer and attach to its elapsed event methods that will cause the next render, it will typically play fine for around 2-20 seconds. Using smaller video, I can get longer playback before the issue starts. If I play back 1080p video, I can get the crash usually within 5 seconds without contest. Playback will start fine, maybe have a hiccup here or there before catching up.
Eventually, I will start to see flickering, as if the background color is rendered but the frame is not. Additional frames may or may not later render, but this is very short-lived.
After the flicker, if I stop the timer quickly enough there will be no crash.
If I do not stop in time:
If I'm lucky (maybe 10% of the time), frames will stop rendering, be stuck on one frame until I resize the control, at which point only black is drawn to the entire control. (I've read that this may indicate a lost device context for DirectX rendering, but have not seen much more.)
If I'm unlucky, the display driver will crash, Windows recovers and restarts the driver, after which point the EndDraw will finally tell me that an error has occurred and return D2DERR_RECREATE_TARGET.
If I am really unlucky, the display will start to look like a kaleidoscope and I'll have to power off my machine, and I just wasted 5 minutes booting, logging on, loading Visual Studio and my solution, loading the video and lost all my debugging data.
I want to think that there is some sort of race condition that I am missing, but every time I run through the rendering code, it appears that it should be properly locked.
Render Control's Code
using System;
using System.Drawing;
using System.Windows.Forms;
using Direct2D = Bardez.Projects.DirectX.Direct2D;
using ExternalPixelEnums = Bardez.Projects.FileFormats.MediaBase.Video.Pixels.Enums;
using Bardez.Projects.DirectX.Direct2D;
using Bardez.Projects.FileFormats.MediaBase.Video;
using Bardez.Projects.FileFormats.MediaBase.Video.Enums;
using Bardez.Projects.Win32;
namespace Bardez.Projects.Output.Visual
{
/// <summary>Represents a rendering target for Direct2D.</summary>
/// <remarks>
/// To use this control, an external component will need to supply bitmap data. This does not need to be a GDI+ Bitmap class.
/// However, the container for this control will need to push data into this control.
/// So, in the case of a movie player, we'd see the following model:
/// * decompress a frame.
/// * push frame to control
/// * invoke Invalidate
/// * control will render the bitmap
/// * sleep just a little bit
/// * go back to first step
/// </remarks>
public class Direct2dRenderControl : VisualRenderControl
{
/*
* Locking orientation:
* There are two rendering targets: a GDI display and a bitmap back buffer.
* There are 5 'real' locking operations:
* Rendering to the GDI display (OnPaint)
* Rendering to the back buffer (DrawBitmapToBuffer, DiscardCurrentBuffer)
* Setting the current displayed frame (SetRenderFrame)
* Resource Freeing (FreeFrameResource)
* Resizing (OnResize)
*
* Briefly, the overarching effects of these five are:
* Rendering
* Utilizes the buffer and the display
* Back Buffer
* Utilizes the buffer and if mid-render could affect this display
* Set Frame
* Sets the buffer's displayed image
* Resource Freeing
* Affects the buffer if a resource is refernced
* Resizing
* Resizes the render control and the bitmap, uses the frame set by set
*
* Locking plan:
* Resize should block set, free and back buffer
* Render should block back buffer, control
* Set frame should block resize and back buffer
* Free should block back buffer,
* Back buffer should block rendering, setting, resizing, freeing
*
* Basically, lock everything at the process level, and not at the access level.
*/
#region Fields
/// <summary>Represents a Win32 HWND render target for Direct2D</summary>
private ControlRenderTarget ctrlRenderTarget;
/// <summary>Represents a drawing buffer</summary>
private BitmapRenderTarget bmpRenderTarget;
/// <summary>Represents a drawing buffer, used solely to create bitmaps</summary>
private BitmapRenderTarget resourceBmpRenderTarget;
/// <summary>Represents a buffer drawing command lock</summary>
private Object controlBufferLock;
/// <summary>Represents a paint/render drawing command lock</summary>
private Object controlPaintRenderLock;
/// <summary>Locking object reference for resource management (memory bitmaps, etc.)</summary>
private Object resourceLock;
/// <summary>Represents the key to accessing the currently set key</summary>
protected Int32 currentFrameKey;
#endregion
#region Properties
/// <summary>Indicates whether there is a frame set for this control</summary>
protected Boolean HasFrameSet
{
get { return currentFrameKey > -1; }
}
/// <summary>Exposes a wrapper for the bitmap render target</summary>
protected BitmapRenderTarget BmpRenderTarget
{
get { return this.bmpRenderTarget; }
set
{
lock (this.controlBufferLock)
{
if (this.bmpRenderTarget != null)
this.bmpRenderTarget.Dispose();
this.bmpRenderTarget = value;
}
}
}
/// <summary>Exposes a wrapper for the bitmap render target</summary>
protected BitmapRenderTarget ResourceBmpRenderTarget
{
get { return this.resourceBmpRenderTarget; }
set
{
lock (this.resourceLock)
{
if (this.resourceBmpRenderTarget != null)
this.resourceBmpRenderTarget.Dispose();
this.resourceBmpRenderTarget = value;
}
}
}
/// <summary>Represents a Win32 HWND render target for Direct2D</summary>
protected ControlRenderTarget CtrlRenderTarget
{
get { return this.ctrlRenderTarget; }
set
{
lock (this.controlPaintRenderLock)
{
if (this.ctrlRenderTarget != null)
this.ctrlRenderTarget.Dispose();
this.ctrlRenderTarget = value;
}
}
}
#endregion
#region Construction
/// <summary>Default constructor</summary>
public Direct2dRenderControl() : base()
{
this.controlBufferLock = new Object();
this.controlPaintRenderLock = new Object();
this.resourceLock = new Object();
this.currentFrameKey = -1;
this.InitializeControlDirect2D();
}
/// <summary>Initializes the Direct2D</summary>
protected void InitializeControlDirect2D()
{
//disable Windows background draw
this.SetStyle(ControlStyles.UserPaint | ControlStyles.AllPaintingInWmPaint, true);
// Build options on the Control render target
PixelFormat format = new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.Unknown); //32-bit color, pure alpha
DpiResolution res = Direct2dResourceManager.Instance.Factory.GetDesktopDpi();
RenderTargetProperties rtProp = new RenderTargetProperties(RenderTargetType.Default, format, res, RenderTargetUsage.GdiCompatible, DirectXVersion.DirectX9);
//Build out control render target properties
HwndRenderTargetProperties hwndProp = new HwndRenderTargetProperties(this.Handle, new SizeU(this.Size), PresentationOptions.RetainContents);
lock (this.controlPaintRenderLock)
{
// populate the Control rendering target
ResultCode result = Direct2dResourceManager.Instance.Factory.CreateHwndRenderTarget(rtProp, hwndProp, out this.ctrlRenderTarget);
lock (this.controlBufferLock)
{
// create a bitmap rendering targets
this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.bmpRenderTarget);
lock (this.resourceLock)
this.CtrlRenderTarget.CreateCompatibleRenderTarget(out this.resourceBmpRenderTarget);
}
}
}
#endregion
#region Destruction
/// <summary>Disposal code; releases unmanaged resources</summary>
/// <param name="disposing">True indicates to dispose managed resources</param>
protected override void Dispose(Boolean disposing)
{
this.ResourceBmpRenderTarget = null; //property disposes
this.BmpRenderTarget = null; //property disposes
this.CtrlRenderTarget = null; //property disposes
base.Dispose(disposing);
}
/// <summary>Disposal</summary>
~Direct2dRenderControl()
{
this.Dispose();
}
#endregion
#region Event Raising
/// <summary>Draws the output, then raises the paint event</summary>
/// <param name="e">Painting Event arguments</param>
protected override void OnPaint(PaintEventArgs e)
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
this.SuspendLayout();
this.OnPaintBackground(e);
Direct2D.Bitmap bmp;
ResultCode result;
result = this.BmpRenderTarget.GetBitmap(out bmp);
Direct2D.RectangleF rect = new Direct2D.RectangleF(e.ClipRectangle);
this.CtrlRenderTarget.BeginDraw();
this.CtrlRenderTarget.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);
result = this.CtrlRenderTarget.EndDraw();
bmp.Dispose();
if (result != ResultCode.Success_OK)
throw new ApplicationException(String.Format("Error encountered during draw: '{0}'", result.ToString()));
base.OnPaint(e);
this.ResumeLayout();
}
}
}
/// <summary>Overides the resize method</summary>
/// <param name="e">Parameters for the resize event</param>
protected override void OnResize(EventArgs e)
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
//Null check since resize fires before it is constructed, with the size event during parent's InitializeComponent
if (this.CtrlRenderTarget != null)
{
this.SuspendLayout();
//resize the existing control rendering target
this.CtrlRenderTarget.Resize(new SizeU(this.Size));
//I also need to resize the buffer, but can't. Instead, create a new one, then copy the existing one. Kind of lame.
this.ResizeBitmapRenderTarget();
base.OnResize(e);
//cause another draw
this.Invalidate(new Rectangle(new Point(0, 0), this.Size));
this.ResumeLayout();
}
}
}
}
/// <summary>Overridden Paint background method</summary>
/// <param name="e">Paint event arguments</param>
/// <remarks>
/// Made empty to avoid GDI and Direct2D writing to the same control; when moving the control around
/// or rendering at high speeds, the dreaded WinForms flicker was introduced.
/// The background painting can be found in the <see cref="FillBufferRenderTarget"/> method,
/// which fills the bitmap back buffer with the control's background color.
/// </remarks>
protected override void OnPaintBackground(PaintEventArgs e) { }
#endregion
#region Drawing
/// <summary>Resizes the bitmap rendering target buffer</summary>
/// <remarks>Does not lock the GDI render target. The OnResize or other callers lock instead.</remarks>
protected void ResizeBitmapRenderTarget()
{
lock (this.controlPaintRenderLock)
{
lock (this.controlBufferLock)
{
using (BitmapRenderTarget bmpCurr = this.BmpRenderTarget)
{
BitmapRenderTarget bmpNew;
ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);
this.DuplicateDoubleBufferContents(bmpNew);
//Property disposes and locks
this.BmpRenderTarget = bmpNew;
}
}
}
}
/// <summary>Draws a Bitmap to the render buffer</summary>
/// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp, Point2dF origin)
{
lock (this.controlBufferLock)
{
lock (this.resourceLock)
{
Direct2D.SizeF bmpSize = bmp.GetSize();
Single width = bmpSize.Width > this.BmpRenderTarget.Size.Width ? this.BmpRenderTarget.Size.Width : bmpSize.Width;
Single height = bmpSize.Height > this.BmpRenderTarget.Size.Height ? this.BmpRenderTarget.Size.Height : bmpSize.Height;
Direct2D.RectangleF destRect = new Direct2D.RectangleF(origin.X, origin.X + width, origin.Y, origin.Y + height);
Direct2D.RectangleF srcRect = new Direct2D.RectangleF(0.0F, width, 0.0F, height);
this.BmpRenderTarget.BeginDraw();
this.FillBufferRenderTarget(this.BmpRenderTarget);
// do the actual draw
this.BmpRenderTarget.DrawBitmap(bmp, destRect, 1.0F, BitmapInterpolationMode.Linear, srcRect);
//tell Direct2D that a paint operation is ending
ResultCode result = this.BmpRenderTarget.EndDraw();
}
}
}
/// <summary>Draws a Bitmap to the render buffer</summary>
/// <param name="bmp">Direct2D bitmap to draw to the buffer.</param>
protected void DrawBitmapToBuffer(Direct2D.Bitmap bmp)
{
this.DrawBitmapToBuffer(bmp, new Point2dF(0.0F, 0.0F));
}
/// <summary>Duplicates the bitmap behind the existing rendering target, and drawing it to a new one, discarding the current and setting the new.</summary>
/// <remarks>Does not lock any references, as the outside method locks</remarks>
protected void DuplicateDoubleBufferContents(BitmapRenderTarget bmpNew)
{
Direct2D.Bitmap bmp = null;
ResultCode result = ResultCode.Success_OK;
if (this.HasFrameSet)
bmp = Direct2dResourceManager.Instance.GetBitmapResource(this.currentFrameKey);
else
result = this.BmpRenderTarget.GetBitmap(out bmp);
bmpNew.BeginDraw();
this.FillBufferRenderTarget(bmpNew);
//calculate the size to copy
Direct2D.SizeF bmpSize = bmp.GetSize();
Single width = bmpSize.Width > this.CtrlRenderTarget.Size.Width ? this.CtrlRenderTarget.Size.Width : bmpSize.Width;
Single height = bmpSize.Height > this.CtrlRenderTarget.Size.Height ? this.CtrlRenderTarget.Size.Height : bmpSize.Height;
//Determine the copy rectangle
Direct2D.RectangleF rect = new Direct2D.RectangleF(0, width, 0, height);
//Copy
bmpNew.DrawBitmap(bmp, rect, 1.0F, BitmapInterpolationMode.Linear, rect);
//conditionally disose the bitmap, don't if it is in the manager
if (!this.HasFrameSet)
bmp.Dispose();
result = bmpNew.EndDraw();
}
/// <summary>Discards the current buffer and replaces it with a blank one.</summary>
protected void DiscardCurrentBuffer()
{
lock (this.controlBufferLock)
{
BitmapRenderTarget bmpNew;
// create a bitmap rendering target
ResultCode result = this.CtrlRenderTarget.CreateCompatibleRenderTarget(out bmpNew);
bmpNew.BeginDraw();
this.FillBufferRenderTarget(bmpNew);
result = bmpNew.EndDraw();
//property locks, so no lock here
this.BmpRenderTarget = bmpNew; //replace the old buffer
}
}
/// <summary>Fills the buffer render target with the background color</summary>
/// <param name="renderTarget">Bitmap render target to fill</param>
protected void FillBufferRenderTarget(BitmapRenderTarget renderTarget)
{
SolidColorBrush brush = null;
ResultCode result = renderTarget.CreateSolidColorBrush(new ColorF(this.BackColor), out brush);
renderTarget.FillRectangle(new Direct2D.RectangleF(new Rectangle(new Point(0, 0), this.Size)), brush);
brush.Dispose();
}
#endregion
#region Frame Resources
/// <summary>Posts a Frame resource to the resource manager and returns a unique key to access it.</summary>
/// <param name="resource">Frame to be posted.</param>
/// <returns>A unique Int32 key</returns>
public override Int32 AddFrameResource(Frame resource)
{
lock (this.resourceLock)
{
//create the bitmap
BitmapProperties properties = new BitmapProperties(new PixelFormat(DXGI_ChannelFormat.FORMAT_B8G8R8A8_UNORM, AlphaMode.PreMultiplied), Direct2dResourceManager.Instance.Factory.GetDesktopDpi());
SizeU dimensions = new SizeU(Convert.ToUInt32(resource.Pixels.Metadata.Width), Convert.ToUInt32(resource.Pixels.Metadata.Height));
Direct2D.Bitmap bmp = null;
ResultCode result = this.ResourceBmpRenderTarget.CreateBitmap(dimensions, properties, out bmp);
Byte[] data = resource.Pixels.GetPixelData(ExternalPixelEnums.PixelFormat.RGBA_B8G8R8A8, ScanLineOrder.TopDown, 0, 0);
result = bmp.CopyFromMemory(new RectangleU(dimensions), data, Convert.ToUInt32(resource.Pixels.Metadata.Width * 4));
return Direct2dResourceManager.Instance.AddFrameResource(bmp);
}
}
/// <summary>Frees a Bitmap resource in the resource manager and Disposes of it.</summary>
/// <param name="frameKey">Direct2D Bitmap key to be Disposed.</param>
public override void FreeFrameResource(Int32 frameKey)
{
lock (this.resourceLock)
{
if (frameKey > -1)
Direct2dResourceManager.Instance.FreeFrameResource(frameKey);
}
}
#endregion
#region Frame Setting
/// <summary>Sets the frame to be rendered to the User Control</summary>
/// <param name="key">Frame key to set as current image</param>
public override void SetRenderFrame(Int32 key)
{
this.SetRenderFrame(key, 0, 0);
}
/// <summary>Sets the frame to be rendered to the User Control</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
public override void SetRenderFrame(Int32 key, Int64 originX, Int64 originY)
{
lock (this.controlBufferLock)
{
if (key > -1)
this.DrawBitmapToBuffer(Direct2dResourceManager.Instance.GetBitmapResource(key), new Point2dF(Convert.ToSingle(originX), Convert.ToSingle(originY)));
else
this.DiscardCurrentBuffer();
this.currentFrameKey = key;
}
}
/// <summary>This method invokes the rendering. For use by the appliation to tell the control to change images on demand.</summary>
public void Render()
{
this.Invalidate();
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
public virtual void SetRenderFrameAndRender(Int32 key)
{
this.SetRenderFrameAndRender(key, false);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="freePreviousFrame">
/// Flag indicating whether to dispose of the previous image set
/// (in the case of transient images, such as composite images for a game or high-frame video playback)
/// </param>
public virtual void SetRenderFrameAndRender(Int32 key, Boolean freePreviousFrame)
{
this.SetRenderFrameAndRender(key, 0, 0, freePreviousFrame);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY)
{
this.SetRenderFrameAndRender(key, originX, originY, false);
}
/// <summary>Sets the frame to be rendered to the User Control and then renders it</summary>
/// <param name="key">Frame key to set as current image</param>
/// <param name="originX">X coordinate to start drawing from</param>
/// <param name="originY">Y coordinate to start drawing from</param>
/// <param name="freePreviousFrame">
/// Flag indicating whether to dispose of the previous image set
/// </param>
public virtual void SetRenderFrameAndRender(Int32 key, Int64 originX, Int64 originY, Boolean freePreviousFrame)
{
lock (this.controlBufferLock)
{
Int32 previousFrameKey = this.currentFrameKey;
this.SetRenderFrame(key, originX, originY);
this.Render();
if (freePreviousFrame)
this.FreeFrameResource(previousFrameKey);
}
}
#endregion
}
}
Win32 Multimedia Timer Callback's Code
Excluded for size, but here is the relevant code, linking to "winmm.dll", EntryPoint = "timeSetEvent"
/// <summary>Callback method for WIn32 API</summary>
protected virtual void Win32Callback(UInt32 timerId, UInt32 message, IntPtr user, IntPtr param1, IntPtr param2)
{
TimeSpan raise;
lock (this.timerLock)
{
//get system time for start time
UInt32 uptime = Win32MultimediaTimeFunctions.GetEnvironmentUptime();
Int32 upTimeSec = (Int32)(uptime / 1000);
Int32 upTimeMilliSec = (Int32)(uptime % 1000);
TimeSpan WinUpTime = new TimeSpan(0, 0, 0, upTimeSec, upTimeMilliSec);
raise = WinUpTime - this.StartTime;
//this.elapsed(raise);
this.RaiseTimer(raise);
}
//Note: outside the lock, this.elapse(raise) kills the display driver
//inside, without threading, it is fine.
//inside, with threading, the display driver will crash after a bit of playback
}
protected void RaiseTimer(TimeSpan elapsedTime)
{
Thread timerEvent = new Thread(() => { this.RunThread(elapsedTime); });
timerEvent.IsBackground = true;
timerEvent.Name = "Timer callback";
timerEvent.Start();
}
/// <summary>Raises the timer on a separate thread</summary>
protected void RunThread(TimeSpan elapsedTime)
{
if (this.threadLock.WaitOne(0))
{
this.elapsed(elapsedTime);
this.threadLock.ReleaseMutex();
}
}
Conclusion
After 3 weeks, I'm not ashamed to say I'm baffled by this behavior, and looking for any sort of input for identifying a race condition, something that I might be missing, architecturally speaking, about Direc2D or where I'm just showing my ignorance.
All locks statements in the code seems to be an over complicated solution. Also creating COM objects (like Direct2D1) from constructors and using them from thread UI methods is usually not a good practice.
I would simplify your application by performing all device/objects creation and rendering on the same thread, also by keeping your rendering code outside the form. Ideally, this should be totally independent, as you could render to any kind of windows/controls via the HWND or to a DXGI surface, or to a Windows 8 Metro surface.
Also, as Aren suggest, instead of using your own Direct2D1 wrapper, you should rely on an existing robust wrapper like SharpDX. It will save you to maintain a costly layer and help you to focus on your specific application.