We Keep Coding

MAUI: How to draw image from byte[] on MAUI

I'm try to use maui develop an application now, but I meet a problem:
MAUI document just offers the way to draw the image from a file, link is here: https://learn.microsoft.com/en-us/dotnet/maui/user-interface/graphics/draw?view=net-maui-7.0#draw-an-image
In my scenario, I will get the yuv/rgb byte[], not the image file, but the PlatformImage just can create the IImage object from an image file that has been encoded. And ICanvas just needs to use IImage object to draw an image.
And PlatformImage isn't support Windows.
using Microsoft.Maui.Graphics.Platform;
...
IImage image;
Assembly assembly = GetType().GetTypeInfo().Assembly;
using (Stream stream = assembly.GetManifestResourceStream("GraphicsViewDemos.Resources.Images.dotnet_bot.png"))
{
image = PlatformImage.FromStream(stream);
}
if (image != null)
{
canvas.DrawImage(image, 10, 10, image.Width, image. Height);
}
So, my question is, is there any way to draw the byte[] into View?
I have tried Microsoft.Maui.Graphics.Skia, but it cannot work.
This is my codes:
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Reflection.Metadata;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Maui.Graphics.Skia;
using Render.Source;
using SkiaSharp;
namespace MauiSample
{
internal class VideoDrawable : IDrawable, IDisposable
{
private SKBitmap? _sKBitmap;
private SkiaImage? _skiaImage;
private Microsoft.Maui.Graphics.IImage? _image;
private uint _width;
private uint _height;
private byte[]? _buffer;
public void Draw(ICanvas canvas, RectF dirtyRect)
{
if (_image is null)
{
return;
}
canvas.DrawImage(_image, dirtyRect.X, dirtyRect.Y, dirtyRect.Width, dirtyRect.Height);
}
private void EnsureBitmap(uint width, uint height)
{
bool needResize = false;
if (_width != width)
{
needResize = true;
_width = width;
}
if (_height != height)
{
needResize = true;
_height = height;
}
if (_sKBitmap is null)
{
_sKBitmap = new SKBitmap((int)width, (int)height, SKColorType.Rgb888x, SKAlphaType.Premul);
}
else if (needResize)
{
_sKBitmap.Resize(new SKImageInfo((int)width, (int)height), SKFilterQuality.None);
}
if (_skiaImage is null)
{
_skiaImage = new SkiaImage(_sKBitmap);
_image = _skiaImage;
}
else if (needResize)
{
_skiaImage.Resize(width, height);
}
if (needResize)
{
_buffer = new byte[width * height << 2];
}
}
public void DrawVideoFrame(byte[] yuvFrame, uint width, uint height)
{
EnsureBitmap(width, height);
byte[] pixels = new byte[width * height * 4];
VideoFrameConverter.YUV2RGBA(yuvFrame, pixels, width, height);
unsafe
{
fixed (byte* p = pixels)
{
_sKBitmap.SetPixels((nint)p);
}
}
}
#region IDisposable
private bool _disposed = false;
// Use C# finalizer syntax for finalization code.
// This finalizer will run only if the Dispose method
// does not get called.
// It gives your base class the opportunity to finalize.
// Do not provide finalizer in types derived from this class.
~VideoDrawable()
{
// Do not re-create Dispose clean-up code here.
// Calling Dispose(disposing: false) is optimal in terms of
// readability and maintainability.
Dispose(disposing: false);
}
// Implement IDisposable.
// Do not make this method virtual.
// A derived class should not be able to override this method.
public void Dispose()
{
Dispose(disposing: true);
// This object will be cleaned up by the Dispose method.
// Therefore, you should call GC.SuppressFinalize to
// take this object off the finalization queue
// and prevent finalization code for this object
// from executing a second time.
GC.SuppressFinalize(this);
}
// Dispose(bool disposing) executes in two distinct scenarios.
// If disposing equals true, the method has been called directly
// or indirectly by a user's code. Managed and unmanaged resources
// can be disposed.
// If disposing equals false, the method has been called by the
// runtime from inside the finalizer and you should not reference
// other objects. Only unmanaged resources can be disposed.
protected virtual void Dispose(bool disposing)
{
// Check to see if Dispose has already been called.
if (!_disposed)
{
// If disposing equals true, dispose all managed
// and unmanaged resources.
if (disposing)
{
// Dispose managed resources.
_skiaImage.Dispose();
_sKBitmap.Dispose();
}
// Note disposing has been done.
_disposed = true;
}
}
#endregion
}
}

Memory will not release in C# singleton mode

I have a puzzle about singleton mode freeing object memory in C# between in C++;
Here C++ Code:
#include<iostream>
using namespace std;
class Rocket
{
private:
Rocket() { speed = 100; }
~Rocket() {}
static Rocket* ms_rocket;
public:
int speed;
static Rocket*ShareRocket()
{
if (ms_rocket == NULL)
ms_rocket = new Rocket();
return ms_rocket;
}
static void Close()
{
if (ms_rocket != NULL)
delete ms_rocket;
}
};
Rocket *Rocket::ms_rocket = NULL;
int main()
{
Rocket* p = Rocket::ShareRocket();
p->speed = 100;
cout << p->speed << endl;
Rocket::Close();
cout << p->speed << endl;
getchar();
}
When I use Rocket::Close(), the memory space which ms_rocket pointed to will be freed, ms_rocket become a wild pointer, and the second "cout<age<<endl" show is not 100, but when I use C# , I also use Dispose(), but still show 100. here the C# code:
class A : IDisposable
{
public int age;
public A() { }
public void Run()
{
Console.WriteLine("Run");
}
#region IDisposable Support
private bool disposedValue = false;
protected virtual void Dispose(bool disposing)
{
if (!disposedValue)
{
if (disposing)
{
Console.WriteLine("A is release");
}
disposedValue = true;
}
}
public void Dispose()
{
Dispose(true);
}
#endregion
}
class B
{
static A a;
private B() { }
public static A Intance
{
get
{
if (a == null)
a = new A();
return a;
}
}
}
class Class1
{
public static void Main(string[] args)
{
A a = B.Intance;
a.age =100;
Console.WriteLine(a.age);
a.Dispose();
A a1 = B.Intance;
Console.WriteLine(a1.age);
Console.Read();
}
}
In C#, I think when I use Dispose(), the memory('a' object in B singleton) will be released, but in the second access, the age value should not be 100, and the static variable 'a' will become like a wild pointer.
Who can tell me why?

In C# Dispose mainly is used to release unmanaged resources as soon as they are not needed and not to free the memory occupied by object itself - it is handled by garbage collector which will free (when GC will decide that it needs to run) it only when it will become unaccessible from so called GC roots (and static variables are one of the GC roots, so B.Intance will hold the reference to this instance of A in the heap).
So first of all to free the memory taken by current instance of A you will need to set B.Instance to null (and wait for GC to run).
Also fundamentals of garbage collection in CLR can be useful.

What determines the surface texture width and height for camera?

Here's a snippet from my implementation of my custom camera page.
For some reason, I keep getting a lower resolution textureView/SurfaceTexture height and width than I expect. I want to keep the native maximum camera resolution (or picture size) my native camera on my phone takes normally. However, with the way I have it setup right now, the resolution of my textureView is lower than the native camera resolution. I'm wondering why? I tried to debug and seems OnSurfaceTextureAvailable() seems to be setting the width and height incorrectly. Where is it grabbing it from?
public class CameraPage : PageRenderer, TextureView.ISurfaceTextureListener, Android.Views.View.IOnTouchListener
{
global::Android.Hardware.Camera camera;
Activity activity;
CameraFacing cameraType;
TextureView textureView;
SurfaceTexture surfaceTexture;
public void OnSurfaceTextureAvailable (SurfaceTexture surface, int width, int height)
{
GetCameraInstance();
if (camera != null) {
textureView.LayoutParameters = new FrameLayout.LayoutParams(width, height);
surfaceTexture = surface;
camera.SetPreviewTexture(surface);
PrepareAndStartCamera();
}
}
private void GetCameraInstance()
{
try {
camera = global::Android.Hardware.Camera.Open((int)CameraFacing.Back);
}
catch (Exception e) {
//ignore any exception
}
}
public bool OnSurfaceTextureDestroyed (SurfaceTexture surface)
{
StopCameraPreviewAndRelease();
return true;
}
private void StopCameraPreview()
{
try {
if (camera != null)
camera.StopPreview();
}
catch { }
}
private void StopCameraPreviewAndRelease()
{
try {
if (camera != null) {
StopCameraPreview();
camera.SetPreviewCallback(null);
camera.Release();
camera = null;
}
}
catch { }
}
public void OnSurfaceTextureSizeChanged (SurfaceTexture surface, int width, int height)
{
PrepareAndStartCamera ();
}
public void OnSurfaceTextureUpdated (SurfaceTexture surface)
{
}
private void PrepareAndStartCamera ()
{
var flashMode = GetFlashMode();
SetCameraParameters(flashMode);
StopCameraPreview();
var display = activity.WindowManager.DefaultDisplay;
if (display.Rotation == SurfaceOrientation.Rotation0) {
camera.SetDisplayOrientation (90);
}
if (display.Rotation == SurfaceOrientation.Rotation270) {
camera.SetDisplayOrientation (180);
}
if (flashOn)
toggleFlashButton.SetBackgroundResource(Resource.Drawable.flash_on);
else
toggleFlashButton.SetBackgroundResource(Resource.Drawable.flash_off);
camera.StartPreview ();
}
}
This is how I'm setting my textureView:
textureView = view.FindViewById<TextureView> (Resource.Id.textureView);
textureView.SurfaceTextureListener = this;
textureView in my cameraLayout:
<TextureView
android:id="#+id/textureView"
android:layout_marginTop="-110dp"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:backgroundTint="#99b4d1ff"
android:layout_marginLeft="0dp" />
That is, I'm expecting 2576x1932 resolution when the photo is taken and saved. But I am getting 1451x720 instead when the photo is taken and saved. Seems like that was determined to be the textureView size (but I want the native camera resolution size).
EDIT: Here's the way the photo is being taken of:
private async void TakePhotoButtonTapped (object sender, EventArgs e)
{
try{
try
{
StopCameraPreview();
}
catch (Exception ex) {
camera.Reconnect();
PrepareAndStartCamera();
StopCameraPreview();
}
var image = textureView.Bitmap;
var imageQuality = AppState.ApplicationInfo.AndroidImageCompressionFactor;
using (var imageStream = new MemoryStream ()) {
await image.CompressAsync(Bitmap.CompressFormat.Jpeg, imageQuality, imageStream);
image.Recycle();
imageBytes = imageStream.ToArray ();
}
count +=1;
textView.Text = Convert.ToString(count);
_images.Add(imageBytes);
camera.StartPreview ();
}
catch(Exception ex)
{
}
}

WPF Memory not released when called from ActiveX

I have a class that populates 1000 ComboBoxes into the UI, the code is
namespace ActiveXMemory
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
PopulateControls();
}
public void PopulateControls() {
var newGrid = new Grid();
for (var i = 0; i < 1000; i++)
{
ComboBox newcombo = new ComboBox();
newGrid.Children.Add(newcombo);
}
this.Content = newGrid;
}
public bool OpenWindow(bool isRunning)
{
Dispatcher.Invoke(DispatcherPriority.Normal, new Action(() =>
{
try
{
this.ShowDialog();
}
catch (Exception exp)
{
Console.WriteLine(exp.Message);
}
}));
return true;
}
public bool CloseWindow()
{
Dispatcher.Invoke(DispatcherPriority.Render, new Action(() =>
{
try
{
this.Close();
}
catch (Exception exp)
{
//Console.WriteLine(exp.Message);
}
}));
return true;
}
private void Window_Closed(object sender, EventArgs e)
{
var grid = Content as Grid;
var children = grid.Children;
while (children.Count > 0)
{
var child = children[0];
grid.Children.Remove(child);
child = null;
}
grid = null;
}
}
}
I created an ActiveX library for the class to be accessible as ActiveX,
namespace ActiveXLibrary
{
[ComVisible(true)]
[Guid("EF2EAD91-68A8-420D-B5C9-E30A6F510BDE")]
public interface IActiveXLib
{
[DispId(1)]
bool Initialize();
[DispId(2)]
bool CloseActiveX();
}
[ComVisible(true)]
[Guid("9DACD44F-0237-4F44-BCB9-0E6B729915D6"),
ClassInterface(ClassInterfaceType.None)]
[ProgId("Samp")]
public class ActiveXLib : IActiveXLib
{
MainWindow form;
Thread thr;
public bool Initialize()
{
int i = 0;
try
{
ThreadStart exeFunc = new ThreadStart(() =>
{
Dispatcher.CurrentDispatcher.Invoke(DispatcherPriority.Send, new Action(() =>
{
start();
}));
});
thr = new Thread(exeFunc);
thr.SetApartmentState(ApartmentState.STA);
thr.Start();
while (form == null)
{
i++;
//Console.WriteLine("form Null");
System.Threading.Thread.Sleep(1000);
if (i > 30)
break;
}
return true;
}
catch (Exception exp)
{
Console.WriteLine("[Initialize]" + exp.Message);
return false;
}
}
public bool CloseActiveX()
{
bool success = false;
try
{
success = form.CloseWindow();
form = null;
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced);
GC.WaitForPendingFinalizers();
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced);
return success;
}
catch (Exception exp)
{
Console.WriteLine("[CloseActiveX]" + exp.Message);
return false;
}
}
private void start()
{
try
{
Console.WriteLine("start() - new MainWindow()");
form = new MainWindow();
Console.WriteLine("OpenWindow()");
form.OpenWindow(true);
}
catch (Exception exp)
{
MessageBox.Show(exp.Message + "\nPossible Reasons: " + exp.Source + "\n" + exp.InnerException, "Error");
}
}
}
}
Then I created a demo project as
ActiveXLib activeXRef;
public MainWindow()
{
InitializeComponent();
}
private void startButton_Click(object sender, RoutedEventArgs e)
{
activeXRef = new ActiveXLib();
activeXRef.Initialize();
}
private void stopButton_Click(object sender, RoutedEventArgs e)
{
activeXRef.CloseActiveX();
GC.Collect();
}
Here the problem is every time I click the start and stop button, the memory keeps on increasing, the memory (17MB) created by the first start of the application did not get released, even though I removed the ComboBoxes and set the grid content null. Even the GC.Collect() has no effect, since it only adds the garbage to a queue. Is there any fix for this.
I also tried flushing the memory as in the link, But even then the memory is still held.(since I am not sure whether the handle of the current process is from demo project/from ActiveXMemory)
Edit
I included the line
this.Dispatcher.Invoke(DispatcherPriority.Render, GCDelegate);
to the Window_Closed event
Now the memory gets cleared
But one problem I am facing now is, if I try to immediately close (call CloseActiveX()), then this is not happening.(i.e)
private void startButton_Click(object sender, RoutedEventArgs e)
{
activeXRef = new ActiveXLib();
activeXRef.Initialize();
activeXRef.CloseActiveX();
activeXRef = null;
GC.Collect();
}
The above code still has the memory locked, I don't understand the difference, it is just another event,Does anyone have any idea on this?

I think I got a repro for this problem although I can't tell what your MainWindow.Open/CloseWindow() methods look like. Using a thread is certainly part of the problem, there is one non-intuitive thing you have to do to prevent leaking internal WPF plumbing objects that have thread affinity.
It is imperative to shutdown the dispatcher for the thread. This normally just happens once in a WPF app when the UI thread terminates. Also very important that the dispatcher does the dispatching, WPF heavily depends on it to ensure that "weak events" are actually weak.
A sample implementation for MainWindow that does not leak:
public class MainWindow : Window {
public void OpenWindow(bool noidea) {
this.ShowDialog();
}
public bool CloseWindow() {
this.Dispatcher.Invoke(() => {
this.Dispatcher.InvokeShutdown(); // <== Important!
});
return true;
}
}
Where ShowDialog() ensures the dispatcher does the dispatching and the InvokeShutdown() ensures that the cleanup occurs. Tested by using a 10 msec DispatcherTimer instead of buttons so this happened at a very high rate. I could remove the GC.Collect() calls and memory usage was stable.

At the moment I've no comment about the ActiveX part (will double check and let you know), but I can suggest something related to the WPF code.
This should make the combo items disposable
public class DispCombo : ComboBox, IDisposable
{
public void Dispose()
{
GC.SuppressFinalize(this);
}
}
public class DispGrid : Grid, IDisposable
{
public void Dispose()
{
GC.SuppressFinalize(this);
}
}
You will populate disposable controls
public void PopulateControls()
{
var newGrid = new DispGrid();
for (var i = 0; i < 1000; i++)
{
DispCombo newcombo = new DispCombo();
newGrid.Children.Add(newcombo);
}
this.Content = newGrid;
}
So the closing phase would become (disposing instead of setting to null)
private void Window_Closed(object sender, EventArgs e)
{
var grid = Content as DispGrid;
var children = grid.Children;
while (children.Count > 0)
{
var child = children[0] as DispCombo;
grid.Children.Remove(child);
child.Dispose();
}
grid.Dispose();
}
That said, now you can wrap the WPF view in a using clause
public class MainLibrary
{
public bool OpenWindow() //found no ref to bool isRunning
{
using (var mw = new MainWindow())
{
mw.ShowDialog();
mw.Close();
}
return true;
}
public bool CloseWindow()
{
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced);
GC.WaitForPendingFinalizers();
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced);
return true;
}
}
with your garbage collection in the close.
These are my memory snapshots after these changes, taken (1) at start, then (2) after the main ShowDialog in the OpenWindow and finally (3) after the CloseWindow
Edit
Finally, if you need to start a Task, calling the Dispatcher, here it is the function
public bool CloseWindow()
{
winClose = () =>
{
mw.Dispatcher.Invoke(() =>
{
mw.Close();
mw.Dispose();
});
};
cleanProc = () =>
{
mw.Dispatcher.Invoke(() =>
{
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced);
GC.WaitForPendingFinalizers();
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced);
});
};
Task.Run(() =>
{
winClose.Invoke();
}).ContinueWith(x =>
{
cleanProc.Invoke();
});
return true;
}
with
public bool OpenWindow()
{
mw = new MainWindow();
mw.Show();
return true;
}

Is there a way to fake a DirectShow filter in a program?

I have an IP Camera that receives a char buffer containing an image over the network. I cant access it until i setup the connection to it in a program. I am trying to dissect windows source filter code and im not going very fast so i thought i'd ask if it was possible to just take a buffer like that and cast it to something that could then connect a pin to AVISplitter or such in Directshow/.net
(video buffer from IP Cam) -> (???) -> (AVI Splitter) -> (Profit)
Update
I have my program capturing video in a namespace, and i have this code from the GSSF in its own namespace. I pass a ptr with an image from the cam namespace to the GSSF namespace. This only occurs once, but the graph streams from this one image, and the camera streams from the network. is there a way to continually pass the buffer from cam to GSSF or should i combine the namespaces somehow? I tried sending the main camera pointer to the GSSF but it crashed because its accessing the pointer and its being written. maybe if i grabbed an image, passed the pointer, waited to grab a new one?
*Update*
I shrunk my code and I don't believe im doing the namespace correctly either now that i look at it.
namespace Cam_Controller
{
static byte[] mainbyte = new byte[1280*720*2];
static IntPtr main_ptr = new IntPtr();
//(this function is threaded)
static void Trial(NPvBuffer mBuffer, NPvDisplayWnd mDisplayWnd, VideoCompression compressor)
{
Functions function = new Functions();
Defines define = new Defines();
NPvResult operationalResult = new NPvResult();
VideoCompression mcompressor = new VideoCompression();
int framecount = 0;
while (!Stopping && AcquiringImages)
{
Mutex lock_video = new Mutex();
NPvResult result = mDevice.RetrieveNextBuffer(mBuffer, operationalResult);
if(result.isOK())
{
framecount++;
wer = (int)mDisplayWnd.Display(mBuffer, wer);
main_ptr = (IntPtr)mBuffer.GetMarshalledBuffer();
Marshal.Copy(main_ptr, mainbyte, 0, 720 * 2560);
}
}
}
private void button7_Click(object sender, EventArgs e)
{
IntPtr dd = (IntPtr)mBuffer.GetMarshalledBuffer();
Marshal.Copy(dd, main_byte1, 0, 720 * 2560);
play = new VisiCam_Controller.DxPlay.DxPlay("", panel9, main_byte1);
play.Start();
}
namespace DxPlay
{
public class DxPlay
{
public DxPlay(string sPath, Control hWin, byte[] color)
{
try
{
// pick one of our image providers
//m_ImageHandler = new ImageFromFiles(sPath, 24);
m_ImageHandler = new ImageFromPixels(20, color);
//m_ImageHandler = new ImageFromMpg(#"c:\c1.mpg");
//m_ImageHandler = new ImageFromMpg(sPath);
//m_ImageHandler = new ImageFromMP3(#"c:\vss\media\track3.mp3");
// Set up the graph
SetupGraph(hWin);
}
catch
{
Dispose();
throw;
}
}
}
abstract internal class imagehandler
internal class imagefrompixels
{
private int[] mainint = new int[720 * 1280];
unsafe public ImageFromPixels(long FPS, byte[] x)
{
long fff = 720 * 1280 * 3;
mainptr = new IntPtr(fff);
for (int p = 0; p < 720 * 640; p++)
{
U = (x[ p * 4 + 0]);
Y = (x[p * 4 + 1]);
V = (x[p * 4 + 2]);
Y2 = (x[p * 4 + 3]);
int one = V << 16 | Y << 8 | U;
int two = V << 16 | Y2 << 8 | U;
mainint[p * 2 + 0] = one;
mainint[p * 2 + 1] = two;
}
m_FPS = UNIT / FPS;
m_b = 211;
m_g = 197;
}
}
}
}
Theres also GetImage but thats relatively the same, copy the buffer into the pointer. What happens is i grab a buffer of the image and send it to the DxPlay class. it is able to process it and put it on the directshow line no problems; but it never updates nor gets updated because its just a single buffer. If i instead send DxPlay a IntPtr holding the address of the image buffer, the code crashes for accessing memory because i assume ImageFromPixels code ( which isn't there now ( change
(x[p * 4 + #])
to
(IntPtr)((x-passed as an IntPtr).toInt64()+p*4 + #)
))
is accessing the memory of the pointer as the Cam_Controller class is editing it. I make and pass copies of the IntPtrs, and new IntPtrs but they fail halfway through the conversion.

If you want to do this in .NET, the following steps are needed:
Use the DirectShow.NET Generic Sample Source Filter (GSSF.AX) from the Misc/GSSF directory within the sample package. A source filter is always a COM module, so you need to register it too using "RegSvr32 GSSF.ax".
Implement a bitmap provider in .NET
Setup a graph, and connect the pin from the GSSF to the implementation of the bitmap provider.
Pray.
I am using the following within a project, and made it reusable for future usage.
The code (not the best, and not finished, but a working start) (this takes a IVideoSource, which is bellow):
public class VideoSourceToVideo : IDisposable
{
object locker = new object();
public event EventHandler<EventArgs> Starting;
public event EventHandler<EventArgs> Stopping;
public event EventHandler<EventArgs> Completed;
/// <summary> graph builder interface. </summary>
private DirectShowLib.ICaptureGraphBuilder2 captureGraphBuilder = null;
DirectShowLib.IMediaControl mediaCtrl = null;
IMediaEvent mediaEvent = null;
bool stopMediaEventLoop = false;
Thread mediaEventThread;
/// <summary> Dimensions of the image, calculated once in constructor. </summary>
private readonly VideoInfoHeader videoInfoHeader;
IVideoSource source;
public VideoSourceToVideo(IVideoSource source, string destFilename, string encoderName)
{
try
{
this.source = source;
// Set up the capture graph
SetupGraph(destFilename, encoderName);
}
catch
{
Dispose();
throw;
}
}
/// <summary> release everything. </summary>
public void Dispose()
{
StopMediaEventLoop();
CloseInterfaces();
}
/// <summary> build the capture graph for grabber. </summary>
private void SetupGraph(string destFilename, string encoderName)
{
int hr;
// Get the graphbuilder object
captureGraphBuilder = new DirectShowLib.CaptureGraphBuilder2() as DirectShowLib.ICaptureGraphBuilder2;
IFilterGraph2 filterGraph = new DirectShowLib.FilterGraph() as DirectShowLib.IFilterGraph2;
mediaCtrl = filterGraph as DirectShowLib.IMediaControl;
IMediaFilter mediaFilt = filterGraph as IMediaFilter;
mediaEvent = filterGraph as IMediaEvent;
captureGraphBuilder.SetFiltergraph(filterGraph);
IBaseFilter aviMux;
IFileSinkFilter fileSink = null;
hr = captureGraphBuilder.SetOutputFileName(MediaSubType.Avi, destFilename, out aviMux, out fileSink);
DsError.ThrowExceptionForHR(hr);
DirectShowLib.IBaseFilter compressor = DirectShowUtils.GetVideoCompressor(encoderName);
if (compressor == null)
{
throw new InvalidCodecException(encoderName);
}
hr = filterGraph.AddFilter(compressor, "compressor");
DsError.ThrowExceptionForHR(hr);
// Our data source
IBaseFilter source = (IBaseFilter)new GenericSampleSourceFilter();
// Get the pin from the filter so we can configure it
IPin ipin = DsFindPin.ByDirection(source, PinDirection.Output, 0);
try
{
// Configure the pin using the provided BitmapInfo
ConfigurePusher((IGenericSampleConfig)ipin);
}
finally
{
Marshal.ReleaseComObject(ipin);
}
// Add the filter to the graph
hr = filterGraph.AddFilter(source, "GenericSampleSourceFilter");
Marshal.ThrowExceptionForHR(hr);
hr = filterGraph.AddFilter(source, "source");
DsError.ThrowExceptionForHR(hr);
hr = captureGraphBuilder.RenderStream(null, null, source, compressor, aviMux);
DsError.ThrowExceptionForHR(hr);
IMediaPosition mediaPos = filterGraph as IMediaPosition;
hr = mediaCtrl.Run();
DsError.ThrowExceptionForHR(hr);
}
private void ConfigurePusher(IGenericSampleConfig ips)
{
int hr;
source.SetMediaType(ips);
// Specify the callback routine to call with each sample
hr = ips.SetBitmapCB(source);
DsError.ThrowExceptionForHR(hr);
}
private void StartMediaEventLoop()
{
mediaEventThread = new Thread(MediaEventLoop)
{
Name = "Offscreen Vid Player Medialoop",
IsBackground = false
};
mediaEventThread.Start();
}
private void StopMediaEventLoop()
{
stopMediaEventLoop = true;
if (mediaEventThread != null)
{
mediaEventThread.Join();
}
}
public void MediaEventLoop()
{
MediaEventLoop(x => PercentageCompleted = x);
}
public double PercentageCompleted
{
get;
private set;
}
// FIXME this needs some work, to be completely in-tune with needs.
public void MediaEventLoop(Action<double> UpdateProgress)
{
mediaEvent.CancelDefaultHandling(EventCode.StateChange);
//mediaEvent.CancelDefaultHandling(EventCode.Starvation);
while (stopMediaEventLoop == false)
{
try
{
EventCode ev;
IntPtr p1, p2;
if (mediaEvent.GetEvent(out ev, out p1, out p2, 0) == 0)
{
switch (ev)
{
case EventCode.Complete:
Stopping.Fire(this, null);
if (UpdateProgress != null)
{
UpdateProgress(source.PercentageCompleted);
}
return;
case EventCode.StateChange:
FilterState state = (FilterState)p1.ToInt32();
if (state == FilterState.Stopped || state == FilterState.Paused)
{
Stopping.Fire(this, null);
}
else if (state == FilterState.Running)
{
Starting.Fire(this, null);
}
break;
// FIXME add abort and stuff, and propagate this.
}
// Trace.WriteLine(ev.ToString() + " " + p1.ToInt32());
mediaEvent.FreeEventParams(ev, p1, p2);
}
else
{
if (UpdateProgress != null)
{
UpdateProgress(source.PercentageCompleted);
}
// FiXME use AutoResetEvent
Thread.Sleep(100);
}
}
catch (Exception e)
{
Trace.WriteLine("MediaEventLoop: " + e);
}
}
}
/// <summary> Shut down capture </summary>
private void CloseInterfaces()
{
int hr;
try
{
if (mediaCtrl != null)
{
// Stop the graph
hr = mediaCtrl.Stop();
mediaCtrl = null;
}
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
if (captureGraphBuilder != null)
{
Marshal.ReleaseComObject(captureGraphBuilder);
captureGraphBuilder = null;
}
GC.Collect();
}
public void Start()
{
StartMediaEventLoop();
}
}
IVideoSource:
public interface IVideoSource : IGenericSampleCB
{
double PercentageCompleted { get; }
int GetImage(int iFrameNumber, IntPtr ip, int iSize, out int iRead);
void SetMediaType(global::IPerform.Video.Conversion.Interops.IGenericSampleConfig psc);
int SetTimeStamps(global::DirectShowLib.IMediaSample pSample, int iFrameNumber);
}
ImageVideoSource (mostly taken from DirectShow.NET examples):
// A generic class to support easily changing between my different sources of data.
// Note: You DON'T have to use this class, or anything like it. The key is the SampleCallback
// routine. How/where you get your bitmaps is ENTIRELY up to you. Having SampleCallback call
// members of this class was just the approach I used to isolate the data handling.
public abstract class ImageVideoSource : IDisposable, IVideoSource
{
#region Definitions
/// <summary>
/// 100 ns - used by a number of DS methods
/// </summary>
private const long UNIT = 10000000;
#endregion
/// <summary>
/// Number of callbacks that returned a positive result
/// </summary>
private int m_iFrameNumber = 0;
virtual public void Dispose()
{
}
public abstract double PercentageCompleted { get; protected set; }
abstract public void SetMediaType(IGenericSampleConfig psc);
abstract public int GetImage(int iFrameNumber, IntPtr ip, int iSize, out int iRead);
virtual public int SetTimeStamps(IMediaSample pSample, int iFrameNumber)
{
return 0;
}
/// <summary>
/// Called by the GenericSampleSourceFilter. This routine populates the MediaSample.
/// </summary>
/// <param name="pSample">Pointer to a sample</param>
/// <returns>0 = success, 1 = end of stream, negative values for errors</returns>
virtual public int SampleCallback(IMediaSample pSample)
{
int hr;
IntPtr pData;
try
{
// Get the buffer into which we will copy the data
hr = pSample.GetPointer(out pData);
if (hr >= 0)
{
// Set TRUE on every sample for uncompressed frames
hr = pSample.SetSyncPoint(true);
if (hr >= 0)
{
// Find out the amount of space in the buffer
int cbData = pSample.GetSize();
hr = SetTimeStamps(pSample, m_iFrameNumber);
if (hr >= 0)
{
int iRead;
// Get copy the data into the sample
hr = GetImage(m_iFrameNumber, pData, cbData, out iRead);
if (hr == 0) // 1 == End of stream
{
pSample.SetActualDataLength(iRead);
// increment the frame number for next time
m_iFrameNumber++;
}
}
}
}
}
finally
{
// Release our pointer the the media sample. THIS IS ESSENTIAL! If
// you don't do this, the graph will stop after about 2 samples.
Marshal.ReleaseComObject(pSample);
}
return hr;
}
}
RawVideoSource (an example of a concrete managed source generator for a DirectShow pipeline):
internal class RawVideoSource : ImageVideoSource
{
private byte[] buffer;
private byte[] demosaicBuffer;
private RawVideoReader reader;
public override double PercentageCompleted
{
get;
protected set;
}
public RawVideoSource(string sourceFile)
{
reader = new RawVideoReader(sourceFile);
}
override public void SetMediaType(IGenericSampleConfig psc)
{
BitmapInfoHeader bmi = new BitmapInfoHeader();
bmi.Size = Marshal.SizeOf(typeof(BitmapInfoHeader));
bmi.Width = reader.Header.VideoSize.Width;
bmi.Height = reader.Header.VideoSize.Height;
bmi.Planes = 1;
bmi.BitCount = 24;
bmi.Compression = 0;
bmi.ImageSize = (bmi.BitCount / 8) * bmi.Width * bmi.Height;
bmi.XPelsPerMeter = 0;
bmi.YPelsPerMeter = 0;
bmi.ClrUsed = 0;
bmi.ClrImportant = 0;
int hr = psc.SetMediaTypeFromBitmap(bmi, 0);
buffer = new byte[reader.Header.FrameSize];
demosaicBuffer = new byte[reader.Header.FrameSize * 3];
DsError.ThrowExceptionForHR(hr);
}
long startFrameTime;
long endFrameTime;
unsafe override public int GetImage(int iFrameNumber, IntPtr ip, int iSize, out int iRead)
{
int hr = 0;
if (iFrameNumber < reader.Header.NumberOfFrames)
{
reader.ReadFrame(buffer, iFrameNumber, out startFrameTime, out endFrameTime);
Demosaic.DemosaicGBGR24Bilinear(buffer, demosaicBuffer, reader.Header.VideoSize);
Marshal.Copy(demosaicBuffer, 0, ip, reader.Header.FrameSize * 3);
PercentageCompleted = ((double)iFrameNumber / reader.Header.NumberOfFrames) * 100.0;
}
else
{
PercentageCompleted = 100;
hr = 1; // End of stream
}
iRead = iSize;
return hr;
}
override public int SetTimeStamps(IMediaSample pSample, int iFrameNumber)
{
reader.ReadTimeStamps(iFrameNumber, out startFrameTime, out endFrameTime);
DsLong rtStart = new DsLong(startFrameTime);
DsLong rtStop = new DsLong(endFrameTime);
int hr = pSample.SetTime(rtStart, rtStop);
return hr;
}
}
And the interops to the GSSF.AX COM:
namespace IPerform.Video.Conversion.Interops
{
[ComImport, Guid("6F7BCF72-D0C2-4449-BE0E-B12F580D056D")]
public class GenericSampleSourceFilter
{
}
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown),
Guid("33B9EE57-1067-45fa-B12D-C37517F09FC0")]
public interface IGenericSampleCB
{
[PreserveSig]
int SampleCallback(IMediaSample pSample);
}
[Guid("CE50FFF9-1BA8-4788-8131-BDE7D4FFC27F"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IGenericSampleConfig
{
[PreserveSig]
int SetMediaTypeFromBitmap(BitmapInfoHeader bmi, long lFPS);
[PreserveSig]
int SetMediaType([MarshalAs(UnmanagedType.LPStruct)] AMMediaType amt);
[PreserveSig]
int SetMediaTypeEx([MarshalAs(UnmanagedType.LPStruct)] AMMediaType amt, int lBufferSize);
[PreserveSig]
int SetBitmapCB(IGenericSampleCB pfn);
}
}
Good luck, try to get it working using this. Or comment with further questions so we can iron out other issues.