We Keep Coding

Convert VLC video stream to bitmap for xzing to decode using C#

I need to convert a VLC video stream (working) in my WPF app to a bmp for Xzing to be able to decode,
I can decode using a usb or the onboard webcam.
But how to get the 2 together??
Been searching all over and have tried some solutions but nothing seems to work, might be me as i'm pretty new to C#
Thanks
I tried this piece of code:
public static void Get_Frame(Vlc.DotNet.Forms.VlcControl vlc)
{
try
{
if (File.Exists(Temp_Frame_Filename))
{
File.Delete(Temp_Frame_Filename);
}
vlc.TakeSnapshot(Temp_Frame_Filename, width: (uint)vlc.Size.Width, (uint)vlc.Size.Height);
}
catch
{
MessageBox.Show("No Code");
}
}
I cant get it to work because I don't know how to call it,
I want to call it from a dispatch timer but I get errors.
Sorry for not answering questions correctly, will get the hang of this forum.
Try get the whole function in:
/// <summary>
/// Draw the video box into an image
/// </summary>
/// <param name="vlc"></param>
/// <returns></returns>
public static void Get_Frame(Vlc.DotNet.Forms.VlcControl vlc)
{
try
{
if (File.Exists(Temp_Frame_Filename))
{
File.Delete(Temp_Frame_Filename);
}
vlc.TakeSnapshot(Temp_Frame_Filename, width: (uint)vlc.Size.Width, (uint)vlc.Size.Height);
}
catch
{
MessageBox.Show("No Code");
}
}
I tried doing this:
void Codetimer_Tick(object sender, EventArgs e)
{
ZXing.BarcodeReader Reader = new ZXing.BarcodeReader();
Result result = Reader.Decode(Temp_Frame_Filename);
if (result != null) TxtScannedResult.Text = result.ToString();
}
Get an error cannot convert from string to system.drawingbitmap.
Thanks
I think I went down the wrong path with that attempt,
I somehow need to convert a VLCVideo stream to bitmap for Xzing
any ideas please

I made a little test with Vlc.DotNet.Wpf 3.0.0 and ZXing.Net 0.16.5.
I'm not sure why you are speaking about a WPF app but then you are using the Vlc.DotNet.Forms namespace. I used the Vlc.DotNet.Wpf.VlcControl.
Here is the proof of concept code (perhaps there is a better solution for threading and locking, but it works):
First, I added the VlcControl to my WPF form:
<Wpf:VlcControl Name="vlcControl" HorizontalAlignment="Left" Height="380" Margin="87,10,0,0" VerticalAlignment="Top" Width="367" Loaded="VlcControl_Loaded"/>
And here is the code behind:
private int isWorking;
private System.Threading.Timer decodingTimer;
private ZXing.IBarcodeReader<BitmapSource> reader = new ZXing.Presentation.BarcodeReader();
private void VlcControl_Loaded(object sender, RoutedEventArgs e)
{
((VlcControl)sender).SourceProvider.CreatePlayer(new DirectoryInfo(#"C:\Program Files\VideoLAN\VLC"));
((VlcControl)sender).SourceProvider.MediaPlayer.SetMedia(new FileInfo(#"How Barcodes Work - YouTube.mp4"));
((VlcControl)sender).SourceProvider.MediaPlayer.Play();
decodingTimer = new System.Threading.Timer(DoDecoding, null, 500, 500);
}
private void DoDecoding(object state)
{
if (Interlocked.Increment(ref isWorking) == 1)
{
this.Dispatcher.BeginInvoke((Action) (() =>
{
try
{
var bitmap = vlcControl.SourceProvider.VideoSource as InteropBitmap;
if (bitmap != null)
{
var result = reader.Decode(bitmap);
if (result != null)
{
MessageBox.Show(result.Text);
}
}
}
catch (Exception )
{
// add handling here
}
finally
{
Interlocked.Decrement(ref isWorking);
}
}));
}
else
{
Interlocked.Decrement(ref isWorking);
}
}

Thank you so much for your help, I tried your solution and it works perfectly.
I have implemented in my solution and am getting the following error:
Object reference not set to an instance of an object.
Any ides why?
Here are some code snippets:
public partial class MainWindow : Window
private ZXing.IBarcodeReader<BitmapSource> reader = new ZXing.Presentation.BarcodeReader();
private System.Threading.Timer decodingTimer;
and
private void DoDecoding(object state)
{
this.Dispatcher.BeginInvoke((Action)(() =>
{
try
{
InteropBitmap bitmap = videoSin.SourceProvider.VideoSource as InteropBitmap;
if (bitmap != null)
{
var result = reader.Decode(bitmap);
if (result != null)
{
MessageBox.Show(result.Text);
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
));
}
The Video stream works and I only get the error when I present a barcode to the camara.
Thanks

C# (MediaFrameReader API) SoftwareBitmap doesn't show

I followed Microsoft's instructions to implement MediaFrameReader API in order to process video captured by camera. The problem is I don't know why the bitmap doesn't show up in XAML, I can't save it to jpeg either (bitmap is not empty). I did convert the bitmap to proper format for XAML:
Here's my code to show the frame in XAML control (no error):
public void ShowSoftwareBitmap(SoftwareBitmap softwareBitmap, SoftwareBitmap backBuffer, Image imageElement, bool taskRunning)
{
try
{
//check if softwareBitmap is in proper format (Bgra8 and premultiplied alpha) to display to XAML Image control
if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Bgra8 ||
softwareBitmap.BitmapAlphaMode != BitmapAlphaMode.Premultiplied)
{
softwareBitmap = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
}
if(softwareBitmap != null)
{
softwareBitmap = Interlocked.Exchange(ref backBuffer, softwareBitmap);
softwareBitmap?.Dispose();
var task = imageElement.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal,
async () =>
{
if (taskRunning)
{
return;
}
taskRunning = true;
SoftwareBitmap latestBitmap;
while ((latestBitmap = Interlocked.Exchange(ref backBuffer, null)) != null)
{
SaveSoftwareBitmapToJpg(latestBitmap, 2);
var imageSource = (SoftwareBitmapSource)imageElement.Source;
await imageSource.SetBitmapAsync(latestBitmap);
latestBitmap.Dispose();
}
taskRunning = false;
}
);
}
}
catch (NullReferenceException e)
{
Debug.WriteLine("Backbuffer is empty." + e.Message);
}
}
Here's my code to save it to jpeg (error unsupported bitmap format):
public async void SaveSoftwareBitmapToJpg(SoftwareBitmap softwareBitmap, int _frameIndex)
{
StorageFolder captureFolder = await FileAccess();
StorageFile outputFile = await captureFolder.CreateFileAsync($"capture{_frameIndex}.jpg", CreationCollisionOption.FailIfExists);
using (IRandomAccessStream stream = await outputFile.OpenAsync(FileAccessMode.ReadWrite))
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);
encoder.SetSoftwareBitmap(softwareBitmap);
encoder.BitmapTransform.ScaledHeight = 300;
encoder.BitmapTransform.ScaledWidth = 300;
encoder.IsThumbnailGenerated = true;
try
{
await encoder.FlushAsync();
}
catch (Exception e)
{
switch (e.HResult)
{
case unchecked((int)0x88982F81): //WINCODEC_ERR_UNSUPPORTEDOPERATION
// If the encoder does not support writing a thumbnail, then try again
// but disable thumbnail generation.
encoder.IsThumbnailGenerated = false;
break;
default:
throw e;
}
}
//catch error generating thumbnail
if (encoder.IsThumbnailGenerated == false)
{
await encoder.FlushAsync();
}
}
}

I tested with your code snippet, if you invoking the above methods correctly with a correct SoftwareBitmap object, the above methods could work well. Simply for example:
<Image x:Name="imgshow" ></Image>
imgshow.Source = new SoftwareBitmapSource();
SoftwareBitmap _backBuffer = new SoftwareBitmap(BitmapPixelFormat.Bgra8, 300, 400);
ShowSoftwareBitmap(softwareBitmap, _backBuffer, imgshow, false);
So that issue may be not happened with the methods above. This could help you narrow the issue, you could check if something wrong when getting the SoftwareBitmap object. I saw you may copy code from ProcessFrame method which is in FrameRenderer class of the official sample. The official sample get the SoftwareBitmap by converting VideoMediaFrame, check if anything wrong when you converting.
Also please debug your code to see if taskRunning is false that your code do run into setting image source steps.

So here's how I changed my code:
In ShowSoftwareBitmap(), I used the same boolean taskRunning in other methods so it was changed crazily in run time. Just create another boolean to solve the problem.
In save SaveSoftwareBitmapToJpg(), the bitmap wasn't properly converted, it has to be Rgba8 or Rgba16 to print to JPEG, so the same conversion used for ShowSoftwareBitmap() wouldn't work. Use this instead:
try
{
if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Rgba16)
{
result = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Rgba16);
}
else
{
result = SoftwareBitmap.Copy(softwareBitmap);
}
}

C# increasing Memory on camera image processing

I have a problem with increasing memory. I have an image, that comes from a camera. The image is processed by a function with unmanaged code. If the expected pattern could not be found, this function takes a long time (some seconds). If the pattern can be found it returns a result very fast (some ms).
I tried to start the postprocessing in a new Thread and Abort it after 200ms. So far this works. Now I have the problem, that my memory grows. Maybe the using clause doesn't work as expected and the image is kept in memory...
private void ImageWorker()
{
while (_imageWorkerRunning)
{
try
{
using (var img = CameraHelper.GetImage())
{
var waiter = new ManualResetEvent(false);
ProcessResult result = null;
var thd = new Thread(() => {
result = UnManagedImageProcessor.Process(img);
waiter.Set();
});
thd.Start();
waiter.WaitOne(200);
if (thd.ThreadState == ThreadState.Running || result == null)
{
thd.Abort();
while (thd.ThreadState != ThreadState.Aborted) new ManualResetEvent(false).WaitOne(10);
}
Application.Current.Dispatcher.Invoke(() => DisplayImage = img);
}
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
}
}
Does the GC work fine on aborted Threads? I think maybe this is my problem...
Solutiuon:
I changed my code to the following. Now I have two parallel tasks. The first is getting images and check, if processing is available.
If it is, the image is provided to this ImageProcessing task. That displays image only on Success (then processing is fast and each picture can be processed).
If processing is not successfull it takes some time and the image is displayed only as live image (by ImageWorker).
private readonly object _imageLock = new object();
private ExtendedImage _sharedProcessingImage;
private readonly ManualResetEvent _processingWaiter = new ManualResetEvent(false);
private bool _processingWaits = false;
private void ImageWorker()
{
while (_imageWorkerRunning) {
try {
var img = CameraHelper.GetImage();
if (_processingWaits) {
//processing available --> process image
lock (_imageLock) {
_sharedProcessingImage = img;
}
_processingWaits = false;
_processingWaiter.Set();
}
else {
//Processing in progress --> Only display image
Application.Current.Dispatcher.Invoke(() => DisplayImage = img);
}
new ManualResetEvent(false).WaitOne(50);
}
catch (Exception e) {
_log.Error(e);
}
}
}
private void ImageProcessing()
{
while (_imageWorkerRunning) {
_processingWaits = true;
_processingWaiter.WaitOne();
_processingWaiter.Reset();
lock (_imageLock) {
try {
result = UnManagedImageProcessor.Process(_sharedProcessingImage);
// ... handle result
if(result.Succeeded) Application.Current.Dispatcher.Invoke(() => DisplayImage = _sharedProcessingImage);
}
catch (Exception e) {
_log.Error(e);
}
}
}
}
Thanks for your help.

What I propose to you is to use BlockingCollection.
To display the images you can use a locker, every time the UI is idle you lock the object, get the last image and dispose the previus one. Also, everytime a image is processed you lock the object and set the var to the last one.
Display logic:
readonly object lastProcessdImageLocker = new object();
Bitmap lastProcessdImage;
//Every time a image process is done:
lock(lastProcessdImageLocker)
lastProcessdImage = imageJustProcessed;
//Every time the UI thread is idle
lock(lastProcessdImageLocker)
myPictureboxImage = lastProcessdImage;
//Here you should also dispose the previus myPictureboxImage so you prevent your memory usage to grow fast!
See this post for more info about WPF and also WinForm render loop
Sample code of the use of BlockingCollection to consume (process) camera images:
BlockingCollection<Bitmap> cameraImages = new BlockingCollection<Bitmap>();
//use another theard to fill up the cameraImages, like that:
//cameraImages.Add(CameraHelper.GetImage());
void StartProcess()
{
if (processImageThread== null || !processImageThread.IsAlive)
{
processImageThread= new Thread(ProcessLoop);
processImageThread.Name = "ProcessLoop";
processImageThread.IsBackground = true;
processImageThread.Start();
Console.TraceInformation("ProcessLoop started");
}
}
private void ProcessLoop()
{
try
{
foreach (img in cameraImages.GetConsumingEnumerable(CancelProcessing.Token))
{
// Do your stuff
}
}
catch (OperationCanceledException)
{
Console.WriteLine("ProcessLoop OperationCanceledException.");
}
finally
{
}
}
If your cameraImages count grows too fast (getting out of memory), you need to reduce the process time or stop the camera for some time.

Add image to lock screen C#

What i am trying to achieve is my canvas to be saved as image in isolated storage and that image then be used as lock screen.
The problem is that it happens only once,after that when the canvas is updated and when i try changing the lock screen with new image, it is not updated. I tried the following code :
Uri uri;
async private void LockScreen_Click(object sender, EventArgs e)
{
bitmap = null;
bitmap = new WriteableBitmap((int)bgCanvas.Width, (int)bgCanvas.Height);
bitmap.Render(bgCanvas,null);
bitmap.Invalidate();
using(IsolatedStorageFile iso = IsolatedStorageFile.GetUserStoreForApplication())
{
//Delete old background file from isolated storage
if (iso.FileExists("Background"))
{
iso.DeleteFile("Background");
}
//Save canvas to isolated storage
using (IsolatedStorageFileStream isostream = iso.CreateFile("Background"))
{
Extensions.SaveJpeg(bitmap, isostream, bitmap.PixelWidth, bitmap.PixelHeight, 0, 100);
//isostream.Close();
}
}
//Access isolated storage file and put as lock screen
try
{
var isProvider = Windows.Phone.System.UserProfile.LockScreenManager.IsProvidedByCurrentApplication;
if (!isProvider)
{
var permission = await Windows.Phone.System.UserProfile.LockScreenManager.RequestAccessAsync();
isProvider = permission == Windows.Phone.System.UserProfile.LockScreenRequestResult.Granted;
}
if (isProvider)
{
uri = new Uri("ms-appdata:///Local/Background", UriKind.Absolute);
Windows.Phone.System.UserProfile.LockScreen.SetImageUri(uri);
MessageBox.Show("Lock Screen updated!");
}
else
{
MessageBox.Show("Couldn't update the lockscreen picture.");
}
}
catch (Exception ex)
{
MessageBox.Show("An error occured while updating the lockscreen picture: " + ex.Message);
}
}
The message box shows that lock screen is updated even though it is not.
I think the file isn't updated in the isolated storage but cant figure out why ?

AccessViolation in WPF with no user code on the stack

I'm getting a strange AccessViolationException in a WPF application. Since there is no user code on the stack I'm not really sure how to go about troubleshooting this. It also only happens every 2-3 days which adds to the troubleshooting complexity.
at MS.Win32.PresentationCore.UnsafeNativeMethods+MILUnknown.Release(IntPtr)
at MS.Win32.PresentationCore.UnsafeNativeMethods+MILUnknown.ReleaseInterface(IntPtr ByRef)
at System.Windows.Media.SafeMILHandle.ReleaseHandle()
at System.Windows.Media.Imaging.BitmapSourceSafeMILHandle.ReleaseHandle()
at System.Runtime.InteropServices.SafeHandle.InternalFinalize()
at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean)
at System.Runtime.InteropServices.SafeHandle.Finalize()
My image manipulation code (slightly simplified)
private int timerCount;
private void TimerProc() // called from a timer obviously
{
if(Interlocked.Increment(ref timerCount) !=0)
{
Interlocked.Decrement(ref timerCount);
return;
}
try
{
byte[] img = FetchImageFromExternalSource(); //returns a jpeg image
this.Image = LoadImage(new MemoryStream(img));
}
finally
{
Interlocked.Decrement(ref timerCount);
}
}
private BitmapImage LoadImage(Stream inputStream)
{
var bitmapImage = new BitmapImage();
bitmapImage.BeginInit();
bitmapImage.CacheOption = BitmapCacheOption.OnLoad;
bitmapImage.StreamSource = inputStream;
bitmapImage.EndInit();
bitmapImage.Freeze();
return bitmapImage;
}
private BitmapImage image;
public BitmapImage Image
{
get
{
return image;
}
set
{
if (image!= value)
{
this.dispatcher.BeginInvoke(new Action(()=>
{
image = value;
PropertyChanged(this,new PropertyChangedEventArgs("Image"));
}),null);
}
}
}
Any advice on if I'm hitting a known bug (.NET 4) or if there is some problem with the code that I've missed would be appreciated.

System.Threading.Timer invokes the callback on a background thread, not on the UI thread, which the BitmapImage class is probably expecting. Try using DispatcherTimer.