We Keep Coding

Convert VLC video stream to bitmap for xzing to decode using C#

I need to convert a VLC video stream (working) in my WPF app to a bmp for Xzing to be able to decode,
I can decode using a usb or the onboard webcam.
But how to get the 2 together??
Been searching all over and have tried some solutions but nothing seems to work, might be me as i'm pretty new to C#
Thanks
I tried this piece of code:
public static void Get_Frame(Vlc.DotNet.Forms.VlcControl vlc)
{
try
{
if (File.Exists(Temp_Frame_Filename))
{
File.Delete(Temp_Frame_Filename);
}
vlc.TakeSnapshot(Temp_Frame_Filename, width: (uint)vlc.Size.Width, (uint)vlc.Size.Height);
}
catch
{
MessageBox.Show("No Code");
}
}
I cant get it to work because I don't know how to call it,
I want to call it from a dispatch timer but I get errors.
Sorry for not answering questions correctly, will get the hang of this forum.
Try get the whole function in:
/// <summary>
/// Draw the video box into an image
/// </summary>
/// <param name="vlc"></param>
/// <returns></returns>
public static void Get_Frame(Vlc.DotNet.Forms.VlcControl vlc)
{
try
{
if (File.Exists(Temp_Frame_Filename))
{
File.Delete(Temp_Frame_Filename);
}
vlc.TakeSnapshot(Temp_Frame_Filename, width: (uint)vlc.Size.Width, (uint)vlc.Size.Height);
}
catch
{
MessageBox.Show("No Code");
}
}
I tried doing this:
void Codetimer_Tick(object sender, EventArgs e)
{
ZXing.BarcodeReader Reader = new ZXing.BarcodeReader();
Result result = Reader.Decode(Temp_Frame_Filename);
if (result != null) TxtScannedResult.Text = result.ToString();
}
Get an error cannot convert from string to system.drawingbitmap.
Thanks
I think I went down the wrong path with that attempt,
I somehow need to convert a VLCVideo stream to bitmap for Xzing
any ideas please

I made a little test with Vlc.DotNet.Wpf 3.0.0 and ZXing.Net 0.16.5.
I'm not sure why you are speaking about a WPF app but then you are using the Vlc.DotNet.Forms namespace. I used the Vlc.DotNet.Wpf.VlcControl.
Here is the proof of concept code (perhaps there is a better solution for threading and locking, but it works):
First, I added the VlcControl to my WPF form:
<Wpf:VlcControl Name="vlcControl" HorizontalAlignment="Left" Height="380" Margin="87,10,0,0" VerticalAlignment="Top" Width="367" Loaded="VlcControl_Loaded"/>
And here is the code behind:
private int isWorking;
private System.Threading.Timer decodingTimer;
private ZXing.IBarcodeReader<BitmapSource> reader = new ZXing.Presentation.BarcodeReader();
private void VlcControl_Loaded(object sender, RoutedEventArgs e)
{
((VlcControl)sender).SourceProvider.CreatePlayer(new DirectoryInfo(#"C:\Program Files\VideoLAN\VLC"));
((VlcControl)sender).SourceProvider.MediaPlayer.SetMedia(new FileInfo(#"How Barcodes Work - YouTube.mp4"));
((VlcControl)sender).SourceProvider.MediaPlayer.Play();
decodingTimer = new System.Threading.Timer(DoDecoding, null, 500, 500);
}
private void DoDecoding(object state)
{
if (Interlocked.Increment(ref isWorking) == 1)
{
this.Dispatcher.BeginInvoke((Action) (() =>
{
try
{
var bitmap = vlcControl.SourceProvider.VideoSource as InteropBitmap;
if (bitmap != null)
{
var result = reader.Decode(bitmap);
if (result != null)
{
MessageBox.Show(result.Text);
}
}
}
catch (Exception )
{
// add handling here
}
finally
{
Interlocked.Decrement(ref isWorking);
}
}));
}
else
{
Interlocked.Decrement(ref isWorking);
}
}

Thank you so much for your help, I tried your solution and it works perfectly.
I have implemented in my solution and am getting the following error:
Object reference not set to an instance of an object.
Any ides why?
Here are some code snippets:
public partial class MainWindow : Window
private ZXing.IBarcodeReader<BitmapSource> reader = new ZXing.Presentation.BarcodeReader();
private System.Threading.Timer decodingTimer;
and
private void DoDecoding(object state)
{
this.Dispatcher.BeginInvoke((Action)(() =>
{
try
{
InteropBitmap bitmap = videoSin.SourceProvider.VideoSource as InteropBitmap;
if (bitmap != null)
{
var result = reader.Decode(bitmap);
if (result != null)
{
MessageBox.Show(result.Text);
}
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
));
}
The Video stream works and I only get the error when I present a barcode to the camara.
Thanks

C# (MediaFrameReader API) SoftwareBitmap doesn't show

I followed Microsoft's instructions to implement MediaFrameReader API in order to process video captured by camera. The problem is I don't know why the bitmap doesn't show up in XAML, I can't save it to jpeg either (bitmap is not empty). I did convert the bitmap to proper format for XAML:
Here's my code to show the frame in XAML control (no error):
public void ShowSoftwareBitmap(SoftwareBitmap softwareBitmap, SoftwareBitmap backBuffer, Image imageElement, bool taskRunning)
{
try
{
//check if softwareBitmap is in proper format (Bgra8 and premultiplied alpha) to display to XAML Image control
if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Bgra8 ||
softwareBitmap.BitmapAlphaMode != BitmapAlphaMode.Premultiplied)
{
softwareBitmap = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Bgra8, BitmapAlphaMode.Premultiplied);
}
if(softwareBitmap != null)
{
softwareBitmap = Interlocked.Exchange(ref backBuffer, softwareBitmap);
softwareBitmap?.Dispose();
var task = imageElement.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal,
async () =>
{
if (taskRunning)
{
return;
}
taskRunning = true;
SoftwareBitmap latestBitmap;
while ((latestBitmap = Interlocked.Exchange(ref backBuffer, null)) != null)
{
SaveSoftwareBitmapToJpg(latestBitmap, 2);
var imageSource = (SoftwareBitmapSource)imageElement.Source;
await imageSource.SetBitmapAsync(latestBitmap);
latestBitmap.Dispose();
}
taskRunning = false;
}
);
}
}
catch (NullReferenceException e)
{
Debug.WriteLine("Backbuffer is empty." + e.Message);
}
}
Here's my code to save it to jpeg (error unsupported bitmap format):
public async void SaveSoftwareBitmapToJpg(SoftwareBitmap softwareBitmap, int _frameIndex)
{
StorageFolder captureFolder = await FileAccess();
StorageFile outputFile = await captureFolder.CreateFileAsync($"capture{_frameIndex}.jpg", CreationCollisionOption.FailIfExists);
using (IRandomAccessStream stream = await outputFile.OpenAsync(FileAccessMode.ReadWrite))
{
BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.JpegEncoderId, stream);
encoder.SetSoftwareBitmap(softwareBitmap);
encoder.BitmapTransform.ScaledHeight = 300;
encoder.BitmapTransform.ScaledWidth = 300;
encoder.IsThumbnailGenerated = true;
try
{
await encoder.FlushAsync();
}
catch (Exception e)
{
switch (e.HResult)
{
case unchecked((int)0x88982F81): //WINCODEC_ERR_UNSUPPORTEDOPERATION
// If the encoder does not support writing a thumbnail, then try again
// but disable thumbnail generation.
encoder.IsThumbnailGenerated = false;
break;
default:
throw e;
}
}
//catch error generating thumbnail
if (encoder.IsThumbnailGenerated == false)
{
await encoder.FlushAsync();
}
}
}

I tested with your code snippet, if you invoking the above methods correctly with a correct SoftwareBitmap object, the above methods could work well. Simply for example:
<Image x:Name="imgshow" ></Image>
imgshow.Source = new SoftwareBitmapSource();
SoftwareBitmap _backBuffer = new SoftwareBitmap(BitmapPixelFormat.Bgra8, 300, 400);
ShowSoftwareBitmap(softwareBitmap, _backBuffer, imgshow, false);
So that issue may be not happened with the methods above. This could help you narrow the issue, you could check if something wrong when getting the SoftwareBitmap object. I saw you may copy code from ProcessFrame method which is in FrameRenderer class of the official sample. The official sample get the SoftwareBitmap by converting VideoMediaFrame, check if anything wrong when you converting.
Also please debug your code to see if taskRunning is false that your code do run into setting image source steps.

So here's how I changed my code:
In ShowSoftwareBitmap(), I used the same boolean taskRunning in other methods so it was changed crazily in run time. Just create another boolean to solve the problem.
In save SaveSoftwareBitmapToJpg(), the bitmap wasn't properly converted, it has to be Rgba8 or Rgba16 to print to JPEG, so the same conversion used for ShowSoftwareBitmap() wouldn't work. Use this instead:
try
{
if (softwareBitmap.BitmapPixelFormat != BitmapPixelFormat.Rgba16)
{
result = SoftwareBitmap.Convert(softwareBitmap, BitmapPixelFormat.Rgba16);
}
else
{
result = SoftwareBitmap.Copy(softwareBitmap);
}
}

c# - wpf - Main UI hang during execution of Thread

I'am using WPF with multi threading. Problem is during execution of the MAIN thread hangs, like its waiting for something or infinite loop in main UI. I don't know if there's any other approach when it comes to multi threading in WPF. See my code below:
Thread myNewThread1 = new Thread(() => ping(IP1, img_m));
Thread myNewThread2 = new Thread(() => ping(IP2, img_h));
Thread myNewThread3 = new Thread(() => ping(IP3, img_c));
Thread myNewThread4 = new Thread(() => ping(IP4, img_b));
Thread myNewThread5 = new Thread(() => ping(IP5, img_e));
myNewThread1.Start();
myNewThread2.Start();
myNewThread3.Start();
myNewThread4.Start();
myNewThread5.Start();
private void ping(string IP, Image img)
{
this.Dispatcher.Invoke(() =>
{
Ping p = new Ping();
var r = p.Send(IP, 1000, new byte[5]);
if (r.Status == IPStatus.Success)
{
var image = new BitmapImage();
image.BeginInit();
image.UriSource = new Uri("subonline.gif", UriKind.Relative);
image.CacheOption = BitmapCacheOption.OnLoad;
image.EndInit();
ImageBehavior.SetAnimatedSource(img, image);
}
else
{
var image = new BitmapImage();
image.BeginInit();
image.UriSource = new Uri("suboffline.gif", UriKind.Relative);
image.CacheOption = BitmapCacheOption.OnLoad;
image.EndInit();
ImageBehavior.SetAnimatedSource(img, image);
}
});
Thread.Sleep(500);
ping(IP, img);
}

Your main thread hangs because of Dispatcher.Invoke wrong usage, it should be used for UI logic only, so you should move out the Ping-oriented logic out of it.
Do not use the BackgroundWorker for this, it's an obsolete and heavy construction you don't really need. Also, do not a thread for pinging, this is wrong approach, and that's why:
Pinging operation is network-related, and the thread you're using for wait the remote server to response simply wastes the system resources, as it does absolutely nothing except wait. So you should switch to asynchronous approach for this.
You either should subscribe to Ping.PingCompleted event and call the SendAsync method, like this:
private void ping(string IP, MediaTypeNames.Image img)
{
Ping p = new Ping();
PingReply r;
// lambda delegate here, may use event handler instead
p.PingCompleted += (sender, args) => { PingCompleted(args, img); };
r = p.SendAsync(IP, 1000, new byte[5], null);
}
private void PingCompleted(PingCompletedEventArgs args, MediaTypeNames.Image img)
{
this.Dispatcher.Invoke(() =>
{
string imageAddress;
if (args.Reply.Status == IPStatus.Success)
{
imageAddress = "subonline.gif";
}
else
{
imageAddress = "suboffline.gif";
}
var image = new BitmapImage();
image.BeginInit();
image.UriSource = new Uri(imageAddress, UriKind.Relative);
image.CacheOption = BitmapCacheOption.OnLoad;
image.EndInit();
ImageBehavior.SetAnimatedSource(img, image);
});
}
or you should use async/await feature which was introduced exactly for such cases (code partially from this answer):
// async event handler
private async void btn_Click(object sender, EventArgs e)
{
// async call to all the ips
var results = await PingAsync(new List<string> { IP1, IP2, IP3, IP4, IP5 });
// here we do not need the Dispatcher as await will restore UI context for you
PingCompleted(results[0], img_m);
// etc
}
private void PingCompleted(PingReply r, MediaTypeNames.Image img)
{
string imageAddress;
if (r.Status == IPStatus.Success)
{
imageAddress = "subonline.gif";
}
else
{
imageAddress = "suboffline.gif";
}
var image = new BitmapImage();
image.BeginInit();
image.UriSource = new Uri(imageAddress, UriKind.Relative);
image.CacheOption = BitmapCacheOption.OnLoad;
image.EndInit();
ImageBehavior.SetAnimatedSource(img, image);
}
// helper method
private async Task<PingReply[]> PingAsync(List<string> theListOfIPs)
{
Ping pingSender = new Ping();
var tasks = theListOfIPs.Select(ip => pingSender.SendPingAsync(ip, 1000, new byte[5]));
return await Task.WhenAll(tasks);
}

Do not include the call to p.Send method in the dispatcher - currently only the Thread.Sleep(500) is done in the background and everything else is done in the UI thread.
Also, I believe that because of the Thread.Sleep the function never gives the UI thread a chance to work so the pinging never occurs. Replace the use of Thread.Sleep with a Thread.Timer object.

C# increasing Memory on camera image processing

I have a problem with increasing memory. I have an image, that comes from a camera. The image is processed by a function with unmanaged code. If the expected pattern could not be found, this function takes a long time (some seconds). If the pattern can be found it returns a result very fast (some ms).
I tried to start the postprocessing in a new Thread and Abort it after 200ms. So far this works. Now I have the problem, that my memory grows. Maybe the using clause doesn't work as expected and the image is kept in memory...
private void ImageWorker()
{
while (_imageWorkerRunning)
{
try
{
using (var img = CameraHelper.GetImage())
{
var waiter = new ManualResetEvent(false);
ProcessResult result = null;
var thd = new Thread(() => {
result = UnManagedImageProcessor.Process(img);
waiter.Set();
});
thd.Start();
waiter.WaitOne(200);
if (thd.ThreadState == ThreadState.Running || result == null)
{
thd.Abort();
while (thd.ThreadState != ThreadState.Aborted) new ManualResetEvent(false).WaitOne(10);
}
Application.Current.Dispatcher.Invoke(() => DisplayImage = img);
}
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
}
}
Does the GC work fine on aborted Threads? I think maybe this is my problem...
Solutiuon:
I changed my code to the following. Now I have two parallel tasks. The first is getting images and check, if processing is available.
If it is, the image is provided to this ImageProcessing task. That displays image only on Success (then processing is fast and each picture can be processed).
If processing is not successfull it takes some time and the image is displayed only as live image (by ImageWorker).
private readonly object _imageLock = new object();
private ExtendedImage _sharedProcessingImage;
private readonly ManualResetEvent _processingWaiter = new ManualResetEvent(false);
private bool _processingWaits = false;
private void ImageWorker()
{
while (_imageWorkerRunning) {
try {
var img = CameraHelper.GetImage();
if (_processingWaits) {
//processing available --> process image
lock (_imageLock) {
_sharedProcessingImage = img;
}
_processingWaits = false;
_processingWaiter.Set();
}
else {
//Processing in progress --> Only display image
Application.Current.Dispatcher.Invoke(() => DisplayImage = img);
}
new ManualResetEvent(false).WaitOne(50);
}
catch (Exception e) {
_log.Error(e);
}
}
}
private void ImageProcessing()
{
while (_imageWorkerRunning) {
_processingWaits = true;
_processingWaiter.WaitOne();
_processingWaiter.Reset();
lock (_imageLock) {
try {
result = UnManagedImageProcessor.Process(_sharedProcessingImage);
// ... handle result
if(result.Succeeded) Application.Current.Dispatcher.Invoke(() => DisplayImage = _sharedProcessingImage);
}
catch (Exception e) {
_log.Error(e);
}
}
}
}
Thanks for your help.

What I propose to you is to use BlockingCollection.
To display the images you can use a locker, every time the UI is idle you lock the object, get the last image and dispose the previus one. Also, everytime a image is processed you lock the object and set the var to the last one.
Display logic:
readonly object lastProcessdImageLocker = new object();
Bitmap lastProcessdImage;
//Every time a image process is done:
lock(lastProcessdImageLocker)
lastProcessdImage = imageJustProcessed;
//Every time the UI thread is idle
lock(lastProcessdImageLocker)
myPictureboxImage = lastProcessdImage;
//Here you should also dispose the previus myPictureboxImage so you prevent your memory usage to grow fast!
See this post for more info about WPF and also WinForm render loop
Sample code of the use of BlockingCollection to consume (process) camera images:
BlockingCollection<Bitmap> cameraImages = new BlockingCollection<Bitmap>();
//use another theard to fill up the cameraImages, like that:
//cameraImages.Add(CameraHelper.GetImage());
void StartProcess()
{
if (processImageThread== null || !processImageThread.IsAlive)
{
processImageThread= new Thread(ProcessLoop);
processImageThread.Name = "ProcessLoop";
processImageThread.IsBackground = true;
processImageThread.Start();
Console.TraceInformation("ProcessLoop started");
}
}
private void ProcessLoop()
{
try
{
foreach (img in cameraImages.GetConsumingEnumerable(CancelProcessing.Token))
{
// Do your stuff
}
}
catch (OperationCanceledException)
{
Console.WriteLine("ProcessLoop OperationCanceledException.");
}
finally
{
}
}
If your cameraImages count grows too fast (getting out of memory), you need to reduce the process time or stop the camera for some time.

Error : Object is currently in use elsewhere.

I have developed an application the continuous read image stream from a DSLR camera.
while (!liveViewExit)
{
// Create a Memory Stream
stream = new IntPtr();
// Get the bitmap image from the DSLR
bmp = GetEvfImage(stream);
if (bmp != null)
{
// Crop the image.
picImage = (System.Drawing.Image)bmp.Clone(new Rectangle(10, 0, 492, 768), bmp.PixelFormat);
try
{
if (picImage != null)
this.picLiveView.Image = (System.Drawing.Image)picImage.Clone();
}
catch (Exception ex)
{
Utility.HandleError(ex);
}
}
}
After running a while, I have this error for this line of code:
this.picLiveView.Image = (System.Drawing.Image)picImage.Clone();
Object is currently in use elsewhere.( at System.Drawing.Image.get_FrameDimensionsList()
at System.Drawing.ImageAnimator.CanAnimate(Image image)
at System.Drawing.ImageAnimator.ImageInfo..ctor(Image image)
at System.Drawing.ImageAnimator.Animate(Image image, EventHandler onFrameChangedHandler)
at System.Windows.Forms.PictureBox.Animate(Boolean animate)
at System.Windows.Forms.PictureBox.Animate()
at System.Windows.Forms.PictureBox.InstallNewImage(Image value, ImageInstallationType installationType)
at System.Windows.Forms.PictureBox.set_Image(Image value)
I think the picLiveView PictureBox control is not yet ready to accept new image.
Any idea how on detect if PictureBox is still in used.
// Added:
It is a single thread. I think the picturebox is not fast enough to process the picture object in the while loop.

Are multiple threads updating the picLiveView image? If so that would explain this problem. Just use one thread instead, and serialize the update - alternatively you could use a lock to access picLiveView:
private readonly object myLock = new object();
...
if (picImage != null)
{
lock(myLock)
{
this.picLiveView.Image = (System.Drawing.Image)picImage.Clone();
}
}

I know am late...but try this, if someone have same issue..
if (bmp != null)
{
// Crop the image.
picImage = (System.Drawing.Image)bmp.Clone(new Rectangle(10, 0, 492, 768), bmp.PixelFormat);
**Bitmap img = new Bitmap(picImage);
picImage.Dispose();
picImage = null;**
try
{
if (picImage != null)
**this.picLiveView.Image = img;**
}
catch (Exception ex)
{
Utility.HandleError(ex);
}
}