We Keep Coding

Is there any way to grab the frames from media element like a video without feeling any delay? - WPF

I have to show the frames from a media element in an Image control like a video. I have tried to grab continues frames from media element using a timer and bind that bitmap to Image source. But when setting the scale as 1, it seems like frames are grabbing very slowly. And when reduce the scale to 0.3 or below, the grabbing is working very fast. but the quality of frame is reducing.
Is there any way to solve this?
In short, I want to display the frames from media element to an Image source without any delay and with original quality.
<MediaElement x:Name="MediaEL" Volume="0" ScrubbingEnabled="True" SnapsToDevicePixels="True" MediaOpened="MediaEL_MediaOpened" LoadedBehavior="Manual" MediaEnded="MediaEL_MediaEnded" MediaFailed="MediaEL_MediaFailed">
</MediaElement>
<Image Name="ImageViewerMediaEL" />
ScreenShotimer = new DispatcherTimer();
ScreenShotimer.Interval = TimeSpan.FromMilliseconds(35);//35//
ScreenShotimer.Tick += ScreenShotimer_Tick;
public Bitmap TakeScreenshot(MediaElement medElement, double scale)
{
Bitmap screenBitmap = null;
double actualHeight = medElement.NaturalVideoHeight;
double actualWidth = medElement.NaturalVideoWidth;
double renderHeight = actualHeight * scale;
double renderWidth = actualWidth * scale;
if ((int)renderWidth > 0 && (int)renderHeight > 0)
{
RenderTargetBitmap renderTarget = new RenderTargetBitmap((int)renderWidth,
(int)renderHeight, 96, 96, PixelFormats.Default);
VisualBrush sourceBrush = new VisualBrush(medElement);
DrawingVisual drawingVisual = new DrawingVisual();
DrawingContext drawingContext = drawingVisual.RenderOpen();
using (drawingContext)
{
drawingContext.PushTransform(new ScaleTransform(scale, scale));
drawingContext.DrawRectangle(sourceBrush, null, new Rect(new System.Windows.Point(0, 0),
new System.Windows.Point(actualWidth, actualHeight)));
}
renderTarget.Render(drawingVisual);
MemoryStream stream = new MemoryStream();
BitmapEncoder encoder = new BmpBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(renderTarget));
encoder.Save(stream);
screenBitmap = new Bitmap(stream);
}
return screenBitmap;
}

Using a WinForms Bitmap in a WPF application doesn't seem to be necessary.
Reuse a single instance of a RenderTargetBitmap that is only created when necessary and assigned to the Source property of the Image element.
As well, reuse the DrawingVisual and draw a rectangle with a VisualBrush only when size changes.
private readonly DrawingVisual visual = new DrawingVisual();
private RenderTargetBitmap bitmap;
...
private void OnTimerTick(object sender, EventArgs e)
{
var width = MediaEL.NaturalVideoWidth;
var height = MediaEL.NaturalVideoHeight;
if (width > 0 && height > 0)
{
if (bitmap == null ||
bitmap.PixelWidth != width ||
bitmap.PixelHeight != height)
{
using (var dc = visual.RenderOpen())
{
dc.DrawRectangle(
new VisualBrush(MediaEL), null,
new Rect(0, 0, width, height));
}
bitmap = new RenderTargetBitmap(
width, height, 96, 96, PixelFormats.Default);
ImageViewerMediaEL.Source = bitmap;
}
bitmap.Render(visual);
}
}
The code above works fine for me with Microsoft's Wildlife.wmv and 35 ms timer interval.

How to set Thickness value for Margin in doubleanimation (code not XAML) [duplicate]

Does anyone know of an easy way to animate a movement from an Image's current location to a new location (X,Y) using WPF animation with no XAML, 100% programmatically? And with no reference to "this" (with RegisterName etc).
I am trying to make an extension class for Image to do animation stuff on it. It is easy enough to change the width and height properties through animation, but after searching for location animation of an object it suddenly becomes more advanced.
As it is an extension class I will only have a reference to the actual Image object and the X and Y I want to move it to.
public static void MoveTo(this Image targetControl, double X, double Y, double Width, double Height){
//code here
...
}
Update:
Thanks. Almost working. It seems The GetTop and GetLeft returns 'NaN' not explicitly set. Found the workaround in this post: Canvas.GetTop() returning NaN
public static void MoveTo(this Image target, double newX, double newY) {
Vector offset = VisualTreeHelper.GetOffset(target);
var top = offset.Y;
var left = offset.X;
TranslateTransform trans = new TranslateTransform();
target.RenderTransform = trans;
DoubleAnimation anim1 = new DoubleAnimation(0, newY - top, TimeSpan.FromSeconds(10));
DoubleAnimation anim2 = new DoubleAnimation(0, newX - left, TimeSpan.FromSeconds(10));
trans.BeginAnimation(TranslateTransform.YProperty, anim1);
trans.BeginAnimation(TranslateTransform.XProperty, anim2);
}
I had to swap two of the values (FROM) with 0. I assume that must be because in this context the upper left corner of the picture is the origin? But now it works.

Try this:
public static void MoveTo(this Image target, double newX, double newY)
{
var top = Canvas.GetTop(target);
var left = Canvas.GetLeft(target);
TranslateTransform trans = new TranslateTransform();
target.RenderTransform = trans;
DoubleAnimation anim1 = new DoubleAnimation(top, newY - top, TimeSpan.FromSeconds(10));
DoubleAnimation anim2 = new DoubleAnimation(left, newX - left, TimeSpan.FromSeconds(10));
trans.BeginAnimation(TranslateTransform.XProperty,anim1);
trans.BeginAnimation(TranslateTransform.YProperty,anim2);
}

Here it is... It changes the size and moves a MediaElement under the Canvas. Just input your parameters:
Storyboard story = new Storyboard();
DoubleAnimation dbWidth = new DoubleAnimation();
dbWidth.From = mediaElement1.Width;
dbWidth.To = 600;
dbWidth.Duration = new Duration(TimeSpan.FromSeconds(.25));
DoubleAnimation dbHeight = new DoubleAnimation();
dbHeight.From = mediaElement1.Height;
dbHeight.To = 400;
dbHeight.Duration = dbWidth.Duration;
story.Children.Add(dbWidth);
Storyboard.SetTargetName(dbWidth, mediaElement1.Name);
Storyboard.SetTargetProperty(dbWidth, new PropertyPath(MediaElement.WidthProperty));
story.Children.Add(dbHeight);
Storyboard.SetTargetName(dbHeight, mediaElement1.Name);
Storyboard.SetTargetProperty(dbHeight, new PropertyPath(MediaElement.HeightProperty));
DoubleAnimation dbCanvasX = new DoubleAnimation();
dbCanvasX.From = 0;
dbCanvasX.To = 5;
dbCanvasX.Duration = new Duration(TimeSpan.FromSeconds(.25));
DoubleAnimation dbCanvasY = new DoubleAnimation();
dbCanvasY.From = 0;
dbCanvasY.To = 5;
dbCanvasY.Duration = dbCanvasX.Duration;
story.Children.Add(dbCanvasX);
Storyboard.SetTargetName(dbCanvasX, mediaElement1.Name);
Storyboard.SetTargetProperty(dbCanvasX, new PropertyPath(Canvas.LeftProperty));
story.Children.Add(dbCanvasY);
Storyboard.SetTargetName(dbCanvasY, mediaElement1.Name);
Storyboard.SetTargetProperty(dbCanvasY, new PropertyPath(Canvas.TopProperty));
story.Begin(this);
<Viewbox Stretch="Uniform" StretchDirection="Both" SnapsToDevicePixels="True">
<Grid Width="640" Height="480" Name="MainLayout" SnapsToDevicePixels="True" Background="Black">
<Canvas Width="640" Height="480" Name="MainCanvas" SnapsToDevicePixels="True">
<MediaElement Height="171" HorizontalAlignment="Left" Name="mediaElement1" VerticalAlignment="Top" Width="337" LoadedBehavior="Manual" Margin="166,140,0,0" Canvas.Left="-162" Canvas.Top="-140" />
<Button Canvas.Left="294" Canvas.Top="196" Content="Button" Height="23" Name="button1" Width="75" Click="button1_Click" />
</Canvas>
</Grid>
</Viewbox>
UPDATE:
Instead of MediaElement use this line:
<Rectangle Height="171" HorizontalAlignment="Left" Name="mediaElement1" VerticalAlignment="Top" Width="337" Margin="166,140,0,0" Canvas.Left="-162" Canvas.Top="-140" Fill="{DynamicResource {x:Static SystemColors.MenuBarBrushKey}}" />
And don't forget to put the C# code to:
private void button1_Click(object sender, RoutedEventArgs e) {}
You can use MediaElement as well but you have to define a VideoClip to see something ;)

Please find a solution that uses the Left and Top properties of Canvas for the extension method. See the following code:
public static void MoveTo(this Image target, Point newP)
{
Point oldP = new Point();
oldP.X = Canvas.GetLeft(target);
oldP.Y = Canvas.GetTop(target);
DoubleAnimation anim1 = new DoubleAnimation(oldP.X, newP.X, TimeSpan.FromSeconds(0.2));
DoubleAnimation anim2 = new DoubleAnimation(oldP.Y, newP.Y , TimeSpan.FromSeconds(0.2));
target.BeginAnimation(Canvas.LeftProperty , anim1);
target.BeginAnimation(Canvas.TopProperty, anim2);
}

This code is based on #DeanChalk's answer.
It moves an Image contained within a Canvas (RFID_Token) diagonally from the top-right to the bottom-left, positioned centrally over another Image within a Canvas (RFID_Reader).
<Canvas>
<Canvas x:Name="RFID_Reader_Canvas">
<Image x:Name="RFID_Reader" Source="RFID-Reader.png" Height="456" Width="682" Canvas.Left="37" Canvas.Top="524"/>
</Canvas>
<Canvas x:Name="RFID_Token_Canvas">
<Image x:Name="RFID_Token" Source="RFID-Token.png" Height="268" Width="343" Canvas.Left="874" Canvas.Top="70"/>
</Canvas>
</Canvas>
var StartX = Canvas.GetLeft(RFID_Token);
var StartY = Canvas.GetTop(RFID_Token);
var EndX = RFID_Reader.Width / 2 + Canvas.GetLeft(RFID_Reader) - StartX - (RFID_Token.Width / 2);
var EndY = RFID_Reader.Height / 2 + Canvas.GetTop(RFID_Reader) - StartY - (RFID_Token.Height / 2);
var AnimationX = new DoubleAnimation(0, EndX, TimeSpan.FromSeconds(1));
var AnimationY = new DoubleAnimation(0, EndY, TimeSpan.FromSeconds(1));
var Transform = new TranslateTransform();
RFID_Token_Canvas.RenderTransform = Transform;
Transform.BeginAnimation(TranslateTransform.XProperty, AnimationX);
Transform.BeginAnimation(TranslateTransform.YProperty, AnimationY);

I kept having NaN or 0 values for my nested elements, here's a modified version of Danny's answer :
public void MoveTo(Canvas canvas, FrameworkElement target, FrameworkElement destination)
{
Point oldPoint = target.TransformToAncestor(canvas).Transform(new Point(0, 0));
Point newPoint = destination.TransformToAncestor(canvas).Transform(new Point(0, 0));
var EndX = destination.Width / 2 + newPoint.X - oldPoint.X - (target.Width / 2);
var EndY = destination.Height / 2 + newPoint.Y - oldPoint.Y - (target.Height / 2);
TranslateTransform trans = new TranslateTransform();
target.RenderTransform = trans;
DoubleAnimation anim1 = new DoubleAnimation(0, EndX, TimeSpan.FromSeconds(0.3));
DoubleAnimation anim2 = new DoubleAnimation(0, EndY, TimeSpan.FromSeconds(0.3));
trans.BeginAnimation(TranslateTransform.XProperty, anim1);
trans.BeginAnimation(TranslateTransform.YProperty, anim2);
}

Improving the rendering performance in drawing heat map

I am trying to draw a heat map with a given input of points and amplitude of each point. The normal input size is around 400,000 points, but the code draws every 15th point. The map is drawn onto a Canvas.
In the XAML:
<Grid x:Name="MainPanel" Width="800" Height="600">
<Textbox Name="textbox" Grid.Column="0"/>
<Canvas Name="canvas" Grid.Column="1" HorizontalAlignment="Left" Height="450" Margin="70,75,0,0" VerticalAlignment="Top" Width="706"/>
</Grid>
Rendering code:
public static void DrawPoints(Parser parser, Canvas canvas, double radiusX, double radiusY, double thickness, double minAmplitude, double maxAmplitude)
{
double cellWidth = canvas.ActualWidth;
double cellHeight = canvas.ActualHeight;
var list = parser.Data;
// Draws every 15th point
int res = 15;
for (int i = 0; i < list.Count; i += res)
{
var location = list[i].Point;
var amplitude = list[i].Amplitude;
// Converts rgb to hsv values for representation
var color= ColorConverter.hsv2rgb(
COLOR_START_H * (1.0 - ((amplitude - minAmplitude) / (maxAmplitude - minAmplitude))),
COLOR_START_S, COLOR_START_V);
Point center = new Point(location.X, location.Y);
DrawEllipse(canvas, center, radiusX, radiusY,
new SolidColorBrush(color), new SolidColorBrush(color), thickness);
}
return;
}
In the DrawEllipse function:
public static void DrawEllipse(Canvas canvas, Point center, double radiusX, double radiusY, Brush fill, Brush stroke, double thickness)
{
var ellipse = new Ellipse();
ellipse.Width = radiusX * 2;
ellipse.Height = radiusY * 2;
ellipse.Fill = fill;
ellipse.Stroke = stroke;
ellipse.StrokeThickness = thickness;
Canvas.SetLeft(ellipse, center.X - radiusX);
Canvas.SetBottom(ellipse, center.Y - radiusY);
canvas.Children.Add(ellipse);
return;
}
In the Main code:
private void btnDraw_Click(object sender, RoutedEventArgs e)
{
Painter.DrawPoints(parser, canvas, 3, 3, 1, 0, 100);
}
The issue is that after rendering all the points onto Canvas, when I try to enter some text into textbox within the same grid, there is significant lag. What can I do to reduce the lag?

What you can do, is draw all of your data into a Visual and render it with a RenderTargetBitmap. This will take a while but will solve your performance-issues.
private List<Point> _points;
private Size _targetarea;
private void btnGenerate_Click(object sender, RoutedEventArgs e)
{
_targetarea = new Size(500, 500);
_points = GeneratePoints(10000, _targetarea);
TheImage.Source = DrawPoints(_points, _targetarea, 5, 5, 1);
}
private List<Point> GeneratePoints(int count, Size size)
{
Random r = new Random();
return Enumerable.Range(0, count)
.Select(j => new Point(r.NextDouble() * size.Width, r.NextDouble() * size.Height))
.ToList();
}
private static ImageSource DrawPoints(IEnumerable<Point> points, Size size, double radiusX, double radiusY, double thickness)
{
DrawingVisual visual = new DrawingVisual();
using (DrawingContext context = visual.RenderOpen())
{
var fill = new SolidColorBrush(Colors.Yellow);
var stroke = new SolidColorBrush(Colors.Red);
foreach(var center in points)
{
context.DrawEllipse(fill, new Pen(stroke, thickness), center, radiusX, radiusY);
}
}
RenderTargetBitmap bitmap = new RenderTargetBitmap((int)size.Width, (int)size.Height, 96, 96, PixelFormats.Pbgra32);
bitmap.Render(visual);
bitmap.Freeze();
return bitmap;
}
To determine the nearest data point when you click on the image, simply use the original data that you used to generate the image.
private void TheImage_OnMouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
Point clickPoint = GetCoordinates(e.GetPosition((IInputElement) sender));
Point closestPoint = new Point();
double minSquareDistance = double.MaxValue;
foreach (Point point in _points)
{
double dX = point.X - clickPoint.X;
double dY = point.Y - clickPoint.Y;
double squareDistance = dX * dX * dY * dY;
if (squareDistance < minSquareDistance)
{
minSquareDistance = squareDistance;
closestPoint = point;
}
}
MessageBox.Show($"Clicked near {closestPoint.X:f0}/{closestPoint.Y:f0}");
}
private Point GetCoordinates(Point position)
{
// Here you need to translate Screen-Coordinates to your internal coordinate system
return position;
}
To speed up the process, install WriteableBitmapEx (NuGet available) and use this instead:
private static ImageSource DrawPoints(IEnumerable<Point> points, Size size, double radiusX, double radiusY, double thickness)
{
WriteableBitmap bitmap = new WriteableBitmap((int)size.Width, (int)size.Height, 96, 96, PixelFormats.Pbgra32, null);
foreach(var center in points)
{
bitmap.FillEllipse((int)(center.X - radiusX), (int)(center.Y - radiusY), (int)(center.X + radiusX), (int)(center.Y + radiusY), Colors.Yellow);
bitmap.DrawEllipse((int) (center.X - radiusX), (int) (center.Y - radiusY), (int) (center.X + radiusX), (int) (center.Y + radiusY), Colors.Red);
}
bitmap.Freeze();
return bitmap;
}
I don't have access to the rest of your code, so the following code is using a lot of simplifications and random data e.g. I'm using an Image instead of a Canvas and simple Points instead of whatever your parser returns.

WPF Image Cropping

I have a problem with image cropping. When I try to crop image with rectangle the cropping area is set to some strange value.
Here is xaml code:
<Grid Name="GridImage">
<Image Name="LoadedImage" SnapsToDevicePixels="True" RenderOptions.BitmapScalingMode="HighQuality">
</Image>
<Canvas x:Name="ImageCanvas" ClipToBounds="True">
<Rectangle x:Name="SelectionRectangle" Stroke="LightBlue" Fill="#220000FF" Visibility="Collapsed" />
</Canvas>
</Grid>
and here how I crop image:
var imagePosition = LoadedImage.TransformToAncestor(GridImage).Transform(new Point(0, 0));
Rect rect1 = new Rect(Math.Max(Canvas.GetLeft(SelectionRectangle) - imagePosition.X, 0), Math.Max(Canvas.GetTop(SelectionRectangle) - imagePosition.Y, 0), SelectionRectangle.Width, SelectionRectangle.Height);
var ratioX = LoadedImage.Source.Width / LoadedImage.ActualWidth;
var ratioY = LoadedImage.Source.Height / LoadedImage.ActualHeight;
Int32Rect rcFrom = new Int32Rect
{
X = (int)(rect1.X * ratioX),
Y = (int)(rect1.Y * ratioY),
Width = (int)(rect1.Width * ratioX),
Height = (int)(rect1.Height * ratioY)
};
try
{
BitmapSource bs = new CroppedBitmap((BitmapSource)LoadedImage.Source, rcFrom);
LoadedImage.Source = bs;
SetImageStretch(LoadedImage);
SetElementVisibility(Visibility.Hidden, Visibility.Visible, SelectionRectangle);
}
private void SetImageStretch(Image image)
{
if (image.Source.Width > image.ActualWidth || image.Source.Height > image.ActualHeight)
image.Stretch = Stretch.Uniform;
else image.Stretch = Stretch.None;
}
Does anybody know how to fix that?
Here how it looks before cropping:
How it looks after cropping:

Most likely the problem is with image resolution, e.g. 300 dpi, versus screen resolution (most likely 96 dpi). Have you checked image's PixelWidth and PixelHeight?

Performance optimization of multiple rotating images

I'm working on a project in C# windows forms where I'm using pictureboxes to display multiple tachometers.
Each picturebox consists of a background image and an image. The background image is static, but the image (displaying the actual indicator) is rotated and updated a couple of times per second.
Background image: http://i.stack.imgur.com/FBsKI.jpg
Fron image: http://i.stack.imgur.com/T0nJU.jpg
It works well as long as I have one or two tachometers running simultaneously, but when I add more it starts to lag a lot.
Here is the code I use for updating the picturebox:
private void timer4_Tick(object sender, EventArgs e)
{
Bitmap image = RotateImage(indicatorImg, i++, false, true, Color.Transparent);
pictureBox4.Image = image;
}
The RotateImage function I'm using (Credit to RenniePet at https://stackoverflow.com/a/16027561/3660713):
public static Bitmap RotateImage(Image inputImage, float angleDegrees, bool upsizeOk,
bool clipOk, Color backgroundColor)
{
lock (lockObject)
{
// Test for zero rotation and return a clone of the input image
if (angleDegrees == 0f)
return (Bitmap)inputImage.Clone();
// Set up old and new image dimensions, assuming upsizing not wanted and clipping OK
int oldWidth = inputImage.Width;
int oldHeight = inputImage.Height;
int newWidth = oldWidth;
int newHeight = oldHeight;
float scaleFactor = 1f;
// If upsizing wanted or clipping not OK calculate the size of the resulting bitmap
if (upsizeOk || !clipOk)
{
double angleRadians = angleDegrees * Math.PI / 180d;
double cos = Math.Abs(Math.Cos(angleRadians));
double sin = Math.Abs(Math.Sin(angleRadians));
newWidth = (int)Math.Round(oldWidth * cos + oldHeight * sin);
newHeight = (int)Math.Round(oldWidth * sin + oldHeight * cos);
}
// If upsizing not wanted and clipping not OK need a scaling factor
if (!upsizeOk && !clipOk)
{
scaleFactor = Math.Min((float)oldWidth / newWidth, (float)oldHeight / newHeight);
newWidth = oldWidth;
newHeight = oldHeight;
}
// Create the new bitmap object. If background color is transparent it must be 32-bit,
// otherwise 24-bit is good enough.
Bitmap newBitmap = new Bitmap(newWidth, newHeight, backgroundColor == Color.Transparent ?
PixelFormat.Format32bppArgb : PixelFormat.Format24bppRgb);
newBitmap.SetResolution(inputImage.HorizontalResolution, inputImage.VerticalResolution);
// Create the Graphics object that does the work
using (Graphics graphicsObject = Graphics.FromImage(newBitmap))
{
graphicsObject.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
graphicsObject.PixelOffsetMode = System.Drawing.Drawing2D.PixelOffsetMode.HighQuality;
graphicsObject.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
// Fill in the specified background color if necessary
if (backgroundColor != Color.Transparent)
graphicsObject.Clear(backgroundColor);
// Set up the built-in transformation matrix to do the rotation and maybe scaling
graphicsObject.TranslateTransform(newWidth / 2f, newHeight / 2f);
if (scaleFactor != 1f)
graphicsObject.ScaleTransform(scaleFactor, scaleFactor);
graphicsObject.RotateTransform(angleDegrees);
graphicsObject.TranslateTransform(-oldWidth / 2f, -oldHeight / 2f);
// Draw the result
graphicsObject.DrawImage(inputImage, 0, 0);
}
return newBitmap;
}
}
Any ideas on how to improve performance? Is there a better way to do this?

I would create a static list of images for each angle of degrees required on startup.
You could then reference the correct index in the list and use that pre rotated image.

Thank you for your help!
What I did was that I added a WPF control to my Winforms project. The WPF control contains the two images and simply contains code for rotating the front image:
C#:
public void Rotate(float deg)
{
RotateTransform rt = new RotateTransform(deg);
image2.RenderTransform = rt;
}
XAML:
<UserControl x:Class="Project.WPF_imgControl"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
mc:Ignorable="d"
d:DesignHeight="300" d:DesignWidth="300">
<Grid>
<Image HorizontalAlignment="Left" Name="image1" Stretch="Fill" VerticalAlignment="Top" Source="/Project;component/Images/Tachometer.png" />
<Image RenderTransformOrigin =".5,.5" HorizontalAlignment="Left" Name="image2" Stretch="Fill" VerticalAlignment="Top" Source="/Project;component/Images/Pointer_80%25.png" />
</Grid>
Now I just have to find out why the image looks jagged in the WPF component compared to my standard form.
Edit: RenderOptions.BitmapScalingMode="HighQuality" did the trick :)
//Jocke