I'm working with c# and OpenCV. I have a Bitmap that I want to write as a frame of video using the VideoWriter provided by OpenCV. I've done this in Python so know it will work. I just need the conversion step from Bitmap to Mat.
My (partial) code looks roughly like this...
VideoWriter video = new VideoWriter(filename, fps, frameSize, false);
Bitmap image = SomethingReturningABitmap();
// NEED CONVERT FROM Bitmap to Mat
Mat frame = new Mat();
video.Write(frame);
I'm using the Bitmap Converter OpenCV Extensions to convert my Bitmap to a Mat object in memory.
PSTK ps = new PSTK();
Image img = ps.CaptureScreen();
Bitmap bmpScreenshot = new Bitmap(img);
image = BitmapConverter.ToMat(bmpScreenshot);
One could also do the below but then you will incur significant overhead for writing and reading the data from disk.
myBitmap.Save("TempFile.PNG");
Mat myMatImage = CvInvoke.Imread("TempFile.PNG");
I wanted to see what the difference looked like, roughly. I was processing a 1920x1080 screen cast of my desktop for 500 frames using each method and here's what I found.
On average, for the Bitmap.Save/Cv2.ImRead method, it took 0.1403 seconds per frame.
On average, for the Bitmap Converter method, it took 0.09604 seconds per frame. It takes roughly 50% longer to save and re-read the file using my home desktop machine. Core I7 2nd gen, 16GB RAM.
Related
I have an image with the color format YCbCr (CMYK). When I load this with the .NET bitmap and save the result, the color format will change to RBG and the filze increase from ~1.600 KB to ~7.500 KB.
using (Image bitmap = Image.FromFile(#"C:\Test\Original.tif"))
{
bitmap.Save(#"C:\Test\result.tif");
}
The sample file is a multi frame Tiff file with the color format YCbCr (CMYK) and it is 24 bit color deep. Its a Tiff container that includes two JPEG files with no compression.
Here you can download the sample file: https://ufile.io/9x21n
I already tried a OpenCV wrapper Nuget Package (https://github.com/shimat/opencvsharp), but the result is the same.
Mat imageForProcessing = Cv2.ImRead(#"C:\Test\Original.tif");
Cv2.ImWrite(#"C:\Test\result.tif", imageForProcessing);
Is there a free third party library, a .NET or OpenCV way to read and write the image without lossing the color format and increasing the file size five times?
Update 1
I tried the TiffBitmapEncoder class with the following source code. The bit deep is now correct (decreases the result from ~7.500 KB to ~6.400 KB), but the output is againg in RGB.
FileStream stream = new FileStream(#"C:\Test\original.tif", FileMode.Open);
TiffBitmapEncoder encoder = new TiffBitmapEncoder();
encoder.Frames.Add(BitmapFrame.Create(stream));
encoder.Save(stream);
I would appreciate any help!
Regards
Sascha
In my program, I compress a BMP into a JPEG like this:
private void ConvertBmpToStreamJPG30(Bitmap b, Stream s)
{
s.Flush();
EncoderParameters encoderParameters = new EncoderParameters(1);
encoderParameters.Param[0] = new EncoderParameter(System.Drawing.Imaging.Encoder.Quality, 30L);
b.Save(s, GetEncoder(ImageFormat.Jpeg), encoderParameters);
}
Then a function is receiving the JPEG in a MemoryStream, I transform it into a Bitmap by doing
Bitmap b = new Bitmap(stream);
When I display the image, there are a lot of lines like this :
What am I doing wrong, people?
Edit 1
Here a small visual studio solution showing the problem: http://www.fast-files.com/getfile.aspx?file=79311
It is the beginning of a screen sharing software. What it does: It takes screenshots, compare them, compress the difference and send it to another part of the program that decompress it and recompose an image with everything received. It opens a window displaying what is "sent" on the left and the recomposed image on the right.
Three things come to mind:
Try setting a better quality than 30 and see if that helps;
Check your RAM (and possibly video RAM, though I doubt that GDI+ might use VGA for compression) for hardware problems;
I've had a similar weird problem where I loaded some JPEG file, modified it a bit, and then saved it again. That produced an exception. The solution was to make a new bitmap based on the old one and save the copy. Try that.
I am developing a program for image processing, and I need to save some pictures from a video and do some processing on them.
When dealing with 1 picture it doesn't really take time,
But when I'm dealing with 100 pictures it makes difference
I'm saving the files to my hard disk, and that's why it takes time
the thing is, the function I'm using is a ready made function and it only accepts (file name)
the function is really complicated so i cannot build my own function ( if that's what you are thinking )
I'm thinking of 2 things right now and would like to have your opinions about them:
change the input of the function, but how ? is there a way to change this input from a ( file name ) to an array which holds these pictures ?
save the file to ram. but how to save files to ram by names, and be able to use them as ( file name ) in the function ?
I appreciate your help , thanks so much
this is my code but i still have problems:
Capture video = new Capture("c:\\5.avi");
Image<Bgr, Byte> Imageframe ;
Dictionary<string, MemoryStream> dict = new Dictionary<string, MemoryStream>();
Imageframe = video.QueryFrame();
Bitmap bmp = Imageframe.ToBitmap();
dict.Add("mypicture.png", new MemoryStream());
bmp.Save(dict["mypicture.png"],imageformat.png);
its saying imageformat does not exist in the context
and this is the function im using :
Image<Bgr, byte> result;
result = DrawMatches.Draw("box.png", "box_in_scene.png", out matchTime,i);
You could save to RAM (in the loosest of senses) using a MemoryStream. If you wanted to name them you could use a Dictionary<string,MemoryStream>, such as:
Dictionary<string,MemoryStream> dict = new Dictionary<string,MemoryStream>();
dict.Add("mypicture.png",new MemoryStream());
image.Save(dict["mypicture.png"]);
However you'll need to write cleanup code for these streams and this doesn't take into account that eventually you could use up all the physical RAM and then start going into the paging file, which is disk based anyhow.
As an aside, if this is on a server you control and you're not releasing this software to end users you could get a RAM disk drive (plenty around just Google) that uses physical RAM as an actual disk available to Windows. Then you could just load/save to that.
Very very rough EmguCv variant:
// Global dictionary of memory streams
Dictionary<string,MemoryStream> dict = new Dictionary<string,MemoryStream>();
// Add image memory stream to dictionary
dict.Add("mypicture.png",new MemoryStream());
// Get bitmap from EmguCv
Bitmap bmp = image.ToBitmap();
// Save bitmap to image memory stream
bmp.Save(dict["mypicture.png"],ImageFormat.Png);
// Get bitmap from memory stream
dict["mypicture.png"].Seek(0,SeekOrigin.Begin);
Bitmap new_bmp = Image.FromStream(dict["mypicture.png"]);
// Convert bitmap to EmguCv image
Image<Bgr,Byte> new_img = new Image<Bgr,Byte>(new_bmp);
Try to use MemoryStream for work with memmory like with file,
and try to save files on harddisk by thread
I understand you are using a DLL you dont have the source for.
You might be able to load it into a reflector, and modify it to take the image as an argument instead of the file name.
I used Red Gate's reflector in the past and it worked for me: http://www.reflector.net/
after a whole day of testing I came up with this code, which captures current screen using DirectX (SlimDX) and saves it into a file:
Device d;
public DxScreenCapture()
{
PresentParameters present_params = new PresentParameters();
present_params.Windowed = true;
present_params.SwapEffect = SwapEffect.Discard;
d = new Device(new Direct3D(), 0, DeviceType.Hardware, IntPtr.Zero, CreateFlags.SoftwareVertexProcessing, present_params);
}
public Surface CaptureScreen()
{
Surface s = Surface.CreateOffscreenPlain(d, Screen.PrimaryScreen.Bounds.Width, Screen.PrimaryScreen.Bounds.Height, Format.A8R8G8B8, Pool.Scratch);
d.GetFrontBufferData(0, s);
return s;
}
Then I do the following:
DxScreenCapture sc = new DxScreenCapture();
..code here
private void button1_Click(object sender, EventArgs e)
{
Stopwatch stopwatch = new Stopwatch();
// Begin timing
stopwatch.Start();
Surface s = sc.CaptureScreen();
Surface.ToFile(s, #"c:\temp\test.png", ImageFileFormat.Png);
s.Dispose();
stopwatch.Stop();
textBox1.Text = ("Elapsed:" + stopwatch.Elapsed.TotalMilliseconds);
}
The results are:
0. when I don't save surface: avg. elapsed time: 80-90ms
1. when I also save Surface to BMP file: format: ImageFileFormat.Bmp , avg. elapsed time: 120ms, file size: 7mb
2. when I also save Surface to PNG file: format: ImageFileFormat.Png , avg. elapsed time: 800ms, file size: 300kb
The questions are:
1. Is it possible to optimise current image capture? According to this article - Directx screen capture should be faster than GDI. For me, GDI usually takes 20ms to get a "Bitmap", whereas it takes 80ms to get "Surfare" using DX (both without saving).
http://www.codeproject.com/Articles/274461/Very-fast-screen-capture-using-DirectX-in-Csharp
2a. How to save Surface to PNG image format faster? When I save surface to 7mb BMP file it takes almost 6 times less time, than when I save the same surface to 300kb PNG file..
2b. Is it possible to save Surface directly to Bitmap so I don't have to create temporary files?
So I don't have to do following: Surface -> image file; image file open -> bitmap;, but instead: Surface -> bitmap
that's all for now. I'll gladly accept any tips, thanks!
Edit:
Just solved 2b by doing:
Bitmap bitmap = new Bitmap(SlimDX.Direct3D9.Surface.ToStream(s, SlimDX.Direct3D9.ImageFileFormat.Bmp));
Edit2:
Surface.ToFile(s, #"C:\temp\test.bmp", ImageFileFormat.Bmp);
Bitmap bitmap = new Bitmap(#"C:\temp\test.bmp");
is faster than:
Bitmap bitmap = new Bitmap(SlimDX.Direct3D9.Surface.ToStream(s, SlimDX.Direct3D9.ImageFileFormat.Bmp));
by 100 ms!!! Yeah, I couldn't believe my eyes too ;) I don't like the idea of temporary file creation, but a 50% performance increase (100-200ms instead of 200-300+) is a very good thing.
If you don't want to use SlimDX library you can also try
public Bitmap GimmeBitmap(Surface s)
{
GraphicsStream gs = SurfaceLoader.SaveToStream(ImageFileFormat.Bmp, s);
return new Bitmap(gs);
}
and try the same for .png - I did not test performance but it have to be faster than using disc temporary file :)
and as for 1st question - try to only once create surface and then on every screenshot only put into it device's buffer data and create the bitmap
d.GetFrontBufferData(0, s);
return new Bitmap(SurfaceLoader.SaveToStream(ImageFileFormat.Bmp, s));
this should save you some time :)
If performance really is an issue, you should consider writing your code in C++ instead. Therefor you dont need an external library but can directly access the backend-buffer of your video card via Windows-API + DirectX.
Accessing the backend(-video)-buffer is a lot faster than reading from the frontend-buffer.
To optimize performance (which also awnsers your question 1) use multithreading (see TPL or threading depending on your needs).
Here is an inside of how to do it in C++ CodeProject examples in C++.
From my personal experience, DirectX was by far the fastest.
These steps
1. reading backend-buffer into a bitmap to process the data
2. spawning new thread to repeat step 1 while previous thread is still busy
take about 10-40ms (together) - implemented in C++ (on NVIDIA GeForce GTX 970M) and depending on the current workload of the hardware
Possible middle course
If you want to stick with C# but also need the performance, writing a C++-dll for .NET (see .NET Programming with C++/CLI (Visual C++)) which reads the video buffer and returns the data to your C#-Code will do the trick.
This example on Code Project is almost exactly what I need... except the saveFrameFromVideo takes a percentage instead of a frame number...
How can I use this to extract frame X from a WMV file?
I've also tried FFmpeg.NET... but there weren't any downloadable builds, and I couldn't get the source to build...
You can also try AsfMojo for this task, it allows you to extract an image by time offset:
Bitmap bitmap = AsfImage.FromFile(videoFileName)
.AtOffset(17.34);
Internally the Media SDK and some custom stream manipulation is used to get frame accurate still frames (up to a 100 millisecond tolerance), so if you know the frame rate of your media file (i.e. 25) you can calculate the time offset of the nearest frame:
int frameX = 400; //get 400th frame
double frameRate = 25.0;
double timeOffset = frameX / frameRate;
Bitmap bitmap = AsfImage.FromFile(videoFileName)
.AtOffset(timeOffset);
The magic is in this line:
mediaDet.WriteBitmapBits(streamLength * percentagePosition,
target.Width, target.Height, outputBitmapFile);
It's calculating the frame number from the percentage and the length of the stream. Since you already know the frame number, use that instead.
I have been working on extracting frames from webcam videos and video files- for both, i used the AForge library- (you need to add references to AForge.Video, AForge.Imaging , AForge.Video.Directshow and AForge.Video.FFMPEG). For live videos, I added a videoSourcePlayer_NewFrame(object sender, ref Bitmap image) to get the frame- Bitmap image contains the required frame in type Bitmap. This is basically the event handler for the videosource player i added in the windows form.
For video from a file, i used:
videoSource=new FileVideoSoource(fileName);
videoSource.Start();
videoSource.NewFrame += new AForge.Video.NewFrameEventHandler(videoSource_NewFrame);
videoSource_NewFrame is the event handler in case there is a new frame.