I need to perform video scaling in my C++ app. I know I can use COM to communicate to C#, and I know C# can perform video scaling, so I am thinking of bridging the two worlds.
How can I, using C#, scale a video image which is in memory, and then get the video data after the image is scaled?
This question seems similar but how do I get the data after scaling instead of showing it to screen?
High Quality Image Scaling Library
C# (using GDI+ a.k.a. System.Drawing) can scale individual images, but it has no built-in way of scaling full videos (like MPEGs or AVIs).
Assuming that you actually only need to scale individual images (i.e. not full videos), then you would be better off doing this from C++ (StretchBlt would be the main API method that you would use for this.
Related
If I have a video file that defines a video image that is 6144 pixels long (x) by 64 pixels high (y) and I want to display that video so that it wraps at the end of the monitor. In other words I want to display the first 1024 pixels of the video starting at position 0,0 on the monitor, then video pixels 1024 to 2047 starting at position 0,64, and repeat this until all 6144 pixels are shown on the monitor. That would mean the video needs to wrap around on a 1024x768 monitor 6 times.
What is the best way to do this? Can DirectX, DirectShow, Media Foundation, or Windows Media Player ActiveX Control handle this wrapping for you automatically? I need to do this preferably in C#, but not opposed to dropping in to C++ native. Or is the only way to do this, is to split the video into 6 separate sections and play them in separate window? If splitting them into 6 separate videos and playing them in 6 separate windows is the only reasonable way, how do you make certain they start at the same time so they are sync'd?
Just thinking about something per comment below could ffmpeg and/or C# transform this 6144 x 64 pixel video file into something like this:
6144 x 64 ---> 0-1023 x 64
1024-2047 x 64
2048-3071 x 64
3072-4095 x 64
4096-5119 x 64
5120-6143 x 64
In other words what looks like it's wrapped but really just one video that's 1024 x 384 ??
You need to develop a transformation which converts your 6144x64 video to resolution in question (1024x768 or different) and integrate it with one of the player pipelines. When you convert the video frames to required resolution, the frames can be presented as usual video playback, esp. fullscreen in you need to span in across the entire monitor, and such playback on its presentation end will not differ from play back a regular video file meaning that you can use standard components and APIs.
All video APIs are native: in most, if not all, cases you would be better off implementing the transformation in C++ rather than C#.
With DirectShow you typically develop a transform filter which accept the video frames and rearranges the pixel data respectively to your requirements. With Media Foundation the similar task is achieved by developing a Media Foundation Transform (with the data processing in CPU or GPU domain). In both cases you are packing your transformation step into API defined form factor and extending the standard pipeline.
Otherwise you can also prepare the frames outside of the playback pipelines and inject them already prepared. Even though possible, it is perhaps a more complicated way but it can be preferred by those who are not well familiar with the mentioned APIs.
When you prepare a re-arranged frame for presentation at once you don't need to do additional presentation synchronization. Presumably, this is the way to achieve the mentioned task as the splitting into parts and managing synchronized presentation is a more complicated for now reason alternative.
DirectShow vs. Media Foundation - both APIs let you play video with the same quality and performance (exception might be that you need GPU only processing, in which case Media Foundation might be a better choice, but in your case it's unlikely that you can leverage this advantage).
DirectShow is older and near stop of its development but offers much more online tutorials, discussions, materials, helpers and samples. Windows SDK 7.1 EzRGB24 Filter Sample is a good start point for a transform filter.
Media Foundation is newer and "current" API, presumably a more reasonable choice. Windows SDK 7.1 offers MFT_Grayscale Sample for a starting development point. It is generally possible to implement a C# MFT for Media Foundation (there are good reasons to not do it - 1, 2). Even though DirectShow is notorious for being an API with a step learning curve, for the video effect developer Media Foundation is even a more complicated API.
Generally speaking the API choice should take into consideration, if not even be defined by, your preference for playback pipeline.
I am working on displaying a Y4M movie to a screen. Parsing the file I can handle however I currently have a 2d byte array with the YUV values of the frame. Is there a built in hardware accelerated way or do I have to convert it to RGB and create a Bitmap? If anyone has a good website for a tutorial in DirectX in WPF it would be greatly appreciated as well. By the way this is in C#.
YUV surfaces are likely special surfaces supported by a specific hardware vendor (IHV) FourCC. You will need to find out from the IHV if they have a FourCC you can use and you can't rely on it working on other platforms.
There is a standard D3DFORMAT for 4:2:2 (D3DFMT_YUY2 or D3DFMT_UYVY) but there is no default one for 4:4:4. I'm sure most IHVs do provide a FourCC for such rendering.
Its worth bearing in mind that if you know the FourCC of the incoming data then it may well be worth simply trying to create a surface with that FourCC directly. If it works, you are ready to go :) If not then you may be able to find an alternative that works or you may have to do some manual format conversion.
I need some help deciding what to use to acquire an image from a webcam. I want to acquire a single image. I know you can typically acquire a still image at a higher resolution than a single video frame.
Currently, I am using MATLAB's image acquisition toolbox.. which apparently only supports obtaining frames in video mode(so lower resolution). Which other libraries do you recommend? Has anyone else encountered this problem?
Are you referring to the fact that the largest resolution reported by the Image Acquisition Toolbox is (for example) 1024x768 but the webcam claims that it can acquire 6 megapixel still images? If so, every webcam that I have ever seen has a note in very small print somewhere that explains that the higher resolution is achieved via software interpolation.
You can just acquire the image in the largest format supported by the toolbox and then use IMRESIZE to scale the image to whatever resolution that you want.
We've used WIA at work before. I can't share our code, but we basically bring up the WIA capture screen (which the user has to interact with before the image is captured). For an automated solution, have a look at this: http://www.codeproject.com/KB/cs/WebCamService.aspx
i'm developping an application in CF 3.5 for windows Mobile 6 Pro using C# and i have a little issue requiring the advice of someone that knows better.
Basically, i want my application to run and scale on multiple device sizes and resolutions. Eveything scales properly but the images.
Some images that are for example 16X16 will look very small on a high resolution screen, so I want to display a 32X32 image, but I don't know what's the best way to decide which image size to display.
I have the option to check the dpi and then manually choose which image to display, but it seems like dirty work.
Isn't there any way to do it otherwise or what's the best way to do it?
I recommend that you create a layer between your forms and the images. Create a new class that would be responsible for returning the correct sized image. The code in your forms will rely on that to get the image and would have to know nothing about the sizes. For example:
mypicturebox.Image = ImageFactory.Image01;
The good thing is that you can use any technique you want inside the ImageFactory without affecting the rest of the code. The easiest thing to do is to check the size of the screen (using Screen.PrimaryScreen.WorkingArea) and do a manual decision.
Is there another way to render graphics in C# beyond GDI+ and XNA?
(For the development of a tile map editor.)
SDL.NET is the solution I've come to love. If you need 3D on top of it, you can use Tao.OpenGL to render inside it. It's fast, industry standard (SDL, that is), and cross-platform.
Yes, I have written a Windows Forms control that wraps DirectX 9.0 and provides direct pixel level manipulation of the video surface.
I actually wrote another post on Stack Overflow asking if there are other better approaches: Unsafe C# and pointers for 2D rendering, good or bad?
While it is relatively high performance, it requires the unsafe compiler option as it uses pointers to access the memory efficiently. Hence the reason for this earlier post.
This is a high level of the required steps:
Download the DirectX SDK.
Create a new C# Windows Forms project and reference the installed
Microsoft DirectX assembly.
Initialize a new DirectX Device object with Presentation Parameters
(windowed, back buffering, etc.) you require.
Create the Device, taking care to record the surface "Pitch" and
current display mode (bits per pixel).
When you need to display something, Lock the backbuffer
surface and store the returned pointer to the start of surface
memory.
Use pointer arithmetic, calculate the actual pixel position in the
data based on the surface pitch,
bits per pixel and the actual x/y pixel coordinate.
In my case for simplicity I am sticking to 32 bpp, meaning setting a pixel is as simple as: *(surfacePointer + (y * pitch + x))=Color.FromARGB(255,0,0);
When finished drawing, Unlock the back buffer surface. Present the surface.
Repeat from step 5 as required.
Be aware that taking this approach you need to be very careful about checking the current display mode (pitch and bits per pxiel) of the target surface. Also you will need to have a strategy in place to deal with window resizing or changes of screen format while your program is running.
Managed DirectX (Microsoft.DirectX namespace) for faster 3D graphics. It's a solid .NET wrapper over DirectX API, which comes with a bit of performance hit for creating .NET objects and marshalling. Unless you are writing a full featured modern 3D engine, it will work fine.
Window Presentation Foundation (WPF) (Windows.Media namespace) - best choice for 2D graphics. Also has limited 3D abilities. Aimed to replace Windows Forms with vector, hardware accelerated resolution-independent framework. Very convenient, supports several flavours of custom controls, resources, data binding, events and commands... also has a few WTFs. Speed is usually faster than GDI and slower than DirectX, and depends greatly on how you do things (seen something to work 60 times faster after rewriting in a sensible way). We had a success implementing 3 1280x1024 screens full of real-time indicators, graphs and plots on a single (and not the best) PC.
You could try looking into WPF, using Visual Studio and/or Expression Blend. I'm not sure how sophisticated you're trying to get, but it should be able to handle a simple editor. Check out this MSDN Article for more info.
You might look into the Cairo graphics library. The Mono project has bindings for C#.
Cairo is an option. I'm currently rewriting my mapping software using both GDI+ and Cairo. It has a tile map generator, among other features.