We Keep Coding

Is it possible to stream by rtsp bitmap images?

I'm working on an application C# which consist in collect images (format bayer8) from a camera and then stream it over the local network by rtsp.
I looked for a framework which allow me to send over the network bitmap images.
Every time I find something, it's a file which is streamed, but i need to stream bitmap images.
Thanks for your help

If you are proficient enough to use native code (either through COM or P/Invoke), Live555 can be used to achieve what you need. I am not aware if it has as C# wrapper (IMHO most likely no). There is also this answer that mentions a codeproject page on RTSP in C# alongside Live555.
That being said I think there is a more important bit of information in your case (which is why I did not simply mark the question as a duplicate). RTP has a predefined set of payload types (so that the clients know how to remux the stream). RFC4175 lists the uncompressed video formats for RTP. As far as I understand bayer8 is not just another name for some kind of RGB compression - it is a totally different one (correct me if I'm wrong)? In that case you just can't put your data in a stream that standard clients (VLC, FFmpeg, etc.) will be able to demux. Then it's up to you to understand if you can recompress or if you don't need RTP at all since it will simply not work.

How to load cmyk image with c++ or c# and to change cmyk values of every pixel?

For a whole month I've been trying to find out how to load a cmyk image (8-bit tiff) and do a UCA, UCR, GCR transformations.
I know the formulas for this transformations, but I have a problem to load an cmyk image with C++ or C#.
My plan is to manipulate the image pixel by pixel and change every value C, M, Y, K of that pixel, and finally save changed image as 8-bit tiff cmyk image. So basically I want to run through each pixel, make change to cmyk values, and save the image.
I have difficulties to find and install such a library for C# or C++, and I'm using Visual Studio Express 2010 for both.
What libraries can I use to load these images?

libtiff will let you manipulate TIFF files. See also this answer on building libtiff using Visual Studio.
You'll need to read the image row by row using TIFFReadScanline to avoid conversion to RGB by the library. The open source tifftopnm utility does this; its source is here - it uses pick_cmyk_pixel to convert the data to RGB after obtaining it, but naturally you can just substitute your own logic instead.

EMGU is a c# wrapper for opencv (c++) either of these will allow Tiff files to be easily used and manipulated. They will allow you to access pixels more readily and uses highly optimised code. Since these are a dedicated image processing libraries they will allow you to do more with your image without having to worry about RGB conversion or reading the file in etc.
Hope they help,
Cheers
Chris

C#: Decoding JPEG images with 12-bit precision using Silverlight FJCore library?

In my C# Silverlight application, I am trying to decode DICOM images in compressed JPEG transfer syntax, using the FJCore class library.
The DICOM images are normally compressed with 12-bit precision. When trying to decode such an image using the original FJCore source code, I get an exception saying "Unsupported codec type", because in the original FJCore implementation only SOF0 (Baseline DCT) and SOF2 (Progressive DCT) Start-of-Frame markers are supported. If I change the implementation to also accept the SOF1 marker (Extended Sequential DCT) and treat SOF1 frames the same way as SOF0 frames, the images are decoded, but only 8 bits are accounted for.
A typical 12-bit precision image now looks like this after decoding with the modified FJCore library:
Ideally, the image should look like this:
As far as I have been able to tell from the FJCore implementation, the image precision is recorded in the JpegFrame class, but it is never used. The original FJCore implementation seems to only fully support grayscale images with 8 bit precision.
I am planning to "take the bull by the horns" and try to extend FJCore myself to support 12-bit precision for grayscale images. But before I do, I thought I should pose the question here in StackOverflow to see if anyone has encountered and solved this problem before? In that case, I would be very happy to learn how you solved the problem.
Many thanks in advance!
Anders # Cureos

I just updated my own JPEG decoder to handle the extended mode and what I needed to change was my inverse DCT. Before changing the code, the output looked similar to your sample image above. I have always stored 16-bit coefficient values from the entropy decode, but my DCT calculation was corrupting the larger values by using 16-bit integers to hold temporary values while doing the math. I changed the DCT code to use 32-bit integers for the calculations and that solved the problem.

How to read using C++ (C#) sound stream sent by flash?

I need to read sound stream sent by flash audio in my C++ application (C++ is not a real limitation, it may be C# or any other desktop language).
Now flash app sends audio to another flash app but I need to receive the same audio by desktop application.
So, is there a standard or best way how to do it?
Thank you for your answers.

How is the sound actually sent? Via the network?
Edit: You'd be either capturing the audio from an HTTP stream, or an RTMP stream. Run Wireshark to find out, but I suspect you're doing something slightly shady...

You could try using the sound system from the Gnash project.

So basically you want to connect to RTMP sound stream from flash media server from an arbitrary non-flash application? Have you taken a look at http://en.wikipedia.org/wiki/Real_Time_Messaging_Protocol ?

Unfortunately, Adobe IS relatively proprietary (hence the apple-adobe wars happening lately), but for several languages, there are projects to help out with RTMP.
WebOrb is commercial, for .NET, Java, PHP:
http://www.themidnightcoders.com/products.html
FluorineFX is open source for .NET only:
http://www.fluorinefx.com/
I haven't used either myself for RTMP, but I have used FluorineFX to connect to a flash remoting (AMF) gateway. I imagine it may do what you need for receiving the audio stream from a .NET-enabled client.

Getting the frames, frame rate and other attributes of video clip
If you have experience with writing applications in Microsoft DirectShow Editing Services (codename Dexter), this will sound very familiar to you. In the Windows environment, traditionally capturing still frames has been done using C++ and Dexter Type Library to access DirectShow COM objects. To do this in .NET Framework, you can make an Interop assembly of DexterLib which is listed under COM References in VS 2005. However it takes you a good amount of work to figure out how to convert your code from C++ to C# .NET. The problem occurs when you need to pass in a pointer reference as an argument to a native function, CLR does not directly support pointers as the memory position can change after each garbage collection cycle. You can find many articles on how to use DirectShow on the CodeProject or other places and we try to keep it simple. Here our goal is to convert a video file into an array of Bitmaps and I tried to keep this as short as possible, of course you can write your own code to get the Bitmaps out of a live stream and buffer them shortly before you send them.
Basically we have two option for using the DirectShow for converting our video file to frames in .NET:
Edit the Interop assembly and change the type references from pointer to C# .NET types.
Use pointers with unsafe keyword.
We chose the unsafe (read fast) method. It means that we extract our frames outside of .NET managed scope. It is important to mention that managed does not always mean better and unsafe does not really mean unsafe!
MediaDetClass mediaClass = new MediaDetClass();
_AMMediaType mediaType;
... //load the video file
int outputStreams = mediaClass.OutputStreams;
outFrameRate=0.0;
for (int i = 0; i < outputStreams; i++)
{
mediaClass.CurrentStream = i;
try{
//If it can the get the framerate, it's enough,
//we accept the video file otherwise it throws an exception here
outFrameRate = mediaClass.FrameRate;
.......
//get the attributes here
.....
}catch
{ // Not a valid meddia type? go to the next outputstream }
}
// No frame rate?
if (outFrameRate==0.0)
throw new NotSupportedException( " The program is unable" +
" to read the video file.");
// we have a framerate? move on...
...
//Create an array to hold Bitmaps and intilize
//other objects to store information...
unsafe {
...
// create a byte pointer to store the BitmapBits
...
while (currentStreamPos < endPosition)
{
mediaClass.GetBitmapBits(currentStreamPos, ref bufferSize,
ref *ptrRefFramesBuffer,
outClipSize.Width, outClipSize.Height);
...
//add frame Bitmap to the frameArray
...
}
}
...
Transfer extracted data over HTTP
So far we have converted our video to an array of Bitmap frames. The next step is to transfer our frames over HTTP all the way to the client�s browser. It would be nice if we could just send our Bitmap bits down to the client but we cannot. HTTP is designed to transport text characters which mean your browser only reads characters that are defined in the HTML page character set. Anything else out of this encoding cannot be directly displayed.
To accomplish this step, we use Base64 encoding to convert our Bitmap to ASCII characters. Traditionally, Base64 encoding has been used to embed objects in emails. Almost all modern browsers including Gecko browsers, Opera, Safari, and KDE (not IE!) support data: URI scheme standard to display Base64 encoded images. Great! Now, we have our frames ready to be transferred over HTTP.
System.IO.MemoryStream memory = new System.IO.MemoryStream();
while (currentStreamPos < endPosition)
{
...
// Save the Bitmpas somewhere in the (managed) memory
vdeoBitmaps.Save(memory, System.Drawing.Imaging.ImageFormat.Jpeg);
//Convert it to Base64
strFrameArray[frameCount] = System.Convert.ToBase64String(memory.ToArray());
//Get ready for the next one
memory.Seek(0, System.IO.SeekOrigin.Begin);
}
memory.Close();
...
But we cannot just send out the encoded frames as a giant string. We create an XML document that holds our frames and other information about the video and then send it to the client. This way the browser can receive our frames as a DOM XML object and easily navigate through them. Just imagine how easy it is to edit a video that is stored in XML format:
14.9850224700412
{Width=160, Height=120}
6.4731334
/9j/4AAQSkZJRgABAQEAYAB....
....
This format also has its own drawbacks. The videos that are converted to Base64 encoded XML files are somewhere between 10% (mostly AVI files) to 300 % or more (some WMV files) bigger than their binary equivalent.
If you are using an XML file, you even don't need a web server , you can open the HTML from a local directory and it should work! I included an executable in the article's download file that can convert your video file to XML document which later can be shown in the browser. However using big files and high resolution videos is not a good idea!
OK, now we can send out our �Base64 encoded video� XML document as we would do with any other type of XML files. Who says XML files always have to be boring record sets anyway

How to create an image from a raw data of DICOM image

I have a raw pixel data in a byte[] from a DICOM image.
Now I would like to convert this byte[] to an Image object.
I tried:
Image img = Image.FromStream(new MemoryStream(byteArray));
but this is not working for me. What else should I be using ?

One thing to be aware of is that a dicom "image" is not necessarily just image data. The dicom file format contains much more than raw image data. This may be where you're getting hung up. Consider checking out the dicom file standard which you should be able to find linked on the wikipedia article for dicom. This should help you figure out how to parse out the information you're actually interested in.

You have to do the following
Identify the PIXEL DATA tag from the file. You may use FileStream to read byte by byte.
Read the pixel data
Convert it to RGB
Create a BitMap object from the RGB
Use Graphics class to draw the BitMap on a panel.

The pixel data usually (if not always) ends up at the end of the DICOM data. If you can figure out width, height, stride and color depth, it should be doable to skip to the (7FE0,0010) data element value and just grab the succeeding bytes. This is the trick that most normal image viewers use when they show DICOM images.

There is a C# library called EvilDicom (http://rexcardan.com/evildicom/) that can be used to pull the image out of a DICOM file. It has a tutorial on how to do it on the website.

You should use GDCM.
Grassroots DiCoM is a C++ library for DICOM medical files. It is automatically wrapped to python/C#/Java (using swig). It supports RAW, JPEG 8/12/16bits (lossy/lossless), JPEG 2000, JPEG-LS, RLE and deflated (zlib).
It is portable and is known to run on most system (Win32, linux, MacOSX).
http://gdcm.sourceforge.net/wiki/index.php/GDCM_Release_2.4
See for example:
http://gdcm.sourceforge.net/html/DecompressImage_8cs-example.html

Are you working with a pure standard DICOM File? I've been maintainning a DICOM parser for over a two years and I came across some realy strange DICOM files that didn't completely fulfill the standard (companies implementing their "own" twisted standard DICOM files) . flush you byte array into a file and test whether your image viewer(irfanview, picassa or whatever) can show it. If your code is working with a normal JPEG stream then from my experience , 99.9999% chance that this simply because the file voilate the standard in some strange way ( and believe me , medical companies does that a lot)
Also note that DICOM standard support several variants of the JPEG standard . could be that the Bitmap class doesn't support the data you get from the DICOM file. Can you please write down the transfer syntax?
You are welcome to send me the file (if it's not big) yossi1981#gmail.com , I can check it out , There was a time I've been hex-editing DICOM file for a half a year.

DICOM is a ridiculous specification and I sincerely hope it gets overhauled in the near future. That said Offis has a software suite "DCMTK" which is fairly good at converting dicoms with the various popular encodings. Just trying to skip ahead in the file x-bytes will probably be fine for a single file but if you have a volume or several volumes a more robust strategy is in order. I used DCMTK's conversion code and just grabbed the image bits before they went into a pnm. The file you'll be looking for in DCMTK is dcm2pnm or possibly dcmj2pnm depending on the encoding scheme.
I had a problem with the scale window that I fixed with one of the runtime flags. DCMTK is open source and comes with fairly simple build instructions.