I have a scenario that requires me to generate a printed page with encrypted data that can be scanned into our application later. Much like a barcode but encrypted.
The volume of text to be encrypted would be between 1KB to 10KB but could increase in the future. Most likely not above 1MB.
I develop document processing systems so I understand the dos and donts of printing. Scanning usually adds properties to printed pages such as skewing, rotation, artifacts, etc.
So the question is:
What is the best format to represent this amount of data visually? Not barcodes I'm assuming.
Whatever the format, real estate is crucial since we want to squeeze the data into a single A4 page. I'm thinking Base64 encoding with code-table compression could be a candidate? Any thoughts.
EDIT: It seems like barcodes are the only viable solution. The data is essentially a series of digits so I could use base 10 and then compress them to a code/index table before encrypting it. Or would using a higher base be wiser here?
My suggestion would be to make a C# program that takes your data, encrypts it, then runs it through a Base64 encoder. Then I would take the Base64 data and create QR code bitmaps from the data. I would then shrink the bitmaps so you can fit a lot on a page, then I would print pages of those QR codes. Later you can scan them in, enlarge each code and then read in the data. Finally, you'd have to Base64 decode it and then decrypt it!
Is the application online? If so, have you thought about storing the data securely (encrypted if you really want it that way) in a server, and on your page printing an access token (or simply a web reference) in the form of a QR code? Upon scanning the token your app could then retrieve the actual data from the server.
EDIT
While the above would give you the higher reliability and scalability, you could still store the data in QR codes directly on the page. You'd just need to encrypt into the alphanumeric space (using Base64 encoding) and store it as a series of QR codes; each can store around 3KB, so you'd get a lot of data on the page (as long as the page wasn't damaged).
High-Capacity Color Barcodes are 2D matrix codes that use colored "bars" (typically triangles) to encode data on printed pages. The Wikipedia article says they can store 3kB per square inch.
Obviously these require color printing and scanning hardware.
Unless you need to be able to run this completely offline from any internet or network access, I would suggest encrypting the text, hashing the encryption and storing the hash in a QR code. Getting a 2d bar or QR code to store a 64 or 128 bit hash should be easy enough, and reliable enough for scanning (as that's what they are designed for). Commercial QR scanners or software are available to make the scanning easy.
If you cannot have access to a common server between printing and scanning, then 2d barcodes are your only real option, and won't have the information density you want. You'd have a better chance storing it on RFID or a mini CD.
Related
For my Unity Game, I want to save player data by using text files. Text files can be easily modified and so can the data in them be modified. So, I would like to convert the text files to 0's and 1's. So that when you open it you should see 0101011 instead of readable and editable data. I know that I can use Read Bytes of File and replace the text in the file with this data, but how in the world do I make it data again? I need help with that.
Text files can be easily modified and so can the data in them be modified.
Is this a problem? games are made to have fun, so why do you want to stop the user from gaming how he wants? If it is a competitive multiplayer game, or any type of game involving actual money, you should not rely on any data on the client.
So, I would like to convert the text files to 0's and 1's. So that when you open it you should see 0101011 instead of readable and editable data
Please do not do this, it will just reduce performance for no benefit. If you do not want trivial modification of data, just use a binary serialization format. Or use the Unity provided storage options. Using binary serialization will likely be faster and produce smaller files, at the cost of making modifications more difficult. But you do not seem concerned about the last point.
Converting objects to serialized data and back again is a common operation, and there are great libraries for it. Json is common for textual data, I have used Protobuf .net for binary data, and find it quite fast and easy to use.
I'm currently trying to figure out how JPEG's are made in depth out of interest. I found documents on the different sections (soi, sof, sos, eoi etc) which are pretty straight forward, but not how to get a single pixel out of there.
My first thought was to make a small image, 2x2 for example, but with all the headers and sections it's still to big to isolate the pixel information without knowing the exact location and method to extract it. I'm sure it's compressed, but is their a way to get it out manually? (as RGB?)
Anyone has a clue on how to do this?
Getting the value of a single pixel of a JPEG image requires parsing some (if not most) of those sections anyway.
There's a good step-by-step guide available at https://www.imperialviolet.org/binary/jpeg/ (though the code is in Haskell, so it might be moderately inscrutable to mere mortals) that explains the concepts behind turning a JPEG into a bunch of RGB values.
This is the only source I know that explains JPEG end-to-end:
https://www.amazon.com/gp/product/B01JXRY4R0/ref=dbs_a_def_rwt_bibl_vppi_i4
Parsing the structure of a JPEG stream is easy. Decoding a JPEG scan is very difficult and involves several compression steps. Plus there are two different types of scan that are commonly in use (progressive & sequential).
I am comfortable with several programming languages (stronger in C#, C, Java than the others) so please feel free to suggest whichever would provide me with a way to read in a (preferably uncompressed) video file and look at the color of each pixel in a frame, for every frame. So what I mean is, say in a 1 pixel display of a trivially small video that runs for 5 frames, are there standard library classes or ways I can access the 5 colors that one pixel will show during the video?
Having never worked with video properly I am not too clued up on the data structure a video file would use to represent the color information, or how one would manipulate this!
Many thanks
For processing uncompressed video data (as it might come off a camera) you get an array of pixel data per-frame; you probably want to read up about pixel formats and how frames are defined within the array, which will depend entirely on what is producing the video. The YUV444, YUV422 and YUV420 formats are quite common; they're expressed in the YUV colour space but you can readily convert between them and RGB (or indeed HSV) if that's what you want to do.
Compressed video formats are a nightmare unto themselves, but you can decompress them into a raw format with ffmpeg or a similar tool. (Be careful - uncompressed video quickly produces vast quantities of data!) Indeed, I would use ffmpeg's libraries to manipulate video, but they're written in C(C++?) for speed - I don't know whether they're available to java or c#.
I am in the process of creating a TCP remote desktop broadcasting application. (Something like Team Viewer or VNC)
the server application will
1. run on a PC listening for multiple clients on one Thread
2. and on another thread it will record the desktop every second
3. and it will broadcast the desktop for each connected client.
i need to make this application possible to run on a connections with a 12KBps upload and 50KBps download DSL connection (client's and server).
so.. i have to reduce the size of the data/image i send per second.
i tried to reduce by doing the following.
I. first i send a Bitmap frame of the desktop and each other time i send only the difference of the previously sent frame.
II. the second way i tried was, each time i send a JPEG frame.
i was unsuccessful to send a JPEG frame and then each next time send the difference of the previously sent JPEG frame.
i tried using lzma compression (7zip SDK) for the when i was transmitting the difference of the Bitmap.
But i was unsuccessful to reduce the data into 12KBps. the maximum i was able to achieve was around 50KBps.
Can someone advice me an algorithm/procedure for doing this?
What you want to do is do what image compression formats do, but in a custom way (Send only the changes, not the whole image over and over). Here is what I would do, in two phases (phase 1: get it done, prove it works, phase 2: optimize)
Proof of concept phase
1) Capture an image of the screen in bitmap format
2) Section the image into blocks of contiguous bytes. You need to play around to find out what the optimal block size is; it will vary by uplink/downlink speed.
3) Get a short hash (crc32, maybe md5, experiment with this as well) for each block
4) Compress (don't forget to do this!) and transfer each changed block (If the hash changed, the block changed and needs to be transferred). Stitch the image together at the receiving end to display it.
5) Use UDP packets for data transfer.
Optimization phase
These are things you can do to optimize for speed:
1) Gather stats and hard code transfer speed vs frame size and hash method for optimal transfer speed
2) Make a self-adjusting mechanism for #1
3) Images compress better in square areas rather then contiguous blocks of bytes, as I explained in #2 of the first phase above. Change your algorithm so you are getting a visual square area rather than sequential blocks of lines. This square method is how the image and video compression people do it.
4) Play around with the compression algorithm. This will give you lots of variables to play with (CPU load vs internet access speed vs compression algorithm choice vs frequency of screen updates)
This is basically a summary of how (roughly) compressed video streaming works (you can see the similarities with your task if you think about it), so it's not an unproven concept.
HTH
EDIT: One more thing you can experiment with: After you capture a bitmap of the screen, reduce the number of colors in it. You can save half the image size if you go from 32 bit color depth to 16 bit, for example.
I have a raw pixel data in a byte[] from a DICOM image.
Now I would like to convert this byte[] to an Image object.
I tried:
Image img = Image.FromStream(new MemoryStream(byteArray));
but this is not working for me. What else should I be using ?
One thing to be aware of is that a dicom "image" is not necessarily just image data. The dicom file format contains much more than raw image data. This may be where you're getting hung up. Consider checking out the dicom file standard which you should be able to find linked on the wikipedia article for dicom. This should help you figure out how to parse out the information you're actually interested in.
You have to do the following
Identify the PIXEL DATA tag from the file. You may use FileStream to read byte by byte.
Read the pixel data
Convert it to RGB
Create a BitMap object from the RGB
Use Graphics class to draw the BitMap on a panel.
The pixel data usually (if not always) ends up at the end of the DICOM data. If you can figure out width, height, stride and color depth, it should be doable to skip to the (7FE0,0010) data element value and just grab the succeeding bytes. This is the trick that most normal image viewers use when they show DICOM images.
There is a C# library called EvilDicom (http://rexcardan.com/evildicom/) that can be used to pull the image out of a DICOM file. It has a tutorial on how to do it on the website.
You should use GDCM.
Grassroots DiCoM is a C++ library for DICOM medical files. It is automatically wrapped to python/C#/Java (using swig). It supports RAW, JPEG 8/12/16bits (lossy/lossless), JPEG 2000, JPEG-LS, RLE and deflated (zlib).
It is portable and is known to run on most system (Win32, linux, MacOSX).
http://gdcm.sourceforge.net/wiki/index.php/GDCM_Release_2.4
See for example:
http://gdcm.sourceforge.net/html/DecompressImage_8cs-example.html
Are you working with a pure standard DICOM File? I've been maintainning a DICOM parser for over a two years and I came across some realy strange DICOM files that didn't completely fulfill the standard (companies implementing their "own" twisted standard DICOM files) . flush you byte array into a file and test whether your image viewer(irfanview, picassa or whatever) can show it. If your code is working with a normal JPEG stream then from my experience , 99.9999% chance that this simply because the file voilate the standard in some strange way ( and believe me , medical companies does that a lot)
Also note that DICOM standard support several variants of the JPEG standard . could be that the Bitmap class doesn't support the data you get from the DICOM file. Can you please write down the transfer syntax?
You are welcome to send me the file (if it's not big) yossi1981#gmail.com , I can check it out , There was a time I've been hex-editing DICOM file for a half a year.
DICOM is a ridiculous specification and I sincerely hope it gets overhauled in the near future. That said Offis has a software suite "DCMTK" which is fairly good at converting dicoms with the various popular encodings. Just trying to skip ahead in the file x-bytes will probably be fine for a single file but if you have a volume or several volumes a more robust strategy is in order. I used DCMTK's conversion code and just grabbed the image bits before they went into a pnm. The file you'll be looking for in DCMTK is dcm2pnm or possibly dcmj2pnm depending on the encoding scheme.
I had a problem with the scale window that I fixed with one of the runtime flags. DCMTK is open source and comes with fairly simple build instructions.