We Keep Coding

How to extract image from TCP Stream

I need your help.
I was creating an application in c# that converts the data from the IP camera to an image (JPEG).
I was able to convert the image using the below code:
hex = "FFD8FFDB008400130D0F1.........";/// supply this with the attached hex dump.
byte[] image = HexString2Bytes(hex);
File.WriteAllBytes("visio.png", image);
Process.Start("visio.png");
private static byte[] HexString2Bytes(string hexString)
{
int bytesCount = (hexString.Length) / 2;
byte[] bytes = new byte[bytesCount];
for (int x = 0; x < bytesCount; ++x)
{
bytes[x] = Convert.ToByte(hexString.Substring(x * 2, 2), 16);
}
return bytes;
}
Sometimes I get a better image as expected:https://ibb.co/pxrwn6p
but sometimes I get a distorted image after converting https://ibb.co/9twx5ZT.
I was wondering if there is a problem with the conversion or the way I save the image.
because as per the supplier what I need to do is to directly save the image from the stream.
but since I receive it as a byte and I still need to convert it maybe there is something wrong with my codes.
the image also starts with ÿØÿÛ FF D8 and ends with ÿ Ùÿÿÿÿ (FF D9 FF FF FF FF)
here's the hex dump from their sample app:
https://drive.google.com/file/d/1CMlQ0xaVjM0jfU5A4MB-_HwK54dUMTOr/view?usp=sharing
using their test application the image can be captured and converted the image perfectly.
captured image using their application:https://ibb.co/2KgyLTc
using the hex from the sniff and convert it using my code:
converted image using my code:https://ibb.co/G0WMjht
sample source code:
please bare with my codes because currently this is only my test app before integrating this feature to another app.
https://drive.google.com/file/d/1Ux7zsR39IVNyd1wrBxQPQKA6yM4YnwJN/view?usp=sharing
Thank You in advance.

Looking at the hex-dump it looks like some kind of XML file with embedded image data. Trying to convert this directly to an image will most likely not work, you would need to parse the XML-data to extract the actual image file. But it looks like you have a valid Jpeg header, so I would guess you have found the start of the image at least. But you probably also need to check the length property from the XML-data to find the length of the image-data block.
However, the datablock looks like it contains large sections of zeros, this should not be present in a jpeg file, so it might indicate some data corruption. Possibly from the way the network data is captured.
I would expect cameras to use some higher level protocol than raw TCP. Like Real Time Streaming Protocol, GigE vision, or mjpeg over http. I have not seen any camera that require you to process a raw TCP streams. But since you do not show how the data is fetched it is difficult to tell if there is any mistakes in that code.

Decryption of Encrypted Vector Tiles Mapbox

I'll try to be brief, but I'll share the whole picture.
Problem Statement
I am using vector tile from tippecanoe from mapbox to create .pbtiles from my geojson data. The issue is, on a web client when I see the inspect element and download the .pbf and run it by this (mapbox-vector-tile-cs) library, I am able to successfully get the data from the tile. Which means that any one with some basic google search can also steal my data from the vector tiles.
What I was able to achieve
To avoid the security concern, with the short timeline I have, I came up with a quick and dirty way. After tippecanoe creates the .mbtiles sqlite db, I run a java utility I made to encrypt the data in the blob using AES 256 encryption and stored it in two different ways in two different sqlite db's:
Stored as bytes into a different .mbtiles sqlite db (which get's stored as Blob). Along with z, x, y and metadata
Encoded the encrypted data as base64 and then stored the base64encoded encrypted tile data into a string data type column. Along with z, x, y and metadata.
and stored the key (base64 encoded) and initialization vector (base64 encoded) into a file.
The API side (Question 1)
Now, when I get the non encrypted .pbf from the API, a header of type gzip and application/x-protobuf is set that helps to convert the unencrypted blob data to a protobuf and returns a .pbf file that gets downloaded.
Now when I try to get the encrypted data from the API with the same header as the non encrypted on, the download of the .pbf fails saying Failed - Network error. I realized that it's being caused as the header application/x-protobuf is trying to package the file into a .pbf while the contents of the blob might not be matching what's expected and hence the result.
I removed the header application/x-protobuf and since I can't gzip now, i removed the header of gzip too. Now the data gets displayed on the chrome browser instead of being downloaded, I figure as now it's just a random response.
The question is, How can I make it to send a .pbf that has encrypted data in it and this((mapbox-vector-tile-cs)) library can parse the data? I know the data will be need to be decrypted first before I pass it for parsing assuming that it's decrypted and I have the data that was stored into the blob of the .mbtiles.
This Library with a UWP project (Question 2)
So now currently as mentioned above (since i don't have a solution to the headers part) I removed the headers and let the API return me a direct response.
The Issue now I am facing is that when I pass in the decryted (I checked the decryption was successful and the decrypted data is an exact match to the what was stored in the Blob) Blob data to the
var layerInfos = VectorTileParser.Parse(stream);
code line returns me an IEnumerable<Tile> that is not null but has 0 layers in it. while the actual tile contains 5 layers in it.
My Question is, how do I get this((mapbox-vector-tile-cs)) library to return me the layers.
The code to fetch the tile from the server and decrypt before I send it for parsing is as below:
//this code downloads the tile, layerInfos is returned as an empty collection
private async Task<bool> ProcessTile(TileData t, int xOffset, int yOffset)
{
var stream = await GetTileFromWeb(EncryptedTileURL,true);
if (stream == null)
return false;
var layerInfos = VectorTileParser.Parse(stream);
if (layerInfos.Count == 0)
return false;
return true;
}
The tiles are fetched from the server using a GetTileFromWeb() method:
private async Task<Stream> GetTileFromWeb(Uri uri, bool GetEnc = false)
{
var handler = new HttpClientHandler();
if (!GetEnc)
handler.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
var gzipWebClient = new HttpClient(handler);
var bytes = gzipWebClient.GetByteArrayAsync(uri).Result;
if (GetEnc)
{
var decBytes = await DecryptData(bytes);
return decBytes;
}
var stream = new MemoryStream(bytes);
return stream;
}
PS: Sorry for such a long question, I am not used to such elaborate detail, but seemed I need to share more as Encryption is my forte while map data vector tiles isn't.

Base64 decoding of a image gives a red question mark

I am encoding and decoding facebook thumbnail image data using base64 strings in unity3d. When the user does not have a profile picture, the resulting image is a red question mark. Is there any way to recognise that an invalid picture is being sent so that I can replace it with a default pic of my choosing?
I am converting the string to image data using Convert.FromBase64String (string encoded) in c#.

I assume that you use some API to retrieve the base64 encoded string from an URL?
In that case, you could just dump the retrieved string once to the console and then copy it into your source code and compare this to the string you get in the future. Of course, this will break if the facebook API you use decides to deliver a different icon, in which case you would have to dump the new "unknown user" thumbnail.
string encoded = ... // however you obtain your thumbnail
print encoded; // dump the string to the console once. remove this statement later
if (encoded == "...here comes the (rather large) string you just copied")
encoded = "...here comes some other image you like to use, encoded as string";
...
Not very elegant, but at least easy to implement.

I suppose, if an error image (e.g. the "red question-mark" image) is predictably returned when no profile picture exists, you could simply test for that case and replace with another image of your choosing. You may choose to store the red question-mark image as a resource and then compare its Base64 string to the Base64 of each image returned from your requests.
In that case, the pivotal bit of code might look something like this (assuming you've already stored the image's Base64 string in a resource):
ResourceManager rm =
new ResourceManager("ExampleAppData", typeof(ExampleApp).Assembly);
String errorImageBase64 = rm.GetString("ErrorImageBase64");
// the image you get from your request
String resultImageBase64 = GetProfileImageBase64();
Image missingProfile;
if(resultImageBase64.Equals(errorImageBase64))
{
missingProfile = ImageFromBase64String(rm.GetString("MissingProfileBase64"));
}
else
{
missingProfile = ImageFromBase64String(resultImageBase64);
}
References:
http://msdn.microsoft.com/en-us/library/2xsy4hac.aspx
http://ozgur.ozcitak.com/snippets/2009/12/21/base64-encoding-an-image-with-csharp.html

In my experience it was due to a wrong request.
In my case the wrong API query returning Red Questionmark was /{fbId}/picture?g&type=square&height=128&width=128
I solved removing "?g", the working query now is
/{fbId}/picture?type=square&height=128&width=128

c# how to write a jpg image from request.binaryread

I have a flash app which sends raw data for a jpg image to a particular url Send.aspx . In Send.aspx I am using request.binaryread() to get the total request length and then read in the data to a byte array.
Then I am writing the data as jpg file to the server. The code is given below:
FileStream f = File.Create(Server.MapPath("~") + "/plugins/handwrite/uploads/" + filename);
byte[] data = Request.BinaryRead(Request.TotalBytes);
f.Write(data, 0, data.Length);
f.Close();
The file is getting created but there is no image in it. It always shows up as empty in any graphic viewer. What part am I missing. Am I supposed to use jpg encoding first before writing it to file? Thanks in advance

Well, you should use a using statement for your file stream, but other than that it looks okay to me.
A few suggestions for how to proceed...
Is it possible that the client isn't providing the data properly? Perhaps it's providing it as base64-encoded data?
Have you already read some data from the request body? (That could mess things up.)
I suggest you look closely at what you end up saving vs the original file:
Are they the same length? If not, which is longer?
If they're the same length, do their MD5 sums match?
If you look at both within a binary file editor, do they match at all? Any obvious differences?

Unable to verify body hash for DKIM

I'm writing a C# DKIM validator and have come across a problem that I cannot solve. Right now I am working on calculating the body hash, as described in Section 3.7 Computing the Message Hashes. I am working with emails that I have dumped using a modified version of EdgeTransportAsyncLogging sample in the Exchange 2010 Transport Agent SDK. Instead of converting the emails when saving, it just opens a file based on the MessageID and dumps the raw data to disk.
I am able to successfully compute the body hash of the sample email provided in Section A.2 using the following code:
SHA256Managed hasher = new SHA256Managed();
ASCIIEncoding asciiEncoding = new ASCIIEncoding();
string rawFullMessage = File.ReadAllText(#"C:\Repositories\Sample-A.2.txt");
string headerDelimiter = "\r\n\r\n";
int headerEnd = rawFullMessage.IndexOf(headerDelimiter);
string header = rawFullMessage.Substring(0, headerEnd);
string body = rawFullMessage.Substring(headerEnd + headerDelimiter.Length);
byte[] bodyBytes = asciiEncoding.GetBytes(body);
byte[] bodyHash = hasher.ComputeHash(bodyBytes);
string bodyBase64 = Convert.ToBase64String(bodyHash);
string expectedBase64 = "2jUSOH9NhtVGCQWNr9BrIAPreKQjO6Sn7XIkfJVOzv8=";
Console.WriteLine("Expected hash: {1}{0}Computed hash: {2}{0}Are equal: {3}",
Environment.NewLine, expectedBase64, bodyBase64, expectedBase64 == bodyBase64);
The output from the above code is:
Expected hash: 2jUSOH9NhtVGCQWNr9BrIAPreKQjO6Sn7XIkfJVOzv8=
Computed hash: 2jUSOH9NhtVGCQWNr9BrIAPreKQjO6Sn7XIkfJVOzv8=
Are equal: True
Now, most emails come across with the c=relaxed/relaxed setting, which requires you to do some work on the body and header before hashing and verifying. And while I was working on it (failing to get it to work) I finally came across a message with c=simple/simple which means that you process the whole body as is minus any empty CRLF at the end of the body. (Really, the rules for Body Canonicalization are quite ... simple.)
Here is the real DKIM email (right click and save it, the browsers eat the ending CRLF) with a signature using the simple algorithm (completely unmodified). Now, using the above code and updating the expectedBase64 hash I get the following results:
Expected hash: VnGg12/s7xH3BraeN5LiiN+I2Ul/db5/jZYYgt4wEIw=
Computed hash: ISNNtgnFZxmW6iuey/3Qql5u6nflKPTke4sMXWMxNUw=
Are equal: False
The expected hash is the value from the bh= field of the DKIM-Signature header. Now, the file used in the second test is a direct raw output from the Exchange 2010 Transport Agent. If so inclined, you can view the modified EdgeTransportLogging.txt.
At this point, no matter how I modify the second email, changing the start position or number of CRLF at the end of the file I cannot get the files to match. What worries me is that I have been unable to validate any body hash so far (simple or relaxed) and that it may not be feasible to process DKIM through Exchange 2010.

I tried this in python-dkim and I get a body hash mismatch too.
I think probably Exchange's GetMimeReadStream is not giving you the actual bytes as they were transmitted, therefore the hash doesn't match. Probably it's disassembling the message into its mime parts, and then GetMimeReadStream gives you a valid representation of the message, but not the one it was originally sent with.
Perhaps there's another API that will give you the real raw bytes?
Or perhaps by this point in the process the message has been torn apart and the original message thrown away, and you need to hook in earlier.
Probably you should try intercepting a DKIM-signed message by sending it to a non-Exchange server, and see if that works with your code. GetContentReadStream might possibly work?
Anyhow, what I would do next is try to find an API that gives you byte-for-byte what was sent.