zlib compressing byte array? - c#
I have this uncompressed byte array:
0E 7C BD 03 6E 65 67 6C 65 63 74 00 00 00 00 00 00 00 00 00 42 52 00 00 01 02 01
00 BB 14 8D 37 0A 00 00 01 00 00 00 00 05 E9 05 E9 00 00 00 00 00 00 00 00 00 00
00 00 00 00 01 00 00 00 00 00 81 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 05 00 00 01 00 00 00
And I need to compress it using the deflate algorithm (implemented in zlib), from what I searched the equivalent in C# would be using GZipStream but I can't match the compressed resulted at all.
Here is the compressing code:
public byte[] compress(byte[] input)
{
using (MemoryStream ms = new MemoryStream())
{
using (GZipStream deflateStream = new GZipStream(ms, CompressionMode.Compress))
{
deflateStream.Write(input, 0, input.Length);
}
return ms.ToArray();
}
}
Here is the result of the above compressing code:
1F 8B 08 00 00 00 00 00 04 00 ED BD 07 60 1C 49 96 25 26 2F 6D CA 7B 7F 4A F5 4A
D7 E0 74 A1 08 80 60 13 24 D8 90 40 10 EC C1 88 CD E6 92 EC 1D 69 47 23 29 AB 2A
81 CA 65 56 65 5D 66 16 40 CC ED 9D BC F7 DE 7B EF BD F7 DE 7B EF BD F7 BA 3B 9D
4E 27 F7 DF FF 3F 5C 66 64 01 6C F6 CE 4A DA C9 9E 21 80 AA C8 1F 3F 7E 7C 1F 3F
22 7E 93 9F F9 FB 7F ED 65 7E 51 E6 D3 F6 D7 30 CF 93 57 BF C6 AF F1 6B FE 5A BF
E6 AF F1 F7 FE 56 7F FC 03 F3 D9 AF FB 5F DB AF 83 E7 0F FE 35 23 1F FE BA F4 FE
AF F1 6B FC 1A FF 0F 26 EC 38 82 5C 00 00 00
Here is the result I am expecting:
78 9C E3 AB D9 CB 9C 97 9A 9E 93 9A 5C C2 00 03 4E 41 0C 0C 8C 4C 8C 0C BB 45 7A
CD B9 80 4C 90 18 EB 4B D6 97 0C 28 00 2C CC D0 C8 C8 80 09 58 21 B2 00 65 6B 08
C8
What I am doing wrong, could some one help me out there ?
First, some information: DEFLATE is the compression algorithm, it is defined in RFC 1951. DEFLATE is used in the ZLIB and GZIP formats, defined in RFC 1950 and 1952 respectively, which essentially are thin wrappers around DEFLATE bytestreams. The wrappers provide metadata such as, the name of the file, timestamps, CRCs or Adlers, and so on.
.NET's base class library implements a DeflateStream that produces a raw DEFLATE bytestream, when used for compression. When used in decompression it consumes a raw DEFLATE bytestream. .NET also provides a GZipStream, which is just a GZIP wrapper around that base. There is no ZlibStream in the .NET base class library - nothing that produces or consumes ZLIB. There are some tricks to doing it, you can search around.
The deflate logic in .NET exhibits a behavioral anomaly, where previously compressed data can actually be inflated, significantly, when "compressed". This was the source of a Connect bug raised with Microsoft, and has been discussed here on SO. This may be what you are seeing, as far as ineffective compression. Microsoft have rejected the bug, because while it is ineffective for saving space, the compressed stream is not invalid, in other words it can be "decompressed" by any compliant DEFLATE engine.
In any case, as someone else posted, the compressed bytestream produced by different compressors may not necessarily be the same. It depends on their default settings, and the application-specified settings for the compressor. Even though the compressed bytestreams are different, they may still decompress to the same original bytestream. On the other hand the thing you used to compress was GZIP, while it appears what you want is ZLIB. While they are related, they are not the same; you cannot use GZipStream to produce a ZLIB bytestream. This is the primary source of the difference you see.
I think you want a ZLIB stream.
The free managed Zlib in the DotNetZip project implements compressing streams for all of the three formats (DEFLATE, ZLIB, GZIP). The DeflateStream and GZipStream work the same way as the .NET builtin classes, and there's a ZlibStream class in there, that does what you think it does. None of these classes exhibit the behavior anomaly I described above.
In code it looks like this:
byte[] original = new byte[] {
0x0E, 0x7C, 0xBD, 0x03, 0x6E, 0x65, 0x67, 0x6C,
0x65, 0x63, 0x74, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x42, 0x52, 0x00, 0x00,
0x01, 0x02, 0x01, 0x00, 0xBB, 0x14, 0x8D, 0x37,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x05, 0xE9, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x81, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x05, 0x00, 0x00,
0x01, 0x00, 0x00, 0x00
};
var compressed = Ionic.Zlib.ZlibStream.CompressBuffer(original);
The output is like this:
0000 78 DA E3 AB D9 CB 9C 97 9A 9E 93 9A 5C C2 00 03 x...........\...
0010 4E 41 0C 0C 8C 4C 8C 0C BB 45 7A CD 61 62 AC 2F NA...L...Ez.ab./
0020 19 B0 82 46 46 2C 82 AC 40 FD 40 0A 00 35 25 07 ...FF,..#.#..5%.
0030 CE .
To decompress,
var uncompressed = Ionic.Zlib.ZlibStream.UncompressBuffer(compressed);
You can see the documentation on the static CompressBuffer method.
EDIT
The question is raised, why is DotNetZip producing 78 DA for the first two bytes instead of 78 9C? The difference is immaterial. 78 DA encodes "max compression", while 78 9C encodes "default compression". As you can see in the data, for this small sample, the actual compressed bytes are exactly the same whether using BEST or DEFAULT. Also, the compression level information is not used during decompression. It has no effect in your application.
If you don't want "max" compression, in other words if you are very set on getting 78 9C as the first two bytes, even though it doesn't matter, then you cannot use the CompressBuffer convenience function, which uses the best compression level under the covers. Instead you can do this:
var compress = new Func<byte[], byte[]>( a => {
using (var ms = new System.IO.MemoryStream())
{
using (var compressor =
new Ionic.Zlib.ZlibStream( ms,
CompressionMode.Compress,
CompressionLevel.Default ))
{
compressor.Write(a,0,a.Length);
}
return ms.ToArray();
}
});
var original = new byte[] { .... };
var compressed = compress(original);
The result is:
0000 78 9C E3 AB D9 CB 9C 97 9A 9E 93 9A 5C C2 00 03 x...........\...
0010 4E 41 0C 0C 8C 4C 8C 0C BB 45 7A CD 61 62 AC 2F NA...L...Ez.ab./
0020 19 B0 82 46 46 2C 82 AC 40 FD 40 0A 00 35 25 07 ...FF,..#.#..5%.
0030 CE .
Quite simply what you got had a GZip header. What you want is the simpler Zlib header. ZLib has options for GZip header, Zlib header or no header. Typically the Zlib header is used unless the data is associated with a disk file (in which case GZip header is used.) Apparently, there is no way with .Net library to write a zlib header (even though this is by far the most common header used in file formats). Try http://dotnetzip.codeplex.com/.
You can quickly test all the different zlib options using HexEdit (Operations->Compression->Settings). See http://www.hexedit.com . It took me 10 minutes to check your data by simply pasting your compressed bytes into HexEdit and decompressing. Also tried compressing your orignal bytes with GZip and ZLib headers as a double-check. Note that you may have to fiddle with the settings to get exactly the bytes you were expecting.
Related
Openssl Command Line for Triple DES HMAC like C# MACTripleDES
Can anyone explain how to make a TDES MAC in OpenSSL command line? I am trying to duplicate some functionality of a working C# program in C for the OpenSSL API, and am having trouble duplicating the .Net MACTripleDES.ComputeHash function in openssl. Here is an example with bogus data and key: using (MACTripleDES hmac = new MACTripleDES(Utilities.HexStringToByteArray("112233445566778899aabbccddeeff00"))) { // Compute the hash of the input file. byte[] hashValue = hmac.ComputeHash(Utilities.HexStringToByteArray("001000000000000000000000000000008000000000000000")); string signature = Utilities.ByteArrayToHexString(hashValue); PrintToFeedback("Bogus Signature = " + signature); } The result is "Bogus Signature = A056D11063084B3E" My new C program has to provide the same hash of that data in order to interoperate with its wider environment. But the way to do this in openSSL eludes me. This shows that the openssl data starts out the same as the C# data: cmd>od -tx1 bsigin 0000000 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000020 80 00 00 00 00 00 00 00 stringified, 001000000000000000000000000000008000000000000000 MATCHes the c# string. cmd>openssl dgst -md5 -mac hmac -macopt hexkey:112233445566778899aabbccddeeff00 bsigin HMAC-MD5(bsigin)= 7071d693451da3f2608531ee43c1bb8a That data is too long, and my expected data is not a substring. Same for -sha1 etc. I tried encrypting and making the digest separately, no good. MS does not say what kind of hash it does, and I can't find documentation of how to set up a MAC with TDES in openssl. So I'm hoping someone here knows enough about both platforms to give me a decent hint.
Command line answer: cmd>openssl enc -des-ede-cbc -K 112233445566778899aabbccddeeff00 -iv 0000000000000000 -in bsigin -out bsigout cmd>od -tx1 bsigout 0000000 7c de 93 c6 5f b4 03 21 aa c0 89 b8 ae f3 da 5d 0000020 a0 56 d1 10 63 08 4b 3e 4c 03 41 d6 dd 9e e4 32 ^^^^^^^^^^^^^^^^^^^^^^^ That is, the command line form returns 32 bytes, and bytes 16..23 contain the hmac. API answer: DES_key_schedule SchKey1,SchKey2; DES_cblock iv = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; DES_set_key((C_Block *)Key1, &SchKey1); DES_set_key((C_Block *)Key2, &SchKey2); DES_ede3_cbc_encrypt( (unsigned char*)input_data, (unsigned char*)cipher, inputLength, &SchKey1, &SchKey2, &SchKey1, &iv, DES_ENCRYPT); Where Key1 is the Lkey or left 8 bytes of the 16 byte TDES key, and Key2 is the Rkey or right 8 bytes of the 16 byte TDES key. This call only populates 24 bytes of cipher, as opposed to the 32 byte return of the command line version. You still take bytes 16..23. Hopefully the supporting declarations are intuitive.
setting the BluetoothLEAdvertisement.Flags in C# for iBeacon advertisemet
MS provided samples to send and receive Bluetooth Low Energy advertisements. I saw this very helpful answer for breaking down the iBeacon packet. There's also an example for setting BluetoothLEAdvertisement.ManufacturerData as the ibeacon standards. May I ask how can I set the Flags of the BluetoothLEAdvertisement? For example set the value to: 02-01-06 Thanks Edit 1: Here's the code: using System; using System.Management; using System.Text.RegularExpressions; using Windows.Devices.Bluetooth.Advertisement; using System.Runtime.InteropServices.WindowsRuntime; namespace BLE_iBeacon { class IBeacon { static void Main() { Console.WriteLine("Advertising as iBeacon. Press Enter to exit"); // Create and initialize a new publisher instance. BluetoothLEAdvertisementPublisher publisher = new BluetoothLEAdvertisementPublisher(); // Add a manufacturer-specific section: var manufacturerData = new BluetoothLEManufacturerData(); // Set the company ID for the manufacturer data. // 0x004C Apple, Inc. manufacturerData.CompanyId = 0x004C; byte[] dataArray = new byte[] { // last 2 bytes of Apple's iBeacon 0x02, 0x15, // UUID E4 C8 A4 FC F6 8B 47 0D 95 9F 29 38 2A F7 2C E7 0xE4, 0xC8, 0xA4, 0xFC, 0xF6, 0x8B, 0x47, 0x0D, 0x95, 0x9F, 0x29, 0x38, 0x2A, 0xF7, 0x2C, 0xE7, // Major 0x00, 0x00, // Minor 0x00, 0x01, // TX power 0xC5 }; manufacturerData.Data = dataArray.AsBuffer(); // Add the manufacturer data to the advertisement publisher: publisher.Advertisement.ManufacturerData.Add(manufacturerData); publisher.Advertisement.Flags = BluetoothLEAdvertisementFlags.GeneralDiscoverableMode; publisher.Start(); Console.Read(); publisher.Stop(); } } } Edit 2: In the C# code if I do not set the Flags, my windows laptop would advertise raw packet like: 04 3E 27 02 01 02 01 0D 45 84 D3 68 21 1B 1A FF 4C 00 02 15 E4 C8 A4 FC F6 8B 47 0D 95 9F 29 38 2A F7 2C E7 00 00 00 01 C5 BA My purpose is to use raspberry pi's as BLE receivers. I used the Radius Networks's code here. You can see in the ibeacon_scan script, they check the packet of the advertisement to see if it's an iBeacon by: if [[ $packet =~ ^04\ 3E\ 2A\ 02\ 01\ .{26}\ 02\ 01\ .{14}\ 02\ 15 ]]; then So the previous raw packet would not be recognized, for missing the flag part. I am wondering if I can advertise the packet with the Flags, like: 04 3E 2A 02 01 02 01 0D 45 84 D3 68 21 1B **02 01 1A** 1A FF 4C 00 02 15 E4 C8 A4 FC F6 8B 47 0D 95 9F 29 38 2A F7 2C E7 00 00 00 01 C5 BA instead of changing the scan script in the pi.
iBeacon on Windows The following code publishes an iBeacon on Windows 10 machines: // Create and initialize a new publisher instance. BluetoothLEAdvertisementPublisher publisher = new BluetoothLEAdvertisementPublisher(); // Add a manufacturer-specific section: var manufacturerData = new BluetoothLEManufacturerData(); // Set the company ID for the manufacturer data. // 0x004C Apple, Inc. manufacturerData.CompanyId = 0x004c; // Create the payload var writer = new DataWriter(); byte[] dataArray = new byte[] { // last 2 bytes of Apple's iBeacon 0x02, 0x15, // UUID e2 c5 6d b5 df fb 48 d2 b0 60 d0 f5 a7 10 96 e0 0xe2, 0xc5, 0x6d, 0xb5, 0xdf, 0xfb, 0x48, 0xd2, 0xb0, 0x60, 0xd0, 0xf5, 0xa7, 0x10, 0x96, 0xe0, // Major 0x00, 0x00, // Minor 0x00, 0x01, // TX power 0xc5 }; writer.WriteBytes(dataArray); manufacturerData.Data = writer.DetachBuffer(); // Add the manufacturer data to the advertisement publisher: publisher.Advertisement.ManufacturerData.Add(manufacturerData); publisher.Start(); Proximity UUID While testing this out, my iOS device would not recognize the Proximity UUID you provided. I'm guessing this is because you generated it yourself, so the app doesn't know what to look for. Instead, I used the proximity UUID from this answer which identifies the Windows 10 device as an AirLocate iBeacon. Flags Windows 10 does not currently allow developers to set the flags for a Bluetooth LE advertisement. Luckily, for the Windows device to be recognized as an iBeacon, you don't need those flags!
Ideally, you want to set the flags byte to 0x1a, but other values may still work. The important flag to set is General Discoverable (0x02). You can use BluetoothLEAdvertisementFlags is an enumeration of bit values here. My C# is very rusty, but you might try setting the flags hex value directly with: publisher.Advertisement.Flags = 0x1A;
string to byte[] without encoding or changing actual bytes at string
assume i got the following byte[] 0C 00 21 08 01 00 00 00 86 1B 06 00 54 51 53 65 72 76 65 72 with bitconverter BitConverter.ToString i can convert it to 0C-00-21-08-01-00-00-00-86-1B-06-00-54-51-53-65-72-76-65-72 how do i convert it back from string to byte[] to get 0C 00 21 08 01 00 00 00 86 1B 06 00 54 51 53 65 72 76 65 72 ascii encoding and other methods always getting me the equivalent bytes to the string but what i really need is the string to be byte[] as it is, i know if i did a reversing operation (using getbytes then tostring) ill end up with the same string but what i care about is while at getbytes to get the exact bytes as i said to put 0C-00-21-08-01-00-00-00-86-1B-06-00-54-51-53-65-72-76-65-72 AS string and get 0C 00 21 08 01 00 00 00 86 1B 06 00 54 51 53 65 72 76 65 72 As byte[] thanks in advance
You need this byte[] bytes = str.Split('-').Select(s => Convert.ToByte(s, 16)).ToArray();
You can use SoapHexBinary class in System.Runtime.Remoting.Metadata.W3cXsd2001 namespace string s = "0C-00-21-08-01-00-00-00-86-1B-06-00-54-51-53-65-72-76-65-72"; byte[] buf = SoapHexBinary.Parse(s.Replace("-"," ")).Value;
Remenber that BitConverter.ToString returns an equivalent hexadecimal string representation,so if you decide to stick with it converting back as follow: string temp = BitConverter.ToString(buf);//buf is your array. byte[] newbuf = temp.Split('-').Select(s => Convert.ToByte(s,16)).ToArray(); But the safest way to convert bytes to string and back is base64: string str = Convert.ToBase64String(buf); byte[] result = Convert.FromBase64String(str);
Passing pointer to a struct?
Hello I'm trying to pass data from a pointer to a struct but the values seem to be different. struct somestruct { public file header; public uint version; } unsafe struct file { public fixed char name[8]; public uint type; public uint size; } Then in code somewhere.. public unsafe int ReadFile(string filepath) { somestruct f = new somestruct(); byte[] fdata = System.IO.ReadAllBytes( filepath ); fixed( byte* src = fdata ) { f.header = *(file*)src; MessageBox.Show( new string(f.header.name) ); //should be 'FILENAME' but it's like japanese. } return 0; } Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 00000000 46 49 4C 45 4E 41 4D 45 00 00 00 01 00 00 00 30 FILENAME.......0 00000010 74 27 9F EF 74 77 F1 D7 C5 86 93 3D 39 0D 72 A9 t'Ÿïtwñ×ņ“=9.r© 00000020 63 8B 92 CF F6 7D 8A 14 45 9D 68 51 A4 8E A4 EE c‹’Ïö}Š.E.hQ¤Ž¤î 00000030 4E FE D0 66 45 0E C9 8D 96 BB F4 EE 52 1F 89 D3 NþÐfE.É.–»ôîR.‰Ó 00000040 5C 80 1A 71 8A 16 B1 8B 3A A8 1B A4 48 11 B8 E8 \€.qŠ.±‹:¨.¤H.¸è Do you have any idea what's going on?
Each char is 2 bytes - a fixed buffer of 8 chars is 16 bytes. You are reading the first 8 bytes as only the first 4 characters in that buffer, and the high bytes will make it look. Like the eastern Unicode ranges. I would say: deserialize it at the stream level. Don't do this. Basically, read (at least) 20 bytes into a buffer, then decode manually, using: string s = Encoding.ASCII.GetString(buffer, 0, 8); For the string, and probably shift operations for the unsigned integers. You could also use unsafe code to read the integers from the buffer, via the other meaning of fixed and a pointer-cast.
A char is UTF-16 and is 2 bytes. You need to convert the UTF-8/ANSI (1 byte) string to a UTF-16 string.
Windows C# implementation of linux dd command
I'm writing a C#.Net app to run on windows that needs to take an image of a removable disk and chuck it onto a Linux Live USB. The Live USB is the inserted into the target machine and boots, on start up it runs a script which uses the dd command like so to flash it onto another drive: dd if=/path/to/file/from/csharp/program of=/dev/sdX The problem I am having is creating the image on the windows side. I have tried my Live Linux out with files I have created on a Linux system using dd and that works fine, but I need to be able to create these files from within a C#.Net application on Windows. I'd rather not have to rely on cygwin or some other dependency so tried to use the Win32 CreateFile function to open the physical device. CreateFile is called with the first arg set to "\.\F:" (if F: is the drive I want to image), like so: SafeFileHandle TheDevice = CreateFile(_DevicePath, (uint)FileAccess.Read, (uint)(FileShare.Write | FileShare.Read | FileShare.Delete), IntPtr.Zero, (uint)FileMode.Open, (uint)FILE_ATTRIBUTE_SYSTEM | FILE_FLAG_SEQUENTIAL_SCAN, IntPtr.Zero); if (TheDevice.IsInvalid) { throw new IOException("Unable to access drive. Win32 Error Code " + Marshal.GetLastWin32Error()); } FileStream Dest = System.IO.File.Open(_SaveFile, FileMode.Create); FileStream Src = new FileStream(TheDevice, FileAccess.Read); Src.CopyTo(Dest); Dest.Flush(); Src.Close(); Dest.Close(); But when the output file is dd'd back onto a disk using the Live Linux USB the result is not as expected (the disk isn't bootable etc, but from examining the output file in a hex editor, it looks like there is an MBR at the beginning etc). Is this a problem with endianess or should I using something other than a FileStream to copy the data into the file. Alternatively is there an example of dd for Windows source code (C# or C++, i've looked at the Delphi for http://www.chrysocome.net/dd and don't totally understand it or have a decent Delphi IDE to pick the code apart) so I can see how that works? UPDATE/EDIT: Here is a hex string of the first 512 Bytes that the dd output contains: 33 C0 FA 8E D8 8E D0 BC 00 7C 89 E6 06 57 8E C0 FB FC BF 00 06 B9 00 01 F3 A5 EA 1F 06 00 00 52 52 B4 41 BB AA 55 31 C9 30 F6 F9 CD 13 72 13 81 FB 55 AA 75 0D D1 E9 73 09 66 C7 06 8D 06 B4 42 EB 15 5A B4 08 CD 13 83 E1 3F 51 0F B6 C6 40 F7 E1 52 50 66 31 C0 66 99 E8 66 00 E8 21 01 4D 69 73 73 69 6E 67 20 6F 70 65 72 61 74 69 6E 67 20 73 79 73 74 65 6D 2E 0D 0A 66 60 66 31 D2 BB 00 7C 66 52 66 50 06 53 6A 01 6A 10 89 E6 66 F7 36 F4 7B C0 E4 06 88 E1 88 C5 92 F6 36 F8 7B 88 C6 08 E1 41 B8 01 02 8A 16 FA 7B CD 13 8D 64 10 66 61 C3 E8 C4 FF BE BE 7D BF BE 07 B9 20 00 F3 A5 C3 66 60 89 E5 BB BE 07 B9 04 00 31 C0 53 51 F6 07 80 74 03 40 89 DE 83 C3 10 E2 F3 48 74 5B 79 39 59 5B 8A 47 04 3C 0F 74 06 24 7F 3C 05 75 22 66 8B 47 08 66 8B 56 14 66 01 D0 66 21 D2 75 03 66 89 C2 E8 AC FF 72 03 E8 B6 FF 66 8B 46 1C E8 A0 FF 83 C3 10 E2 CC 66 61 C3 E8 62 00 4D 75 6C 74 69 70 6C 65 20 61 63 74 69 76 65 20 70 61 72 74 69 74 69 6F 6E 73 2E 0D 0A 66 8B 44 08 66 03 46 1C 66 89 44 08 E8 30 FF 72 13 81 3E FE 7D 55 AA 0F 85 06 FF BC FA 7B 5A 5F 07 FA FF E4 E8 1E 00 4F 70 65 72 61 74 69 6E 67 20 73 79 73 74 65 6D 20 6C 6F 61 64 20 65 72 72 6F 72 2E 0D 0A 5E AC B4 0E 8A 3E 62 04 B3 07 CD 10 3C 0A 75 F1 CD 18 F4 EB FD 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 19 16 9F 29 00 00 80 01 01 00 06 FE 3F 0E 3F 00 00 00 61 C8 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA and here is what my code produces: EB 76 90 4D 53 44 4F 53 35 2E 30 00 02 04 04 00 02 00 02 00 00 F8 F2 00 3F 00 FF 00 3F 00 00 00 61 C8 03 00 80 00 29 7A E8 21 04 4E 4F 20 4E 41 4D 45 20 20 20 20 46 41 54 31 36 20 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 E9 05 01 B4 0E 53 33 DB CD 10 5B C3 8A 07 3C 00 74 06 E8 EE FF 43 EB F4 C3 0D 4E 6F 20 42 50 42 3A 20 43 61 6E 27 74 20 62 6F 6F 74 20 75 73 69 6E 67 20 43 48 53 20 66 75 6E 63 74 69 6F 6E 73 00 50 B0 2E E8 BC FF 58 33 DB 8E 06 E4 01 F6 06 DC 01 02 75 42 F6 06 DC 01 04 75 07 80 3E E8 01 80 72 34 53 53 52 50 06 53 55 6A 10 8B F4 52 50 8A 16 E8 01 B8 00 42 F9 CD 13 8A EC 58 5A 8D 64 10 72 14 80 FD 00 75 0F 03 C5 83 D2 00 C3 BB 91 00 E8 78 FF F4 EB FD 83 3E 18 00 00 74 F0 52 50 8B CD F7 36 18 00 8B F2 03 D1 3B 16 18 00 76 06 8B 0E 18 00 2B CE 33 D2 F7 36 1A 00 88 16 E9 01 8B F8 8B D7 51 8A C1 8D 4C 01 C0 E6 06 0A CE 8A EA 8B 16 E8 01 B4 02 CD 13 59 73 15 80 FC 09 75 0A 49 EB DE 8A C4 04 30 E8 18 FF B4 00 CD 13 EB D1 58 5A 03 C1 83 D2 00 2B E9 74 07 C1 E1 09 03 D9 EB 94 C3 00 00 00 00 FA FC E8 00 00 5E 81 EE 85 01 2E 8B 84 E4 01 8E D8 8E C0 8E D0 2E C7 84 7C 01 AF 01 2E 89 84 7E 01 B9 00 01 BF 00 00 F3 2E A5 2E FF AC 7C FF BC 00 0A FB 80 3E E8 01 FF 75 04 88 16 E8 01 83 06 E4 01 20 A1 E0 01 8B 16 E2 01 BD 02 00 E8 E9 FE 50 52 EB 74 90 00 00 00 00 00 00 00 00 00 00 00 D3 20 00 00 00 30 80 00 FF 00 68 41 00 40 09 FF 40 5A AC 04 00 00 AC 04 00 00 00 00 12 00 55 AA This was taken from exactly the same CF card without any editing/writing etc happening, so i'm confused as to why they are so different, but both end with the correct 55 AA bytes too. Does Windows mangle the MBR's on cards when they're accessed this way or is some other weird under the hood stuff happening that I'm not aware of?
I think what you have should work - I've tried this myself using a bootable floppy disk image (mounted as a virtual drive using ImDisk) and the resulting file is binary identical to the original image. For completeness here is the code I used (in its entirity): using System; using System.IO; using System.Runtime.InteropServices; using Microsoft.Win32.SafeHandles; namespace ConsoleApplication1 { public class Program { const int FILE_ATTRIBUTE_SYSTEM = 0x4; const int FILE_FLAG_SEQUENTIAL_SCAN = 0x8; [DllImport("Kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)] public static extern SafeFileHandle CreateFile(string fileName, [MarshalAs(UnmanagedType.U4)] FileAccess fileAccess, [MarshalAs(UnmanagedType.U4)] FileShare fileShare, IntPtr securityAttributes, [MarshalAs(UnmanagedType.U4)] FileMode creationDisposition, int flags, IntPtr template); [STAThread] static void Main() { using (SafeFileHandle device = CreateFile(#"\\.\E:", FileAccess.Read, FileShare.Write | FileShare.Read | FileShare.Delete, IntPtr.Zero, FileMode.Open, FILE_ATTRIBUTE_SYSTEM | FILE_FLAG_SEQUENTIAL_SCAN, IntPtr.Zero)) { if (device.IsInvalid) { throw new IOException("Unable to access drive. Win32 Error Code " + Marshal.GetLastWin32Error()); } using (FileStream dest = File.Open("TempFile.bin", FileMode.Create)) { using (FileStream src = new FileStream(device, FileAccess.Read)) { src.CopyTo(dest); } } } } } } If this doesn't work then it seems to indicate that: There is a problem with the original image. The problem is with whatever is using the disk image that you've just written. There is some subtle differences in dealing with the specific device you are accessing (although I can't think what) The most likely culprit is step 2. What exactly is it that you are doing with the resulting disk image? Update: This is written in the comments, but for completeness I thought I'd add it to my answer - it looks like whats happening is that the contents of the first partition of the disk is being written, when instead what is wanted is the contents of the entire disk. When you take a look at the second hex string (the one produced by sample code) in something like HxD we see this: ëv.MSDOS5.0..........øò.?.ÿ.?...aÈ..€.)zè!.NO NAME FAT16 .. ........................................................é..´.S3Û Í.[Ê.<.t.èîÿCëôÃ.No BPB: Can't boot using CHS functions.P°.è¼ÿX 3ÛŽ.ä.ö.Ü..uBö.Ü..u.€>è.€r4SSRP.SUj.‹ôRPŠ.è.¸.BùÍ.ŠìXZ.d.r.€ý.u. .ŃÒ.û‘.èxÿôëýƒ>...tðRP‹Í÷6..‹ò.Ñ;...v.‹...+Î3Ò÷6..ˆ.é.‹ø‹×QŠÁ. L.Àæ..Ίê‹.è.´.Í.Ys.€ü.u.IëÞŠÄ.0è.ÿ´.Í.ëÑXZ.ÁƒÒ.+ét.Áá..Ùë”Ã.... úüè..^.î…..‹„ä.ŽØŽÀŽÐ.Ç„|.¯..‰„~.¹..¿..ó.¥.ÿ¬|ÿ¼..û€>è.ÿu.ˆ.è.ƒ. ä. ¡à.‹.â.½..èéþPRët............Ó ...0€.ÿ.hA.#.ÿ#Z¬...¬.......Uª This looks to me like the boot sector of a FAT16 partition - the presence of the strings "MSDOS5.0", "NO NAME" and "FAT16" near the start is a dead giveaway. Compare this to the output of the first hex string (the one produced by dd): 3ÀúŽØŽÐ¼.|‰æ.WŽÀûü¿..¹..ó¥ê....RR´A»ªU1É0öùÍ.r..ûUªu.Ñés.fÇ...´B ë.Z´.Í.ƒá?Q.¶Æ#÷áRPf1Àf™èf.è!.Missing operating system...f`f1Ò». |fRfP.Sj.j.‰æf÷6ô{Àä.ˆáˆÅ’ö6ø{ˆÆ.áA¸..Š.ú{Í..d.faÃèÄÿ¾¾}¿¾.¹ .ó¥ Ãf`‰å»¾.¹..1ÀSQö.€t.#‰ÞƒÃ.âóHt[y9Y[ŠG.<.t.$.<.u"f‹G.f‹V.f.Ðf!Òu. f‰Âè¬ÿr.è¶ÿf‹F.è ÿƒÃ.âÌfaÃèb.Multiple active partitions...f‹D.f. F.f‰D.è0ÿr..>þ}Uª.….ÿ¼ú{Z_.úÿäè..Operating system load error...^ ¬´.Š>b.³.Í.<.uñÍ.ôëý......................................Ÿ)..€. ...þ?.?...aÈ..................................................Uª And we see something that looks to me a lot like a master boot record. Why? Because in the MBR all of the first 440 bytes is boot code, unlike a FAT boot sector which contains the distinctive bios parameter block (it looks like garbage above, but if you put that through a disassembler you get something that looks like valid 16 bit code). Also, both of those look like valid and completely different boot sectors (complete with error messages). There is no way that a programming error could have "mangled" one to look like the other - it must just be that the wrong thing is being read. In order to get CreateFile to return the disk instead of the partition it looks like you just need to pass it a different string, for example #"\\.\PhysicalDrive0" opens the first physical disk. See: Low Level Disk Access INFO: Direct Drive Access Under Win32
This is what i've written to do get the \.\PhysicalDriveX path for a given drive letter. If Pass the drive letter into this and take the return value and pass into CreateFile as the first Param I should now get something similar to dd under Linux. using System.Management; //Add in a reference to this as well in the project settings public static string GetPhysicalDevicePath(char DriveLetter) { ManagementClass devs = new ManagementClass( #"Win32_Diskdrive"); { ManagementObjectCollection moc = devs.GetInstances(); foreach(ManagementObject mo in moc) { foreach (ManagementObject b in mo.GetRelated("Win32_DiskPartition")) { foreach (ManagementBaseObject c in b.GetRelated("Win32_LogicalDisk")) { string DevName = string.Format("{0}", c["Name"]); if (DevName[0] == DriveLetter) return string.Format("{0}", mo["DeviceId"]); } } } } return ""; }