I am writing a code that calculates the MD5/SHA256 of a program and later I want to be able to change it.
I wrote the code for calculating the MD5/SHA256, which is:
using (var md5 = MD5.Create())
{
using (var stream = File.OpenRead(textBox1.Text))
{
MessageBox.Show(BitConverter.ToString(md5.ComputeHash(stream)).Replace("-", ""));
}
}
using (var sha256 = SHA256.Create())
{
using (var stream = File.OpenRead(textBox1.Text))
{
MessageBox.Show(BitConverter.ToString(sha256.ComputeHash(stream)).Replace("-", ""));
}
}
Next I want to be able to change the values of MD5/SHA256 for the specified file. I have searched and all I found was this class:
class FileUtils
{
#region VARIABLES
private const int OFFSET_CHECKSUM = 0x12;
#endregion
#region METHODS
public static ushort GetCheckSum(string fileName)
{
if (!File.Exists(fileName))
throw new FileNotFoundException("Invalid fileName");
return GetCheckSum(File.ReadAllBytes(fileName));
}
public static ushort GetCheckSum(byte[] fileData)
{
if (fileData.Length < OFFSET_CHECKSUM + 1)
throw new ArgumentException("Invalid fileData");
return BitConverter.ToUInt16(fileData, OFFSET_CHECKSUM);
}
public static void WriteCheckSum(string sourceFile, string destFile, ushort checkSum)
{
if (!File.Exists(sourceFile))
throw new FileNotFoundException("Invalid fileName");
WriteCheckSum(File.ReadAllBytes(sourceFile), destFile, checkSum);
}
public static void WriteCheckSum(byte[] data, string destFile, ushort checkSum)
{
byte[] checkSumData = BitConverter.GetBytes(checkSum);
checkSumData.CopyTo(data, OFFSET_CHECKSUM);
File.WriteAllBytes(destFile, data);
}
#endregion
}
Which I don't really understand how it works and only changes the MD5. Is there an easier way to do this, for not so advanced users? If this class works for what I need, could someone explain to me how can I use it?
Edit: I am aware that the MD5 of the file can't be changed, my goal is not to change the MD5 of the actual file, I want to add some contents to the file which would change the MD5 and by doing that I want the file to remain unchanged in functionalities.
As far as I understand, you have or want two copies of the same PE executable file. Now you want to change either or both of these files, so that when you calculate a hash of the file's contents, they are different.
If you change the checksum, chances are the executable won't run anymore. If you're OK with that, you can easily use the class that you showed. It seems to assume a checksum consists of two bytes and is offset at byte 0x12 in the executable. I can't verify right now that it is correct, but at a glance it doesn't seem to be.
Anyway you can create your unique checksum per file and set it:
FileUtils.WriteCheckSum(sourceFile, destFile1, 1);
FileUtils.WriteCheckSum(sourceFile, destFile2, 2);
Now the two files will bear different contents, so the hash of their contents will be different.
You can't just decide that you want your file to have a different hash because the hash is a direct result of the data stored in that file. Two identical files, in terms of what they contain, will always produce the same hash, regardless of what their names are.
Any changes to the content the file itself will result in an entirely different hash value.
MD5 is computed by passing bytes(a file for example) and representing them uniquely in hexadecimal, You don't change the "MD5" of a file, the result MD5 will change as the file changes.
Related
I am submitting the ACA forms(tax year:2016) to the IRS, getting the below error
<ns3:FormBCTransmitterSubmissionDtl xmlns="urn:us:gov:treasury:irs:ext:aca:air:ty16" xmlns:ns2="urn:us:gov:treasury:irs:common" xmlns:ns3="urn:us:gov:treasury:irs:msg:form1094-1095BCtransmittermessage">
<ACATransmitterSubmissionDetail>
<TransmitterErrorDetailGrp>
<ns2:ErrorMessageDetail>
<ns2:ErrorMessageCd>MANIFEST-025</ns2:ErrorMessageCd>
<ns2:ErrorMessageTxt>Manifest 'ChecksumAugmentationNum' must match the IRS-calculated 'ChecksumAugmentationNum' value of the transmission</ns2:ErrorMessageTxt>
</ns2:ErrorMessageDetail>
</TransmitterErrorDetailGrp>
</ACATransmitterSubmissionDetail>
Attached is our MTOM format we are using to send it through A2A.
https://www.dropbox.com/home?preview=samplemtom.txt
I am also tried the ChecksumAugmentationNum value set as Lower case also.
Have you successfully transmitted for Tax Year 2015? I have seen another post related to this issue, but have not run into this issue while sending TY2015 (to AATS or Production) or TY2016 records to AATS. My checksum calculation has not changed, and is very simple.
I have two methods I use to create the checksum: GetChecksum(string) and GetMD5Hash(MD5, string). This approach worked for TY2015, and I expect it to work for TY2016. IIRC, I took this approach directly from MSDN.
The string I pass into the GetChecksum method is the contents of the Form Data Attachment. In my process, I output the XML document into the file system for audit purposes, so the attachment is a physical file for me to use and reference. I read the attachment into a string variable using File.ReadAllText(string path) method.
After generating the checksum my process also will check the checksum against the database and return whether or not that checksum exists (meaning it was used by another form). In the case where this is true, then I update the Contact Suffix, regenerate the Form Data and then regenerate the checksum; this is per the IRS rules for transmission.
This is what is currently working for me, and hopefully this helps you.
Application Callers:
This is what I am doing to call the Checksum calculation functions/routines. It should be noted, I am physically writing each Form Data XML file to the File System, then reading from that.
string AttachmentFileContents = "";
AttachmentFileContents = File.ReadAllText(FormDataFilePath);
string checkSumAugmentationNumber = new Checksum().GetChecksum(AttachmentFileContents);
Checksum Methods:
These are the two methods I use for Checksum Calculation.
public string GetChecksum(string stringToEncrpyt)
{
string hash = "";
using(MD5 md5Hash = MD5.Create())
{
hash = GetMD5Hash(md5Hash, stringToEncrpyt);
}
return hash;
}
private string GetMD5Hash(MD5 md5Hash, string input)
{
byte[] data = md5Hash.ComputeHash(Encoding.UTF8.GetBytes(input));
StringBuilder sb = new StringBuilder();
for (int i = 0; i < data.Length; i++)
{
sb.Append(data[i].ToString("x2"));
}
return sb.ToString();
}
I need to combine multiple files (video & music & text) into single file with a custom file type (for example: *.abcd) and custom file data structure , and the file should only be read by my program and my program should be able to separate parts of this file. How do this in .net & c#?
https://www.imageupload.co.uk/image/ZWnh
Like M463 rightly pointed out, you could use System.IO.Compression to compress those files together and encrypt them...although encryption is a completely different art and another headache.
A better option would be to have some metadata in, say, the first few bytes of the file and then store the files' bytes as raw bytes. This would avoid any person from figuring the contents by just looking at it in a text editor. Again, if you want to really protect your data, encryption is unavoidable. But a simple algorithm to begin with would be this:
using System;
using System.IO;
using System.Collections.Generic;
namespace Compression
{
public class ClassName
{
public static void Compress(string[] fileNames, string resultantFileName)
{
List<byte> bytesToWrite = new List<byte>();
//add metadata about the number of files
int filesLength = fileNames.Length;
bytesToWrite.AddRange(BitConverter.GetBytes(filesLength));
List<byte[]> files = new List<byte[]>();
foreach(string fileName in fileNames)
{
byte[] bytes = File.ReadAllBytes(fileName);
//add metadata about the size of each file
bytesToWrite.AddRange(BitConverter.GetBytes(bytes.Length));
files.Add(bytes);
}
foreach(byte[] bytes in files)
{
//write the actual files itself
bytesToWrite.AddRange(bytes);
}
File.WriteAllBytes(resultantFileName, bytesToWrite.ToArray());
}
public static void Decompress(string fileName)
{
List<byte> bytes = new List<byte>(File.ReadAllBytes(fileName));
//this int represents the number of files in the byte array
int filesLength = BitConverter.ToInt32(bytes.ToArray(), 0);
List<int> sizes = new List<int>();
//get the size of each file
for(int i = 0; i < filesLength; i++)
{
//the first 2 bytes represent the number of files
//then each succeding int represents the size of each file
int size = BitConverter.ToInt32(bytes.ToArray(), 2 + i * 2);
sizes.Add(size);
}
//now read all the files
for(int i = 0; i < filesLength; i++)
{
int lastByteTillNow = 0;
for(int j = 0; j < i; j++)
lastByteTillNow += sizes[j];
File.WriteAllBytes("file " + i, bytes.GetRange(2 + 2 * filesLength + lastByteTillNow, sizes[i]).ToArray());
}
}
}
}
Now obviously this is not the best algorithm you've come across nor is it the most optimized snippet of code. After all, it is just what I could come up with in 10-15 mins. So I haven't even tested it yet. However, the point is, it gives you the idea doesn't it? I have limited the size of each file to the maximum length of an Int32 (however, changing it to Int64 a.k.a long wouldnt be much of a trouble). But it gives you an idea. You can even modify the snippet to load and write to and from the RAM via MemoryStreams (Systtem.IO.MemorySteam I think). But whatever, this should give you a start!
How about an encrypted .zip-container containing your files? Handling of .zip-files is already available in the .NET-Framework, take a look at the System.IO.Compression namespace. Or you could use some third party library.
You could even force a different file extension by just renaming the file, if you really want to...
I find this quite odd. I have a utility I'm working on which will be pointed at a folder, index the folder with relative path / filename / filesize / md5 hash / some other things. If the md5 hashes don't match, it updates the hash in the database, backs the file up again, and continues on it's way with the rest of the files. This is primarily for backup purposes, but also me leaning.
The first time I ran the program aimed at some of my web projects it used disk IO and grabbed file handles, both visible in Process Hacker. However, the second time of running it (as in, I shut it down and restarted it), it doesn't appear to be taking up any disk IO, and only periodically grabs a handle. Yet, a hash is appearing.
The code which iterates over the files:
foreach (string path in paths)
{
try
{
string relativePath = path.Replace(#"Z:\99_Projects\web\de.com\", "");
BackupFile backupFile = BackupFile.GetFile(relativePath, connection);
string md5hash = "";
long filesize = (new FileInfo(path)).Length;
using (var file = File.OpenRead(path))
{
md5hash = Hasher.ComputeMD5Hash(file);
//Console.WriteLine(md5hash);
if (backupFile == null)
{
BackupFile.NewBackupFile(relativePath, Path.GetFileName(path), md5hash, filesize, connection);
}
else
{
if (backupFile.md5 != md5hash)
backupFile.flags = CoreLib.Utils.Backup.Enums.BackupFileFlags.CHANGED;
else
backupFile.flags = CoreLib.Utils.Backup.Enums.BackupFileFlags.UNCHANGED;
backupFile.filesize = filesize;
backupFile.md5 = md5hash;
backupFile.Save(connection);
}
file.Close();
}
}
catch (IOException e)
{
Console.WriteLine("Access: " + Path.GetFileName(path));
}
catch (SQLiteException e)
{
Console.WriteLine("|E|");
}
catch (Exception e)
{
Console.WriteLine("|EG|");
throw e;
}
}
The code for the Hasher class as used, which is just really a small method wrapper for the cryptography MD5 hash calculator so I could reuse it (and other hash methods I stick in it) elsewhere in other code.
public class Hasher
{
public static string ComputeMD5Hash(Stream stream)
{
string hash = "";
using (var md5 = System.Security.Cryptography.MD5.Create())
{
hash = BitConverter.ToString(md5.ComputeHash(stream)).Replace("-", "").ToLower();
}
return hash;
}
}
I've tried several things, including debugging the application and verified that it's actually opening up file streams and computing a hash. I've also had it print out the hash to the console, as shown by the commented out line under where it's computed, but even when it's printing it out it's showing no disk IO at all.
Well, it looks as though it's simple file system cache mixed with Process Hacker's polling rate being too slow to pick up on it once cached. I changed a file, it detected the change but showed no measurable disk IO at all (which it does in bytes), so I'm assuming at this point it just didn't pick up on it due to how fast it was going.
Let's say I have a folder with five hundred pictures in it, and I want to check for repeats and delete them.
Here's the code I have right now:
using (var md5 = MD5.Create())
{
using (var stream = File.OpenRead(filename))
{
return md5.ComputeHash(stream);
}
}
Would this be viable to spot repeated MD5s in a specific folder, provided I loop it accordingly?
Creating hashes in order to identify identical files is OK, in any programming language, on any OS. It is slow, though, because you read the whole file even if that is not necessary.
I would recommend several passes for finding duplicates:
get the size of all files
for all files of equal size: get the hash of the first, say, 1k bytes
for all files of equal size and equal hash of first 1k: get the hash of the entire file
There is a risk of hash collisions. You cannot avoid it with hash algorithms. As MD5 uses 128 bits, the risk is 1 : (1 << 128) (roughly 0.0000000000000000000000000000000000000001) for two random files. Your chances of getting the jackpot in your national lottery four times in a row, using only one lottery ticket each week, are much better than getting a hash collision on a random pair of files.
Though the probability of a hash collision raises somewhat, if you compare the hash of many files. The mathematically interested and people implementing hash containers should look up the "birthday problem". Mere mortals trust MD5 hashes when they are not implementing cryptographic algorithms.
using System;
using System.IO;
using System.Collections.Generic;
internal static class FileComparer
{
public static void Compare(string directoryPath)
{
if(!Directory.Exists(directoryPath))
{
return;
}
FileComparer.Compare(new DirectoryInfo(directoryPath));
}
private static void Compare(DirectoryInfo info)
{
List<FileInfo> files = new List<FileInfo>(info.EnumerateFiles());
foreach(FileInfo file in files)
{
if(file.Exists)
{
byte[] array = File.ReadAllBytes(file.FullName);
foreach(FileInfo file2 in files)
{
int length = array.Length;
byte[] array2 = File.ReadAllBytes(file2.FullName);
if(array2.Length == length)
{
bool flag = true;
for(int current = 0; current < length; current++)
{
if(array[current] != array2[current])
{
flag = false;
break;
}
}
if(flag)
{
file2.Delete();
}
}
}
}
}
}
}
I need two methods one to encrypt and one to decrypt an xml file with a key= "hello world",the key hello world should be used to encrypt and decrypt the xml file.These methods should work on all machines!!! Any encryption methods will do. XML File contents below:
<root>
<lic>
<number>19834209</number>
<expiry>02/02/2002</expiry>
</lic>
</root>
Can some give me a sample?The issue is the msdn sample encyptions make a xml file encypted but when I decrypt on another machine it doesn't work.For example
I tried this sample:
How to: Encrypt XML Elements with Asymmetric Keys,
but here there is some kinda session and on another machine it says bad data phewf!
If you want the same key for encrypting and decrypting you should use a symmetric method (that's the definition, really). Here's the closest one to your sample (same source).
http://msdn.microsoft.com/en-us/library/sb7w85t6.aspx
The posted sample isn't working because they aren't using the same keys. Not only on different machines: running the program on the same machine twice should not work either (didn't work for me), because they use different random keys every time.
try adding this code after creating your key:
key = new RijndaelManaged();
string password = "Password1234"; //password here
byte[] saltBytes = Encoding.UTF8.GetBytes("Salt"); // salt here (another string)
var p = new Rfc2898DeriveBytes(password, saltBytes); //TODO: think about number of iterations (third parameter)
// sizes are devided by 8 because [ 1 byte = 8 bits ]
key.IV = p.GetBytes(key.BlockSize / 8);
key.Key = p.GetBytes(key.KeySize / 8);
Now the program is using the same key and initial vector, and Encrypt and Decrypt should work on all machines.
Also, consider renaming key to algorithm, otherwise this is very misleading. I'd say it's a bad, not-working-well example from MSDN.
NOTE: PasswordDeriveBytes.GetBytes() has been deprecated because of serious (security) issues within the PasswordDeriveBytes class. The code above has been rewritten to use the safer Rfc2898DeriveBytes class instead (PBKDF2 instead of PBKDF1). Code generated with the above using PasswordDeriveBytes may be compromised.
See also: Recommended # of iterations when using PKBDF2-SHA256?
First of all, if you want to use the same key for encrypting and decrypting, you should look at symmetric cryptography. Asymmetric cryptography is when the keys for encrypting and decrypting are different. Just so that you know - RSA is asymmetric, TripleDES and Rijndael are symmetric. There are others too, but .NET does not have default implementations for them.
I'd advise studying the System.Security.Cryptography namespace. And learning a bit about all that stuff. It has all you need to encrypt and decrypt files, as well as generate a password. In particular, you might be interested in these classes:
CryptoStream
PasswordDeriveBytes
RijndaelManaged
There are also examples for usage in MSDN for each of them. You can use these classes to encrypt any file, not just XML. If however you want to encrypt just a select few elements, you can take a look at System.Security.Cryptography.Xml namespace. I see you've already found one article about it. Keep following the links on that page and you will learn more about those classes.
Would be cooler if you used a private key to sign the <lic> element and added the result to the file (in a <hash> element perhaps). This would make it possibly for everyone to read the xml file in case your support needs to know the license number, or the date of expiry, but they can not change any values without the private key.
The public key needed to verify the signature would be common knowledge.
Clarification
Signing your code will only protect it against changes, it will not keep any information in it hidden. Your original question mentions encryption, but I am not sure that it is a requirement to hide the data, or just protect it from modification.
Example code: (Never publish PrivateKey.key. ServerMethods are only needed when signing the xml file, ClientMethods are only needed when verifying the xml file.)
using System;
using System.Diagnostics;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using System.Xml;
public static class Program {
public static void Main() {
if (!File.Exists("PublicKey.key")) {
// Assume first run, generate keys and sign document.
ServerMethods.GenerateKeyPair();
var input = new XmlDocument();
input.Load("input.xml");
Debug.Assert(input.DocumentElement != null);
var licNode = input.DocumentElement["lic"];
Debug.Assert(licNode != null);
var licNodeXml = licNode.OuterXml;
var signedNode = input.CreateElement("signature");
signedNode.InnerText = ServerMethods.CalculateSignature(licNodeXml);
input.DocumentElement.AppendChild(signedNode);
input.Save("output.xml");
}
if (ClientMethods.IsValidLicense("output.xml")) {
Console.WriteLine("VALID");
} else {
Console.WriteLine("INVALID");
}
}
public static class ServerMethods {
public static void GenerateKeyPair() {
var rsa = SharedInformation.CryptoProvider;
using (var keyWriter = File.CreateText("PublicKey.key"))
keyWriter.Write(rsa.ToXmlString(false));
using (var keyWriter = File.CreateText("PrivateKey.key"))
keyWriter.Write(rsa.ToXmlString(true));
}
public static string CalculateSignature(string data) {
var rsa = SharedInformation.CryptoProvider;
rsa.FromXmlString(File.ReadAllText("PrivateKey.key"));
var dataBytes = Encoding.UTF8.GetBytes(data);
var signatureBytes = rsa.SignData(dataBytes, SharedInformation.HashAlgorithm);
return Convert.ToBase64String(signatureBytes);
}
}
public static class ClientMethods {
public static bool IsValid(string data, string signature) {
var rsa = SharedInformation.CryptoProvider;
rsa.FromXmlString(File.ReadAllText("PublicKey.key"));
var dataBytes = Encoding.UTF8.GetBytes(data);
var signatureBytes = Convert.FromBase64String(signature);
return rsa.VerifyData(dataBytes, SharedInformation.HashAlgorithm, signatureBytes);
}
public static bool IsValidLicense(string filename) {
var doc = new XmlDocument();
doc.Load(filename);
var licNode = doc.SelectSingleNode("/root/lic") as XmlElement;
var signatureNode = doc.SelectSingleNode("/root/signature") as XmlElement;
if (licNode == null || signatureNode == null) return false;
return IsValid(licNode.OuterXml, signatureNode.InnerText);
}
}
public static class SharedInformation {
public static int KeySize {
get { return 1024; }
}
public static string HashAlgorithm {
get { return "SHA512"; }
}
public static RSACryptoServiceProvider CryptoProvider {
get { return new RSACryptoServiceProvider(KeySize, new CspParameters()); }
}
}
}
this is how you digitally sign and verify XML documents Sign XML Documents