Good morning all,
I'm working on an MD5 file integrity check tool in C#.
How long should it take for a file to be given an MD5 checksum value?
For example, if I try to get a 2gb .mpg file, it is taking around 5 mins+ each time.
This seems overly long.
Am I just being impatient?
Below is the code I'm running
public string getHash(String #fileLocation)
{
FileStream fs = new FileStream(#fileLocation, FileMode.Open);
HashAlgorithm alg = new HMACMD5();
byte[] hashValue = alg.ComputeHash(fs);
string md5Result = "";
foreach (byte x in hashValue)
{
md5Result += x;
}
fs.Close();
return md5Result;
}
Any suggestions will be appreciated.
Regards
See this on how to calculate file hash value in a most efficient way. You basically have to wrap FileStream into a BufferedStream and than feed that into HMACMD5.ComputeHash(Stream) overload:
HashAlgorithm hmacMd5 = new HMACMD5();
byte[] hash;
using(Stream fileStream = new FileStream(fileLocation, FileMode.Open))
using(Stream bufferedStream = new BufferedStream(fileStream, 1200000))
hash = hmacMd5.ComputeHash(bufferedStream);
Related
Using the following code I always get the same hash regardless of the input. Any ideas why that might be?
private static SHA256 sha256;
internal static byte[] HashForCDCR(this string value)
{
byte[] hash;
using (var myStream = new System.IO.MemoryStream())
{
using (var sw = new System.IO.StreamWriter(myStream))
{
sw.Write(value);
hash = sha256.ComputeHash(myStream);
}
}
return hash;
}
You are computing hash of empty portion of the stream (the one immediately after content you wrote with sw.Write) so it always the same.
Cheap fix: sw.Flush();myStream.Position = 0;. Better fix is to finish writing and create new read only stream for encryption based on original stream:
using (var myStream = new System.IO.MemoryStream())
{
using (var sw = new System.IO.StreamWriter(myStream))
{
sw.Write(value);
}
using (var readonlyStream = new MemoryStream(myStream.ToArray(), writable:false)
{
hash = sha256.ComputeHash(readonlyStream);
}
}
You may need to flush your stream. For optimal performance StreamWriter doesn't write to stream immediately . It waits for its internal buffer to fill. Flushing the writer immediately flush the content of the internal buffer to underline stream.
sw.Write(value);
sw.Flush();
myStream.Position = 0;
hash = sha256.ComputeHash(myStream);
I will probably use the solution that Alexei Levenkov called a "cheap fix". However, I did come across one other way to make it work, which I will post for future readers:
var encoding = new System.Text.UTF8Encoding();
var bytes = encoding.GetBytes(value);
var hash = sha256.ComputeHash(bytes);
return hash;
Jacob
What is the best solution in C# for computing an "on the fly" md5 like hash of a stream of unknown length? Specifically, I want to compute a hash from data received over the network. I know I am done receiving data when the sender terminates the connection, so I don't know the length in advance.
[EDIT] - Right now I am using md5 and am doing a second pass over the data after it's been saved and written to disk. I'd rather hash it in place as it comes in from the network.
MD5, like other hash functions, does not require two passes.
To start:
HashAlgorithm hasher = ..;
hasher.Initialize();
As each block of data arrives:
byte[] buffer = ..;
int bytesReceived = ..;
hasher.TransformBlock(buffer, 0, bytesReceived, null, 0);
To finish and retrieve the hash:
hasher.TransformFinalBlock(new byte[0], 0, 0);
byte[] hash = hasher.Hash;
This pattern works for any type derived from HashAlgorithm, including MD5CryptoServiceProvider and SHA1Managed.
HashAlgorithm also defines a method ComputeHash which takes a Stream object; however, this method will block the thread until the stream is consumed. Using the TransformBlock approach allows an "asynchronous hash" that is computed as data arrives without using up a thread.
Further to #peter-mourfield 's answer, here is the code that uses ComputeHash():
private static string CalculateMd5(string filePathName) {
using (var stream = File.OpenRead(filePathName))
using (var md5 = MD5.Create()) {
var hash = md5.ComputeHash(stream);
var base64String = Convert.ToBase64String(hash);
return base64String;
}
}
Since both the stream as well as MD5 implement IDisposible, you need to use using(...){...}
The method in the code example returns the same string that is used for the MD5 checksum in Azure Blob Storage.
The System.Security.Cryptography.MD5 class contains a ComputeHash method that takes either a byte[] or Stream. Check out the documentation.
This seems like a perfect use case for CryptoStream (docs).
I've used CryptoStream for processing unknown-length streams of database results that need to be gzipped and then transferred across the network along with a hash of the compressed file. Inserting a CryptoStream between the compressor and the file writer allows you to compute the hash on the fly so that it's ready as soon as the file is written.
The basic approach looks like this:
var hasher = MD5.Create();
using (FileStream outFile = File.Create(filePath))
using (CryptoStream crypto = new CryptoStream(outFile, hasher, CryptoStreamMode.Write))
using (GZipStream compress = new GZipStream(crypto, CompressionMode.Compress))
using (StreamWriter writer = new StreamWriter(compress))
{
foreach (string line in GetLines())
writer.WriteLine(line);
}
// at this point the streams are closed so the hash is ready
string hash = BitConverter.ToString(hasher.Hash).Replace("-", "").ToLowerInvariant();
Necromancing.
Two possibilitites in C# .NET Core:
private static System.Security.Cryptography.HashAlgorithm GetHashAlgorithm(System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.MD5.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA1.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA256.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA384.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA512.Create();
throw new System.Security.Cryptography.CryptographicException($"Unknown hash algorithm \"{hashAlgorithmName.Name}\".");
}
protected override byte[] HashData(System.IO.Stream data,
System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
using (System.Security.Cryptography.HashAlgorithm hashAlgorithm1 =
GetHashAlgorithm(hashAlgorithm))
return hashAlgorithm1.ComputeHash(data);
}
or with BouncyCastle:
private static Org.BouncyCastle.Crypto.IDigest GetBouncyAlgorithm(
System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
return new Org.BouncyCastle.Crypto.Digests.MD5Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
return new Org.BouncyCastle.Crypto.Digests.Sha1Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
return new Org.BouncyCastle.Crypto.Digests.Sha256Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
return new Org.BouncyCastle.Crypto.Digests.Sha384Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
return new Org.BouncyCastle.Crypto.Digests.Sha512Digest();
throw new System.Security.Cryptography.CryptographicException(
$"Unknown hash algorithm \"{hashAlgorithmName.Name}\"."
);
} // End Function GetBouncyAlgorithm
protected override byte[] HashData(System.IO.Stream data,
System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
Org.BouncyCastle.Crypto.IDigest digest = GetBouncyAlgorithm(hashAlgorithm);
byte[] buffer = new byte[4096];
int cbSize;
while ((cbSize = data.Read(buffer, 0, buffer.Length)) > 0)
digest.BlockUpdate(buffer, 0, cbSize);
byte[] hash = new byte[digest.GetDigestSize()];
digest.DoFinal(hash, 0);
return hash;
}
Another option could be to use the System.Security.Cryptography.IncrementalHash class instead.
byte[] DataBrick;
var IncMD5 = IncrementalHash.CreateHash(HashAlgorithmName.MD5);
then you can: accumulate data in the hasher
IncMD5.AppendData(DataBrick,0,DataBrick.Length);
,check the hash value for the data accumulated so far
byte[] hash = IncMD5.GetCurrentHash();
bytesReceived = netStream.Read(DataBrick,0,DataBrick.Length);
IncMD5.AppendData(DataBrick,0,bytesReceived);
,or stop and reset to start accumulating a new hash value
byte[] hash = IncMD5.GetHashAndReset();
Note: it implements iDisposable
IncMD5.Dispose(); // when done, or using(IncMD5){..} if that makes more sense in your scope
okay so I have this code for decrypting files
public static byte[] DecryptFile(string inputFile, string skey)
{
RijndaelManaged aes = new RijndaelManaged();
byte[] key = ASCIIEncoding.UTF8.GetBytes(skey);
using (FileStream fsCrypt = new FileStream(inputFile, FileMode.Open))
{
using (CryptoStream cs =
new CryptoStream(fsCrypt, aes.CreateDecryptor(key, key),
CryptoStreamMode.Read))
{
using (BinaryReader reader = new BinaryReader(cs))
{
byte[] str = reader.ReadBytes(Convert.ToInt32(cs.Length));
reader.Close();
cs.Close();
return (str);
}
}
}
}
}
NOW i've got a problem with it, i can't determine the byte length! I tried
cs.Length
but it says the Stream doesn't support seeking (something like tht)
I also tried counting the bytes of the file by
File.ReadAllBytes(encrypted_file_path).Length
but it says the file is in use...it is indeed in use because of the FileStream fsCrypt
for the meantime I replaced cs.Length with some large integer to make it work..like 1000000..the maximum integer that doesn't cause any exception..it does work that way.
You cannot know the length until after you decrypt the entire file.
Therefore, you need to start with a small array, and make it bigger as it gets full.
The MemoryStream class does just that; you can just cs.CopyTo() into a new MemoryStream and call ToArray().
[SOLVED]: I copied the file and ran the hasher on that copy.
I need my app to find the EXE's current MD5. I can get the MD5 of any file.
However, no matter what I do, I cannot get a FileStream to read the open EXE. I tried using FileOptions.Asynchronous but that didn't help.
EDIT: I guess I'm not very clear. I want my app to be be able to read itself.
EDIT to code:
private void GetMd5()
{
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
FileInfo fi = new FileInfo(Process.GetCurrentProcess().MainModule.FileName);
FileStream stream = File.Create(Process.GetCurrentProcess().MainModule.FileName, (int)fi.Length, FileOptions.Asynchronous);
md5.ComputeHash(stream);
stream.Close();
string rtrn = "";
for (int i = 0; i < md5.Hash.Length; i++)
{
rtrn += (md5.Hash[i].ToString("x2"));
}
MessageBox.Show(rtrn.ToUpper());
}
The File.Create Method (String, Int32, FileOptions, FileSecurity):
Creates or overwrites the specified file with the specified buffer
size, file options, and file security.
I'm fairly sure that's not what you intended to do. Presumably you want FileInfo.Open Method (FileMode, FileAccess):
FileInfo fi = new FileInfo(path);
FileStream stream = File.Open(path, FileMode.Open);
Bit late to the party, but was recently trying to do this myself. In .NET 4.5 the following works quite nicely without the need for making a temporary copy. As said, if you can read the file to make a copy of it, you can read the file to generate a hash for it.
private string GetMD5()
{
System.Security.Cryptography.MD5CryptoServiceProvider md5 = new System.Security.Cryptography.MD5CryptoServiceProvider();
System.IO.FileStream stream = new System.IO.FileStream(System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName, System.IO.FileMode.Open, System.IO.FileAccess.Read);
md5.ComputeHash(stream);
stream.Close();
System.Text.StringBuilder sb = new System.Text.StringBuilder();
for (int i = 0; i < md5.Hash.Length; i++)
sb.Append(md5.Hash[i].ToString("x2"));
return sb.ToString().ToUpperInvariant();
}
Change: FileStream stream = File.Create(path, (int)fi.Length, FileOptions.Asynchronous); to FileStream stream = File.Open(path, FileMode.Open);
Since I've tried these answers and found that none of them change every edit to the exe, or change at all, I found something that actually does work.
I did not edit any code here, all of this was from the referred page below.
Reference: http://www.vcskicks.com/self-hashing.php
internal static class ExecutingHash
{
public static string GetExecutingFileHash()
{
return MD5(GetSelfBytes());
}
private static string MD5(byte[] input)
{
return MD5(ASCIIEncoding.ASCII.GetString(input));
}
private static string MD5(string input)
{
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
byte[] originalBytes = ASCIIEncoding.Default.GetBytes(input);
byte[] encodedBytes = md5.ComputeHash(originalBytes);
return BitConverter.ToString(encodedBytes).Replace("-", "");
}
private static byte[] GetSelfBytes()
{
string path = Application.ExecutablePath;
FileStream running = File.OpenRead(path);
byte[] exeBytes = new byte[running.Length];
running.Read(exeBytes, 0, exeBytes.Length);
running.Close();
return exeBytes;
}
}
Every test seems to output properly. I'd recommend to anyone seeing this to use this class or make something of it.
How do I use the SHA1CryptoServiceProvider() on a file to create a SHA1 Checksum of the file?
using (FileStream fs = new FileStream(#"C:\file\location", FileMode.Open))
using (BufferedStream bs = new BufferedStream(fs))
{
using (SHA1Managed sha1 = new SHA1Managed())
{
byte[] hash = sha1.ComputeHash(bs);
StringBuilder formatted = new StringBuilder(2 * hash.Length);
foreach (byte b in hash)
{
formatted.AppendFormat("{0:X2}", b);
}
}
}
formatted contains the string representation of the SHA-1 hash. Also, by using a FileStream instead of a byte buffer, ComputeHash computes the hash in chunks, so you don't have to load the entire file in one go, which is helpful for large files.
With the ComputeHash method. See here:
ComputeHash
Example snippet:
using(var cryptoProvider = new SHA1CryptoServiceProvider())
{
string hash = BitConverter
.ToString(cryptoProvider.ComputeHash(buffer));
//do something with hash
}
Where buffer is the contents of your file.
If you are already reading the file as a stream, then the following technique calculates the hash as you read it. The only caveat is that you need to consume the whole stream.
class Program
{
static void Main(string[] args)
{
String sourceFileName = "C:\\test.txt";
Byte[] shaHash;
//Use Sha1Managed if you really want sha1
using (var shaForStream = new SHA256Managed())
using (Stream sourceFileStream = File.Open(sourceFileName, FileMode.Open))
using (Stream sourceStream = new CryptoStream(sourceFileStream, shaForStream, CryptoStreamMode.Read))
{
//Do something with the sourceStream
//NOTE You need to read all the bytes, otherwise you'll get an exception ({"Hash must be finalized before the hash value is retrieved."})
while(sourceStream.ReadByte() != -1);
shaHash = shaForStream.Hash;
}
Console.WriteLine(Convert.ToBase64String(shaHash));
}
}
Also you can try:
FileStream fop = File.OpenRead(#"C:\test.bin");
string chksum = BitConverter.ToString(System.Security.Cryptography.SHA1.Create().ComputeHash(fop));