Using the following code I always get the same hash regardless of the input. Any ideas why that might be?
private static SHA256 sha256;
internal static byte[] HashForCDCR(this string value)
{
byte[] hash;
using (var myStream = new System.IO.MemoryStream())
{
using (var sw = new System.IO.StreamWriter(myStream))
{
sw.Write(value);
hash = sha256.ComputeHash(myStream);
}
}
return hash;
}
You are computing hash of empty portion of the stream (the one immediately after content you wrote with sw.Write) so it always the same.
Cheap fix: sw.Flush();myStream.Position = 0;. Better fix is to finish writing and create new read only stream for encryption based on original stream:
using (var myStream = new System.IO.MemoryStream())
{
using (var sw = new System.IO.StreamWriter(myStream))
{
sw.Write(value);
}
using (var readonlyStream = new MemoryStream(myStream.ToArray(), writable:false)
{
hash = sha256.ComputeHash(readonlyStream);
}
}
You may need to flush your stream. For optimal performance StreamWriter doesn't write to stream immediately . It waits for its internal buffer to fill. Flushing the writer immediately flush the content of the internal buffer to underline stream.
sw.Write(value);
sw.Flush();
myStream.Position = 0;
hash = sha256.ComputeHash(myStream);
I will probably use the solution that Alexei Levenkov called a "cheap fix". However, I did come across one other way to make it work, which I will post for future readers:
var encoding = new System.Text.UTF8Encoding();
var bytes = encoding.GetBytes(value);
var hash = sha256.ComputeHash(bytes);
return hash;
Jacob
Related
What is the best solution in C# for computing an "on the fly" md5 like hash of a stream of unknown length? Specifically, I want to compute a hash from data received over the network. I know I am done receiving data when the sender terminates the connection, so I don't know the length in advance.
[EDIT] - Right now I am using md5 and am doing a second pass over the data after it's been saved and written to disk. I'd rather hash it in place as it comes in from the network.
MD5, like other hash functions, does not require two passes.
To start:
HashAlgorithm hasher = ..;
hasher.Initialize();
As each block of data arrives:
byte[] buffer = ..;
int bytesReceived = ..;
hasher.TransformBlock(buffer, 0, bytesReceived, null, 0);
To finish and retrieve the hash:
hasher.TransformFinalBlock(new byte[0], 0, 0);
byte[] hash = hasher.Hash;
This pattern works for any type derived from HashAlgorithm, including MD5CryptoServiceProvider and SHA1Managed.
HashAlgorithm also defines a method ComputeHash which takes a Stream object; however, this method will block the thread until the stream is consumed. Using the TransformBlock approach allows an "asynchronous hash" that is computed as data arrives without using up a thread.
Further to #peter-mourfield 's answer, here is the code that uses ComputeHash():
private static string CalculateMd5(string filePathName) {
using (var stream = File.OpenRead(filePathName))
using (var md5 = MD5.Create()) {
var hash = md5.ComputeHash(stream);
var base64String = Convert.ToBase64String(hash);
return base64String;
}
}
Since both the stream as well as MD5 implement IDisposible, you need to use using(...){...}
The method in the code example returns the same string that is used for the MD5 checksum in Azure Blob Storage.
The System.Security.Cryptography.MD5 class contains a ComputeHash method that takes either a byte[] or Stream. Check out the documentation.
This seems like a perfect use case for CryptoStream (docs).
I've used CryptoStream for processing unknown-length streams of database results that need to be gzipped and then transferred across the network along with a hash of the compressed file. Inserting a CryptoStream between the compressor and the file writer allows you to compute the hash on the fly so that it's ready as soon as the file is written.
The basic approach looks like this:
var hasher = MD5.Create();
using (FileStream outFile = File.Create(filePath))
using (CryptoStream crypto = new CryptoStream(outFile, hasher, CryptoStreamMode.Write))
using (GZipStream compress = new GZipStream(crypto, CompressionMode.Compress))
using (StreamWriter writer = new StreamWriter(compress))
{
foreach (string line in GetLines())
writer.WriteLine(line);
}
// at this point the streams are closed so the hash is ready
string hash = BitConverter.ToString(hasher.Hash).Replace("-", "").ToLowerInvariant();
Necromancing.
Two possibilitites in C# .NET Core:
private static System.Security.Cryptography.HashAlgorithm GetHashAlgorithm(System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.MD5.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA1.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA256.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA384.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA512.Create();
throw new System.Security.Cryptography.CryptographicException($"Unknown hash algorithm \"{hashAlgorithmName.Name}\".");
}
protected override byte[] HashData(System.IO.Stream data,
System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
using (System.Security.Cryptography.HashAlgorithm hashAlgorithm1 =
GetHashAlgorithm(hashAlgorithm))
return hashAlgorithm1.ComputeHash(data);
}
or with BouncyCastle:
private static Org.BouncyCastle.Crypto.IDigest GetBouncyAlgorithm(
System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
return new Org.BouncyCastle.Crypto.Digests.MD5Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
return new Org.BouncyCastle.Crypto.Digests.Sha1Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
return new Org.BouncyCastle.Crypto.Digests.Sha256Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
return new Org.BouncyCastle.Crypto.Digests.Sha384Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
return new Org.BouncyCastle.Crypto.Digests.Sha512Digest();
throw new System.Security.Cryptography.CryptographicException(
$"Unknown hash algorithm \"{hashAlgorithmName.Name}\"."
);
} // End Function GetBouncyAlgorithm
protected override byte[] HashData(System.IO.Stream data,
System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
Org.BouncyCastle.Crypto.IDigest digest = GetBouncyAlgorithm(hashAlgorithm);
byte[] buffer = new byte[4096];
int cbSize;
while ((cbSize = data.Read(buffer, 0, buffer.Length)) > 0)
digest.BlockUpdate(buffer, 0, cbSize);
byte[] hash = new byte[digest.GetDigestSize()];
digest.DoFinal(hash, 0);
return hash;
}
Another option could be to use the System.Security.Cryptography.IncrementalHash class instead.
byte[] DataBrick;
var IncMD5 = IncrementalHash.CreateHash(HashAlgorithmName.MD5);
then you can: accumulate data in the hasher
IncMD5.AppendData(DataBrick,0,DataBrick.Length);
,check the hash value for the data accumulated so far
byte[] hash = IncMD5.GetCurrentHash();
bytesReceived = netStream.Read(DataBrick,0,DataBrick.Length);
IncMD5.AppendData(DataBrick,0,bytesReceived);
,or stop and reset to start accumulating a new hash value
byte[] hash = IncMD5.GetHashAndReset();
Note: it implements iDisposable
IncMD5.Dispose(); // when done, or using(IncMD5){..} if that makes more sense in your scope
[SOLVED]: I copied the file and ran the hasher on that copy.
I need my app to find the EXE's current MD5. I can get the MD5 of any file.
However, no matter what I do, I cannot get a FileStream to read the open EXE. I tried using FileOptions.Asynchronous but that didn't help.
EDIT: I guess I'm not very clear. I want my app to be be able to read itself.
EDIT to code:
private void GetMd5()
{
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
FileInfo fi = new FileInfo(Process.GetCurrentProcess().MainModule.FileName);
FileStream stream = File.Create(Process.GetCurrentProcess().MainModule.FileName, (int)fi.Length, FileOptions.Asynchronous);
md5.ComputeHash(stream);
stream.Close();
string rtrn = "";
for (int i = 0; i < md5.Hash.Length; i++)
{
rtrn += (md5.Hash[i].ToString("x2"));
}
MessageBox.Show(rtrn.ToUpper());
}
The File.Create Method (String, Int32, FileOptions, FileSecurity):
Creates or overwrites the specified file with the specified buffer
size, file options, and file security.
I'm fairly sure that's not what you intended to do. Presumably you want FileInfo.Open Method (FileMode, FileAccess):
FileInfo fi = new FileInfo(path);
FileStream stream = File.Open(path, FileMode.Open);
Bit late to the party, but was recently trying to do this myself. In .NET 4.5 the following works quite nicely without the need for making a temporary copy. As said, if you can read the file to make a copy of it, you can read the file to generate a hash for it.
private string GetMD5()
{
System.Security.Cryptography.MD5CryptoServiceProvider md5 = new System.Security.Cryptography.MD5CryptoServiceProvider();
System.IO.FileStream stream = new System.IO.FileStream(System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName, System.IO.FileMode.Open, System.IO.FileAccess.Read);
md5.ComputeHash(stream);
stream.Close();
System.Text.StringBuilder sb = new System.Text.StringBuilder();
for (int i = 0; i < md5.Hash.Length; i++)
sb.Append(md5.Hash[i].ToString("x2"));
return sb.ToString().ToUpperInvariant();
}
Change: FileStream stream = File.Create(path, (int)fi.Length, FileOptions.Asynchronous); to FileStream stream = File.Open(path, FileMode.Open);
Since I've tried these answers and found that none of them change every edit to the exe, or change at all, I found something that actually does work.
I did not edit any code here, all of this was from the referred page below.
Reference: http://www.vcskicks.com/self-hashing.php
internal static class ExecutingHash
{
public static string GetExecutingFileHash()
{
return MD5(GetSelfBytes());
}
private static string MD5(byte[] input)
{
return MD5(ASCIIEncoding.ASCII.GetString(input));
}
private static string MD5(string input)
{
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
byte[] originalBytes = ASCIIEncoding.Default.GetBytes(input);
byte[] encodedBytes = md5.ComputeHash(originalBytes);
return BitConverter.ToString(encodedBytes).Replace("-", "");
}
private static byte[] GetSelfBytes()
{
string path = Application.ExecutablePath;
FileStream running = File.OpenRead(path);
byte[] exeBytes = new byte[running.Length];
running.Read(exeBytes, 0, exeBytes.Length);
running.Close();
return exeBytes;
}
}
Every test seems to output properly. I'd recommend to anyone seeing this to use this class or make something of it.
How do I use the SHA1CryptoServiceProvider() on a file to create a SHA1 Checksum of the file?
using (FileStream fs = new FileStream(#"C:\file\location", FileMode.Open))
using (BufferedStream bs = new BufferedStream(fs))
{
using (SHA1Managed sha1 = new SHA1Managed())
{
byte[] hash = sha1.ComputeHash(bs);
StringBuilder formatted = new StringBuilder(2 * hash.Length);
foreach (byte b in hash)
{
formatted.AppendFormat("{0:X2}", b);
}
}
}
formatted contains the string representation of the SHA-1 hash. Also, by using a FileStream instead of a byte buffer, ComputeHash computes the hash in chunks, so you don't have to load the entire file in one go, which is helpful for large files.
With the ComputeHash method. See here:
ComputeHash
Example snippet:
using(var cryptoProvider = new SHA1CryptoServiceProvider())
{
string hash = BitConverter
.ToString(cryptoProvider.ComputeHash(buffer));
//do something with hash
}
Where buffer is the contents of your file.
If you are already reading the file as a stream, then the following technique calculates the hash as you read it. The only caveat is that you need to consume the whole stream.
class Program
{
static void Main(string[] args)
{
String sourceFileName = "C:\\test.txt";
Byte[] shaHash;
//Use Sha1Managed if you really want sha1
using (var shaForStream = new SHA256Managed())
using (Stream sourceFileStream = File.Open(sourceFileName, FileMode.Open))
using (Stream sourceStream = new CryptoStream(sourceFileStream, shaForStream, CryptoStreamMode.Read))
{
//Do something with the sourceStream
//NOTE You need to read all the bytes, otherwise you'll get an exception ({"Hash must be finalized before the hash value is retrieved."})
while(sourceStream.ReadByte() != -1);
shaHash = shaForStream.Hash;
}
Console.WriteLine(Convert.ToBase64String(shaHash));
}
}
Also you can try:
FileStream fop = File.OpenRead(#"C:\test.bin");
string chksum = BitConverter.ToString(System.Security.Cryptography.SHA1.Create().ComputeHash(fop));
Good morning all,
I'm working on an MD5 file integrity check tool in C#.
How long should it take for a file to be given an MD5 checksum value?
For example, if I try to get a 2gb .mpg file, it is taking around 5 mins+ each time.
This seems overly long.
Am I just being impatient?
Below is the code I'm running
public string getHash(String #fileLocation)
{
FileStream fs = new FileStream(#fileLocation, FileMode.Open);
HashAlgorithm alg = new HMACMD5();
byte[] hashValue = alg.ComputeHash(fs);
string md5Result = "";
foreach (byte x in hashValue)
{
md5Result += x;
}
fs.Close();
return md5Result;
}
Any suggestions will be appreciated.
Regards
See this on how to calculate file hash value in a most efficient way. You basically have to wrap FileStream into a BufferedStream and than feed that into HMACMD5.ComputeHash(Stream) overload:
HashAlgorithm hmacMd5 = new HMACMD5();
byte[] hash;
using(Stream fileStream = new FileStream(fileLocation, FileMode.Open))
using(Stream bufferedStream = new BufferedStream(fileStream, 1200000))
hash = hmacMd5.ComputeHash(bufferedStream);
What is the best method of writing a StringBuilder to a System.IO.Stream?
I am currently doing:
StringBuilder message = new StringBuilder("All your base");
message.Append(" are belong to us");
System.IO.MemoryStream stream = new System.IO.MemoryStream();
System.Text.ASCIIEncoding encoding = new ASCIIEncoding();
stream.Write(encoder.GetBytes(message.ToString()), 0, message.Length);
Don't use a StringBuilder, if you're writing to a stream, do just that with a StreamWriter:
using (var memoryStream = new MemoryStream())
using (var writer = new StreamWriter(memoryStream ))
{
// Various for loops etc as necessary that will ultimately do this:
writer.Write(...);
}
That is the best method. Other wise loss the StringBuilder and use something like following:
using (MemoryStream ms = new MemoryStream())
{
using (StreamWriter sw = new StreamWriter(ms, Encoding.Unicode))
{
sw.WriteLine("dirty world.");
}
//do somthing with ms
}
Perhaps it will be usefull.
var sb= new StringBuilder("All your money");
sb.Append(" are belong to us, dude.");
var myString = sb.ToString();
var myByteArray = System.Text.Encoding.UTF8.GetBytes(myString);
var ms = new MemoryStream(myByteArray);
// Do what you need with MemoryStream
Depending on your use case it may also make sense to just start with a StringWriter:
StringBuilder sb = null;
// StringWriter - a TextWriter backed by a StringBuilder
using (var writer = new StringWriter())
{
writer.WriteLine("Blah");
. . .
sb = writer.GetStringBuilder(); // Get the backing StringBuilder out
}
// Do whatever you want with the StringBuilder
EDIT: As #ramon-smits points out, if you have access to StringBuilder.GetChunks(), you will also have access to StreamWriter.WriteAsync(StringBuilder). So you can just do this instead:
StringBuilder stringBuilder = new StringBuilder();
// Write data to StringBuilder...
Stream stream = GetStream(); // Get output stream from somewhere.
using (var streamWriter = new StreamWriter(stream, Encoding.UTF8, leaveOpen: true))
{
await streamWriter.WriteAsync(stringBuilder);
}
Much simpler.
I recently had to do exactly this thing and found this question with unsatisfactory answers.
You can write a StringBuilder to a Stream without materializing the entire string:
StringBuilder stringBuilder = new StringBuilder();
// Write data to StringBuilder...
Stream stream = GetStream(); // Get output stream from somewhere.
using (var streamWriter = new StreamWriter(stream, Encoding.UTF8, leaveOpen: true))
{
foreach (ReadOnlyMemory<char> chunk in stringBuilder.GetChunks())
{
await streamWriter.WriteAsync(chunk);
}
}
N.B. This API (StringBuilder.GetChunks()) is only available in .NET Core 3.0 and above
If this operation happens frequently, you could further reduce GC pressure by using a StringBuilder object pool.
If you want to use something like a StringBuilder because it is cleaner to pass around and work with, then you can use something like the following StringBuilder alternate I created.
The most important thing it does different is that it allows access to the internal data without having to assemble it into a String or ByteArray first. This means you don't have to double up the memory requirements and risk trying to allocate a contiguous chunk of memory that fits your entire object.
NOTE: I am sure there are better options then using a List<string>() internally but this was simple and proved to be good enough for my purposes.
public class StringBuilderEx
{
List<string> data = new List<string>();
public void Append(string input)
{
data.Add(input);
}
public void AppendLine(string input)
{
data.Add(input + "\n");
}
public void AppendLine()
{
data.Add("\n");
}
/// <summary>
/// Copies all data to a String.
/// Warning: Will fail with an OutOfMemoryException if the data is too
/// large to fit into a single contiguous string.
/// </summary>
public override string ToString()
{
return String.Join("", data);
}
/// <summary>
/// Process Each section of the data in place. This avoids the
/// memory pressure of exporting everything to another contiguous
/// block of memory before processing.
/// </summary>
public void ForEach(Action<string> processData)
{
foreach (string item in data)
processData(item);
}
}
Now you can dump the entire contents to file using the following code.
var stringData = new StringBuilderEx();
stringData.Append("Add lots of data");
using (StreamWriter file = new System.IO.StreamWriter(localFilename))
{
stringData.ForEach((data) =>
{
file.Write(data);
});
}