How do I use the SHA1CryptoServiceProvider() on a file to create a SHA1 Checksum of the file?
using (FileStream fs = new FileStream(#"C:\file\location", FileMode.Open))
using (BufferedStream bs = new BufferedStream(fs))
{
using (SHA1Managed sha1 = new SHA1Managed())
{
byte[] hash = sha1.ComputeHash(bs);
StringBuilder formatted = new StringBuilder(2 * hash.Length);
foreach (byte b in hash)
{
formatted.AppendFormat("{0:X2}", b);
}
}
}
formatted contains the string representation of the SHA-1 hash. Also, by using a FileStream instead of a byte buffer, ComputeHash computes the hash in chunks, so you don't have to load the entire file in one go, which is helpful for large files.
With the ComputeHash method. See here:
ComputeHash
Example snippet:
using(var cryptoProvider = new SHA1CryptoServiceProvider())
{
string hash = BitConverter
.ToString(cryptoProvider.ComputeHash(buffer));
//do something with hash
}
Where buffer is the contents of your file.
If you are already reading the file as a stream, then the following technique calculates the hash as you read it. The only caveat is that you need to consume the whole stream.
class Program
{
static void Main(string[] args)
{
String sourceFileName = "C:\\test.txt";
Byte[] shaHash;
//Use Sha1Managed if you really want sha1
using (var shaForStream = new SHA256Managed())
using (Stream sourceFileStream = File.Open(sourceFileName, FileMode.Open))
using (Stream sourceStream = new CryptoStream(sourceFileStream, shaForStream, CryptoStreamMode.Read))
{
//Do something with the sourceStream
//NOTE You need to read all the bytes, otherwise you'll get an exception ({"Hash must be finalized before the hash value is retrieved."})
while(sourceStream.ReadByte() != -1);
shaHash = shaForStream.Hash;
}
Console.WriteLine(Convert.ToBase64String(shaHash));
}
}
Also you can try:
FileStream fop = File.OpenRead(#"C:\test.bin");
string chksum = BitConverter.ToString(System.Security.Cryptography.SHA1.Create().ComputeHash(fop));
Related
When I try to do the following with .zip folder witch contains some videos I get out of memory exeption.
Byte[] bytes = File.ReadAllBytes(#"C:\folderWithVideos.zip");
String base64File= Convert.ToBase64String(bytes);//<----- out of memory exception
How to handle this exception properly? I mean without try-catch, I have tried something like:
String base64File;
if (bytes.Length <= System.Int32.MaxValue)
base64File = Convert.ToBase64String(bytes);
But it didn't helped, but bytes.Length <= 255 did helped, but I'm not sure that 255 is the right number.
Based on the code shown in the blog the following code works.
// using System.Security.Cryptography
private void ConvertLargeFile()
{
//encode
var filein = #"C:\Users\test\Desktop\my.zip";
var fileout = #"C:\Users\test\Desktop\Base64Zip";
using (FileStream fs = File.Open(fileout, FileMode.Create))
using (var cs = new CryptoStream(fs, new ToBase64Transform(),
CryptoStreamMode.Write))
using (var fi = File.Open(filein, FileMode.Open))
{
fi.CopyTo(cs);
}
// the zip file is now stored in base64zip
// and decode
using (FileStream f64 = File.Open(fileout, FileMode.Open))
using (var cs = new CryptoStream(f64, new FromBase64Transform(),
CryptoStreamMode.Read))
using (var fo = File.Open(filein + ".orig", FileMode.Create))
{
cs.CopyTo(fo);
}
// the original file is in my.zip.orig
// use the commandlinetool
// fc my.zip my.zip.orig
// to verify that the start file and the encoded and decoded file
// are the same
}
he code uses standard classes found in System.Security.Cryptography namespace and uses a CryptoStream and the FromBase64Transform and its counterpart ToBase64Transform
One way or another, all digital data is stored in 0 and 1. That's the principle of binary data, I guess.
Is there a method or package that can show you the binary code of a file/single-exe-program of how it is actually being stored in the 0/1 format??
I would see it like:
- import a certain, random file
- convert it to it's 0/1 format
- store the the 1/0-data in a txt (streamwriter/binarywriter)
if yes, is this available in any .NET language (pref: c#)?
Essentially you just need to break this into two steps:
Convert a file into bytes
Convert a byte into a binary string
The first step is easy:
var fileBytes = File.ReadAllBytes(someFileName);
The second step is less straightforward, but still pretty easy:
var byteString = string.Concat(fileBytes.Select(x => Convert.ToString(x, 2).PadLeft(8, '0')))
The idea here is that you select each byte individually, converting each one to a binary string (pad left so each one is 8 characters, since many bytes have leading zeroes), and concatenate all of those into a single string. (Courtesy in part of #xanatos' comment below.)
I think this is something what you are looking for:
byte [] contents = File.ReadAllBytes(filePath);
StringBuilder builder = new StringBuilder();
for(int i = 0; i<contents .Length; i++)
{
builder.Append( Convert.ToString(contents[i], 2).PadLeft(8, '0') );
}
Now, you can for example write builder contents to a text file.
this will stream the conversion, useful if you have huge file.
using System;
using System.IO;
using System.Linq;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
var buffer = new byte[1024];
int pos = 0;
using (var fileIn = new FileStream(#"c:\test.txt", FileMode.Open, FileAccess.Read))
using (var fileOut = new FileStream(#"c:\test.txt.binary", FileMode.Create, FileAccess.Write))
while((pos = fileIn.Read(buffer,0,buffer.Length)) > 0)
foreach (var value in buffer.Take(pos).Select(x => Convert.ToString(x, 2).PadLeft(8, '0')))
fileOut.Write(value.Select(x => (byte)x).ToArray(), 0, 8);
}
}
}
You can open the file in binary mode. Didn't test it but it should work :
BitArray GetBits(string fuleSrc)
{
byte[] bytesFile;
using (FileStream file = new FileStream(fuleSrc, FileMode.Open, FileAccess.Read))
{
bytesFile = new byte[file.Length];
file.Read(bytes, 0, (int)file.Length);
}
return new BitArray(bytesFile);
}
A solution using FileStream, StreamWriter, StringBuilder and Convert
static void Main(string[] args)
{
StringBuilder sb = new StringBuilder();
using (FileStream fs = new FileStream(InputFILEPATH, FileMode.Open))
{
while (fs.Position != fs.Length)
{
sb.Append(Convert.ToString(fs.ReadByte(),2));
}
}
using (StreamWriter stw = new StreamWriter(File.Open(OutputFILEPATH,FileMode.OpenOrCreate)))
{
stw.WriteLine(sb.ToString());
}
Console.ReadKey();
}
Using the following code I always get the same hash regardless of the input. Any ideas why that might be?
private static SHA256 sha256;
internal static byte[] HashForCDCR(this string value)
{
byte[] hash;
using (var myStream = new System.IO.MemoryStream())
{
using (var sw = new System.IO.StreamWriter(myStream))
{
sw.Write(value);
hash = sha256.ComputeHash(myStream);
}
}
return hash;
}
You are computing hash of empty portion of the stream (the one immediately after content you wrote with sw.Write) so it always the same.
Cheap fix: sw.Flush();myStream.Position = 0;. Better fix is to finish writing and create new read only stream for encryption based on original stream:
using (var myStream = new System.IO.MemoryStream())
{
using (var sw = new System.IO.StreamWriter(myStream))
{
sw.Write(value);
}
using (var readonlyStream = new MemoryStream(myStream.ToArray(), writable:false)
{
hash = sha256.ComputeHash(readonlyStream);
}
}
You may need to flush your stream. For optimal performance StreamWriter doesn't write to stream immediately . It waits for its internal buffer to fill. Flushing the writer immediately flush the content of the internal buffer to underline stream.
sw.Write(value);
sw.Flush();
myStream.Position = 0;
hash = sha256.ComputeHash(myStream);
I will probably use the solution that Alexei Levenkov called a "cheap fix". However, I did come across one other way to make it work, which I will post for future readers:
var encoding = new System.Text.UTF8Encoding();
var bytes = encoding.GetBytes(value);
var hash = sha256.ComputeHash(bytes);
return hash;
Jacob
What is the best solution in C# for computing an "on the fly" md5 like hash of a stream of unknown length? Specifically, I want to compute a hash from data received over the network. I know I am done receiving data when the sender terminates the connection, so I don't know the length in advance.
[EDIT] - Right now I am using md5 and am doing a second pass over the data after it's been saved and written to disk. I'd rather hash it in place as it comes in from the network.
MD5, like other hash functions, does not require two passes.
To start:
HashAlgorithm hasher = ..;
hasher.Initialize();
As each block of data arrives:
byte[] buffer = ..;
int bytesReceived = ..;
hasher.TransformBlock(buffer, 0, bytesReceived, null, 0);
To finish and retrieve the hash:
hasher.TransformFinalBlock(new byte[0], 0, 0);
byte[] hash = hasher.Hash;
This pattern works for any type derived from HashAlgorithm, including MD5CryptoServiceProvider and SHA1Managed.
HashAlgorithm also defines a method ComputeHash which takes a Stream object; however, this method will block the thread until the stream is consumed. Using the TransformBlock approach allows an "asynchronous hash" that is computed as data arrives without using up a thread.
Further to #peter-mourfield 's answer, here is the code that uses ComputeHash():
private static string CalculateMd5(string filePathName) {
using (var stream = File.OpenRead(filePathName))
using (var md5 = MD5.Create()) {
var hash = md5.ComputeHash(stream);
var base64String = Convert.ToBase64String(hash);
return base64String;
}
}
Since both the stream as well as MD5 implement IDisposible, you need to use using(...){...}
The method in the code example returns the same string that is used for the MD5 checksum in Azure Blob Storage.
The System.Security.Cryptography.MD5 class contains a ComputeHash method that takes either a byte[] or Stream. Check out the documentation.
This seems like a perfect use case for CryptoStream (docs).
I've used CryptoStream for processing unknown-length streams of database results that need to be gzipped and then transferred across the network along with a hash of the compressed file. Inserting a CryptoStream between the compressor and the file writer allows you to compute the hash on the fly so that it's ready as soon as the file is written.
The basic approach looks like this:
var hasher = MD5.Create();
using (FileStream outFile = File.Create(filePath))
using (CryptoStream crypto = new CryptoStream(outFile, hasher, CryptoStreamMode.Write))
using (GZipStream compress = new GZipStream(crypto, CompressionMode.Compress))
using (StreamWriter writer = new StreamWriter(compress))
{
foreach (string line in GetLines())
writer.WriteLine(line);
}
// at this point the streams are closed so the hash is ready
string hash = BitConverter.ToString(hasher.Hash).Replace("-", "").ToLowerInvariant();
Necromancing.
Two possibilitites in C# .NET Core:
private static System.Security.Cryptography.HashAlgorithm GetHashAlgorithm(System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.MD5.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA1.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA256.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA384.Create();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA512.Create();
throw new System.Security.Cryptography.CryptographicException($"Unknown hash algorithm \"{hashAlgorithmName.Name}\".");
}
protected override byte[] HashData(System.IO.Stream data,
System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
using (System.Security.Cryptography.HashAlgorithm hashAlgorithm1 =
GetHashAlgorithm(hashAlgorithm))
return hashAlgorithm1.ComputeHash(data);
}
or with BouncyCastle:
private static Org.BouncyCastle.Crypto.IDigest GetBouncyAlgorithm(
System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
return new Org.BouncyCastle.Crypto.Digests.MD5Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
return new Org.BouncyCastle.Crypto.Digests.Sha1Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
return new Org.BouncyCastle.Crypto.Digests.Sha256Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
return new Org.BouncyCastle.Crypto.Digests.Sha384Digest();
if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
return new Org.BouncyCastle.Crypto.Digests.Sha512Digest();
throw new System.Security.Cryptography.CryptographicException(
$"Unknown hash algorithm \"{hashAlgorithmName.Name}\"."
);
} // End Function GetBouncyAlgorithm
protected override byte[] HashData(System.IO.Stream data,
System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
Org.BouncyCastle.Crypto.IDigest digest = GetBouncyAlgorithm(hashAlgorithm);
byte[] buffer = new byte[4096];
int cbSize;
while ((cbSize = data.Read(buffer, 0, buffer.Length)) > 0)
digest.BlockUpdate(buffer, 0, cbSize);
byte[] hash = new byte[digest.GetDigestSize()];
digest.DoFinal(hash, 0);
return hash;
}
Another option could be to use the System.Security.Cryptography.IncrementalHash class instead.
byte[] DataBrick;
var IncMD5 = IncrementalHash.CreateHash(HashAlgorithmName.MD5);
then you can: accumulate data in the hasher
IncMD5.AppendData(DataBrick,0,DataBrick.Length);
,check the hash value for the data accumulated so far
byte[] hash = IncMD5.GetCurrentHash();
bytesReceived = netStream.Read(DataBrick,0,DataBrick.Length);
IncMD5.AppendData(DataBrick,0,bytesReceived);
,or stop and reset to start accumulating a new hash value
byte[] hash = IncMD5.GetHashAndReset();
Note: it implements iDisposable
IncMD5.Dispose(); // when done, or using(IncMD5){..} if that makes more sense in your scope
Good morning all,
I'm working on an MD5 file integrity check tool in C#.
How long should it take for a file to be given an MD5 checksum value?
For example, if I try to get a 2gb .mpg file, it is taking around 5 mins+ each time.
This seems overly long.
Am I just being impatient?
Below is the code I'm running
public string getHash(String #fileLocation)
{
FileStream fs = new FileStream(#fileLocation, FileMode.Open);
HashAlgorithm alg = new HMACMD5();
byte[] hashValue = alg.ComputeHash(fs);
string md5Result = "";
foreach (byte x in hashValue)
{
md5Result += x;
}
fs.Close();
return md5Result;
}
Any suggestions will be appreciated.
Regards
See this on how to calculate file hash value in a most efficient way. You basically have to wrap FileStream into a BufferedStream and than feed that into HMACMD5.ComputeHash(Stream) overload:
HashAlgorithm hmacMd5 = new HMACMD5();
byte[] hash;
using(Stream fileStream = new FileStream(fileLocation, FileMode.Open))
using(Stream bufferedStream = new BufferedStream(fileStream, 1200000))
hash = hmacMd5.ComputeHash(bufferedStream);