I am developing an application and I would like to be able to search the whole drive for a regular expression. I would prefer to do this in c# but I can call other languages. Is there any easy way to just seek through all the binary data on a drive from begining to end?
Here's an implementation of grep in C#
http://dotnet.jku.at/applications/Grep/Src.aspx
You can modify to follow subdirectories -- it works off of an array of filenames.
AFAIK there is no simple way to do this on raw binary data (You would need direct disk control).
If file-basis is enough enumerating all files, opening them for binary shared reading (catch the exceptions for the ones that are system protected) and then looking for the data should be straightforward. However this will be quite slow as enumerating and opening all files will take some time.
I don't think C# can read all files / data for the drive the OS is on, since the OS locks some files.
You could use the System.IO namespace to enumerate all files, and then scan them individually byte by byte, this obviously would take a long time.
Do you really want to do this ? How are you going to search:
.doc
.xls
.pdf
.html
etc.? Each file type will represent the string you're searching for in different ways.
This article shows how to read data directly from the disk. Everything they do from C++ could be done from C# using PInvoke.
Related
I'm in a situation where I'd like to, using C#, look at .iso files that are in a directory and determine if they are indeed video discs (DVD/BD or similar).
I don't need to actually distinguish the type, just a blanket "yes this is a video disc". Is there a way to do this?
the ISO file is actually a CD Image in file format. The easiest way to determine what is on it is to mount it with a Virtual CD program. Or you can look at the file contents.
Here is the Specifications for ISO files
http://users.telenet.be/it3.consultants.bvba/handouts/ISO9960.html
After you are able to determine what information is on the disk then you can determine if there is video information on it by finding out what the contents of those files are.
That is a much more daunting task then just determining the file structure.
This specification file will only define ISO files. Other cd formats will need to be read using their own Specifications...
You can determine if the file is of type ISO using the header data
Here is a Stack Question explaining in a little more detail.
Using .NET, how can you find the mime type of a file based on the file signature not the extension
EDIT
Looking into the Mime type thing a little more reveals that Microsoft will have to have a registered mime type for that header data. It may not know that it is an ISO and may tell you application/octet-stream If this is the case then you can instead use your own judgement with the same first 256 bytes. Determine some things that tell you that it is an ISO file that you can handle. Usually you can tell what type and version a file is with the first 20 bytes or so.
I did some searching around for a library that you could use to read/write ISO files. You just need the read part obviously and this project is something you could probably use http://discutils.codeplex.com/
As another mentioned, an ISO file contains a file system. The easiest way to read it is to mount it as a virtual drive, using any one of a number of utilities. Once you've mounted it as a drive, then you can determine that it likely contains a movie by inspecting the file system (i.e. using Directory.GetFiles and similar methods in C#).
If you want to read the file's contents directly (without mounting it), I'm not sure what to tell you. I've heard that 7-zip has an API that will let you read the files. You might also check out DiscUtils, which claims to be able to read ISO files.
Once you can read the contents of the file system, see the "Filesystem" section of http://en.wikipedia.org/wiki/DVD-Video. That will tell you what files and directories you should expect to see in the ISO of a DVD movie.
Note that the files' existence is an indication that the image probably contains a DVD movie. There's no way to tell for sure without examining the files' contents individually. Tracking down the specifications for the individual file types might be a more difficult task.
try using IMAPIv2 to interrogate the iso.
This link doesnt do that.. but it should get you started in the right direction.
How to create optical ISO using IMAPIv2
All,
I'm making a training kit that has content given to use with 2 VOB files that I need the software to automatically merge to 1. We'll be getting upto 10-15 vob files from this vender and our requirements are to move to a single file.
Is merging these files as easy as opening byte streams and combining them?
Thanks!
If the specifications of the files match it should be possible to use the header from the first file and copy the remaining files minus their header into one file. But the specifications needs to match exactly on everything from encoding type and parameters to number of audio channels.
If so, then all you need to do is read all the files and skip the first xxx bytes of every file except the first one.
It won't work if the VOB-files are encrypted (DVD encryption).
Note: This is a job specialized tools do well. They are optimized and (more or less) bug free. So if you can, use them (i.e. from the command line).
No, it is not simple merging. Otherwise old DOS command >type 1.VOB, 2.VOB > Final.VOB would have done the job.
Unless it is for some learning, just use any VOB merging tool to merge these two.
A lot of this is probably going to depend on if the VOB files have the same resolution and bit rate, as well ensuring a lot of other encoding parameters are the same. If they are using the exact same encoding parameters, simply doing a concatenation of the files will probably work. My experience with DVDs shows that files from the DVD work fine when this is done. However, my first guess is that this wouldn't work if there was any format differences between the files.
I need to know if my file is audio file: mp3, wav, etc...
How to do this?
Well, the most robust way would be to write a parser for the file types you want to detect and then just try – if there are no errors, it's obviously of the type you tried. This is an expensive approach, however, but it would ensure that you can successfully load the file as well since it will also check the rest of the file for semantic soundness.
A much less expensive variant would be to look for “magic” bytes – signatures at the start or known offsets of the file. For example, if a file starts with an ID3 tag you can be reasonably sure it's an MP3 file. If a file starts with RIFF¼↕☻ WAVEfmt, then it's a WAV file. However, such detection cannot guarantee you that the file is really of that type – it could just be the signature and following that garbage.
While you can use the extension to make a reasonable guess as to what the file is it's not guaranteed to work 100% of the time. If you are targeting Windows then it will work 99.9% of the time as that's how Windows keeps track of what file is what type.
If you are getting your files from non-Windows sources the only sure way is to open the file and look for a specific string or set of bytes which will unambiguously identify it. For example, you could look for the ID3 tags in an mp3 file:
The ID3v1 tag occupies 128 bytes, beginning with the string TAG.
or
ID3v2 tags are of variable size, and usually occur at the start of the file, to aid streaming media.
How far you go depends on how robust you want your solution to be, and does rely on there being a header or pattern that's always present.
Doing it this way can help guard against malicious content where someone posts a piece of malware as a mp3 file (say) and hopes that it will just be run by a program prone to some exploit (a buffer overrun perhaps).
You can use the file extension to figure it out:
using System.IO;
class Program
{
static void Main()
{
string filepath = #"C:\Users\Sam\Documents\Test.txt";
string extension = Path.GetExtension(filepath);
if (extension == ".mp3")
{
Console.WriteLine(extension);
}
}
}
The file extension is the first point of call for the OS to figure out what file type it's dealing with, if you really want to know the file type 100% the only way to do it is read into the file. But this comes with a catch, image files are easy as they include headers in a pretty easy to read format, however it can get a little more complex with a completely variable file type.
You could check out this post on an old post for a bit of help. Here is a post about finding just media file types.
Ultimately it depends on why your trying to do this.
Path.GetExtension(PathToFile)
See this post. You end up passing the first (up to) 256 bytes of data from the file to FindMimeFromData (part of the Urlmon.dll).
I am trying to split a large AVI 2.0 (OpenDML format) file in smaller parts (under 1GB in my case) in order to be able to open the parts with VFW (avifil32.dll).
What is the best way to achieve this splitting (preferably in C#)?
One of the options is to copy it frame by frame. I found some examples on the net, which do the same. But most of these use VFW which can't read files above 2GB in general and AVI 2.0 files above 1GB because of the max RIFF part size of 1GB.
I would need DirectShow instead of VFW. I am pretty sure that I would also mess up the audio sync if I try to manually copy frames.
I am looking for something similar to what VirtualDub does with "direct stream copy" that doesn't affect the current compression, just splits the file and creates proper AVI indexes.
Avi files can be encoded in many different ways, depending on the codec used. Avi is a wrapper file, not an encoding method. This means there isn't really an easy generic way to split avi files using C#.
To do it in code from scratch would be a major undertaking. That said, you can cheat by using mencoder and calling it from c# - not ideal, but far easier and more reliable than trying to re-invent the wheel. Alternatively, there are a number of ffmpeg c# wrappers that will give you access the ffmpeg tools (but I haven't found one that isn't buggy as hell)
What are you trying to do, exactly? Why do you need avifil32.dll and how are you using it? If you are just trying to play a very large avi file, there are alternatives. Try aforge.net, for example.
mencoder can split files for you. Another option is ffmpeg
I have a table with a binary column which stores files of a number of different possible filetypes (PDF, BMP, JPEG, WAV, MP3, DOC, MPEG, AVI etc.), but no columns that store either the name or the type of the original file. Is there any easy way for me to process these rows and determine the type of each file stored in the binary column? Preferably it would be a utility that only reads the file headers, so that I don't have to fully extract each file to determine its type.
Clarification: I know that the approach here involves reading just the beginning of each file. I'm looking for a good resource (aka links) that can do this for me without too much fuss. Thanks.
Also, just C#/.NET on Windows, please. I'm not using Linux and can't use Cygwin (doesn't work on Windows CE, among other reasons).
you can use these tools to find the file format.
File Analyser
http://www.softpedia.com/get/Programming/Other-Programming-Files/File-Analyzer.shtml
What Format
http://www.jozy.nl/whatfmt.html
PE file format analyser
http://peid.has.it/
This website may be helpful for you.
http://mark0.net/onlinetrid.aspx
Note:
i have included the download links to make sure that you are getting the right tool name and information.
please verify the source before you download them.
i have used a tool in the past i think it is File Analyser, which will tell you the closest match.
happy tooling.
This is not a complete answer, but a place to start would be a "magic numbers" library. This examines the first few bytes of a file to determine a "magic number", which is compared against a known list of them. This is (at least part) of how the file command on Linux systems works.
Someone else asked a similar question and posted the code used to do exactly this. You should be able to take what is posted here, and slightly modify it so that it pulls from your database.
https://stackoverflow.com/questions/58510
In addition to that, it looks like someone has written a library based off of magic numbers to do this, however, it looks like the site requires registration, and some form of alternate access in order to download this lirbary. The documentation is avaliable for free without registration, that may be helpful.
http://software.topcoder.com/catalog/c_component.jsp?comp=13249160&ver=2
The easiest way I know is to use file command that it is also available in Windows with Cygwin .
A lot of filetypes have well defined headers that begin the file. You could check the first few bytes to check to see how the file begins.
Easiest way to do this would be through access to a *nix (or cygwin) system that has the 'file' command:
$ file visitors.*
visitors.html: HTML document text
visitors.png: PNG image data, 5360 x 2819, 8-bit colormap, non-interlaced
You could write a C# application that piped the first X bytes of each binary column to the file command (using - as the file name)
You need to use some p/invoke interop code to call the SHGetFileInfo method from the Win32 API. This article may also help.