I was wondering from a programmers perspective preferably in C# or Java, how hard is it to manipulate a folder within the operating system to do what you want, for instance to act like an FTP?
image http://www.jacks-log.de/wp-content/uploads/2011/02/Dropbox-Wurmloch-1.png
Since I heard of dropbox a few years ago, I've always been wondering whats involved in doing this?
Talking about Dropbox in particular, they don't do anything tricky - just a regular folder being monitored using the mechanisms provided by Windows user-mode API.
It is possible to do much more powerful and interesting things using kernel-mode drivers, though.
First thing you can do is create a virtual disk and map it to the folder. This way your code (either kernel-mode or user-mode with help of our products) will be able to handle all OS requests coming for files in this folder. The folder won't exist on the disk but will be virtual. You can pick data from the remote server on the fly, write them to the remote server etc. In particular, you can map a remote SFTP server to the folder (with FTP things are a bit more complicated as FTP doesn't support partial uploads). Some cloud providers do exactly what I described - they offer a virtual disk or a virtual folder which is a "gate" to their cloud service, and when you read or write data from such virtual folder or disk, data goes from/to the cloud server.
Next thing is that with help of the filesystem filter driver you can have a real folder but provide file contents dynamically. This is how some folder encryptors do (they encrypt file data when writing it to the real disk and without this encryptor active you will read just encrypted "junk" from files in this folder).
Now, with filesystem filter driver you can control who and how can access the folder, i.e. you get fine-grain control over access to folder contents (combine this with encryption and you get a handy data protection mechanism).
Related
I understand this folder App_Data is normally for database files etc but I want this now for images, the idea being users upload images into this folder and they can be accessed from the website, I basically want App_Data to be used/thought of as a normal folder now, anyone know how to do this? Is it just permission settings or can this folder not be used like a normal folder. Thanks in advance :)
ApplicationData is a folder for Application Data. What kind of data you store there is up to you. Note that there are 3 on a Windows:
ApplicationData
CommonApplicationData
LocalApplicationData
Generally data in there it should be data specific to this user - except for CommonApplicationData, of course. Being shared across users, is what the "Common" Prefixed Folders are there for.
However the rest of the question makes no sense. You want the user to manually put stuff there, so a WebSite can upload it? You also seem to think it is somehow not a "normal" folder?
WebSites do not have random access to the file System. So it would really just be annoying for the user to navigate there. And if there is another programm in the loop, you have not told us of it.
And the folder is quite normal. The OS stores a path to it wich can be changed (and the file moved Automagically), but beyond that it is as normal as can be. The unknown position is why you should always retreive the real values from the OS with https://learn.microsoft.com/en-us/dotnet/api/system.environment.specialfolder
Edit:
Based on your comment, I understand now. You are writing a Website. And you wonder why the server has no access to the AppData Folder. Of course only now I noticed MVC properly.
WebServers are uniquely vulnerable to hacking. Online 24/7, a few well known frameworks and widespread reachability as a core goal. As a result they generally run under the most restrictive userrights possible.
Read access to the servers programm and the Instances content directory - any more can not be expected and should never be granted. Maybe write access in a subfolder of content for Temp files - but there are better solutions, that involve Databases and HTTP Handlers.
Solution:
If you want your images to be avalible, put them into a subfolder of the Content directory for this instance. However you really should be considered Database Storage with HTTP Handlers: https://www.red-gate.com/simple-talk/sql/learn-sql-server/an-introduction-to-sql-server-filestream/ Some even go as far as having a seperate, dedicated Webserver just for Images. But I doubt you are on that scale yet.
I'm trying to create software that will add a computer to an Active Directory domain. One criteria I need to meet is the machine must be added to the proper OU. In order to do this I have a set list of site locations with addresses (this is how we determine OU). This list is currently in the form of an ACCDB file, and I want to include this within the application as the Access list will not be changed.
Everything I see wants the DB file to be connected to in a different location such as server or on the local machine. My preference is to use the DB file as a reference or something inside the program's .exe file itself. I may be missing something horribly obvious, but it's been messing with me for a couple days so I'm reaching out for help.
To clarify, this software MUST be self contained (no installer). It must also be able to determine the proper OU to join to the domain (no access to shares until the PC joins the domain). It must also be user-friendly enough to avoid mistakes, meaning I want to avoid copying distributing multiple files that must go to a correct location. This is why I want to embed the ACCDB file into the application for on the fly use.
Things get much easier because this is static data. You don't have to worry about persisting this data, reclaiming changes into your program, or users accidentally deleting something, etc. You will be able to just use an embedded resource in your application. In the link, follow the examples using the image file. Text file examples will corrupt your database.
However, there is still a trick to doing this. The problem is the Access engine included with Windows will not be able to open the database as a resource, and so you will need to save this file to the local hard drive. The good news is its not as bad as it seems, because your program can do this as needed, and make sure it's right, rather than asking the user to put a file in a specific place.
As for where to put the file when you extract it... the safest and best place is the Application Data folder. You can easily get the path for this folder by checking the results of this call:
Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData)
This will give you a path where standard privilege users do have write access, no matter which user.
In summary:
Embed the ACCDB as a resource.
When your program runs, get the Application Data path using the Environment object.
Open a FileStream for writing to a file based on the path from #2
Open the embedded resource as a Stream object in C#
Copy the stream from #4 to the stream from #3. Make sure to use a mechanism intended for binary data, rather than text.
Close/Dispose your streams, so no locks remain on the file. This is as simple as putting using blocks in the right places.
Open a normal Access connection to the file you just made, using any of the thousands of tutorials or examples available on the web as a guide.
In this way, you only need to distribute the final .exe file, and users won't need any special permissions to use the database. You don't have to worry if a user deletes your file; it's still embedded in the application, which will recreate it if needed every time it starts up.
The downside is a clever user may be able to manipulate the database to end up in an undesirable OU. If you need to worry about this, you should consider having the program check a web service, rather than using embedded data. Anything you embed can ultimately be altered by an end user. A web service is also nice because you can update your mapping data as your organization evolves, without needing to rebuild or redistribute the program.
It may seem like this question has been asked a thousand times before, but I can't find an answer that actually points to a path that is common AND writable for all users.
In any case - in my application the users can download some fairly large data files that need to be accessible to the application after download. The application is used in multi user enviroments, so I'm not interested in storing a copy of the downloaded files for each user.
So far I've been using CommonApplicationData (C:\ProgramData\[Vendor]\[Application] created using Wix installer), but I've recently learned that this is meant as readonly storage and several users have experienced issues with this because CommonApplicationData is write protected.
I've been checking different options and the only viable one I have found so far is the %public% folder (https://blogs.msdn.microsoft.com/cjacks/2008/02/05/where-should-i-write-program-data-instead-of-program-files/).
Is this the best/correct place to put the files or is there another solution? Needs to work in Windows XP/Vista/7/8/10.
CommonApplicationData is indeed the correct place to store this data, unless the users should be able to see them as documents to double-click.
When you create the folder (or the individual files or subfolders within it) you should set the permissions according to your needs.
One alternative approach, to provide better cross-user security, is to install a system service to download and install these files on the users behalf.
I am allowing users to upload files to my server. What possible security threats do I face and how can I eliminate them?
Let's say I am allowing users to upload images to my server either from their system or from net. Now to check even the size of these images I have to store them in my /tmp folder. Isn't it risky? How can I minimize the risk?
Also let's say I am using wget to download the images from the link that the users upload in my form. I first have to save those files in my server to check if they actually are images. Also what if a prankster gives me a URL and I end up downloading an entire website full of malware?
First of all, realize that uploading a file means that the user is giving you a lot of data in various formats, and that the user has full control over that data. That's even a concern for a normal form text field, file uploads are the same and a lot more. The first rule is: Don't trust any of it.
What you get from the user with a file upload:
the file data
a file name
a MIME type
These are the three main components of the file upload, and none of it is trustable.
Do not trust the MIME type in $_FILES['file']['type']. It's an entirely arbitrary, user supplied value.
Don't use the file name for anything important. It's an entirely arbitrary, user supplied value. You cannot trust the file extension or the name in general. Do not save the file to the server's hard disk using something like 'dir/' . $_FILES['file']['name']. If the name is '../../../passwd', you're overwriting files in other directories. Always generate a random name yourself to save the file as. If you want you can store the original file name in a database as meta data.
Never let anybody or anything access the file arbitrarily. For example, if an attacker uploads a malicious.php file to your server and you're storing it in the webroot directory of your site, a user can simply go to example.com/uploads/malicious.php to execute that file and run arbitrary PHP code on your server.
Never store arbitrary uploaded files anywhere publicly, always store them somewhere where only your application has access to them.
Only allow specific processes access to the files. If it's supposed to be an image file, only allow a script that reads images and resizes them to access the file directly. If this script has problems reading the file, it's probably not an image file, flag it and/or discard it. The same goes for other file types. If the file is supposed to be downloadable by other users, create a script that serves the file up for download and does nothing else with it.
If you don't know what file type you're dealing with, detect the MIME type of the file yourself and/or try to let a specific process open the file (e.g. let an image resize process try to resize the supposed image). Be careful here as well, if there's a vulnerability in that process, a maliciously crafted file may exploit it which may lead to security breaches (the most common example of such attacks is Adobe's PDF Reader).
To address your specific questions:
[T]o check even the size of these images I have to store them in my /tmp folder. Isn't it risky?
No. Just storing data in a file in a temp folder is not risky if you're not doing anything with that data. Data is just data, regardless of its contents. It's only risky if you're trying to execute the data or if a program is parsing the data which can be tricked into doing unexpected things by malicious data if the program contains parsing flaws.
Of course, having any sort of malicious data sitting around on the disk is more risky than having no malicious data anywhere. You never know who'll come along and do something with it. So you should validate any uploaded data and discard it as soon as possible if it doesn't pass validation.
What if a prankster gives me a url and I end up downloading an entire website full of malware?
It's up to you what exactly you download. One URL will result at most in one blob of data. If you are parsing that data and are downloading the content of more URLs based on that initial blob that's your problem. Don't do it. But even if you did, well, then you'd have a temp directory full of stuff. Again, this is not dangerous if you're not doing anything dangerous with that stuff.
1 simple scenario will be :
If you use a upload interface where there are no restrictions about the type of files allowed for upload then an attacker can upload a PHP or .NET file with malicious code that can lead to a server compromise.
refer:
http://www.acunetix.com/websitesecurity/upload-forms-threat.htm
Above link discusses the common issues
also refer:
http://php.net/manual/en/features.file-upload.php
Here are some of them:
When a file is uploaded to the server, PHP will set the variable $_FILES[‘uploadedfile’][‘type’] to the mime-type provided by the web browser the client is using. However, a file upload form validation cannot depend on this value only. A malicious user can easily upload files using a script or some other automated application that allows sending of HTTP POST requests, which allow him to send a fake mime-type.
It is almost impossible to compile a list that includes all possible extensions that an attacker can use. E.g. If the code is running in a hosted environment, usually such environments allow a large number of scripting languages, such as Perl, Python, Ruby etc, and the list can be endless.
A malicious user can easily bypass such check by uploading a file called “.htaccess”, which contains a line of code similar to the below: AddType application/x-httpd-php .jpg
There are common rules to avoid general issues with files upload:
Store uploaded files not under your website root folder - so users won't be able to rewrite your application files and directly access uploaded files (for example in /var/uploads while your app is in /var/www).
Store sanitated files names in database and physical files give name of file hash value (this also resolves issue of storing files duplicates - they'll have equal hashes).
To avoid issues with filesystem in case there are too many files in /var/uploads folder, consider to store files in folders tree like that:
file hash = 234wffqwdedqwdcs -> store it in /var/uploads/23/234wffqwdedqwdcs
common rule: /var/uploads/<first 2 hash letters>/<hash>
install nginx if you haven't done its already - it serves static like magic and its 'X-Accel-Redirect' header will allow you to serve files with permissions being checked first by custom script
My company created an application that can send large attachements from one mail recipient to another (because most mailboxes are very limited).
But we were wondering how we can prevent the uploading of warez?
For now all extentions are allowed, but we could restrict the extentions to zip and images.
But if you zip warez you can still upload these.
Are there any tools, methods or something like it to prevent the uploading of warez through our system?
Some more info:
This project is semi-public. It will mostly be used for the communication between customer and company. Therefore an email address of our company is always required (either within the receivers as that of the senders, but you all know how easy it is to manipulate this).
Define what "warez" is first.
I'm pretty sure you're going to have problems with that.
You can probably implement heuristics that figure out that you're sending applications and just ban that, but there's no way you're going to figure out that one application is a pirated copy and another isn't and allow the legal one while ban the pirated one.
If you control the server, and is afraid that people will upload pirated copies of applications onto your server and use it to spread it with, then I'm pretty sure your only option is to check with a lawyer what you're obligated to do.
I think it boils down to that you need a system where copyright owners can inform you of pirated copies being present and that you have a system to remove said content within a time frame. I think that's all that is required.
EDIT
If as you said in your edit, that this is for customers to send stuff to you, then I'd be very careful about the allowed email addresses. Is there anything to stop somebody putting in Distribution Email addresses. e.g. If some naughty person sent a large file to All#YourCompany.com, will it be distributed or will it be blocked
ORIGINAL
If this is an open/public system, then its going to be abused. There are ways to unpack zip files on the fly to check their contents, and even to check the file mimetype headers to perform more restrictions, but it doesn't change the fact that someone might want to legitimately send an AVI file of a presentation, while someone else whats to upload a pirated movie.
If this is for internal use in your company, I'd suggest restricting access in someway (tie the system into your Company LDAP/ADSI system and make users login to the system.
Also putting some file size restrictions in place might be necessary as theres nothing to stop some script kiddie just sending 1Gb Junk Text files around, just to be a nuisance and eating up your bandwidth
You can always just rename, for example, a .rar extension to .jpg and let the downloader know to rename the file to open the "Warez".
There is no way to block it other than to take random samplings, test it your self, and then manually delete whatever it is you don't want.
Short answer: No, you can't.
You could look for filenames from a list, but that will fail (e.g. you might ban "MS Word", but then if someone uplads an innocent "MS Word.doc" you fail. Or if the bad guy renames his exe to "MS W0rd" you fail.
You could look for recognised sequences in the file - that fails as soon as they apply even simple encryption or compression.
You can create user accounts and ban users who misbehave, but this fails because you have to spend a lot of effort policing it and in any case users can just creat multiple accounts using web mail addresses.
My suggestion would be to make this someone else's problem. Get users to upload files to someone elses system (Microsoft SkyDrive, Amazon S3 etc) and then they can worry about the legal side.
What if someone sends a password-protected RAR archive to ensure security of some documents? You can't possibly look inside of it - and you shouldn't - not your business.
For example, we had a couple of times some access right issues with our network. And I needed to install some third-party components on my developer machine. As I was unable to access our repository at the time, I just got the installation package sent to me per email. Now, how can an outsider possibly decide whether a file "SomeCoolComponent.msi" is a warez copy downloaded from the intenet or a 100% legal copy which I have the rights to use?
We once had at university our email account suddenly block all password-protected archives as attachments. You guess what? I didn't stop encrypting them. I stopped using that account.
No - you cannot prevent this with a tool or framework.
You can prevent this by banning / blacklisting users who violate the policy.
Trying to do everything in code isn't always the best idea - sometime a simple "break the rules and you get banned" policy is best.
Assuming you can build decompression support in, you could use this heuristic method to determine whether a given uploaded archive is warez (derived from real world warez distribution methods):
get the filenames contained in the archive
if the archive contains (an .nfo OR a .diz file) and (an exe OR an msi or an archive containing one of the above) block the upload
if the archive contains a series of zip/rars/00X files block the upload
if the file is an exe, check whether it's a SFX and if it's a zip, rar or 7z SFX check the embedded archive
Otherwise just accept the upload, making sure you clearly state in your TOS that the company is not responsible for user uploads.