I am writing unit test for this method. I have tried lot of times but still can not write any code for it. Please suggest me how to unit test it. I am using C# , nunit framework and rhino mock.
Thanks in advance.
public FileUploadJsonResult AjaxUploadProfile(int id, string branchName, string filepath, HttpPostedFileBase file)
{
// TODO: Add your business logic here and/or save the file
string statusCode = "1";
string profilePicture = string.Empty;
string fileExtension = System.IO.Path.GetExtension(file.FileName.ToLower());
string fileName = id + "_" + branchName;
string fileNameWithOriginalExtension = fileName + fileExtension;
string fileNameWithJPGExtension = fileName + ".jpg";
string fileServerPath = this.Server.MapPath("~/LO_ProfilePicture/" + fileNameWithJPGExtension);
string statusMessage = string.Empty;
if (string.IsNullOrEmpty(fileExtension) || !Utility.isCorrectExtension(fileExtension))
{
statusMessage = "Profile picture should be of JPG, BMP, PNG, GIF or JPEG format.";
return new FileUploadJsonResult { Data = new { message = string.Format(statusMessage, fileNameWithOriginalExtension), filename = string.Empty, profilepic = profilePicture, statusCode = "0" } };
}
if (file.ContentLength > PageConstants.PROFILE_PICTURE_FILE_SIZE)
{
statusMessage = "Profile picture size should be less than 2MB";
return new FileUploadJsonResult { Data = new { message = string.Format(statusMessage, fileNameWithOriginalExtension), filename = string.Empty, profilepic = profilePicture, statusCode = "0" } };
}
Utility.SaveThumbnailImage(fileServerPath, file.InputStream, PageConstants.BRANCH_PROFILE_PICTURE_FILE_HEIGTH, PageConstants.BRANCH_PROFILE_PICTURE_FILE_WIDTH);
profilePicture = PageConstants.IMAGE_PATH + "LO_ProfilePicture/" + fileNameWithJPGExtension;
// Return JSON
return new FileUploadJsonResult { Data = new { message = string.Format("Profile Picture is successfully uploaded.", fileNameWithOriginalExtension), filename = fileNameWithJPGExtension, profilepic = profilePicture, statusCode } };
}
Make it do just the essential part. Split anything that has nothing to do with the operation you're trying to handle to other classes. Put those behind interfaces, so you can mock these in your unittests. This way you'll notice you don't have to test anything with file i/o in this class. In the class below I split up the function in the essential part, some file i/o and retrieving of settings. Even these settings have nothing to do with the current method you're trying to test. The method just needs verification on, for example, the extension, but it doesn't matter on how it does this.
Tip: try to avoid static utility classes. Give them their own class. Also avoid external components such as network communication or file i/o.
As I don't have a lot of context and it may not compile. But I would go with something like:
class Controller {
public FileUploadJsonResult AjaxUploadProfile(int id, string branchName, string filepath, HttpPostedFileBase file) {
string fileName = id + "_" + branchName;
string fileExtension = _fileIO.GetExtensionForFile(file);
if (!_extensionManager.IsValidExtension(fileExtension)) {
return CreateAjaxUploadProfileError("Profile picture should be of JPG, BMP, PNG, GIF or JPEG format.");
}
if (file.ContentLength > _settingsManager.GetMaximumFileSize()) {
return CreateAjaxUploadProfileError("Profile picture size should be less than 2MB");
}
string fileNameWithJPGExtension = fileName + ".jpg";
string fileServerPath = _fileIO.GetServerProfilePicture(Server, fileNameWithJPGExtension);
string fileClientPath = _fileIO.GetClientProfilePicture(fileNameWithJPGExtension);
var dimensions = _settingsManager.GetThumbnailDimensions();
_fileIO.SaveThumbnailImage(fileServerPath, file, dimensions.Item1, dimensions.Item2);
// Return JSON
var data = new {
message = "Profile Picture is successfully uploaded.",
filename = fileClientPath,
profilepic = profilePicture,
statusCode = "1"
};
return new FileUploadJsonResult { Data = data };
}
private static CreateAjaxUploadProfileError(string message) {
var data = new {
message = message,
filename = string.Empty,
profilepic = string.Empty,
statusCode = "0"
};
return new FileUploadJsonResult { Data = data };
}
}
class FileIO : IFileIO {
public string GetExtensionForFile(HttpPostedFileBase file) {
return System.IO.Path.GetExtension(filePath.FileName.ToLower());
}
public string GetServerProfilePicture(T server, string file) {
return server.MapPath( "~/LO_ProfilePicture/" + file);
}
public void SaveThumbnailImage(string path, HttpPostedFileBase file, int height, int width) {
Utility.SaveThumbnailImage(path, file.InputStream, height, width); // or even inline
}
public string GetClientProfilePicture(string fileName) {
return _settingsManager.GetClientImagePath() + "LO_ProfilePicture/" + fileNameWithJPGExtension;
}
}
class ExtensionManager : IExtensionManager {
public bool IsValidExtension(string extension) {
return Utility.isCorrectExtension(fileExtension); // or even inline
}
}
class SettingsManager : ISettingsManager {
public Tuple<int, int> GetThumbnailDimensions() {
return Tuple.Create<int, int>(PageConstants.BRANCH_PROFILE_PICTURE_FILE_HEIGTH, PageConstants.BRANCH_PROFILE_PICTURE_FILE_WIDTH);
}
public int GetMaximumFileSize() {
return PageConstants.PROFILE_PICTURE_FILE_SIZE;
}
}
You can look at this function as a combination of multiple functions doing specific work. One function is getting target file path, another is validating extension, another is validating size, another creates thumbnail, etc.
The goal is to breakdown the complex code into small testable functions (units) which you can test independently. So when you put them together you have better confidence that your big function works as expected.
Related
I'm making a simple webserver to serve html, css, js & images (done in c#). I am using HttpListener and I can get the html, javascript and css files to work properly. I am just having trouble with the images. This is what I'm using currently:
if (request.RawUrl.ToLower().Contains(".png") || request.RawUrl.Contains(".ico") || request.RawUrl.ToLower().Contains(".jpg") || request.RawUrl.ToLower().Contains(".jpeg"))
{
string dir = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
string[] img = request.RawUrl.Split('/');
string path = dir + #"\public\imgs\" + img[img.Length - 1];
FileInfo fileInfo = new FileInfo(path);
long numBytes = fileInfo.Length;
FileStream fileStream = new FileStream(path, FileMode.Open, FileAccess.Read);
BinaryReader binaryReader = new BinaryReader(fileStream);
byte[] output = binaryReader.ReadBytes((int)numBytes);
binaryReader.Close();
fileStream.Close();
var temp = System.Text.Encoding.UTF8.GetString(output);
return temp;
}
I am converting the image into a string to return them (it's the way my boss suggested). This is the method where I am handling these requests.
private static string SendResponse(HttpListenerRequest request)
This is my WebServer classes Run() method. The call to SetContentType just goes through the request.RawUrl and determines the content type.
public void Run()
{
ThreadPool.QueueUserWorkItem((o) =>
{
Console.WriteLine("StackLight Web Server is running...");
try
{
while (_listener.IsListening)
{
ThreadPool.QueueUserWorkItem((c) =>
{
var ctx = c as HttpListenerContext;
try
{
// store html content in a byte array
string responderString = _responderMethod(ctx.Request);
// set the content type
ctx.Response.Headers[HttpResponseHeader.ContentType] = SetContentType(ctx.Request.RawUrl);
byte[] buffer = buffer = Encoding.UTF8.GetBytes(responderString);
// this writes the html out from the byte array
ctx.Response.ContentLength64 = buffer.Length;
using(Stream stream = ctx.Response.OutputStream)
{
stream.Write(buffer, 0, buffer.Length);
}
}
catch (Exception ex)
{
ConfigLogger.Instance.LogCritical(LogCategory, ex);
}
}, _listener.GetContext());
}
}
catch (Exception ex)
{
ConfigLogger.Instance.LogCritical(LogCategory, ex);
}
});
}
My html page needs to display an image to the screen, it displays a broken image so far. I know the images directory is correct, I tested that.
This is where I got my code for the webserver: here
I was thinking that maybe I have to change the SendResponse method to not return a string
I figured it out. I created a class to hold the data, content type and the request.RawUrl. Then, where I was passing a string, I changed it to pass the object I created.
So, for my WebServer class, my Run method looks like this:
public void Run()
{
ThreadPool.QueueUserWorkItem((o) =>
{
Console.WriteLine("StackLight Web Server is running...");
try
{
while (_listener.IsListening)
{
ThreadPool.QueueUserWorkItem((c) =>
{
var ctx = c as HttpListenerContext;
try
{
// set the content type
ctx.Response.Headers[HttpResponseHeader.ContentType] = SetContentType(ctx.Request.RawUrl);
WebServerRequestData data = new WebServerRequestData();
// store html content in a byte array
data = _responderMethod(ctx.Request);
string res = "";
if(data.ContentType.Contains("text"))
{
char[] chars = new char[data.Content.Length/sizeof(char)];
System.Buffer.BlockCopy(data.Content, 0, chars, 0, data.Content.Length);
res = new string(chars);
data.Content = Encoding.UTF8.GetBytes(res);
}
// this writes the html out from the byte array
ctx.Response.ContentLength64 = data.Content.Length;
ctx.Response.OutputStream.Write(data.Content, 0, data.Content.Length);
}
catch (Exception ex)
{
ConfigLogger.Instance.LogCritical(LogCategory, ex);
}
finally
{
ctx.Response.OutputStream.Close();
}
}, _listener.GetContext());
}
}
catch (Exception ex)
{
ConfigLogger.Instance.LogCritical(LogCategory, ex);
}
});
}
And my SendResponse method looks like this:
private static WebServerRequestData SendResponse(HttpListenerRequest request)
{
string dir = System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
string[] fileUrl = request.RawUrl.Split('/');
// routes
if (request.RawUrl.Contains("/"))
{
// this is the main page ('/'), all other routes can be accessed from here (including css, js, & images)
if (request.RawUrl.ToLower().Contains(".png") || request.RawUrl.ToLower().Contains(".ico") || request.RawUrl.ToLower().Contains(".jpg") || request.RawUrl.ToLower().Contains(".jpeg"))
{
try
{
string path = dir + Properties.Settings.Default.ImagesPath + fileUrl[fileUrl.Length - 1];
FileInfo fileInfo = new FileInfo(path);
path = dir + #"\public\imgs\" + fileInfo.Name;
byte[] output = File.ReadAllBytes(path);
_data = new WebServerRequestData() {Content = output, ContentType = "image/png", RawUrl = request.RawUrl};
//var temp = System.Text.Encoding.UTF8.GetString(output);
//return Convert.ToBase64String(output);
return _data;
}
catch(Exception ex)
{
ConfigLogger.Instance.LogError(LogCategory, "File could not be read.");
ConfigLogger.Instance.LogCritical(LogCategory, ex);
_errorString = string.Format("<html><head><title>Test</title></head><body>There was an error processing your request:<br />{0}</body></html>", ex.Message);
_byteData = new byte[_errorString.Length * sizeof(char)];
System.Buffer.BlockCopy(_errorString.ToCharArray(), 0, _byteData, 0, _byteData.Length);
_data = new WebServerRequestData() { Content = _byteData, ContentType = "text/html", RawUrl = request.RawUrl };
return _data;
}
}
I'm still cleaning up the code a bit but it now serves the images!
Oh... And here is the object I'm using:
public class WebServerRequestData
{
public string RawUrl { get; set; }
public string ContentType { get; set; }
public byte[] Content { get; set; }
public string RawData { get; set; }
}
Some really bad stuff here:
Empty catch. You'll never find out about many bugs.
Stuffing binary data into a string. Why? There's no encoding that is able to roundtrip binary data.
You're not disposing of ctx. I don't see why you need a manual finally block. Use using.
Untrusted callers can inject arbitrary paths into path. I could request your web.config file by navigating to /img/..\..\web.config (something like that).
Consider factoring out some common expressions into variables. You've got a Copy&Paste error with ToLower. Don't do dirty stuff and you'll have less bugs.
I have an method which is download images via URL and sometimes URL become like this, which contains very long path (i removed some string,it just didnt beacome long question :) : data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBxITEhUSEBAVFhMXGRcXFRgWFRcVGRcYFhcXFhUVGBgYISggGBslGxgaITEhJSkrLi8uGCAzODMsNygtLisBCgoKDg0OFxAQGysfHSUtLS8tLS0tLS0tLSstLS0wLS4tLS0tLS0tLS0tKystNy0uLy0rKy0tLS0rNS0tLS0tLf/AABEIALoBDwMBIgACEQEDEQH/xAAcAAEAAgIDAQAAAAAAAAAAAAAABAUCBgEDBwj/xABLEAABAwMCAgYDCgsHAwUAAAABAAIDBBESBSExQQYTFCJRYXGBkQcjMkJTVJOh0fAVJDNSYpKUscHS4RYXQ1VjcoKEorJEg6TCw//EABkBAQEBAQEBAAAAAAAAAAAAAAABAgMEBf/EACURAQEAAgIABQQDAAAAAAAAAAABAhEDIQQSMWFxQYGh0QUTUf/aAAwDAQACEQMRAD8A9xREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERAREQERaH0h90yGnnfAyB8hYcXnIMGQ4gbG/p2Qb4i8p1v3S5JYsKZjoJLtOd2v2B7zcS224Vezp9WBtjISfGzRf1Yq6Tb2ZF4vN7o1WxhJJNud2j6sFd9HPdJc98UUrLukIsTxsSG/FAHHyTRt6aiIooiIgIiICIiAiIgIiICIiAiIgIiICLQPdM6eu09zIomgyOAeSd7NJc22J8S3jflwWtRe6lOWA2Nz+ixWQ29kReR/wB5suNrSZ+OMdvYrLT/AHVImsAnhmc/mWiMA/8AcP3Jqm3pSLU+jPT6mrJepY2SN5BLRIG96wuQC0ncAX3W2KAiIgIiIC+Zdbqc6uodfjNKfV1hsvpeeQNaXHgASfULr5WgeS5zjxO59JN1YlV2txh1S+9+DedvihRuzDll+sV36i78Yf8A8f8AxauHnYrUgvNa6MGmfKXOqXxRmMZX6okv3dfLLhwsAd3i9sSDC0mgZO4hjp2hjDI976lrGNawxhzr9X+c4gDxezwJPoPS8Na2okfA17Gy0Tne9NbsDGHMBLiRk1rmk+TmnjYU/Seth1LrYaGJ8kgpy8PdHHE4lk0TnRNYxoLhgBxN+4OO6zbpdKJjbgW1GUE346jHsQDx8r2+tZNikc5rY9Qnc5xxaG1zXnibEhlzb4PAcneVuqHR6kNv2V4cRnbstmhx7mHwdm4d7h8McPjKVQ6HKZYwYJQ0PYS50JjIZEQWuyHxyC644ktbudrZVArqqWJ5Y6rrDaxDm1Jc1zXAOa5p2uC0+A34gWIXXBqErnNa2qri5xa0ATm7nOLQAN+JJcB/x81ues6tRR0r6eeEds6ppwELXWlkizDnTD4OJcBYEEY8CFpPRvTTUztiLywWfI5zW5ODYmOkd1bR8J/dsB4lal63UqZqT6mnf1dRUV8cmIdi6e5s6+JNn7bj2KFJq7x/6yu5/wCMf0rX7/8As/7vJTtboGOpoq6CeeSJ73QWqQOtY6MF1g4Eh7LHa3C9uRVl0a1bTYqGojrIgZiXFvvHWueMW9W1ribMs4OJ4cRv4tzXSR3w9G68sa91ZVR5Nza2SoIfgT3SWh5xuOF7IejFf/mMvK/4y/8ARvxeP0/Y3xNrbpTptdN1U0MdTIHRtPvdzjtvcRkcrAHyB5Kh/A+pgginrfU2Yf8A22F7gLkdTe3GXA6pYdW1+boW2uSQY+HEWHO/eGwFyOuXVIopDBT0TJWxho6yXKR0pc3IOFm37wu4AkbA7AbLlusnb8Tp7DicHgG++RIbwDO/tbu778F6cPBc3JjMscer8JeTGdWq3WqWoe4NfWCYC+5aWAOyc3bEHIWFAiIgIiICIiD/2Q==
And then when i debug or when its come to download image, i get this Exception:
var DL = webClient.DownloadData(base64)
The specified path and / or file name is too long. The fully qualified
name must be less than 260 characters and the folder name must be less
than 248 characters.
i did also research which is suggested to add <httpRuntime maxUrlLength="260" /> in webconfig or used different library , but unfortunately it didnt help to solve the problem.
Can anyone please help me or point me into the right direction :)
Thanks in advance.
Controller:
[HttpPost]
public string DownloadImagesFromLinkViaURL(ImagesViewModel model)
{
var RandomName = Guid.NewGuid().ToString("N").Substring(0,12);
using (WebClient webClient = new WebClient())
{
try
{
string base64 = model.ImageURL.Substring(model.ImageURL.IndexOf(',') + 1);
byte[] data = Convert.FromBase64String(base64);
var DL = webClient.DownloadData(base64);
using (MemoryStream mem = new MemoryStream(DL))
{
using (var content = Image.FromStream(mem))
{
var format = ImageFormat.Png.ToString().ToLower();
var PathIMG = "https://SomeName.com/folder/" + RandomName + "." + format;
content.Save(Path.Combine(Server.MapPath(PathIMG)));
ImageStore img = new ImageStore();
img.ProducentVarenr = model.ImageName;
img.ImageOrginalURL = model.ImageURL;
img.ImageRandomName = RandomName;
img.LinktilBillede = PathIMG;
db.ImageStoreList.Add(img);
db.SaveChanges();
}
}
}
catch (ArgumentException)
{
return "content is not image";
}
}
return "saved";
}
ViewModal:
public class ImagesViewModel
{
public int ImageID { get; set; }
public string ImageURL { get; set; }
public string ImageName { get; set; }
public string ImagePath { get; set; }
public string RandomName { get; set; }
}
Data you see is not url or path. It is image data presented as Base64 string. So no any downloading is needed since you have image data already.
If you paste that very long string i.e this base64-to-image converter tool, you see actual image.
With given base64-string you can save it to file with following style:
File.WriteAllBytes(#"c:\yourfile", Convert.FromBase64String(base64));
Here is (non-tested and non-refactored) fixed version of your original method, as requested in answer comments.
[HttpPost]
public string DownloadImagesFromLinkViaURL(ImagesViewModel model)
{
var RandomName = Guid.NewGuid().ToString("N").Substring(0, 12);
var format = ImageFormat.Png.ToString().ToLower();
var PathIMG = "https://SomeName.com/folder/" + RandomName + "." + format;
if (model.ImageURL.StartsWith("data:image"))
{
string base64 = model.ImageURL.Substring(model.ImageURL.IndexOf(',') + 1);
File.WriteAllBytes($#"c:\temp\{RandomName}.jpeg", Convert.FromBase64String(base64));
ImageStore img = new ImageStore();
img.ProducentVarenr = model.ImageName;
img.ImageOrginalURL = model.ImageURL;
img.ImageRandomName = RandomName;
img.LinktilBillede = PathIMG;
db.ImageStoreList.Add(img);
db.SaveChanges();
return "saved";
}
using (WebClient webClient = new WebClient())
{
try
{
byte[] data = Convert.FromBase64String(base64);
var DL = webClient.DownloadData(base64);
using (MemoryStream mem = new MemoryStream(DL))
{
using (var content = Image.FromStream(mem))
{
content.Save(Path.Combine(Server.MapPath(PathIMG)));
ImageStore img = new ImageStore();
img.ProducentVarenr = model.ImageName;
img.ImageOrginalURL = model.ImageURL;
img.ImageRandomName = RandomName;
img.LinktilBillede = PathIMG;
db.ImageStoreList.Add(img);
db.SaveChanges();
}
}
}
catch (ArgumentException)
{
return "content is not image";
}
}
return "saved";
}
I am using C# ASP.NET code and trying to download file on the post request of a form. here is my sample code.
[HttpPost]
public ActionResult PostMethodName(PostModel inputModel)
{
if (ModelState.IsValid)
{
//other code is removed.
//Writing this for the test
//Download Method call
DownloadCertificate("This is the test file to download.");
var statusHtml = RenderViewToString("Status",
new ErrorMsgModel
{
IsSuccess = true,
ErrorDesc = "desc"
});
return Json(new { IsSuccess = true, ErrorDescription =
statusHtml}, JsonRequestBehavior.AllowGet);
}
var statusHtml1 = RenderViewToString("Status",
new ErrorMsgModel
{
IsSuccess = false,
ErrorDesc = "desc"
});
statusHtml1 = statusHtml1.Replace("'", "\\'");
statusHtml1 = statusHtml1.Replace(Environment.NewLine, "");
return Json(new { IsSuccess = false, ErrorDescription = statusHtml1
}, JsonRequestBehavior.AllowGet);
}
Download method which is called from this method.
public ActionResult DownloadCertificate(string content)
{
//Certificate Download
const string fileType = "application/pkcs10";
string fileName = "Certificate" + DateTime.Today.ToString(#"yyyy-MM-dd") + ".csr";
var fileContent = String.IsNullOrEmpty(contrnt) ? "" : contrnt;
byte[] fileContents = Encoding.UTF8.GetBytes(fileContent);
var result = new FileContentResult(fileContents, fileType) { FileDownloadName = fileName };
return result;
}
file download is not working, post functionality is working as desired.
[HttpPost]
public ActionResult DownloadCertificate(PostModel inputModel, string content)
{
if(!ModelState.IsValid){return Json(new {Success=false,//error descr})}
//Certificate Download
const string fileType = "application/pkcs10";
string fileName = "Certificate" + DateTime.Today.ToString(#"yyyy-MM-dd") + ".csr";
var fileContent = String.IsNullOrEmpty(contrnt) ? "" : contrnt;
byte[] fileContents = Encoding.UTF8.GetBytes(fileContent);
var result = new FileContentResult(fileContents, fileType) { FileDownloadName = fileName };
return result;
}
In your previous code you don`t use DownloadCertificate result, you simly execute it.
Your DownloadCertificate method returns a value, but you never use the return value in your PostMethodName method.
Given that you are returning json from that method I would suggest that you return a direct link to the file result in the response. The consuming client can then initiate the download. Something like:
return Json(new { IsSuccess = true, Location = Url.Action("DownloadContent")});
Alternatively you could consider a more restful approach and return a 302 response from the post action:
if (ModelState.IsValid)
{
// you code here
return RedirectToAction("Controller", "DownloadContent", new {content = "myContent"});
}
This may well proceed with the download transparently depending on your client whilst keeping to the Post-Redirect-Get pattern
I enable my users to download files, some times the files are archived or do not exist at the location, I currently display a 404, I created a specific error page for this specific scenario but I am unable to display it because I have to return something in the FileDownload Result and I am unable to do a response redirect. Also I tried to return content with javascript but that is also not compatible with FileDownloadResult type. How do I route the user to the intended error page which I have will render from its own controller/action?
public DownloadFileResult Download(string file)
{
try
{
string loadLististFileName = file;
// get the displayed filename, extract the file name
string fileNamePath = loadLististFileName;
string fileName = Path.GetFileName(fileNamePath);
string dirName = Path.GetDirectoryName(fileNamePath);
string dirPath = dirName.Replace("\\", ",");
string[] dir = dirPath.Split(',');
int dirlength = dir.Length;
string year = dir[dirlength - 3];
string month = dir[dirlength - 2];
string day = dir[dirlength - 1];
var fileData = IOHelper.GetFileData(fileName, dirName);
fileName = IOHelper.GetPrimaryFileName(file);
return new DownloadFileResult(fileName, fileData);
}
catch (FileNotFoundException)
{
//return Content("<script language='javascript' type='text/javascript'>alert('Image not found!');</script>");
Response.Redirect("Exception/FileIndex");
}
}
try to return ActionResult instead, something like this:
public ActionResult DownLoadFile(int Id)
{
var model = new PreviewFileAttachmentViewModel(Id, _attachFileProvider);
if (model.FileExist)
return File(model.AbsFileNamePath, model.ContentType, model.FileName);
else
return RedirectToAction("FileNotFound");
}
using .NET SDK v.1.5.21.0
I'm trying to upload a large file (63Mb) and I'm following the example at:
http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFileDotNet.html
But using a helper instead the hole code and using jQuery File Upload
https://github.com/blueimp/jQuery-File-Upload/blob/master/basic-plus.html
what I have is:
string bucket = "mybucket";
long totalSize = long.Parse(context.Request.Headers["X-File-Size"]),
maxChunkSize = long.Parse(context.Request.Headers["X-File-MaxChunkSize"]),
uploadedBytes = long.Parse(context.Request.Headers["X-File-UloadedBytes"]),
partNumber = uploadedBytes / maxChunkSize + 1,
fileSize = partNumber * inputStream.Length;
bool lastPart = inputStream.Length < maxChunkSize;
// http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFileDotNet.html
if (partNumber == 1) // initialize upload
{
iView.Utilities.Amazon_S3.S3MultipartUpload.InitializePartToCloud(fileName, bucket);
}
try
{
// upload part
iView.Utilities.Amazon_S3.S3MultipartUpload.UploadPartToCloud(fs, fileName, bucket, (int)partNumber, uploadedBytes, maxChunkSize);
if (lastPart)
// wrap it up and go home
iView.Utilities.Amazon_S3.S3MultipartUpload.CompletePartToCloud(fileName, bucket);
}
catch (System.Exception ex)
{
// Huston, we have a problem!
//Console.WriteLine("Exception occurred: {0}", exception.Message);
iView.Utilities.Amazon_S3.S3MultipartUpload.AbortPartToCloud(fileName, bucket);
}
and
public static class S3MultipartUpload
{
private static string accessKey = System.Configuration.ConfigurationManager.AppSettings["AWSAccessKey"];
private static string secretAccessKey = System.Configuration.ConfigurationManager.AppSettings["AWSSecretKey"];
private static AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretAccessKey);
public static InitiateMultipartUploadResponse initResponse;
public static List<UploadPartResponse> uploadResponses;
public static void InitializePartToCloud(string destinationFilename, string destinationBucket)
{
// 1. Initialize.
uploadResponses = new List<UploadPartResponse>();
InitiateMultipartUploadRequest initRequest =
new InitiateMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'));
initResponse = client.InitiateMultipartUpload(initRequest);
}
public static void UploadPartToCloud(Stream fileStream, string destinationFilename, string destinationBucket, int partNumber, long uploadedBytes, long maxChunkedBytes)
{
// 2. Upload Parts.
UploadPartRequest request = new UploadPartRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId)
.WithPartNumber(partNumber)
.WithPartSize(maxChunkedBytes)
.WithFilePosition(uploadedBytes)
.WithInputStream(fileStream) as UploadPartRequest;
uploadResponses.Add(client.UploadPart(request));
}
public static void CompletePartToCloud(string destinationFilename, string destinationBucket)
{
// Step 3: complete.
CompleteMultipartUploadRequest compRequest =
new CompleteMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId)
.WithPartETags(uploadResponses);
CompleteMultipartUploadResponse completeUploadResponse =
client.CompleteMultipartUpload(compRequest);
}
public static void AbortPartToCloud(string destinationFilename, string destinationBucket)
{
// abort.
client.AbortMultipartUpload(new AbortMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId));
}
}
my maxChunckedSize is 6Mb (6 * (1024*1024)) as I have read that the minimum is 5Mb...
why am I getting "Your proposed upload is smaller than the minimum allowed size" exception? What am I doing wrong?
The error is:
<Error>
<Code>EntityTooSmall</Code>
<Message>Your proposed upload is smaller than the minimum allowed size</Message>
<ETag>d41d8cd98f00b204e9800998ecf8427e</ETag>
<MinSizeAllowed>5242880</MinSizeAllowed>
<ProposedSize>0</ProposedSize>
<RequestId>C70E7A23C87CE5FC</RequestId>
<HostId>pmhuMXdRBSaCDxsQTHzucV5eUNcDORvKY0L4ZLMRBz7Ch1DeMh7BtQ6mmfBCLPM2</HostId>
<PartNumber>1</PartNumber>
</Error>
How can I get ProposedSize if I'm passing the stream and stream length?
Here is a working solution for the latest Amazon SDK (as today: v.1.5.37.0)
Amazon S3 Multipart Upload works like:
Initialize the request using client.InitiateMultipartUpload(initRequest)
Send chunks of the file (loop until the end) using client.UploadPart(request)
Complete the request using client.CompleteMultipartUpload(compRequest)
If anything goes wrong, remember to dispose the client and request, as well fire the abort command using client.AbortMultipartUpload(abortMultipartUploadRequest)
I keep the client in Session as we need this for each chunk upload as well, keep an hold of the ETags that are now used to complete the process.
You can see an example and simple way of doing this in Amazon Docs itself, I ended up having a class to do everything, plus, I have integrated with the lovely jQuery File Upload plugin (Handler code below as well).
The S3MultipartUpload is as follow
public class S3MultipartUpload : IDisposable
{
string accessKey = System.Configuration.ConfigurationManager.AppSettings.Get("AWSAccessKey");
string secretAccessKey = System.Configuration.ConfigurationManager.AppSettings.Get("AWSSecretKey");
AmazonS3 client;
public string OriginalFilename { get; set; }
public string DestinationFilename { get; set; }
public string DestinationBucket { get; set; }
public InitiateMultipartUploadResponse initResponse;
public List<PartETag> uploadPartETags;
public string UploadId { get; private set; }
public S3MultipartUpload(string destinationFilename, string destinationBucket)
{
if (client == null)
{
System.Net.WebRequest.DefaultWebProxy = null; // disable proxy to make upload quicker
client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretAccessKey, new AmazonS3Config()
{
RegionEndpoint = Amazon.RegionEndpoint.EUWest1,
CommunicationProtocol = Protocol.HTTP
});
this.OriginalFilename = destinationFilename.TrimStart('/');
this.DestinationFilename = string.Format("{0:yyyy}{0:MM}{0:dd}{0:HH}{0:mm}{0:ss}{0:fffff}_{1}", DateTime.UtcNow, this.OriginalFilename);
this.DestinationBucket = destinationBucket;
this.InitializePartToCloud();
}
}
private void InitializePartToCloud()
{
// 1. Initialize.
uploadPartETags = new List<PartETag>();
InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest();
initRequest.BucketName = this.DestinationBucket;
initRequest.Key = this.DestinationFilename;
// make it public
initRequest.AddHeader("x-amz-acl", "public-read");
initResponse = client.InitiateMultipartUpload(initRequest);
}
public void UploadPartToCloud(Stream fileStream, long uploadedBytes, long maxChunkedBytes)
{
int partNumber = uploadPartETags.Count() + 1; // current part
// 2. Upload Parts.
UploadPartRequest request = new UploadPartRequest();
request.BucketName = this.DestinationBucket;
request.Key = this.DestinationFilename;
request.UploadId = initResponse.UploadId;
request.PartNumber = partNumber;
request.PartSize = fileStream.Length;
//request.FilePosition = uploadedBytes // remove this line?
request.InputStream = fileStream; // as UploadPartRequest;
var up = client.UploadPart(request);
uploadPartETags.Add(new PartETag() { ETag = up.ETag, PartNumber = partNumber });
}
public string CompletePartToCloud()
{
// Step 3: complete.
CompleteMultipartUploadRequest compRequest = new CompleteMultipartUploadRequest();
compRequest.BucketName = this.DestinationBucket;
compRequest.Key = this.DestinationFilename;
compRequest.UploadId = initResponse.UploadId;
compRequest.PartETags = uploadPartETags;
string r = "Something went badly wrong";
using (CompleteMultipartUploadResponse completeUploadResponse = client.CompleteMultipartUpload(compRequest))
r = completeUploadResponse.ResponseXml;
return r;
}
public void AbortPartToCloud()
{
// abort.
client.AbortMultipartUpload(new AbortMultipartUploadRequest()
{
BucketName = this.DestinationBucket,
Key = this.DestinationFilename,
UploadId = initResponse.UploadId
});
}
public void Dispose()
{
if (client != null) client.Dispose();
if (initResponse != null) initResponse.Dispose();
}
}
I use DestinationFilename as the destination file so I can avoid the same name, but I keep the OriginalFilename as I needed later.
Using jQuery File Upload Plugin, all works inside a Generic Handler, and the process is something like this:
// Upload partial file
private void UploadPartialFile(string fileName, HttpContext context, List<FilesStatus> statuses)
{
if (context.Request.Files.Count != 1)
throw new HttpRequestValidationException("Attempt to upload chunked file containing more than one fragment per request");
var inputStream = context.Request.Files[0].InputStream;
string contentRange = context.Request.Headers["Content-Range"]; // "bytes 0-6291455/14130271"
int fileSize = int.Parse(contentRange.Split('/')[1]);,
maxChunkSize = int.Parse(context.Request.Headers["X-Max-Chunk-Size"]),
uploadedBytes = int.Parse(contentRange.Replace("bytes ", "").Split('-')[0]);
iView.Utilities.AWS.S3MultipartUpload s3Upload = null;
try
{
// ######################################################################################
// 1. Initialize Amazon S3 Client
if (uploadedBytes == 0)
{
HttpContext.Current.Session["s3-upload"] = new iView.Utilities.AWS.S3MultipartUpload(fileName, awsBucket);
s3Upload = (iView.Utilities.AWS.S3MultipartUpload)HttpContext.Current.Session["s3-upload"];
string msg = System.String.Format("Upload started: {0} ({1:N0}Mb)", s3Upload.DestinationFilename, (fileSize / 1024));
this.Log(msg);
}
// cast current session object
if (s3Upload == null)
s3Upload = (iView.Utilities.AWS.S3MultipartUpload)HttpContext.Current.Session["s3-upload"];
// ######################################################################################
// 2. Send Chunks
s3Upload.UploadPartToCloud(inputStream, uploadedBytes, maxChunkSize);
// ######################################################################################
// 3. Complete Upload
if (uploadedBytes + maxChunkSize > fileSize)
{
string completeRequest = s3Upload.CompletePartToCloud();
this.Log(completeRequest); // log S3 response
s3Upload.Dispose(); // dispose all objects
HttpContext.Current.Session["s3-upload"] = null; // we don't need this anymore
}
}
catch (System.Exception ex)
{
if (ex.InnerException != null)
while (ex.InnerException != null)
ex = ex.InnerException;
this.Log(string.Format("{0}\n\n{1}", ex.Message, ex.StackTrace)); // log error
s3Upload.AbortPartToCloud(); // abort current upload
s3Upload.Dispose(); // dispose all objects
statuses.Add(new FilesStatus(ex.Message));
return;
}
statuses.Add(new FilesStatus(s3Upload.DestinationFilename, fileSize, ""));
}
Keep in mind that to have a Session object inside a Generic Handler, you need to implement IRequiresSessionState so your handler will look like:
public class UploadHandlerSimple : IHttpHandler, IRequiresSessionState
Inside fileupload.js (under _initXHRData) I have added an extra header called X-Max-Chunk-Size so I can pass this to Amazon and calculate if it's the last part of the uploaded file.
Fell free to comment and make smart edits for everyone to use.
I guess you didn't set the content-length of the part inside the UploadPartToCloud() function.