Save progress to a text file C# - c#

I have a problem here...so that's what I wanna do:
I have a program that saves information about user progress, ex: Calls, Answered Calls... and the user run this program every day and save the iformation to the text file. So the problem is that when the user hit's the Save button it add's a new stat's for that day. But I want those data to be modified if user save's in that day 2 times.
What I wanna do is to create a new file where to save the last time saved, and if the date are not diferent Append to file, else modify existing for that day saves.
What I did so far is:
string input3 = string.Format("{0:yyyy-MM-dd}", DateTime.Now);
StreamWriter t,tw;
if(File.Exists(filename))
{
tw=File.AppendText(filename);
t = new StreamWriter("lasttimesaved.txt");
t.WriteLine(input3);
}
else
{
tw=new StreamWriter(filename);
t = new StreamWriter("lasttimesaved.txt");
t.WriteLine(input3);
}
tw.WriteLine();
tw.Write("Stats for Name ");
tw.Write(input);
tw.Write("_");
tw.WriteLine(input3);
tw.WriteLine();
tw.Write("Total Calls: "); tw.WriteLine(calls);
tw.Write("Total Answered: "); tw.WriteLine(answ);
tw.Close();
the only thing now that I don't know ho to do is how to add above all that a check instance to see if the user allready saved today info and to modify existing data.
it's like:
try
{
using (StreamReader sr = new StreamReader("lasttimesaved.txt"))
{
String line = sr.ReadToEnd();
}
}
catch (Exception e)
{
Console.WriteLine("The file could not be read:");
Console.WriteLine(e.Message);
}
if(String.Compare(input3,line) == 0)
{
// that's where I need to modify the existing data.
}
else
{
// do the code above
}
Can anyone help me to modify curent recorded data without losing previous records.
in text file is:
Stats for Name_2013-11-26
Total Calls: 25
Total Answered: 17
Stats for Name_2013-11-27
Total Calls: 32
Total Answered: 15
Stats for Name_2013-11-28
Total Calls: 27
Total Answered: 13

I would say use XML, it will still be readable and modifiable without code and you have some neat way to modify the file with code.
With XML you can easily query the file to see if the date of today is already mentioned in the file, if so you could edit that node if not you could easily append a node.
To append nodes to an xml file i would look at this link:
C#, XML, adding new nodes

Hope this helps, use it like here:
void main()
{
var uw = new UserInformationWriter(#"C:\temp\stats.txt");
var user = new UserInfomration { Calls = "111", Answered = "110" };
uw.Save(user);
}
Here the class(es):
public class UserInformationWriter
{
public string CentralFile { get; set; }
public UserInformationWriter(string centraFile)
{
CentralFile = centraFile;
}
public void Save(UserInfomration newUserInformation)
{
try
{
var streamReader = new StreamReader(CentralFile);
var sourceInformation = streamReader.ReadToEnd();
streamReader.Close();
var userCollection = (List<UserInfomration>)(sourceInformation.ToUserInfomation());
var checkItem = ShouldModify(userCollection);
if (checkItem.Item1)
userCollection.Remove(checkItem.Item2);
newUserInformation.DateTime = DateTime.Today;
userCollection.Add(newUserInformation);
File.Delete(CentralFile);
foreach (var userInfomration in userCollection)
WriteToFile(userInfomration);
}
catch (Exception) { }
}
private Tuple<bool, UserInfomration> ShouldModify(IEnumerable<UserInfomration> userInfomations)
{
try
{
foreach (var userInfomration in userInfomations)
if (userInfomration.DateTime == DateTime.Today)
return new Tuple<bool, UserInfomration>(true, userInfomration);
}
catch (Exception) { }
return new Tuple<bool, UserInfomration>(false, null);
}
private void WriteToFile(UserInfomration newUserInformation)
{
using (var tw = new StreamWriter(CentralFile, true))
{
tw.WriteLine("*Stats for Name_{0}", newUserInformation.DateTime.ToShortDateString());
tw.WriteLine();
tw.WriteLine("*Total Calls: {0}", newUserInformation.Calls);
tw.WriteLine("*Total Answered: {0}#", newUserInformation.Answered);
tw.WriteLine();
}
}
}
public class UserInfomration
{
public DateTime DateTime { get; set; }
public string Calls { get; set; }
public string Answered { get; set; }
}
public static class StringExtension
{
private const string CallText = "TotalCalls:";
private const string AnsweredText = "TotalAnswered:";
private const string StatsForName = "StatsforName_";
private const char ClassSeperator = '#';
private const char ItemSeperator = '*';
public static IEnumerable<UserInfomration> ToUserInfomation(this string input)
{
var splited = input.RemoveUnneededStuff().Split(ClassSeperator);
splited = splited.Where(x => !string.IsNullOrEmpty(x)).ToArray();
var userInformationResult = new List<UserInfomration>();
foreach (var item in splited)
{
if (string.IsNullOrEmpty(item)) continue;
var splitedInformation = item.Split(ItemSeperator);
splitedInformation = splitedInformation.Where(x => !string.IsNullOrEmpty(x)).ToArray();
var userInformation = new UserInfomration
{
DateTime = ConvertStringToDateTime(splitedInformation[0]),
Calls = splitedInformation[1].Substring(CallText.Length),
Answered = splitedInformation[2].Substring(AnsweredText.Length)
};
userInformationResult.Add(userInformation);
}
return userInformationResult;
}
private static DateTime ConvertStringToDateTime(string input)
{
var date = input.Substring(StatsForName.Length);
return DateTime.ParseExact(date, "dd.MM.yyyy", CultureInfo.InvariantCulture);
}
private static string RemoveUnneededStuff(this string input)
{
input = input.Replace("\n", String.Empty);
input = input.Replace("\r", String.Empty);
input = input.Replace("\t", String.Empty);
return input.Replace(" ", string.Empty);
}
}
Let me know If you need help or I understood you wrong.

Related

Writing and converting Data from csv file to an array to work with the data c#

I try to continue my project for the school.
I was able to import the data from a CSV file in the command line. I am really not entirely sure if I wrote it to the array already. But for the next step I need to go ahead and filter the data from the written class. As first example I try to filter the written data over the date (datum) from the class.
I have really no idea what to do now to work. So in the section:
if (decideTyp == "Z")
{
Console.WriteLine("Decide to filtering date");
var inputWert = Console.ReadLine();
Console.WriteLine("Filtering the date and show the date: ");
//Console.WriteLine($" {eineAktie.Aktienkurs_datum} {eineAktie.Aktien_id} {eineAktie.Name} {eineAktie.Wert}" /*+ eineAktie.Aktienkurs_datum*/);
//Console.WriteLine($" {zweiteAktie.Aktienkurs_datum} {zweiteAktie.Aktien_id} {zweiteAktie.Name} {zweiteAktie.Wert}" /*+ eineAktie.Aktienkurs_datum*/);
From my code I try to give out all the dates from my CSV file.
using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
namespace aktienportfolio
{
public class Aktie
{
public int Id { get; set; }
public string Datum { get; set; }
public string Markt { get; set; }
public string Symbol { get; set; }
public string Name { get; set; }
public double Wert { get; set; }
}
class Program
{
public static void Main()
{
Console.WriteLine("AktienProgramm: ");
Console.WriteLine("--------------- ");
Console.WriteLine("\n");
string firstDecide; //StockExchange or Portfolio
do
{
Console.WriteLine("Do you like take a look at (A)ktienKurse or (P)ortfolio abfragen? \n");
firstDecide = Console.ReadKey(true).KeyChar.ToString().ToUpper(); //Kein Casesensitiv mit ToLower.
Console.WriteLine(firstDecide);
}
while (firstDecide != "A" && firstDecide != "P");
if (firstDecide == "A")
{
Console.WriteLine("\nDo you choose StockExchange, please push enter and the data will loaded adnd shown! \n");
var inputWert = Console.ReadLine();
Console.WriteLine("\nhere you see your data\n");
//Read File
string path = #"C:\Auftrag\Aktienkurse.csv";
string[] readText = File.ReadAllLines(path);
foreach (string s in readText)
{
Console.WriteLine(s);
}
}
else
{
Console.WriteLine("\nyou choose Portfolio, push Enter and the data will be loaded and shown! \n");
var inputWert = Console.ReadLine();
Console.WriteLine("\nhere you see your portfolio \n");
//read File
string path = #"C:\Auftrag\Portfolio.csv";
string[] readText = File.ReadAllLines(path);
foreach (string s in readText)
{
Console.WriteLine(s);
}
}
//Progamm proceed
string decideTyp; //Question what the user like to do now
{
Console.WriteLine("\nDo you like filter the (Z)data through the date or the (I)content ");
entscheidungsTyp = Console.ReadKey(true).KeyChar.ToString().ToUpper();
Console.WriteLine(entscheidungsTyp);
}
while (decideTyp != "Z" && decideTyp != "I");
if (decideTyp == "Z")
{
Console.WriteLine("you've chosen the date filtering ");
var inputWert = Console.ReadLine();
Console.WriteLine("Show the data date: ");
}
else
{
Console.WriteLine("you decided to show the content");
var inputWert = Console.ReadLine();
Console.WriteLine("show the content: ");
}
Console.ReadKey(false);
}
}
}
I am really a beginner so excuse my such dumb questions. I appreciate any help or suggestions to go ahead with my project for the school.
Here are the CSV files:
Aktienkurse
Id ";" Datum ";" Markt ";" Symbol ";" Name ";" Wert
1;2021-01-10;DowJones;BA;Boeing;135.20
2;2021-01-10;Nasdaq;MSFT;Microsoft;165.13
3;2021-01-10;Nasdaq;AMD;Advanced Micro Devices;48.79
4;2021-01-10;SMI;ABB;ASEA BROWN BOVERI;17.20
5;2021-01-10;DAX;BMW;Bayerische Motoren Werke AG;49.10
6;2021-01-11;Nasdaq;MSFT;Microsoft;166.13
Portfolio
Id ";" Datum ";" Symbol ";" Anzahl
1;2021-01-10;MSFT;45
2;2021-01-6;AMD;23
3;2021-01-7;BMW;10
4;2021-01-5;AMD;30
I just refactor a project that deals with CSV, but I couldn't refactor everything to pretend it's your CSV, so u can use this as inspiration
Try something like this:
public class CSVModel
{
public string FirstRow { get; set; }
public string SecondRow { get; set; }
public string ThirdRow { get; set; }
}
private void ImportFromCSV()
{
var csv = new CSVModel();
var csvFile = File.ReadAllLines(txtPath.Text);
//let's say it's separated by comma
var file = csvFile[i].Split(',');
for (int i = 0; i < csvFile.Count(); i++)
{
csv.FirstRow = file[i];
csv.SecondRow = file[i];
csv.ThirdRow = file[i];
//PS: All properties will have the same value, this isn't the exact solution,
//just a enlightening
}
// here you use the data as you want
}
Thanks, I am sure, some good insights will help me a lot to continue my project.
I think i will not be able to solve all problems, because the time is really short to finish the project. But everything I will be able to do or understand c# better will help me through this project.
Once Again, thanks for the input. Will try tomorrow. Was the entire day with struggling at this project :)

How do I display mysql data in csv file after I sucessfully read the data in c#

Below is my Connectionstring and sucessfully read the data. It will return total rows of my data.
private static async Task<List<OperatorErrorTransaction>> GetDevIndex()
{
try
{
var currentConnectionDev = new CurrentConnection(Configuration["ConnectionStrings:Default"], currentRequest);
Console.WriteLine("\nPress the Enter key to exit the application...\n");
var response = await currentConnectionDev.DbConnection.QuerySafeAsync<OperatorErrorTransaction>(GenerateGetDatabaseIndexQuery());
return response.ToList();
}
catch (Exception ex)
{
return new List<OperatorErrorTransaction>();
}
}
private static string GenerateGetDatabaseIndexQuery()
{
return #"SELECT * FROM test.operator_error_transaction";
}
Below is the csv CreateFile function. Right now i looking a way how to implement mysql data into the csv file.
public static void CreateFile(List<OperatorErrorTransaction> result)
{
string myFileName = String.Format("{0:yyyy-MM-dd-HHmm}{1}", DateTime.Now, ".csv");
string myFullPath = Path.Combine("D:\\", myFileName);
using (var mem = new MemoryStream())
using (StreamWriter writer = File.CreateText(myFullPath))
using (var csvWriter = new CsvWriter(writer))
{
csvWriter.Configuration.Delimiter = ";";
csvWriter.WriteField(result);
csvWriter.NextRecord();
writer.Flush();
var result1 = Encoding.UTF8.GetString(mem.ToArray());
Console.WriteLine(result1);
}
}
I have created a class for the variables as well such as public string BetId { get; set; } etc...

C# pass file text to list when file is entered via command line argument

I am writing a program for an assignment that is meant to read two text files and use their data to write to a third text file. I was instructed to pass the contents of the one file to a list. I have done something similar, passing the contents to an array (see below). But I can't seem to get it to work with a list.
Here is what I have done in the past with arrays:
StreamReader f1 = new StreamReader(args[0]);
StreamReader f2 = new StreamReader(args[1]);
StreamWriter p = new StreamWriter(args[2]);
double[] array1 = new double[20];
double[] array2 = new double[20];
double[] array3 = new double[20];
string line;
int index;
double value;
while ((line = f1.ReadLine()) != null)
{
string[] currentLine = line.Split('|');
index = Convert.ToInt16(currentLine[0]);
value = Convert.ToDouble(currentLine[1]);
array1[index] = value;
}
If it is of any interest, this is my current setup:
static void Main(String[] args)
{
// Create variables to hold the 3 elements of each item that you will read from the file
// Create variables for all 3 files (2 for READ, 1 for WRITE)
int ID;
string InvName;
int Number;
string IDString;
string NumberString;
string line;
List<InventoryNode> Inventory = new List<InventoryNode>();
InventoryNode Item = null;
StreamReader f1 = new StreamReader(args[0]);
StreamReader f2 = new StreamReader(args[1]);
StreamWriter p = new StreamWriter(args[2]);
// Read each item from the Update File and process the data
//Data is separated by pipe |
If you want to convert Array to List, you can just call Add or Insert to make it happen.
According to your code, you can do Inventory.Add(Item).
while ((line = f1.ReadLine()) != null)
{
string[] currentLine = line.Split('|');
Item = new InventoryItem {
Index = Convert.ToInt16(currentLine[0]),
Value = Convert.ToDouble(currentLine[1])
};
Inventory.Add(Item);
}
like this.
If I understand it correctly all you want to do is read two input file, parse the data in these file in a particular format (in this case int|double) and then write it to a new file. If this is the requirement, please try out the following code, as it is not sure how you want the data to be presented in the third file I have kept the format as it is (i.e. int|double)
static void Main(string[] args)
{
if (args == null || args.Length < 3)
{
Console.WriteLine("Wrong Input");
return;
}
if (!ValidateFilePath(args[0]) || !ValidateFilePath(args[1]))
{
return;
}
Dictionary<int, double> parsedFileData = new Dictionary<int, double>();
//Read the first file
ReadFileData(args[0], parsedFileData);
//Read second file
ReadFileData(args[1], parsedFileData);
//Write to third file
WriteFileData(args[2], parsedFileData);
}
private static bool ValidateFilePath(string filePath)
{
try
{
return File.Exists(filePath);
}
catch (Exception)
{
Console.WriteLine($"Failed to read file : {filePath}");
return false;
}
}
private static void ReadFileData(string filePath, Dictionary<int, double> parsedFileData)
{
try
{
using (StreamReader fileStream = new StreamReader(filePath))
{
string line;
while ((line = fileStream.ReadLine()) != null)
{
string[] currentLine = line.Split('|');
int index = Convert.ToInt16(currentLine[0]);
double value = Convert.ToDouble(currentLine[1]);
parsedFileData.Add(index, value);
}
}
}
catch (Exception ex)
{
Console.WriteLine($"Exception : {ex.Message}");
}
}
private static void WriteFileData(string filePath, Dictionary<int, double> parsedFileData)
{
try
{
using (StreamWriter fileStream = new StreamWriter(filePath))
{
foreach (var parsedLine in parsedFileData)
{
var line = parsedLine.Key + "|" + parsedLine.Value;
fileStream.WriteLine(line);
}
}
}
catch (Exception ex)
{
Console.WriteLine($"Exception : {ex.Message}");
}
}
There are few things you should always remember while writing a C# code :
1) Validate command line inputs before using.
2) Always lookout for any class that has dispose method, instantiate it inside using block.
3) Proper mechanism in the code to catch exceptions, else your program would crash at runtime with invalid inputs or inputs that you could not validate!

Parsing performance of row data from files to SQL Server database

I have the PAF raw data in several files (list of all addresses in the UK).
My goal is to create a PostCode lookup in our software.
I have created a new database but there is no need to understand it for the moment.
Let's take a file, his extension is ".c01" and can be open with a text editor. The data in this file are in the following format :
0000000123A
With (according to the developer guide), 8 char for the KEY, 50 char for the NAME.
This file contains 2,449,652 rows (it's a small one !)
I create a Parsing class for this
private class SerializedBuilding
{
public int Key
{
get; set;
}
public string Name
{
get; set;
}
public bool isValid = false;
public Building ToBuilding()
{
Building b = new Building();
b.BuildingKey = Key;
b.BuildingName = Name;
return b;
}
private readonly int KEYLENGTH = 8;
private readonly int NAMELENGTH = 50;
public SerializedBuilding(String line)
{
string KeyStr = null;
string Name = null;
try
{
KeyStr = line.Substring(0, KEYLENGTH);
}
catch (Exception e)
{
Console.WriteLine("erreur parsing key line " + line);
return;
}
try
{
Name = line.Substring(KEYLENGTH - 1, NAMELENGTH);
}
catch (Exception e)
{
Console.WriteLine("erreur parsing name line " + line);
return;
}
int value;
if (!Int32.TryParse(KeyStr, out value))
return;
if (value == 0 || value == 99999999)
return;
this.Name = Name;
this.Key = value;
this.isValid = true;
}
}
I use this method to read the file
public void start()
{
AddressDataContext d = new AddressDataContext();
Count = 0;
string line;
// Read the file and display it line by line.
System.IO.StreamReader file =
new System.IO.StreamReader(filename);
SerializedBuilding sb = null;
Console.WriteLine("Number of line detected : " + File.ReadLines(filename).Count());
while ((line = file.ReadLine()) != null)
{
sb = new SerializedBuilding(line);
if (sb.isValid)
{
d.Buildings.InsertOnSubmit(sb.ToBuilding());
if (Count % 100 == 0)
d.SubmitChanges();
}
Count++;
}
d.SubmitChanges();
file.Close();
Console.WriteLine("building added");
}
I use Linq to SQL classes to insert data to my database. The connection string is the default one.
This seems to work, I have added 67200 lines. It just crashed but my questions are not about that.
My estimations :
33,647,015 rows to parse
Time needed for execution : 13 hours
It's a one-time job (just needs to be done on my sql and on the client server later) so I don't really care about performances but I think it can be interesting to know how it can be improved.
My questions are :
Is readline() and substring() the most powerful ways to read these huge files ?
Can the performance be improved by modifying the connection string ?

EntityTooSmall in CompleteMultipartUploadResponse

using .NET SDK v.1.5.21.0
I'm trying to upload a large file (63Mb) and I'm following the example at:
http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFileDotNet.html
But using a helper instead the hole code and using jQuery File Upload
https://github.com/blueimp/jQuery-File-Upload/blob/master/basic-plus.html
what I have is:
string bucket = "mybucket";
long totalSize = long.Parse(context.Request.Headers["X-File-Size"]),
maxChunkSize = long.Parse(context.Request.Headers["X-File-MaxChunkSize"]),
uploadedBytes = long.Parse(context.Request.Headers["X-File-UloadedBytes"]),
partNumber = uploadedBytes / maxChunkSize + 1,
fileSize = partNumber * inputStream.Length;
bool lastPart = inputStream.Length < maxChunkSize;
// http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFileDotNet.html
if (partNumber == 1) // initialize upload
{
iView.Utilities.Amazon_S3.S3MultipartUpload.InitializePartToCloud(fileName, bucket);
}
try
{
// upload part
iView.Utilities.Amazon_S3.S3MultipartUpload.UploadPartToCloud(fs, fileName, bucket, (int)partNumber, uploadedBytes, maxChunkSize);
if (lastPart)
// wrap it up and go home
iView.Utilities.Amazon_S3.S3MultipartUpload.CompletePartToCloud(fileName, bucket);
}
catch (System.Exception ex)
{
// Huston, we have a problem!
//Console.WriteLine("Exception occurred: {0}", exception.Message);
iView.Utilities.Amazon_S3.S3MultipartUpload.AbortPartToCloud(fileName, bucket);
}
and
public static class S3MultipartUpload
{
private static string accessKey = System.Configuration.ConfigurationManager.AppSettings["AWSAccessKey"];
private static string secretAccessKey = System.Configuration.ConfigurationManager.AppSettings["AWSSecretKey"];
private static AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretAccessKey);
public static InitiateMultipartUploadResponse initResponse;
public static List<UploadPartResponse> uploadResponses;
public static void InitializePartToCloud(string destinationFilename, string destinationBucket)
{
// 1. Initialize.
uploadResponses = new List<UploadPartResponse>();
InitiateMultipartUploadRequest initRequest =
new InitiateMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'));
initResponse = client.InitiateMultipartUpload(initRequest);
}
public static void UploadPartToCloud(Stream fileStream, string destinationFilename, string destinationBucket, int partNumber, long uploadedBytes, long maxChunkedBytes)
{
// 2. Upload Parts.
UploadPartRequest request = new UploadPartRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId)
.WithPartNumber(partNumber)
.WithPartSize(maxChunkedBytes)
.WithFilePosition(uploadedBytes)
.WithInputStream(fileStream) as UploadPartRequest;
uploadResponses.Add(client.UploadPart(request));
}
public static void CompletePartToCloud(string destinationFilename, string destinationBucket)
{
// Step 3: complete.
CompleteMultipartUploadRequest compRequest =
new CompleteMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId)
.WithPartETags(uploadResponses);
CompleteMultipartUploadResponse completeUploadResponse =
client.CompleteMultipartUpload(compRequest);
}
public static void AbortPartToCloud(string destinationFilename, string destinationBucket)
{
// abort.
client.AbortMultipartUpload(new AbortMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId));
}
}
my maxChunckedSize is 6Mb (6 * (1024*1024)) as I have read that the minimum is 5Mb...
why am I getting "Your proposed upload is smaller than the minimum allowed size" exception? What am I doing wrong?
The error is:
<Error>
<Code>EntityTooSmall</Code>
<Message>Your proposed upload is smaller than the minimum allowed size</Message>
<ETag>d41d8cd98f00b204e9800998ecf8427e</ETag>
<MinSizeAllowed>5242880</MinSizeAllowed>
<ProposedSize>0</ProposedSize>
<RequestId>C70E7A23C87CE5FC</RequestId>
<HostId>pmhuMXdRBSaCDxsQTHzucV5eUNcDORvKY0L4ZLMRBz7Ch1DeMh7BtQ6mmfBCLPM2</HostId>
<PartNumber>1</PartNumber>
</Error>
How can I get ProposedSize if I'm passing the stream and stream length?
Here is a working solution for the latest Amazon SDK (as today: v.1.5.37.0)
Amazon S3 Multipart Upload works like:
Initialize the request using client.InitiateMultipartUpload(initRequest)
Send chunks of the file (loop until the end) using client.UploadPart(request)
Complete the request using client.CompleteMultipartUpload(compRequest)
If anything goes wrong, remember to dispose the client and request, as well fire the abort command using client.AbortMultipartUpload(abortMultipartUploadRequest)
I keep the client in Session as we need this for each chunk upload as well, keep an hold of the ETags that are now used to complete the process.
You can see an example and simple way of doing this in Amazon Docs itself, I ended up having a class to do everything, plus, I have integrated with the lovely jQuery File Upload plugin (Handler code below as well).
The S3MultipartUpload is as follow
public class S3MultipartUpload : IDisposable
{
string accessKey = System.Configuration.ConfigurationManager.AppSettings.Get("AWSAccessKey");
string secretAccessKey = System.Configuration.ConfigurationManager.AppSettings.Get("AWSSecretKey");
AmazonS3 client;
public string OriginalFilename { get; set; }
public string DestinationFilename { get; set; }
public string DestinationBucket { get; set; }
public InitiateMultipartUploadResponse initResponse;
public List<PartETag> uploadPartETags;
public string UploadId { get; private set; }
public S3MultipartUpload(string destinationFilename, string destinationBucket)
{
if (client == null)
{
System.Net.WebRequest.DefaultWebProxy = null; // disable proxy to make upload quicker
client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretAccessKey, new AmazonS3Config()
{
RegionEndpoint = Amazon.RegionEndpoint.EUWest1,
CommunicationProtocol = Protocol.HTTP
});
this.OriginalFilename = destinationFilename.TrimStart('/');
this.DestinationFilename = string.Format("{0:yyyy}{0:MM}{0:dd}{0:HH}{0:mm}{0:ss}{0:fffff}_{1}", DateTime.UtcNow, this.OriginalFilename);
this.DestinationBucket = destinationBucket;
this.InitializePartToCloud();
}
}
private void InitializePartToCloud()
{
// 1. Initialize.
uploadPartETags = new List<PartETag>();
InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest();
initRequest.BucketName = this.DestinationBucket;
initRequest.Key = this.DestinationFilename;
// make it public
initRequest.AddHeader("x-amz-acl", "public-read");
initResponse = client.InitiateMultipartUpload(initRequest);
}
public void UploadPartToCloud(Stream fileStream, long uploadedBytes, long maxChunkedBytes)
{
int partNumber = uploadPartETags.Count() + 1; // current part
// 2. Upload Parts.
UploadPartRequest request = new UploadPartRequest();
request.BucketName = this.DestinationBucket;
request.Key = this.DestinationFilename;
request.UploadId = initResponse.UploadId;
request.PartNumber = partNumber;
request.PartSize = fileStream.Length;
//request.FilePosition = uploadedBytes // remove this line?
request.InputStream = fileStream; // as UploadPartRequest;
var up = client.UploadPart(request);
uploadPartETags.Add(new PartETag() { ETag = up.ETag, PartNumber = partNumber });
}
public string CompletePartToCloud()
{
// Step 3: complete.
CompleteMultipartUploadRequest compRequest = new CompleteMultipartUploadRequest();
compRequest.BucketName = this.DestinationBucket;
compRequest.Key = this.DestinationFilename;
compRequest.UploadId = initResponse.UploadId;
compRequest.PartETags = uploadPartETags;
string r = "Something went badly wrong";
using (CompleteMultipartUploadResponse completeUploadResponse = client.CompleteMultipartUpload(compRequest))
r = completeUploadResponse.ResponseXml;
return r;
}
public void AbortPartToCloud()
{
// abort.
client.AbortMultipartUpload(new AbortMultipartUploadRequest()
{
BucketName = this.DestinationBucket,
Key = this.DestinationFilename,
UploadId = initResponse.UploadId
});
}
public void Dispose()
{
if (client != null) client.Dispose();
if (initResponse != null) initResponse.Dispose();
}
}
I use DestinationFilename as the destination file so I can avoid the same name, but I keep the OriginalFilename as I needed later.
Using jQuery File Upload Plugin, all works inside a Generic Handler, and the process is something like this:
// Upload partial file
private void UploadPartialFile(string fileName, HttpContext context, List<FilesStatus> statuses)
{
if (context.Request.Files.Count != 1)
throw new HttpRequestValidationException("Attempt to upload chunked file containing more than one fragment per request");
var inputStream = context.Request.Files[0].InputStream;
string contentRange = context.Request.Headers["Content-Range"]; // "bytes 0-6291455/14130271"
int fileSize = int.Parse(contentRange.Split('/')[1]);,
maxChunkSize = int.Parse(context.Request.Headers["X-Max-Chunk-Size"]),
uploadedBytes = int.Parse(contentRange.Replace("bytes ", "").Split('-')[0]);
iView.Utilities.AWS.S3MultipartUpload s3Upload = null;
try
{
// ######################################################################################
// 1. Initialize Amazon S3 Client
if (uploadedBytes == 0)
{
HttpContext.Current.Session["s3-upload"] = new iView.Utilities.AWS.S3MultipartUpload(fileName, awsBucket);
s3Upload = (iView.Utilities.AWS.S3MultipartUpload)HttpContext.Current.Session["s3-upload"];
string msg = System.String.Format("Upload started: {0} ({1:N0}Mb)", s3Upload.DestinationFilename, (fileSize / 1024));
this.Log(msg);
}
// cast current session object
if (s3Upload == null)
s3Upload = (iView.Utilities.AWS.S3MultipartUpload)HttpContext.Current.Session["s3-upload"];
// ######################################################################################
// 2. Send Chunks
s3Upload.UploadPartToCloud(inputStream, uploadedBytes, maxChunkSize);
// ######################################################################################
// 3. Complete Upload
if (uploadedBytes + maxChunkSize > fileSize)
{
string completeRequest = s3Upload.CompletePartToCloud();
this.Log(completeRequest); // log S3 response
s3Upload.Dispose(); // dispose all objects
HttpContext.Current.Session["s3-upload"] = null; // we don't need this anymore
}
}
catch (System.Exception ex)
{
if (ex.InnerException != null)
while (ex.InnerException != null)
ex = ex.InnerException;
this.Log(string.Format("{0}\n\n{1}", ex.Message, ex.StackTrace)); // log error
s3Upload.AbortPartToCloud(); // abort current upload
s3Upload.Dispose(); // dispose all objects
statuses.Add(new FilesStatus(ex.Message));
return;
}
statuses.Add(new FilesStatus(s3Upload.DestinationFilename, fileSize, ""));
}
Keep in mind that to have a Session object inside a Generic Handler, you need to implement IRequiresSessionState so your handler will look like:
public class UploadHandlerSimple : IHttpHandler, IRequiresSessionState
Inside fileupload.js (under _initXHRData) I have added an extra header called X-Max-Chunk-Size so I can pass this to Amazon and calculate if it's the last part of the uploaded file.
Fell free to comment and make smart edits for everyone to use.
I guess you didn't set the content-length of the part inside the UploadPartToCloud() function.

Categories