Question:
I need to read a CSV file.
I use the FileHelpers library to achieve this.
The problem is I need a dynamic delimiter (user defined), meaning anything can be delimiter (Comma, semicolon, tab, newline, but also anything else).
The problem is, FileHelpers defines the delimiter in an attribute, which means at compile-time. This makes it impossible to do it dynamically.
What I can do is declare a new class, which inherits from one base class, and set the delimiter on this new class.
[FileHelpers.DelimitedRecord(",")]
public class CommaCustomer : BaseCustomer
{
}
That way I only have to make changes in the base class for every new delimiter.
The problem is, this is I can't (and don't want to) create a child class for every possible delimiter.
This is the code I have so far:
using System;
using System.Data;
using System.IO;
//using FileHelpers;
//using FileHelpers.RunTime;
namespace Examples
{
class MainClass
{
[STAThread]
static void Main()
{
FileHelpers.FileHelperEngine engine = new FileHelpers.FileHelperEngine(typeof(SemicolonCustomer));
// To read use:
string str = #"D:\Username\Desktop\FileHelpers_Examples_CSharp_VbNet\Data\SemicolonCustomers.txt";
//str = #"D:\Username\Desktop\FileHelpers_Examples_CSharp_VbNet\Data\CustomersDelimited.txt";
SemicolonCustomer[] custs = (SemicolonCustomer[])engine.ReadFile(str);
//Customer[] custs = (Customer[]) engine.ReadFile("yourfile.txt");
foreach (SemicolonCustomer cli in custs)
{
Console.WriteLine();
Console.WriteLine("Customer: " + cli.CustId.ToString() + " - " + cli.Name);
Console.WriteLine("Added Date: " + cli.AddedDate.ToString("d-M-yyyy"));
Console.WriteLine("Balance: " + cli.Balance.ToString());
Console.WriteLine();
Console.WriteLine("-----------------------------");
} // Next cli
Console.ReadKey();
Console.WriteLine("Writing data to a delimited file...");
Console.WriteLine();
// To write use:
//engine.WriteFile("myyourfile.txt", custs);
//If you are using .NET 2.0 or greater is
//better if you use the Generics version:
// FileHelperEngine engine = new FileHelperEngine<Customer>();
// To read use (no casts =)
// Customer[] custs = engine.ReadFile("yourfile.txt");
// To write use:
// engine.WriteFile("yourfile.txt", custs);
} // End Sub Main
} // End Class MainClass
//------------------------
// RECORD CLASS (Example, change at your will)
// TIP: Remember to use the wizard to generate this class
public class BaseCustomer
{
public int CustId;
public string Name;
public decimal Balance;
[FileHelpers.FieldConverter(FileHelpers.ConverterKind.Date, "ddMMyyyy")]
public DateTime AddedDate;
}
[FileHelpers.DelimitedRecord(";")]
public class SemicolonCustomer : BaseCustomer
{
}
[FileHelpers.DelimitedRecord(",")]
public class CommaCustomer : BaseCustomer
{
}
}
Is it somehow possible at runtime to compile a child class
[FileHelpers.DelimitedRecord(\"" + delimiter + "\")]
public class AnyDelimiterCustomer : BaseCustomer
{
}
And then reference this runtime compiled class in code ?
I just realized there is a DelimitedFileEngine which solves your problem another way.
You can just go
var engine = new DelimitedFileEngine(typeof(BaseCustomer));
engine.Options.Delimiter = ",";
It seems that BaseCustomer needs to be decorated with a [DelimitedRecord] attribute, otherwise an exception is raised but the delimiter is overridden by whatever is supplied to engine.Options.Delimiter.
The following example imports a comma delimited record using a format which is marked as bar delimited.
[DelimitedRecord("|")]
public class Format1
{
public string Field1;
public string Field2;
public string Field3;
public string Field4;
}
static void Main(string[] args)
{
var engine = new DelimitedFileEngine(typeof(Format1));
// change the delimiter
engine.Options.Delimiter = ",";
// import a comma separated record
object[] importedObjects = engine.ReadString(#"a,b,c,d");
foreach (object importedObject in importedObjects)
{
if (importedObject is Format1)
{
Format1 format1 = (Format1)importedObject;
// process it (for example, check the values)
Assert.AreEqual("a", format1.Field1);
Assert.AreEqual("b", format1.Field2);
Assert.AreEqual("c", format1.Field3);
Assert.AreEqual("d", format1.Field4);
}
}
}
No thats not possible.
But you can use the FileHelper DelimitedClassBuilder to build a dynamic file parser where you can set the delimiter at runtime:
DelimitedClassBuilder dcb = new DelimitedClassBuilder("Name",
"Here goes your col separator");
// You have to build your field definitions by hand now
dcb.AddField("FieldName", typeof(decimal));
...
// build the engine
DelimitedFileEngine fileEngine = new DelimitedFileEngine(dcb.CreateRecordClass());
// read the file
dynamic[] data = fileEngine.ReadFile(filePath);
You can use runtime classes. You have two choices. Either compile your class from a string
For instance
// The class definition
public string mClass =
#"
[DelimitedRecord(""" + delimiter + #""")]
public class BaseCustomer
{
public int CustId;
public string Name;
public decimal Balance;
[FileHelpers.FieldConverter(FileHelpers.ConverterKind.Date, ""ddMMyyyy"")]
public DateTime AddedDate;
}
";
Type t = ClassBuilder.ClassFromString(mClass);
FileHelperEngine engine = new FileHelperEngine(t);
DataTable = engine.ReadFileAsDT("test.txt");
Or alternatively, you can use the DelimitedClassBuilder class.
DelimitedClassBuilder cb = new DelimitedClassBuilder("BaseCustomer", delimiter);
cb.AddField("CustId", typeof(int));
cb.LastField.TrimMode = TrimMode.Both;
cb.LastField.FieldNullValue = 0;
cb.AddField("Balance", typeof(Decimal));
cb.AddField("AddedDate", typeof(DateTime));
engine = new FileHelperEngine(cb.CreateRecordClass());
DataTable dt = engine.ReadFileAsDT("test.txt");
It is possible.
But only by moving the serialization type into a separate assembly.
Like this:
using System;
using System.Collections.Generic;
using System.Windows.Forms;
namespace FlaechenupdateScript
{
static class Program
{
// http://www.codeproject.com/KB/cs/runtimecompiling.aspx
private static System.Reflection.Assembly BuildAssembly(string code)
{
Microsoft.CSharp.CSharpCodeProvider provider =
new Microsoft.CSharp.CSharpCodeProvider();
System.CodeDom.Compiler.ICodeCompiler compiler = provider.CreateCompiler();
System.CodeDom.Compiler.CompilerParameters compilerparams = new System.CodeDom.Compiler.CompilerParameters();
string strLocation = System.Reflection.Assembly.GetExecutingAssembly().Location;
string strBasePath = System.IO.Path.GetDirectoryName(strLocation);
string strSerializationTypes = System.IO.Path.Combine(strBasePath, "SerializationTypes.dll");
string strFileHelpersLocation = System.IO.Path.Combine(strBasePath, "FileHelpers.dll");
compilerparams.ReferencedAssemblies.Add(strSerializationTypes);
compilerparams.ReferencedAssemblies.Add(strFileHelpersLocation);
compilerparams.GenerateExecutable = false;
compilerparams.GenerateInMemory = true;
System.CodeDom.Compiler.CompilerResults results =
compiler.CompileAssemblyFromSource(compilerparams, code);
if (results.Errors.HasErrors)
{
System.Text.StringBuilder errors = new System.Text.StringBuilder("Compiler Errors :\r\n");
foreach (System.CodeDom.Compiler.CompilerError error in results.Errors)
{
errors.AppendFormat("Line {0},{1}\t: {2}\n",
error.Line, error.Column, error.ErrorText);
}
throw new Exception(errors.ToString());
}
else
{
return results.CompiledAssembly;
}
} // End Function BuildAssembly
public static Type GetClassType(Type tt, string strDelimiter)
{
string strFullTypeName = tt.FullName;
string strTypeUniqueName = System.Guid.NewGuid().ToString() + System.Guid.NewGuid().ToString() + System.Guid.NewGuid().ToString() + System.Guid.NewGuid().ToString();
strTypeUniqueName = "_" + strTypeUniqueName.Replace("-", "_");
string xx = #"
namespace CrapLord
{
[FileHelpers.IgnoreFirst]
[FileHelpers.IgnoreEmptyLines]
[FileHelpers.DelimitedRecord(""" + strDelimiter + #""")]
public class " + strTypeUniqueName + #" : " + strFullTypeName + #"
{
}
}
";
System.Reflection.Assembly a = BuildAssembly(xx);
var o = a.CreateInstance("CrapLord." + strTypeUniqueName);
Type t = o.GetType();
//System.Reflection.MethodInfo mi = t.GetMethod("EvalCode");
//var s = mi.Invoke(o, null);
return t;
}
/// <summary>
/// Der Haupteinstiegspunkt für die Anwendung.
/// </summary>
[STAThread]
static void Main()
{
//Application.EnableVisualStyles();
//Application.SetCompatibleTextRenderingDefault(false);
//Application.Run(new Form1());
Type t = GetClassType(typeof(Tools.Serialization.CSV.Customer), ",");
//FileHelpers.FileHelperEngine engine = new FileHelpers.FileHelperEngine(typeof(SemicolonCustomer));
FileHelpers.FileHelperEngine engine = new FileHelpers.FileHelperEngine(t);
string str = "path/to/datafile";
Tools.Serialization.CSV.Customer[] custs = (Tools.Serialization.CSV.Customer[])engine.ReadFile(str);
//Customer[] custs = (Customer[]) engine.ReadFile("yourfile.txt");
foreach (Tools.Serialization.CSV.Customer cli in custs)
{
Console.WriteLine();
Console.WriteLine("Customer: " + cli.CustId.ToString() + " - " + cli.Name);
Console.WriteLine("Added Date: " + cli.AddedDate.ToString("d-M-yyyy"));
Console.WriteLine("Balance: " + cli.Balance.ToString());
Console.WriteLine();
Console.WriteLine("-----------------------------");
} // Next cli
Console.WriteLine(Environment.NewLine);
Console.WriteLine(" --- Press any key to continue --- ");
Console.ReadKey();
}
}
}
SerializationTypes Assembly:
using System;
using System.Collections.Generic;
using System.Text;
namespace Tools.Serialization.CSV
{
//------------------------
// RECORD CLASS (Example, change at your will)
// TIP: Remember to use the wizard to generate this class
public class Customer
{
public int CustId;
public string Name;
public decimal Balance;
[FileHelpers.FieldConverter(FileHelpers.ConverterKind.Date, "ddMMyyyy")]
public DateTime AddedDate;
}
}
Maybe you want to use the TextFieldParser from Microsoft.VisualBasic.FileIO Namespace:
string[] fields;
string[] delimiter = new string[] { "|" };
using (Microsoft.VisualBasic.FileIO.TextFieldParser parser =
new Microsoft.VisualBasic.FileIO.TextFieldParser(filename))
{
parser.Delimiters = delimiter;
parser.HasFieldsEnclosedInQuotes = false;
while (!parser.EndOfData)
{
fields = parser.ReadFields();
//Do what you need
}
}
adding a cast solves the problem for me (FileHelpers V3.5.1)
var engine = new DelimitedFileEngine(typeof(BaseCustomer));
((FileHelpers.Options.DelimitedRecordOptions)engine.Options).Delimiter=",";
Related
This is a function to work with lists in string interpolation. It takes a List and an inner Func, and it appends the string result of the inner Func called for each member of the list, with a separator.
So the following builds a valid start of an Insert statement...
static void Main(string[] args)
{
var tableName = "customers";
var cols = new List<dynamic>
{
new { Name = "surname"},
new { Name = "firstname"},
new { Name = "dateOfBirth"}
};
Func<List<dynamic>, Func<dynamic, string>, string, string> ForEach = (list, func, separator) =>
{
var bldr = new StringBuilder();
var first = true;
foreach (var obj in list)
{
if (!first)
bldr.Append(separator);
first = false;
bldr.Append(func(obj));
}
return bldr.ToString();
};
var InsertStatement = $"Insert into { tableName } ( {ForEach(cols, col => col.Name, ", ")} )";
Console.WriteLine(InsertStatement);
Console.ReadLine();
}
Outputs...
Insert into customers ( surname, firstname, dateOfBirth )
It works for dynamic. How do I make it work for any type? The outer Func shouldn't care about the Type in the list, it just passes it through to the inner Func.
The .NET framework already gives you a generic function to achieve what you are trying to do String.Join and you can combine it with a LINQ Select statement, which will allow you to use a lambda on a generic type to select the property that you want to print. You can view the source code of these methods if you are interested as they are open source.
using System;
using System.Collections.Generic;
using System.Linq;
public class MyType
{
public string Name { get; set; }
}
public class Program
{
public static void Main()
{
var tableName = "customers";
var cols = new List<MyType>
{
new MyType { Name = "surname"},
new MyType { Name = "firstname"},
new MyType { Name = "dateOfBirth"}
};
var InsertStatement = $"Insert into { tableName } ( {String.Join(", ", cols.Select(col => col.Name))} )";
Console.WriteLine(InsertStatement);
}
}
Replace dynamic with object, or TValue with a type constraint stipulating it must be a class (where TValue : class), and call obj.ToString() instead of just obj
However, this doesn't guarantee it would "work with any type" - for that you need to know that those types all follow a contract to output the desired column name as their string representation. To get more specificity, require that your accepted types must implement some interface eg IColumnName and put that interface into the type constraint instead
You can create the text easily like this:
var query = $"INSERT INTO {tableName}({string.Join(",", cols.Select(x=>x.Name))})";
However, if for learning purpose you are going to handle the case using a generic method, you can create a generic function like the following and then easily use a for loop and strip additional separator using TrimEnd, or as a better option, like String.Join implementation of .NET Framework get enumerator like this:
string Join<TItem>(
IEnumerable<TItem> items, Func<TItem, string> itemTextSelecor, string separator)
{
var en = items.GetEnumerator();
if (!en.MoveNext())
return String.Empty;
var builder = new StringBuilder();
if (en.Current != null)
builder.Append(itemTextSelecor(en.Current));
while (en.MoveNext())
{
builder.Append(separator);
if (en.Current != null)
builder.Append(itemTextSelecor(en.Current));
}
return builder.ToString();
}
And use it this way:
var tableName = "customers";
var cols = new[]
{
new { Name = "surname"},
new { Name = "firstname"},
new { Name = "dateOfBirth"}
};
var InsertStatement = $"INSERT INTO {tableName} ({Join(cols, col => col.Name, ", ")})"
+ $"VALUES({Join(cols, col => $"#{col.Name}", ", ")})";
I need to write many sample codes for various API's, which demonstrate how to write the code to use those particular REST API's in C# language.
Now for the API's which are HTTP POST calls, a request body is first created as a class object which is then later serialized into a JSON string and passed to the REST Client.
My Requirement is in this part: "Creating the request Body as a class object".
Following example will make the requirement crystal clear:-
Suppose i have the below JSON Data:
{
"workingDays": ["Monday","Wednesday","Friday"],
"employeeInformation": {
"employeeID": "12345",
"name": {
"firstName": "foo",
"lastName": "bar"
}
},
"joiningDate":"23061984"
}
I need to parse the above data and generate the below code (which currently i am writing manually):
// Create Main Request Body Object
var requestBodyObj = new RequestBody();
// Array Case
var workingDaysObj = new List<string>();
workingDaysObj.Add("Monday");
workingDaysObj.Add("Wednesday");
workingDaysObj.Add("Friday");
requestBodyObj.workingDays = workingDaysObj;
// Nested Object Case
var employeeInformationObj = new employeeInformation();
employeeInformationObj.employeeID = "12345";
var nameObj = new name();
nameObj.firstName = "foo";
nameObj.lastName = "bar";
employeeInformationObj.name = nameObj;
requestBodyObj.employeeInformation = employeeInformationObj;
// Simple Name/Value Pair
requestBodyObj.joiningDate = "23061984";
So as per the above example the JSON Data can be in one of the following 2 forms as well (apart from simple name/value pairs):
Array
Nested Object
And both these cases should be handled as shown in the above code.
Note: User will not be provided with a JSON file so i can't write any code which directly reads a JSON file, deserializes it and assigns the values to a class object using (for example) a NewtonSoft function:
// read file into a string and deserialize JSON to a type
Movie movie1 = JsonConvert.DeserializeObject<Movie>(File.ReadAllText(#"c:\movie.json"));
// deserialize JSON directly from a file
using (StreamReader file = File.OpenText(#"c:\movie.json"))
{
JsonSerializer serializer = new JsonSerializer();
Movie movie2 = (Movie)serializer.Deserialize(file, typeof(Movie));
}
I just want a simple "JSON parser and C# Code generator" (Preferably written in C# language itself).
Any suggestions or pointers will be appreciated.
Edit Update
Pascal Case setting for variable names
Json Object names can be mapped to Project model classes
Write the output to text file for easy copy
Updated Code Below:-
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Newtonsoft.Json;
namespace SampleCodeCreator
{
class Program
{
// Declaring the variables
static string jsonFilePath = #"[Your File Path]";
static string outputFilePath = #"[Your File Path]";
static string jsonData;
static Dictionary<string, string> classMap = new Dictionary<string, string>(StringComparer.InvariantCultureIgnoreCase);
static void Main(string[] args)
{
// Initializing Class map which is used to map the json object to Project Model Class
InitializeClassMap();
// clear current data in the output file
using (System.IO.StreamWriter file = new System.IO.StreamWriter(outputFilePath, false))
{
file.Write(String.Empty);
}
// read the json data file and store the data in a simple string
using (StreamReader r = new StreamReader(jsonFilePath))
{
jsonData = r.ReadToEnd();
}
// Call the method for the whole json data
PrintJsonObject("RequestBody", jsonData);
}
static void PrintJsonObject(string parentObject, string jsonData)
{
// if the parent object has any mapped class, then set the object name
parentObject = MappedClass(parentObject);
Console.WriteLine("var {0}Obj = new {1}();", ToCamelCase(parentObject), parentObject);
SetOutput("var " + ToCamelCase(parentObject) + "Obj = new " + parentObject + "();");
Console.WriteLine("");
SetOutput("");
// Deserialize the Json data and iterate through each of its sub-sections
var jsonSubData = JsonConvert.DeserializeObject<Dictionary<string, object>>(jsonData);
foreach (var data in jsonSubData)
{
var dataKey = data.Key;
var dataValue = data.Value;
// array case (if the sub element is an array)
if (dataValue.ToString().Contains("["))
{
PrintArrayCase(dataKey, dataValue);
Console.WriteLine("{0}Obj.{1} = {1}Obj;", ToCamelCase(parentObject), dataKey);
SetOutput(ToCamelCase(parentObject) + "Obj." + dataKey + " = " + dataKey + "Obj;");
Console.WriteLine("");
SetOutput("");
}
// nested object case (if the sub element itself contains another json format body)
else if (dataValue.ToString().Contains("{"))
{
// Recursive Call
PrintJsonObject(dataKey, dataValue.ToString());
Console.WriteLine("{0}Obj.{1} = {1}Obj;", ToCamelCase(parentObject), dataKey);
SetOutput(ToCamelCase(parentObject) + "Obj." + dataKey + " = " + dataKey + "Obj;");
Console.WriteLine("");
SetOutput("");
}
// simple key value pair case
else
{
PrintKeyValuePairCase(parentObject, dataKey, dataValue.ToString());
}
}
}
static void PrintArrayCase(string key, object obj)
{
Console.WriteLine("var {0}Obj = new List<string>();", key);
SetOutput("var " + key + "Obj = new List<string>();");
// The array value is split into its values
// e.g. [abc, def, ghi] -> [abc] [def] [ghi]
var valueString = obj.ToString();
var valueSplitArray = valueString.Split(',');
for (int k = 0; k < valueSplitArray.Count(); k++)
{
string listValue = "";
if (k != valueSplitArray.Count() - 1)
{
var startIndex = valueSplitArray[k].IndexOf("\"");
listValue = valueSplitArray[k].Substring(startIndex + 1, valueSplitArray[k].Length - startIndex - 2);
}
else
{
var startIndex = valueSplitArray[k].IndexOf("\"");
listValue = valueSplitArray[k].Substring(startIndex + 1, valueSplitArray[k].Length - startIndex - 5);
}
// each value is then added to the main list object
Console.WriteLine(#"{0}Obj.Add(""{1}"");", ToCamelCase(key), listValue);
SetOutput(#""+ToCamelCase(key)+#"Obj.Add("""+listValue+#""");");
}
Console.WriteLine("");
SetOutput("");
}
static void PrintKeyValuePairCase(string parentObj, string key, string value)
{
Console.WriteLine("{0}Obj.{1} = \"{2}\";", ToCamelCase(parentObj), key, value);
SetOutput("" + ToCamelCase(parentObj) + "Obj." + key + " = \"" + value + "\";");
}
static string ToCamelCase(string str)
{
if (!String.IsNullOrEmpty(str))
{
return str.Substring(0, 1).ToLower() + str.Substring(1, str.Length - 1);
}
else
{
return null;
}
}
static string MappedClass(string str)
{
if (classMap.ContainsKey(str))
return classMap[str];
else
return str;
}
static void InitializeClassMap()
{
classMap.Add("employeeInformation", "EmployeeInfo");
}
static void SetOutput(string str)
{
using (System.IO.StreamWriter file = new System.IO.StreamWriter(outputFilePath, true))
{
file.WriteLine(str);
}
}
}
}
I wrote a simple .dll class library C#. My file takes a values of my main program and create a html file with this values. Look at my code: (I am describing it so that you can check if I have a mistake)
public class MyClass1 { public void HTMLGen(int number, string name)
{
var html1 = string.Format("<p>This is number: {0} and this is name: {1}</p>", number, name);
var xDocument = new XDocument(
new XDocumentType("html", null, null, null),
new XElement("html",
new XElement("head"),
new XElement("body",
XElement.Parse(html1))));
var settings = new XmlWriterSettings
{
OmitXmlDeclaration = true,
Indent = true,
IndentChars = "\t"
};
using (var writer = XmlWriter.Create(#"C:\Users\Desktop\test.html", settings))
{
xDocument.WriteTo(writer);
}
} }
Okay, this code works well. so I need add to collection. I create a new class-file in this .dll. This is my new class:
public class Collection : IList {
public int value_one { get; set; }
public int value_two { get; set; }}
and I added this parameter to my class:
public class MyClass1 { public void HTMLGen(int number, string name, IList<Collection> collection)
so I create a new 'var html2' and I want to add this to XElement.Parse. This is my whole code:
public class MyClass1 { public void HTMLGen(int number, string name, IList<Collection> collection) {
var html1 = string.Format("<p>This is number: {0} and this is name: {1}</p>", number, name);
var html2 = string.Format("<p> /* value_one and value_two from Collection */ </p>");
var xDocument = new XDocument(
new XDocumentType("html", null, null, null),
new XElement("html",
new XElement("head"),
new XElement("body",
XElement.Parse(html1), XElement.Parse(html2))));
var settings = new XmlWriterSettings
{
OmitXmlDeclaration = true,
Indent = true,
IndentChars = "\t"
};
using (var writer = XmlWriter.Create(#"C:\Users\Desktop\test.html", settings))
{
xDocument.WriteTo(writer);
} } }
and please, look at my comment. I don't know, how Can I add this values of my collection to this var html2.
Firstly, I don't understand why you would need to create a class named Collection with two integers. You can either use a int Array or list. Or may be a structure would be the best thing to use if you have not just int but other datatypes.
Lets suppose you need that class, then why should it inherit IList? I don't see any usage of it at all.
Anyway, just to answer your question:
var html2 = string.Format("<p> /* {0} and {1} from Collection */ </p>", collection.value_one, collection.value_two);
Or, if you have different types for your values, just convert it to string to be safe.
var html2 = string.Format("<p> /* {0} and {1} from Collection */ </p>", collection.value_one.ToString(), collection.value_two.ToString());
You can use Interpolated Strings to avoid long String.Format method usage like below.
var html1 = $"<p>This is number: {number} and this is name: {name}</p>");
var html2 = $"<p> This is first value: {collection[0]} and this is second value: {collection[1]}</p>";
Maybe you are trying to achive this?
public class MyClass1
{
public void HTMLGen(int number, string name, IList<int> collection)
{
var html1 = string.Format("<p>This is number: {0} and this is name: {1}</p>", number, name);
string html2 = "";
foreach (var item in collection)
{
html2 += item + " ";
}
html2 = "<p>" + html2 + "</p>";
//The rest of the code
}
}
usage:
MyClass1 myClass1 = new MyClass1();
List<int> numbers = new List<int> { 4, 3, 21, 123, 6 };
myClass1.HTMLGen(10, "name", numbers);
This is my first time posting so I apologize for any ignorance or failed use of examples.
I have a console app project to create where I have been given a fair few CSV files and I need to create some kind of Parent/Child/Grandchild relationship out of them (XML? maybe? - then I can use that to do the uploads and writes to the DMS with minimal calls - I don't want to be querying if a folder exists over and over)
I am a little out of my depth on this one
I need to know the best way to do this without 3rd party library dependencies, pure C#, using the OLEDB JET provider is most likely required as it will handle the parsing required, there is no order to the CSV files in regards to date, previous years could appear down the list and vice versa.
Here's an example of the CSV output
"DESCRIPTION1","8, 5/8\" X 6.4MM","STRING","filename001.pdf","2016-09-19","1"
"DESCRIPTION2","12, 3/4\" X 6.4MM","STRING","filename001.pdf","2016-09-19","1"
"DESCRIPTION3","12, 3/4\" X 6.4MM","STRING","filename001.pdf","2016-09-19","1"
"another description 20# gw","1","388015","Scan123.pdf","2015-10-24","1"
"another description 20# gw","3","385902","Scan456.pdf","2015-04-14","1"
"STRINGVAL1","273.10 X 9.27 X 6000","45032-01","KHJDWNEJWKFD9101529.pdf","2012-02-03","1"
"STRINGVAL2","273.10 X 21.44 X 6000","7-09372","DJSWH68767681540.pdf","2017-02-03","1"
The end output will be (YEAR/MONTH/FILENAME + (Attributes for each file - these are for eventually updating columns inside a DMS))
Year and Month retrieved from the column with the date
If the YEAR alread exists then it will not be created again
If the month under that year exists it will not be created again
If the filename already exists under that YEAR/MONTH it will not be created again BUT the additional ATTRIBUTES for that FileName will be added to the attributes - "line seperated?"
Required Output:
I have attempted a Linq query to begin to output the possible required XML for me to progress but it outputs every row and does no grouping, I am not familiar with Linq at the moment.
I also ran into issues with the basic escaping on the .Split(',') doing it this way (see original CSV examples above compared to me using TAB separation in my test file and example below) which is why I want the Oledb provider to handle it.
string[] source = File.ReadAllLines(#"C:\Processing\In\mockCsv.csv");
XElement item = new XElement("Root",
from str in source
let fields = str.Split('\t')
select new XElement("Year", fields[4].Substring(0, 4),
new XElement("Month", fields[4].Substring(5, 2),
new XElement("FileName", fields[3]),
new XElement("Description",fields[0]),
new XElement("Length", fields[1]),
new XElement("Type", fields[2]),
new XElement("FileName", fields[3]),
new XElement("Date", fields[4]),
new XElement("Authorised", fields[5]))
)
);
I also need to log every step of the process so I have setup a Logger class
private class Logger
{
private static string LogFile = null;
internal enum MsgType
{
Info,
Debug,
Error
}
static Logger()
{
var processingDetails = ConfigurationManager.GetSection(SECTION_PROCESSINGDETAILS) as NameValueCollection;
LogFile = Path.Combine(processingDetails[KEY_WORKINGFOLDER],
String.Format("Log_{0}.txt", StartTime.ToString("MMMyyyy")));
if (File.Exists(LogFile))
File.Delete(LogFile);
}
internal static void Write(string msg, MsgType msgType, bool isNewLine, bool closeLine)
{
if (isNewLine)
msg = String.Format("{0} - {1} : {2}", DateTime.Now.ToString("dd/MM/yyyy HH:mm:ss"), msgType, msg);
if (closeLine)
Console.WriteLine(msg);
else
Console.Write(msg);
if (String.IsNullOrEmpty(LogFile))
return;
try
{
using (StreamWriter sw = new StreamWriter(LogFile, true))
{
if (closeLine)
sw.WriteLine(msg);
else
sw.Write(msg);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
Used as such
Logger.Write(String.Format("Reading records from csv file ({0})... ",
csvFile), Logger.MsgType.Info, true, false);
Try following. If you are reading from a file use StreamReader instead of StringReader :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
using System.IO;
using System.Text.RegularExpressions;
namespace ConsoleApplication74
{
class Program
{
static void Main(string[] args)
{
string input =
"\"DESCRIPTION1\",\"8, 5/8 X 6.4MM\",\"STRING\",\"filename001.pdf\",\"2016-09-19\",\"1\"\n" +
"\"DESCRIPTION2\",\"12, 3/4 X 6.4MM\",\"STRING\",\"filename001.pdf\",\"2016-09-19\",\"1\"\n" +
"\"DESCRIPTION3\",\"12, 3/4 X 6.4MM\",\"STRING\",\"filename001.pdf\",\"2016-09-19\",\"1\"\n" +
"\"another description 20# gw\",\"1\",\"388015\",\"Scan123.pdf\",\"2015-10-24\",\"1\"\n" +
"\"another description 20# gw\",\"3\",\"385902\",\"Scan456.pdf\",\"2015-04-14\",\"1\"\n" +
"\"STRINGVAL1\",\"273.10 X 9.27 X 6000\",\"45032-01\",\"KHJDWNEJWKFD9101529.pdf\",\"2012-02-03\",\"1\"\n" +
"\"STRINGVAL2\",\"273.10 X 21.44 X 6000\",\"7-09372\",\"DJSWH68767681540.pdf\",\"2017-02-03\",\"1\"\n";
string pattern = "\\\"\\s*,\\s*\\\"";
string inputline = "";
StringReader reader = new StringReader(input);
XElement root = new XElement("Root");
while ((inputline = reader.ReadLine()) != null)
{
string[] splitLine = Regex.Split(inputline,pattern);
Item newItem = new Item() {
description = splitLine[0].Replace("\"",""),
length = splitLine[1],
type = splitLine[2],
filename = splitLine[3],
date = DateTime.Parse(splitLine[4]),
authorized = splitLine[5].Replace("\"", "") == "1" ? true : false
};
Item.items.Add(newItem);
}
foreach(var year in Item.items.GroupBy(x => x.date.Year).OrderBy(x => x.Key))
{
XElement newYear = new XElement("_" + year.Key.ToString());
root.Add(newYear);
foreach(var month in year.GroupBy(x => x.date.Month).OrderBy(x => x.Key))
{
XElement newMonth = new XElement("_" + month.Key.ToString());
newYear.Add(newMonth);
newMonth.Add(
month.OrderBy(x => x.date).Select(x => new XElement(
x.filename,
string.Join("\r\n", new object[] {
x.description,
x.length,
x.type,
x.date.ToString(),
x.authorized.ToString()
}).ToList()
)));
}
}
}
}
public class Item
{
public static List<Item> items = new List<Item>();
public string description { get; set; }
public string length { get; set; }
public string type { get; set; }
public string filename { get; set; }
public DateTime date { get; set; }
public Boolean authorized { get; set; }
}
}
Given this code:
private ObjectQuery<E> GetEntity()
{ // Pluralization concern. Table and Type need to be consistently named.
// TODO: Don't get cute with database table names. XXX and XXXs for pluralization
return _dc.CreateQuery<E>("[" + typeof(E).Name + "s]");
}
Is there any way to determine an Entity type's plural name so I can access the table, rather than just adding an 's' to the name?
For example, Medium is singular and Media is plural.
You can also use the PluralizationService provided by EF 4. Here is a blog post that covers the service in good detail.
http://web.archive.org/web/20130521044050/http://www.danrigsby.com/blog/index.php/2009/05/19/entity-framework-40-pluralization
I'm not sure how entity framework does this, but I use the pluralizer from Ruby on Rails. You can find this at http://dev.rubyonrails.org/browser/trunk/activesupport/lib/active_support/inflector.rb#L106. This is easy enough to implement in C#.
The entire source for a translation to C# is:
using System;
using System.Collections.Generic;
using System.Text;
using System.Text.RegularExpressions;
public static class Inflector
{
private static List<KeyValuePair<Regex, string>> _pluralRules = new List<KeyValuePair<Regex, string>>();
private static List<KeyValuePair<Regex, string>> _singularRules = new List<KeyValuePair<Regex, string>>();
private static List<KeyValuePair<string, string>> _irregulars = new List<KeyValuePair<string, string>>();
private static List<string> _uncountables = new List<string>();
static Inflector()
{
_uncountables.Add("equipment");
_uncountables.Add("information");
_uncountables.Add("rice");
_uncountables.Add("money");
_uncountables.Add("species");
_uncountables.Add("series");
_uncountables.Add("fish");
_uncountables.Add("sheep");
AddPlural("$", "s", true);
AddPlural("s$", "s");
AddPlural("(ax|test)is$", "$1es");
AddPlural("(octop|vir)us$", "$1i");
AddPlural("(alias|status)$", "$1es");
AddPlural("(bu)s$", "$1ses");
AddPlural("(buffal|tomat)o$", "$1oes");
AddPlural("([ti])um$", "$1a");
AddPlural("sis$", "ses");
AddPlural("(?:([^f])fe|([lr])f)$", "$1$2ves");
AddPlural("(hive)$", "$1s");
AddPlural("([^aeiouy]|qu)y$", "$1ies");
AddPlural("(x|ch|ss|sh)$", "$1es");
AddPlural("(matr|vert|ind)(?:ix|ex)$", "$1ices");
AddPlural("([m|l])ouse$", "$1ice");
AddPlural("^(ox)$", "$1en");
AddPlural("(quiz)$", "$1zes");
AddSingular("s$", "");
AddSingular("(n)ews$", "$1ews");
AddSingular("([ti])a$", "$1um");
AddSingular("((a)naly|(b)a|(d)iagno|(p)arenthe|(p)rogno|(s)ynop|(t)he)ses$", "$1$2sis");
AddSingular("(^analy)ses$", "$1sis");
AddSingular("([^f])ves$", "$1fe");
AddSingular("(hive)s$", "$1");
AddSingular("(tive)s$", "$1");
AddSingular("([lr])ves$", "$1f");
AddSingular("([^aeiouy]|qu)ies$", "$1y");
AddSingular("(s)eries$", "$1eries");
AddSingular("(m)ovies$", "$1ovie");
AddSingular("(x|ch|ss|sh)es$", "$1");
AddSingular("([m|l])ice$", "$1ouse");
AddSingular("(bus)es$", "$1");
AddSingular("(o)es$", "$1");
AddSingular("(shoe)s$", "$1");
AddSingular("(cris|ax|test)es$", "$1is");
AddSingular("(octop|vir)i$", "$1us");
AddSingular("(alias|status)es$", "$1");
AddSingular("^(ox)en", "$1");
AddSingular("(vert|ind)ices$", "$1ex");
AddSingular("(matr)ices$", "$1ix");
AddSingular("(quiz)zes$", "$1");
AddIrregular("person", "people");
AddIrregular("man", "men");
AddIrregular("child", "children");
AddIrregular("sex", "sexes");
AddIrregular("move", "moves");
AddIrregular("cow", "kine");
}
private static void AddIrregular(string singular, string plural)
{
AddPlural(singular.Substring(0, 1).ToLower() + singular.Substring(1) + "$", plural.Substring(0, 1).ToLower() + plural.Substring(1));
AddPlural(singular.Substring(0, 1).ToUpper() + singular.Substring(1) + "$", plural.Substring(0, 1).ToUpper() + plural.Substring(1));
AddSingular(plural.Substring(0, 1).ToLower() + plural.Substring(1) + "$", singular.Substring(0, 1).ToLower() + singular.Substring(1));
AddSingular(plural.Substring(0, 1).ToUpper() + plural.Substring(1) + "$", singular.Substring(0, 1).ToUpper() + singular.Substring(1));
}
private static void AddPlural(string expression, string replacement)
{
AddPlural(expression, replacement, false);
}
private static void AddPlural(string expression, string replacement, bool caseSensitive)
{
var re = caseSensitive ? new Regex(expression) : new Regex(expression, RegexOptions.IgnoreCase);
_pluralRules.Insert(0, new KeyValuePair<Regex, string>(re, replacement));
}
private static void AddSingular(string expression, string replacement)
{
AddSingular(expression, replacement, false);
}
private static void AddSingular(string expression, string replacement, bool caseSensitive)
{
var re = caseSensitive ? new Regex(expression) : new Regex(expression, RegexOptions.IgnoreCase);
_singularRules.Insert(0, new KeyValuePair<Regex, string>(re, replacement));
}
public static string Pluralize(string value)
{
if (_uncountables.Contains(value))
return value;
foreach (var rule in _pluralRules)
{
if (rule.Key.Match(value).Success)
{
return rule.Key.Replace(value, rule.Value);
}
}
return value;
}
public static string Singularize(string value)
{
if (_uncountables.Contains(value))
return value;
foreach (var rule in _singularRules)
{
if (rule.Key.Match(value).Success)
{
return rule.Key.Replace(value, rule.Value);
}
}
return value;
}
public static string Camelize(string value, bool firstLetterUppercase = true)
{
if (firstLetterUppercase)
{
return
Regex.Replace(
Regex.Replace(value, "/(.?)", p => "::" + p.Groups[1].Value.ToUpperInvariant()),
"(?:^|_)(.)", p => p.Groups[1].Value.ToUpperInvariant()
);
}
else
{
return
value.Substring(0, 1).ToLowerInvariant() +
Camelize(value.Substring(1));
}
}
public static string Underscore(string value)
{
value = value.Replace("::", "/");
value = Regex.Replace(value, "([A-Z]+)([A-Z][a-z])", p => p.Groups[1].Value + "_" + p.Groups[2].Value);
value = Regex.Replace(value, "([a-z\\d])([A-Z])", p => p.Groups[1].Value + "_" + p.Groups[2].Value);
value = value.Replace("-", "_");
return value.ToLowerInvariant();
}
}
Have you tried
YourEntityObject.EntityKey.EntitySetName
Assuming your table names are plural.
If the generic method you have takes an entity (that inherits from EntityObject), then you can access the EntityKey from it.
private ObjectQuery<E> GetEntity()
{ // Pluralization concern. Table and Type need to be consistently named.
// TODO: Don't get cute with database table names. XXX and XXXs for pluralization
return _dc.CreateQuery<E>("[" + e.EntityKey.EntitySetName + "]");
}