With code similar to, I receive the exception:
An element with the same key '' already exists in the ExpandoObject
using (var reader = new StreamReader("SampleData.csv"))
using (var csv = new CsvReader(reader))
{
var records = csv.GetRecords<dynamic>();
}
This is simply due to CsvHelper is using the column headers by default as the name of the dynamic object's properties:
It is important to ensure csvReaderConfig.HasHeaderRecord = false; is set or to use another technique such as mapping to a class.
var csvReaderConfig = new Configuration();
csvReaderConfig.HasHeaderRecord = false;
using (var reader = new StreamReader("SampleData.csv"))
using (var csv = new CsvReader(reader, csvReaderConfig))
{
var records = csv.GetRecords<dynamic>();
}
Related
Can anyone help me to understand how we can sort data in csvHelper before we write to string object? I am trying with Key but it is not working in case when there is dynamic data.
string source = "James,Anthony,Mary\n,10,\n21,,212";
using var reader = new StringReader(source);
using var csvReader = new CsvReader(reader, CultureInfo.InvariantCulture);
var records = csvReader.GetRecords<dynamic>().ToList();
var orderedRecords = records.OrderBy(r => r.Key);
using (var writer = new StringWriter())
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(orderedRecords);
Console.WriteLine( writer.ToString());
}
I am getting following exception.
CsvHelper.WriterException: 'An unexpected error occurred.
IWriter state:
Row: 1
Index: 0
HeaderRecord:
1
'
The expected output I am looking for is,
Anthony,James,Mary //--> Header Sorted Alphabetically
,10, //--> First Row Sorted as per Header
21,,212 //--> Second Row Sorted As Per Header
I appreciate your help on this.
After updating the question.
You need to use James, Antony or Mary instead of 'Key' to sort data and the code works:
var source = $"James,Anthony,Mary\n,10,\n21,,212";
using var reader = new StringReader(source);
using var csvReader = new CsvReader(reader, CultureInfo.InvariantCulture);
var records = csvReader.GetRecords<dynamic>().ToList();
var orderedRecords = records.OrderBy(r => r.James);
using (var writer = new StringWriter())
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(orderedRecords);
Console.WriteLine(writer.ToString());
}
P.S. It's very dangerous when you use dynamic to read data, usually we use stable data table.
I am using CsvHelper to read CSV files into Dynamic C# object and I would like to iterate the List<dynamic> using foreach and get property names and values.
FileInfo file = new FileInfo("C:\\Temp\\CSVTest.csv");
List<dynamic> dynObj;
using (var reader = new StreamReader(file.FullName))
using (var csv = new CsvReader(reader))
{
dynObj = csv.GetRecords<dynamic>().ToList();
foreach (var d in dynObj)
{
var properties = d.GetType().GetProperties();
foreach (var property in properties)
{
var PropertyName = property.Name;
var PropetyValue = d.GetType().GetProperty(property.Name).GetValue(d, null);
}
}
}
var properties = d.GetType().GetProperties(); always return 0 but I can see at debug that there are properties.
the CSV file contains this data:
Id,Name,Barnd
1,one,abc
2,two,xyz
Normally, dynamic type has a System.Dynamic.ExpandoObject type object while using it. Its same as a KeyValuePairtype in C#.
Following solution will return list of keys and values from dynamic type.
using (var csv = new CsvReader(reader))
{
dynObj = csv.GetRecords<dynamic>().ToList();
foreach (var d in dynObj)
{
var obj = d as System.Dynamic.ExpandoObject;
if (obj != null)
{
var keys = obj.Select(a => a.Key).ToList();
var values = obj.Select(a => a.Value).ToList();
}
}
}
I've put together a CSV importer which I assume works, though I get this error, how do I allow this column to be null so when it adds it to the table it automatically sets the ID? I've tried:
csv.Configuration.WillThrowOnMissingFields = false;
but it doesn't recognise it, this is the error I get when attempting to upload:
CsvHelper.ValidationException: 'Header matching ['ID'] names at index 0 was not found. If you are expecting some headers to be missing and want to ignore this validation, set the configuration HeaderValidated to null. You can also change the functionality to do something else, like logging the issue.'
[HttpPost]
[ActionName("CreateBulk")]
public ActionResult CreateBulkUpload()
{
object db;
var file = Request.Files["attachmentcsv"];
using (var csv = new CsvReader(new StreamReader(file.InputStream), true))
{
var records = csv.GetRecords<Client>().ToList();
foreach (var item in records)
{
var strip = item.homePage.Replace("https://www.", "").Replace("http://www.", "")
.Replace("https://", "").Replace("http://", "").Replace("www.", "");
string[] URLtests =
{"https://www." + strip, "http://www." + strip, "https://" + strip, "http://" + strip};
string[] Metric = MajesticFunctions.MajesticChecker(URLtests);
var userId = User.Identity.GetHashCode();
var UserTableID = 1;
var newclient = new Client
{
clientN = item.clientN,
homePage = Metric[0],
clientEmail = item.clientEmail,
monthlyQuota = item.monthlyQuota,
TrustFlow = Int32.Parse(Metric[1]),
CitationFlow = Int32.Parse(Metric[2]),
RI = Int32.Parse(Metric[3]),
MJTopicsID = item.MJTopicsID,
UserTableID = UserTableID
};
ViewBag.newdomain = newclient;
return RedirectToAction("Index");
}
}
return RedirectToAction("Index");
}
Did you try out the suggestion mentioned in the error message?
like this?
csv.configuration.HeaderValidated = null;
The developer made some breaking changes this year, so the accepted answer will no longer work.
Instead, you have to create a configuration object in advance and inject it in the constructor:
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
HeaderValidated = null
};
using (var reader = new StreamReader(file))
using (var csv = new CsvReader(reader, config))
Make sure to include both these lines:
csv.Configuration.HeaderValidated = null;
csv.Configuration.MissingFieldFound = null;
I have an excel file which contains lots of data along with icon sets and data bars based on the values in the cell. It looks like this:
I want to import this excel sheet along with the conditional formatting. Is there any library for this?? I went through this http://www.sitecorecleveland.com/resources/blogs-posts/easy_excel_interaction_pt6 but it only imports data not format.
If that's not possible is there code in epplus to have these iconsets in excel sheet. I can have arrows, traffic lights, etc but not these.
I dont think EPP supports custom conditional formatting which are stored as "Workbook Extensions" in the xml of the Excel file. You could copy the xml node of the "extLst" which contains the custom formatting from one worksheet to another. Just make sure there is nothing else beside the cond formatting xml in the node that you do not want copied in which case you will have to select only the child nodes you want.
To test, i created the following excel sheet (temp.xlsx), did a copy.paste of values only and saved to a new file (temp2.xlsx):
Then ran the following and it successfully copied the formatting over:
public void Custom_Condition_Copy_Test()
{
//http://stackoverflow.com/questions/28493050/importing-excel-file-with-all-the-conditional-formatting-rules-to-epplus
//File with custom conditional formatting
var existingFile = new FileInfo(#"c:\temp\temp.xlsx");
//Copy of the file with the conditonal formatting removed
var existingFile2 = new FileInfo(#"c:\temp\temp2.xlsx");
using (var package = new ExcelPackage(existingFile))
using (var package2 = new ExcelPackage(existingFile2))
{
//Make sure there are document element for the source
var worksheet = package.Workbook.Worksheets.First();
var xdoc = worksheet.WorksheetXml;
if (xdoc.DocumentElement == null)
return;
//Make sure there are document element for the destination
var worksheet2 = package2.Workbook.Worksheets.First();
var xdoc2 = worksheet2.WorksheetXml;
if (xdoc2.DocumentElement == null)
return;
//get the extension list node 'extLst' from the ws with the formatting
var extensionlistnode = xdoc
.DocumentElement
.GetElementsByTagName("extLst")[0];
//Create the import node and append it to the end of the xml document
var newnode = xdoc2.ImportNode(extensionlistnode, true);
xdoc2.LastChild.AppendChild(newnode);
package2.Save();
}
}
Might want to put some try's in there but this should get you close.
UPDATE: Based on OPs comment.
If you want to be able to add the custom conditional format without the need of the original file that contains it, I see two options.
Option 1, you do it the more "correct" way and use the DocumentFormat.OpenXml namespace. BUT, this would require you to have the Office Open XML library available which may or may not be so easy depending on the environment you are running this in. You can get it from here http://www.microsoft.com/en-us/download/details.aspx?id=30425 and it comes with a Reflection tool that can generate the code you want which gets you this:
using DocumentFormat.OpenXml.Packaging;
using DocumentFormat.OpenXml.Spreadsheet;
using DocumentFormat.OpenXml;
using X14 = DocumentFormat.OpenXml.Office2010.Excel;
using Excel = DocumentFormat.OpenXml.Office.Excel;
......
WorksheetExtensionList worksheetExtensionList1 = new WorksheetExtensionList();
WorksheetExtension worksheetExtension1 = new WorksheetExtension(){ Uri = "{78C0D931-6437-407d-A8EE-F0AAD7539E65}" };
worksheetExtension1.AddNamespaceDeclaration("x14", "http://schemas.microsoft.com/office/spreadsheetml/2009/9/main");
X14.ConditionalFormattings conditionalFormattings1 = new X14.ConditionalFormattings();
X14.ConditionalFormatting conditionalFormatting1 = new X14.ConditionalFormatting();
conditionalFormatting1.AddNamespaceDeclaration("xm", "http://schemas.microsoft.com/office/excel/2006/main");
X14.ConditionalFormattingRule conditionalFormattingRule1 = new X14.ConditionalFormattingRule(){ Type = ConditionalFormatValues.IconSet, Priority = 2, Id = "{CD6B2710-0474-449D-881A-22CFE15D011D}" };
X14.IconSet iconSet1 = new X14.IconSet(){ IconSetTypes = X14.IconSetTypeValues.FiveArrows, Custom = true };
X14.ConditionalFormattingValueObject conditionalFormattingValueObject1 = new X14.ConditionalFormattingValueObject(){ Type = X14.ConditionalFormattingValueObjectTypeValues.Percent };
Excel.Formula formula1 = new Excel.Formula();
formula1.Text = "0";
conditionalFormattingValueObject1.Append(formula1);
X14.ConditionalFormattingValueObject conditionalFormattingValueObject2 = new X14.ConditionalFormattingValueObject(){ Type = X14.ConditionalFormattingValueObjectTypeValues.Percent };
Excel.Formula formula2 = new Excel.Formula();
formula2.Text = "20";
conditionalFormattingValueObject2.Append(formula2);
X14.ConditionalFormattingValueObject conditionalFormattingValueObject3 = new X14.ConditionalFormattingValueObject(){ Type = X14.ConditionalFormattingValueObjectTypeValues.Percent };
Excel.Formula formula3 = new Excel.Formula();
formula3.Text = "40";
conditionalFormattingValueObject3.Append(formula3);
X14.ConditionalFormattingValueObject conditionalFormattingValueObject4 = new X14.ConditionalFormattingValueObject(){ Type = X14.ConditionalFormattingValueObjectTypeValues.Percent };
Excel.Formula formula4 = new Excel.Formula();
formula4.Text = "60";
conditionalFormattingValueObject4.Append(formula4);
X14.ConditionalFormattingValueObject conditionalFormattingValueObject5 = new X14.ConditionalFormattingValueObject(){ Type = X14.ConditionalFormattingValueObjectTypeValues.Percent };
Excel.Formula formula5 = new Excel.Formula();
formula5.Text = "80";
conditionalFormattingValueObject5.Append(formula5);
X14.ConditionalFormattingIcon conditionalFormattingIcon1 = new X14.ConditionalFormattingIcon(){ IconSet = X14.IconSetTypeValues.ThreeSymbols, IconId = (UInt32Value)0U };
X14.ConditionalFormattingIcon conditionalFormattingIcon2 = new X14.ConditionalFormattingIcon(){ IconSet = X14.IconSetTypeValues.ThreeTrafficLights1, IconId = (UInt32Value)0U };
X14.ConditionalFormattingIcon conditionalFormattingIcon3 = new X14.ConditionalFormattingIcon(){ IconSet = X14.IconSetTypeValues.ThreeTriangles, IconId = (UInt32Value)0U };
X14.ConditionalFormattingIcon conditionalFormattingIcon4 = new X14.ConditionalFormattingIcon(){ IconSet = X14.IconSetTypeValues.ThreeTriangles, IconId = (UInt32Value)1U };
X14.ConditionalFormattingIcon conditionalFormattingIcon5 = new X14.ConditionalFormattingIcon(){ IconSet = X14.IconSetTypeValues.ThreeTriangles, IconId = (UInt32Value)2U };
iconSet1.Append(conditionalFormattingValueObject1);
iconSet1.Append(conditionalFormattingValueObject2);
iconSet1.Append(conditionalFormattingValueObject3);
iconSet1.Append(conditionalFormattingValueObject4);
iconSet1.Append(conditionalFormattingValueObject5);
iconSet1.Append(conditionalFormattingIcon1);
iconSet1.Append(conditionalFormattingIcon2);
iconSet1.Append(conditionalFormattingIcon3);
iconSet1.Append(conditionalFormattingIcon4);
iconSet1.Append(conditionalFormattingIcon5);
conditionalFormattingRule1.Append(iconSet1);
Excel.ReferenceSequence referenceSequence1 = new Excel.ReferenceSequence();
referenceSequence1.Text = "A1:C201";
conditionalFormatting1.Append(conditionalFormattingRule1);
conditionalFormatting1.Append(referenceSequence1);
conditionalFormattings1.Append(conditionalFormatting1);
worksheetExtension1.Append(conditionalFormattings1);
worksheetExtensionList1.Append(worksheetExtension1);
....
worksheet1.Append(worksheetExtensionList1);
Option 2 would be to do as you are asking and perform string manipulation. This is much easier but it is a slightly dirty in that you are messing with strings rather then objects but if the only thing you need to set is the cell range that doesnt seem so bad. I used the test method above to extract the string with = extensionlistnode.OuterXml:
[TestMethod]
public void Custom_Condition_From_String_Test()
{
//http://stackoverflow.com/questions/28493050/importing-excel-file-with-all-the-conditional-formatting-rules-to-epplus
//Throw in some data
var datatable = new DataTable("tblData");
datatable.Columns.Add(new DataColumn("Col1", typeof(int)));
datatable.Columns.Add(new DataColumn("Col2", typeof(int)));
datatable.Columns.Add(new DataColumn("Col3", typeof(int)));
for (var i = 0; i < 20; i++)
{
var row = datatable.NewRow();
row["Col1"] = i;
row["Col2"] = i * 10;
row["Col3"] = i * 100;
datatable.Rows.Add(row);
}
//Copy of the file with the conditonal formatting removed
var existingFile2 = new FileInfo(#"c:\temp\temp2.xlsx");
if (existingFile2.Exists)
existingFile2.Delete();
using (var package2 = new ExcelPackage(existingFile2))
{
//Add the data
var ws = package2.Workbook.Worksheets.Add("Content");
ws.Cells.LoadFromDataTable(datatable, true);
//The XML String extracted from the orginal excel doc using '= extensionlistnode.OuterXml'
var cellrange = "A1:C201";
var rawxml = String.Format(
"<extLst xmlns=\"http://schemas.openxmlformats.org/spreadsheetml/2006/main\"><ext uri=\"{{78C0D931-6437-407d-A8EE-F0AAD7539E65}}\" xmlns:x14=\"http://schemas.microsoft.com/office/spreadsheetml/2009/9/main\"><x14:conditionalFormattings><x14:conditionalFormatting xmlns:xm=\"http://schemas.microsoft.com/office/excel/2006/main\"><x14:cfRule type=\"iconSet\" priority=\"2\" id=\"{{CD6B2710-0474-449D-881A-22CFE15D011D}}\"><x14:iconSet iconSet=\"5Arrows\" custom=\"1\"><x14:cfvo type=\"percent\"><xm:f>0</xm:f></x14:cfvo><x14:cfvo type=\"percent\"><xm:f>20</xm:f></x14:cfvo><x14:cfvo type=\"percent\"><xm:f>40</xm:f></x14:cfvo><x14:cfvo type=\"percent\"><xm:f>60</xm:f></x14:cfvo><x14:cfvo type=\"percent\"><xm:f>80</xm:f></x14:cfvo><x14:cfIcon iconSet=\"3Symbols\" iconId=\"0\" /><x14:cfIcon iconSet=\"3TrafficLights1\" iconId=\"0\" /><x14:cfIcon iconSet=\"3Triangles\" iconId=\"0\" /><x14:cfIcon iconSet=\"3Triangles\" iconId=\"1\" /><x14:cfIcon iconSet=\"3Triangles\" iconId=\"2\" /></x14:iconSet></x14:cfRule><xm:sqref>{0}</xm:sqref></x14:conditionalFormatting></x14:conditionalFormattings></ext></extLst>"
, cellrange);
var newxdoc = new XmlDocument();
newxdoc.LoadXml(rawxml);
//Create the import node and append it to the end of the xml document
var xdoc2 = ws.WorksheetXml;
var newnode = xdoc2.ImportNode(newxdoc.FirstChild, true);
xdoc2.LastChild.AppendChild(newnode);
package2.Save();
}
}
I am building an MVC App in C# and am currently working on a method to load an xml document into a file stream and iterate through the nodes and save the data into my model objects.
It is all working fine but I am hard coding the relationship between the object property and the xml attribute name. I was wondering if there is a smart way to associated the two so I can run it all through a for each loop.
This is the code I have currently and it works, but I would like to make it more generic
OLD CODE
var xmlDoc = LoadXmlFileIntoStream("WSAPayCode.xml");
var elementCollection = ExtractDescendants(xmlDoc, "WSAPayCode");
foreach (var element in elementCollection)
{
var abbreviationChar = element.Attribute("AbbreviationChar");
var payCode = new PayCode
{
Name = element.Attribute("Name").Value,
AutoResolved = element.Attribute("AutoResolved").Value.IsBool(),
EditExcuseAbsn = element.Attribute("EditExcuseAbsn").Value.IsBool(),
PersistPceSw = element.Attribute("PersistPceSw").Value.IsBool(),
AbbreviationChar = (string)element.Attribute("AbbreviationChar"),
EditCntToCdotSw = element.Attribute("EditCntToCdotSw").Value.IsBool(),
EditAffShfTotal = element.Attribute("EditAffShfTotal").Value.IsBool(),
EditCntToOt = element.Attribute("EditCntToOt").Value.IsBool(),
PayUsingWeightedAverageRate = element.Attribute("PayUsingWeightedAverageRate").Value.IsBool(),
RequiresMgrApproval = element.Attribute("RequiresMgrApproval").Value.IsBool(),
WeightedAverageRateIsComputedDaily =
element.Attribute("WeightedAverageRateIsComputedDaily").Value.IsBool(),
JustAutoResExpAsWorked = element.Attribute("JustAutoResExpAsWorked").Value.IsBool(),
AssociatedDurationPayCodeName = element.Attribute("AssociatedDurationPayCodeName").Value,
WeightedAverageRateContributionsUseAnAdjustedRate =
element.Attribute("WeightedAverageRateContributionsUseAnAdjustedRate").Value.IsBool(),
ScheduleHoursType = element.Attribute("ScheduleHoursType").Value,
CheckAvlbltySw = element.Attribute("CheckAvlbltySw").Value.IsBool(),
WageAddition = (string)element.Attribute("WageAddition"),
VisibleInMainArea = element.Attribute("VisibleInMainArea").Value.IsBool(),
IsMoneyCategory = element.Attribute("IsMoneyCategory").Value.IsBool(),
AmountType = element.Attribute("AmountType").Value,
VisibleInReport = element.Attribute("VisibleInReport").Value.IsBool(),
ContributesToWeightedAverageRates =
element.Attribute("ContributesToWeightedAverageRates").Value.IsBool(),
UnjustAutoResExpAsWorked = (bool)element.Attribute("UnjustAutoResExpAsWorked"),
WageMultiply = (string)element.Attribute("WageMultiply"),
Type = (string)element.Attribute("Type"),
VisibleToUser = (bool)element.Attribute("VisibleToUser"),
CustomerId = _customerId,
};
_db.PayCodes.Add(payCode);
_db.SaveChanges();
}
New Code
I have written some code to interate through the xml file and pull out the names of the attributes - Code Below (Also works)
var xmlDoc = LoadXmlFileIntoStream("WSAPayCode.xml");
var elementCollection = ExtractDescendants(xmlDoc, "WSAPayCode");
var nodeAttributes = xmlDoc.Descendants("WSAPayCode").Select(x => x.Attributes());
foreach (var attrs in nodeAttributes)
{
var _attribute = "";
foreach (var attr in attrs)
{
// This successfully reads through each attribute and pulls out the name of the attribut
_attribute = attr.Name.ToString();
}
}
Problem I would like to solve
What I would like to do now is instantiate an object and iterate through the attribute names and save the values to the corresponding property in the object. i.e. replace the OLD code with something that dynamically assigns the values to the object properties.
I would check into using the XML Serializers / Deserializers. On your class, you have provide attributes as to whether the property is an element or attribute, and then let it handle the rest of it for you.
You could set the attributes to match what the XML document is providing, such as if the name is an element, or attribute, etc.
For example:
[XmlRoot("PayCode")]
public class PayCode
{
[XmlElement("Name")
public string Name { get; set;}
....
}
Then, to deserialize to your object:
XmlSerializer serializer = new
XmlSerializer(typeof(PayCode));
FileStream fs = new FileStream(filename, FileMode.Open);
XmlReader reader = XmlReader.Create(fs);
PayCode payCode = (PayCode)serializer.Deserialize(reader);