We Keep Coding

CsvHelper delimeter character same as end of line character

I've run into an issue while parsing some csv-like files that I know how to fix, but like to confirm if that's the appropriate way to do.
The file structure
The file I'm trying to parse has a structure similar to .csv in that it's values are separated with a delimeter (in my case it's |), but different to the ones I've previously seen is that it also has a delimeter at the end of the line, e.g:
Column1|Column2|Column3|
Row1Val1|Row1Val2|Row1Val3|
Row2Val1|Row2Val2|Row2Val3|
The issue
The problem arose when I wrote some unit tests to cover my service that wraps over the CsvHelper library. Apparently there is some issue when I provide the following configuration:
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
Delimiter = "|",
HasHeaderRecord = true,
NewLine = "|\r\n"
};
With the above configuration, csvReader.GetRecords() returns no results. I believe that's because the order of operations for the parser is to first look for columns, then end of line - and it tries to parse empty column without realizing it's actually part of the delimeter.
(I can paste the code for the getRecords call as well, but it's basically generic code taken from examples - the only difference is I'm using System.IO.Abstractions library for easier unit testing)
The attempts to solve the problem
If I remove the NewLine configuration value, parser works fine when reading the file (even if it has end-of-line delimeter character at the end). Then, however, my "write CSV" tests break, since CsvHelper no longer is adding proper line endings to the file.
The question(s)
Is there any way I can configure CsvHelper to cover both cases with one configuration, or should I basically use two different configurations, depending on whether I'm writing to CSV or reading from it? This seems a little bit counter-intuitive for me, since it's basically the same format I'm trying to follow, but different configurations are expected?

You could manually write the empty column for each line and then you could keep the configuration the same for reading and writing.
void Main()
{
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
Delimiter = "|"
};
var records = new List<MyClass>
{
new MyClass {Column1 = "Row1Val1", Column2 = "Row1Val2", Column3 = "Row1Val3"},
new MyClass {Column1 = "Row2Val1", Column2 = "Row2Val2", Column3 = "Row2Val3"}
};
using (var writer = new StreamWriter("file.csv"))
using (var csv = new CsvWriter(writer, config))
{
csv.WriteHeader<MyClass>();
csv.WriteField(string.Empty);
foreach (var record in records)
{
csv.NextRecord();
csv.WriteRecord(record);
csv.WriteField(string.Empty);
}
}
using (var reader = new StreamReader("file.csv"))
using (var csv = new CsvReader(reader, config))
{
var importRecords = csv.GetRecords<MyClass>();
importRecords.Dump();
}
}
public class MyClass
{
public string Column1 { get; set; }
public string Column2 { get; set; }
public string Column3 { get; set; }
}

C# - LinQ - Read text files, group by first column and order by last column?

hi have a text files that contains 3 columns something like this:
contract1;pdf1;63
contract1;pdf2;5
contract1;pdf3;2
contract1;pdf4;00
contract2;pdf1;2
contract2;pdf2;30
contract2;pdf3;5
contract2;pdf4;80
now, i want to write those information into another text files ,and the output will be order put for first the records with the last column in "2,5", something like this:
contract1;pdf3;2
contract1;pdf2;5
contract1;pdf1;63
contract1;pdf4;00
contract2;pdf1;2
contract2;pdf3;5
contract2;pdf2;30
contract2;pdf4;80
how can i do?
thanks

You can use LINQ to group and sort the lines after reading, then put them back together:
var output = File.ReadAllLines(#"path-to-file")
.Select(s => s.Split(';'))
.GroupBy(s => s[0])
.SelectMany(sg => sg.OrderBy(s => s[2] == "2" ? "-" : s[2] == "5" ? "+" : s[2]).Select(sg => String.Join(";", sg)));
Then just write them to a file.

I'm not going to write your program for you, but I would recommend this library for reading and writing delimited files:
https://joshclose.github.io/CsvHelper/getting-started/
When you new up the reader make sure to specify your semi-colon delimiter:
using (var reader = new StreamReader("path\\to\\input_file.csv"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
csv.Configuration.Delimiter = ";";
var records = csv.GetRecords<Row>();
// manipulate the data as needed here
}
Your "Row" class (choose a more appropriate name for clarity) will specify the schema of the flat file. It sounds like you don't have headers? If not, you can specify the Order of each item.
public class Row
{
[Index(1)]
public string MyValue1 { get; set; }
[Index(2)]
public string MyValue2 { get; set; }
}
After reading the data in, you can manipulate it as needed. If the output format is different from the input format, you should convert the input class into an output class. You can use the Automapper library if you would like. However, for a simple project I would suggest to just manually convert the input class into the output class.
Lastly, write the data back out:
using (var writer = new StreamWriter("path\\to\\output_file.csv"))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(records);
}

C# Creating Base Class to Write to CSV file with differnt headers, while reading from an Object [duplicate]

I'm using CsvHelper. To write to a .csv file, I need a header based off of a class.
I write the header manually and it works, but I need to be done automatically when it is read.
All the documentation I can find says to use this expression, writer.WriteHeader<CSVDataFormat>(); but that isn't working because it needs more work done to it.
Here is the class that the header should be based off:
public class CSVDataFormat
{
public string FirstName { get; set; }
public string LastName { get; set; }
public float Wage { get; set; }
}
Here is the code for the reading and writing:
private void ReadCSV(string ogCsvFile)
{
using (var streamReaederfileDir = new StreamReader(#ogCsvFile))
{
using (var streamWriterFileDir = new StreamWriter(Path.Combine(Path.GetDirectoryName(ogCsvFile), "New" + Path.GetFileName(ogCsvFile))))
{
var reader = new CsvReader(streamReaederfileDir);
var writer = new CsvWriter(streamWriterFileDir);
writer.WriteHeader<CSVDataFormat>();
IEnumerable records = reader.GetRecords<CSVDataFormat>().ToList();
foreach (CSVDataFormat record in records)
{
record.Wage = record.Wage + (record.Wage / 10);
writer.WriteField(record.FirstName);
writer.WriteField(record.LastName);
writer.WriteField(record.Wage);
writer.NextRecord();
}
}
}
}
Update
This is the error I am getting when I run my code:
An unhandled exception of type 'CsvHelper.CsvMissingFieldException' occurred in CsvHelper.dll
Additional information: Fields 'FirstName' do not exist in the CSV file.

You may be confused how CSVHelper works. This code handles the write aspect of your read in-write out loop:
List<Employee> empList = new List<Employee>();
empList.Add(new Employee { FirstName = "Ziggy", LastName = "Walters", Wage = 132.50F });
empList.Add(new Employee { FirstName = "Zoey", LastName = "Strand", Wage = 76.50F });
using (StreamWriter sw = new StreamWriter(#"C:\Temp\emp.csv"))
using (CsvWriter cw = new CsvWriter(sw))
{
cw.WriteHeader<Employee>();
foreach (Employee emp in empList)
{
emp.Wage *= 1.1F;
cw.WriteRecord<Employee>(emp);
}
}
CSVWriter implements IDisposable, so I put it into a using block as well.
The wage adjustment is slightly streamlined
Result:
FirstName,LastName,Wage
Ziggy,Walters,145.75
Zoey,Strand,84.15
Write header just writes the first line - the names of the columns/items. Notice that the wages listed are different than what I used to create each one.
For what you are doing, I would read in a typed object in place of iterating the empList. For the error listed in the edit, that means that it could not find a column by that name in the input file (probably because you didnt use the Types overload). The class property names should match the column names exactly (you may also want to configure CSVHelper).
The full in-out loop is only slightly more complex:
using (StreamReader sr = new StreamReader(#"C:\Temp\empIN.csv"))
using (StreamWriter sw = new StreamWriter(#"C:\Temp\empOUT.csv"))
using (CsvWriter cw = new CsvWriter(sw))
using (CsvReader cr = new CsvReader(sr))
{
cw.WriteHeader<Employee>();
var records = cr.GetRecords<Employee>();
foreach (Employee emp in records)
{
emp.Wage *= 1.1F;
cw.WriteRecord<Employee>(emp);
}
}
Results using the output from the first loop as input:
FirstName,LastName,Wage
Ziggy,Walters,160.325
Zoey,Strand,92.565
If there is no header record in the incoming CSV it wont know how to map data to the class. You need to add a map:
public class EmployeeMap : CsvHelper.Configuration.CsvClassMap<Employee>
{
public EmployeeMap()
{
Map(m => m.FirstName).Index(0);
Map(m => m.LastName).Index(1);
Map(m => m.Wage).Index(2);
}
}
Mine is nested inside the Employee class. Then give CSVHelper that map:
... before your try to read from the incoming CSV:
cr.Configuration.RegisterClassMap<Employee.EmployeeMap>();
cw.WriteHeader<Employee>();
...
Now it knows how to map csv columns to the properties in your class.

I believe this exception is from the CsvReader and not the CsvWriter. Default CsvConfiguration expects a header and uses AutoMap to generate a PropertyName_to_Index mapping.
From the documentation you may need to define a map
see mapping

Writing a Header using CsvHelper? C#

I'm using CsvHelper. To write to a .csv file, I need a header based off of a class.
I write the header manually and it works, but I need to be done automatically when it is read.
All the documentation I can find says to use this expression, writer.WriteHeader<CSVDataFormat>(); but that isn't working because it needs more work done to it.
Here is the class that the header should be based off:
public class CSVDataFormat
{
public string FirstName { get; set; }
public string LastName { get; set; }
public float Wage { get; set; }
}
Here is the code for the reading and writing:
private void ReadCSV(string ogCsvFile)
{
using (var streamReaederfileDir = new StreamReader(#ogCsvFile))
{
using (var streamWriterFileDir = new StreamWriter(Path.Combine(Path.GetDirectoryName(ogCsvFile), "New" + Path.GetFileName(ogCsvFile))))
{
var reader = new CsvReader(streamReaederfileDir);
var writer = new CsvWriter(streamWriterFileDir);
writer.WriteHeader<CSVDataFormat>();
IEnumerable records = reader.GetRecords<CSVDataFormat>().ToList();
foreach (CSVDataFormat record in records)
{
record.Wage = record.Wage + (record.Wage / 10);
writer.WriteField(record.FirstName);
writer.WriteField(record.LastName);
writer.WriteField(record.Wage);
writer.NextRecord();
}
}
}
}
Update
This is the error I am getting when I run my code:
An unhandled exception of type 'CsvHelper.CsvMissingFieldException' occurred in CsvHelper.dll
Additional information: Fields 'FirstName' do not exist in the CSV file.

You may be confused how CSVHelper works. This code handles the write aspect of your read in-write out loop:
List<Employee> empList = new List<Employee>();
empList.Add(new Employee { FirstName = "Ziggy", LastName = "Walters", Wage = 132.50F });
empList.Add(new Employee { FirstName = "Zoey", LastName = "Strand", Wage = 76.50F });
using (StreamWriter sw = new StreamWriter(#"C:\Temp\emp.csv"))
using (CsvWriter cw = new CsvWriter(sw))
{
cw.WriteHeader<Employee>();
foreach (Employee emp in empList)
{
emp.Wage *= 1.1F;
cw.WriteRecord<Employee>(emp);
}
}
CSVWriter implements IDisposable, so I put it into a using block as well.
The wage adjustment is slightly streamlined
Result:
FirstName,LastName,Wage
Ziggy,Walters,145.75
Zoey,Strand,84.15
Write header just writes the first line - the names of the columns/items. Notice that the wages listed are different than what I used to create each one.
For what you are doing, I would read in a typed object in place of iterating the empList. For the error listed in the edit, that means that it could not find a column by that name in the input file (probably because you didnt use the Types overload). The class property names should match the column names exactly (you may also want to configure CSVHelper).
The full in-out loop is only slightly more complex:
using (StreamReader sr = new StreamReader(#"C:\Temp\empIN.csv"))
using (StreamWriter sw = new StreamWriter(#"C:\Temp\empOUT.csv"))
using (CsvWriter cw = new CsvWriter(sw))
using (CsvReader cr = new CsvReader(sr))
{
cw.WriteHeader<Employee>();
var records = cr.GetRecords<Employee>();
foreach (Employee emp in records)
{
emp.Wage *= 1.1F;
cw.WriteRecord<Employee>(emp);
}
}
Results using the output from the first loop as input:
FirstName,LastName,Wage
Ziggy,Walters,160.325
Zoey,Strand,92.565
If there is no header record in the incoming CSV it wont know how to map data to the class. You need to add a map:
public class EmployeeMap : CsvHelper.Configuration.CsvClassMap<Employee>
{
public EmployeeMap()
{
Map(m => m.FirstName).Index(0);
Map(m => m.LastName).Index(1);
Map(m => m.Wage).Index(2);
}
}
Mine is nested inside the Employee class. Then give CSVHelper that map:
... before your try to read from the incoming CSV:
cr.Configuration.RegisterClassMap<Employee.EmployeeMap>();
cw.WriteHeader<Employee>();
...
Now it knows how to map csv columns to the properties in your class.

I believe this exception is from the CsvReader and not the CsvWriter. Default CsvConfiguration expects a header and uses AutoMap to generate a PropertyName_to_Index mapping.
From the documentation you may need to define a map
see mapping

Read all values from CSV into a List using CsvHelper

So I've been reading that I shouldn't write my own CSV reader/writer, so I've been trying to use the CsvHelper library installed via nuget. The CSV file is a grey scale image, with the number of rows being the image height and the number columns the width. I would like to read the values row-wise into a single List<string> or List<byte>.
The code I have so far is:
using CsvHelper;
public static List<string> ReadInCSV(string absolutePath)
{
IEnumerable<string> allValues;
using (TextReader fileReader = File.OpenText(absolutePath))
{
var csv = new CsvReader(fileReader);
csv.Configuration.HasHeaderRecord = false;
allValues = csv.GetRecords<string>
}
return allValues.ToList<string>();
}
But allValues.ToList<string>() is throwing a:
CsvConfigurationException was unhandled by user code
An exception of type 'CsvHelper.Configuration.CsvConfigurationException' occurred in CsvHelper.dll but was not handled in user code
Additional information: Types that inherit IEnumerable cannot be auto mapped. Did you accidentally call GetRecord or WriteRecord which acts on a single record instead of calling GetRecords or WriteRecords which acts on a list of records?
GetRecords is probably expecting my own custom class, but I'm just wanting the values as some primitive type or string. Also, I suspect the entire row is being converted to a single string, instead of each value being a separate string.

According to #Marc L's post you can try this:
public static List<string> ReadInCSV(string absolutePath) {
List<string> result = new List<string>();
string value;
using (TextReader fileReader = File.OpenText(absolutePath)) {
var csv = new CsvReader(fileReader);
csv.Configuration.HasHeaderRecord = false;
while (csv.Read()) {
for(int i=0; csv.TryGetField<string>(i, out value); i++) {
result.Add(value);
}
}
}
return result;
}

If all you need is the string values for each row in an array, you could use the parser directly.
var parser = new CsvParser( textReader );
while( true )
{
string[] row = parser.Read();
if( row == null )
{
break;
}
}
http://joshclose.github.io/CsvHelper/#reading-parsing
Update
Version 3 has support for reading and writing IEnumerable properties.

The whole point here is to read all lines of CSV and deserialize it to a collection of objects. I'm not sure why do you want to read it as a collection of strings. Generic ReadAll() would probably work the best for you in that case as stated before. This library shines when you use it for that purpose:
using System.Linq;
...
using (var reader = new StreamReader(path))
using (var csv = new CsvReader(reader))
{
var yourList = csv.GetRecords<YourClass>().ToList();
}
If you don't use ToList() - it will return a single record at a time (for better performance), please read https://joshclose.github.io/CsvHelper/examples/reading/enumerate-class-records

Please try this. This had worked for me.
TextReader reader = File.OpenText(filePath);
CsvReader csvFile = new CsvReader(reader);
csvFile.Configuration.HasHeaderRecord = true;
csvFile.Read();
var records = csvFile.GetRecords<Server>().ToList();
Server is an entity class. This is how I created.
public class Server
{
private string details_Table0_ProductName;
public string Details_Table0_ProductName
{
get
{
return details_Table0_ProductName;
}
set
{
this.details_Table0_ProductName = value;
}
}
private string details_Table0_Version;
public string Details_Table0_Version
{
get
{
return details_Table0_Version;
}
set
{
this.details_Table0_Version = value;
}
}
}

You are close. It isn't that it's trying to convert the row to a string. CsvHelper tries to map each field in the row to the properties on the type you give it, using names given in a header row. Further, it doesn't understand how to do this with IEnumerable types (which string implements) so it just throws when it's auto-mapping gets to that point in testing the type.
That is a whole lot of complication for what you're doing. If your file format is sufficiently simple, which yours appear to be--well known field format, neither escaped nor quoted delimiters--I see no reason why you need to take on the overhead of importing a library. You should be able to enumerate the values as needed with System.IO.File.ReadLines() and String.Split().
//pseudo-code...you don't need CsvHelper for this
IEnumerable<string> GetFields(string filepath)
{
foreach(string row in File.ReadLines(filepath))
{
foreach(string field in row.Split(',')) yield return field;
}
}

static void WriteCsvFile(string filename, IEnumerable<Person> people)
{
StreamWriter textWriter = File.CreateText(filename);
var csvWriter = new CsvWriter(textWriter, System.Globalization.CultureInfo.CurrentCulture);
csvWriter.WriteRecords(people);
textWriter.Close();
}