does anyone know how to read from file to array(container with inheritance) when the line length is different?(I hope language barrier won't make any problems:))
Sportas Skaitymas(Sportas sportas)
{
SportininkasCointainer sportininkai = new SportininkasCointainer();
KomandaContainer komandos = new KomandaContainer();
using (StreamReader reader = new StreamReader("Duomenys.txt"))
{
string line = null;
while ((line = reader.ReadLine()) != null)
{
string[] values;
values = line.Split(';');
string a = values[0];
string b = values[1];
string c = values[2];
string d = values[3];
string e = values[4];
string f = values[5];
string g = values[6];
switch (values.Length)
{
case 7:
Krepsininkas krepsininkas = new Krepsininkas(a, b, c, int.Parse(d), int.Parse(e), int.Parse(f), int.Parse(g));
sportininkai.AddSportinkas(krepsininkas as Krepsininkas);
break;
case 6:
Futbolininkas futbolininkas = new Futbolininkas(a, b, c, int.Parse(d), int.Parse(e), int.Parse(f));
sportininkai.AddSportinkas(futbolininkas as Futbolininkas);
break;
case 4:
Komanda komanda = new Komanda(a, b, c, int.Parse(d));
komandos.AddKomanda(komanda);
break;
}
}
return sportas;
}
I would be very grateful :)
You can use:
string a = values.Length > 0 ? values[0] : "";
string b = values.Length > 1 ? values[1] : "";
Here, string is only set when value is present else it will be ""
Is there a default/official/recommended way to parse CSV files in C#? I don't want to roll my own parser.
Also, I've seen instances of people using ODBC/OLE DB to read CSV via the Text driver, and a lot of people discourage this due to its "drawbacks." What are these drawbacks?
Ideally, I'm looking for a way through which I can read the CSV by column name, using the first record as the header / field names. Some of the answers given are correct but work to basically deserialize the file into classes.
A CSV parser is now a part of .NET Framework.
Add a reference to Microsoft.VisualBasic.dll (works fine in C#, don't mind the name)
using (TextFieldParser parser = new TextFieldParser(#"c:\temp\test.csv"))
{
parser.TextFieldType = FieldType.Delimited;
parser.SetDelimiters(",");
while (!parser.EndOfData)
{
//Process row
string[] fields = parser.ReadFields();
foreach (string field in fields)
{
//TODO: Process field
}
}
}
The docs are here - TextFieldParser Class
P.S. If you need a CSV exporter, try CsvExport (discl: I'm one of the contributors)
CsvHelper (a library I maintain) will read a CSV file into custom objects.
using (var reader = new StreamReader("path\\to\\file.csv"))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
var records = csv.GetRecords<Foo>();
}
Sometimes you don't own the objects you're trying to read into. In this case, you can use fluent mapping because you can't put attributes on the class.
public sealed class MyCustomObjectMap : CsvClassMap<MyCustomObject>
{
public MyCustomObjectMap()
{
Map( m => m.Property1 ).Name( "Column Name" );
Map( m => m.Property2 ).Index( 4 );
Map( m => m.Property3 ).Ignore();
Map( m => m.Property4 ).TypeConverter<MySpecialTypeConverter>();
}
}
Let a library handle all the nitty-gritty details for you! :-)
Check out FileHelpers and stay DRY - Don't Repeat Yourself - no need to re-invent the wheel a gazillionth time....
You basically just need to define that shape of your data - the fields in your individual line in the CSV - by means of a public class (and so well-thought out attributes like default values, replacements for NULL values and so forth), point the FileHelpers engine at a file, and bingo - you get back all the entries from that file. One simple operation - great performance!
In a business application, i use the Open Source project on codeproject.com, CSVReader.
It works well, and has good performance. There is some benchmarking on the link i provided.
A simple example, copied from the project page:
using (CsvReader csv = new CsvReader(new StreamReader("data.csv"), true))
{
int fieldCount = csv.FieldCount;
string[] headers = csv.GetFieldHeaders();
while (csv.ReadNextRecord())
{
for (int i = 0; i < fieldCount; i++)
Console.Write(string.Format("{0} = {1};", headers[i], csv[i]));
Console.WriteLine();
}
}
As you can see, it's very easy to work with.
I know its a bit late but just found a library Microsoft.VisualBasic.FileIO which has TextFieldParser class to process csv files.
Here is a helper class I use often, in case any one ever comes back to this thread (I wanted to share it).
I use this for the simplicity of porting it into projects ready to use:
public class CSVHelper : List<string[]>
{
protected string csv = string.Empty;
protected string separator = ",";
public CSVHelper(string csv, string separator = "\",\"")
{
this.csv = csv;
this.separator = separator;
foreach (string line in Regex.Split(csv, System.Environment.NewLine).ToList().Where(s => !string.IsNullOrEmpty(s)))
{
string[] values = Regex.Split(line, separator);
for (int i = 0; i < values.Length; i++)
{
//Trim values
values[i] = values[i].Trim('\"');
}
this.Add(values);
}
}
}
And use it like:
public List<Person> GetPeople(string csvContent)
{
List<Person> people = new List<Person>();
CSVHelper csv = new CSVHelper(csvContent);
foreach(string[] line in csv)
{
Person person = new Person();
person.Name = line[0];
person.TelephoneNo = line[1];
people.Add(person);
}
return people;
}
[Updated csv helper: bug fixed where the last new line character created a new line]
If you need only reading csv files then I recommend this library: A Fast CSV Reader
If you also need to generate csv files then use this one: FileHelpers
Both of them are free and opensource.
This solution is using the official Microsoft.VisualBasic assembly to parse CSV.
Advantages:
delimiter escaping
ignores Header
trim spaces
ignore comments
Code:
using Microsoft.VisualBasic.FileIO;
public static List<List<string>> ParseCSV (string csv)
{
List<List<string>> result = new List<List<string>>();
// To use the TextFieldParser a reference to the Microsoft.VisualBasic assembly has to be added to the project.
using (TextFieldParser parser = new TextFieldParser(new StringReader(csv)))
{
parser.CommentTokens = new string[] { "#" };
parser.SetDelimiters(new string[] { ";" });
parser.HasFieldsEnclosedInQuotes = true;
// Skip over header line.
//parser.ReadLine();
while (!parser.EndOfData)
{
var values = new List<string>();
var readFields = parser.ReadFields();
if (readFields != null)
values.AddRange(readFields);
result.Add(values);
}
}
return result;
}
I have written TinyCsvParser for .NET, which is one of the fastest .NET parsers around and highly configurable to parse almost any CSV format.
It is released under MIT License:
https://github.com/bytefish/TinyCsvParser
You can use NuGet to install it. Run the following command in the Package Manager Console.
PM> Install-Package TinyCsvParser
Usage
Imagine we have list of Persons in a CSV file persons.csv with their first name, last name and birthdate.
FirstName;LastName;BirthDate
Philipp;Wagner;1986/05/12
Max;Musterman;2014/01/02
The corresponding domain model in our system might look like this.
private class Person
{
public string FirstName { get; set; }
public string LastName { get; set; }
public DateTime BirthDate { get; set; }
}
When using TinyCsvParser you have to define the mapping between the columns in the CSV data and the property in you domain model.
private class CsvPersonMapping : CsvMapping<Person>
{
public CsvPersonMapping()
: base()
{
MapProperty(0, x => x.FirstName);
MapProperty(1, x => x.LastName);
MapProperty(2, x => x.BirthDate);
}
}
And then we can use the mapping to parse the CSV data with a CsvParser.
namespace TinyCsvParser.Test
{
[TestFixture]
public class TinyCsvParserTest
{
[Test]
public void TinyCsvTest()
{
CsvParserOptions csvParserOptions = new CsvParserOptions(true, new[] { ';' });
CsvPersonMapping csvMapper = new CsvPersonMapping();
CsvParser<Person> csvParser = new CsvParser<Person>(csvParserOptions, csvMapper);
var result = csvParser
.ReadFromFile(#"persons.csv", Encoding.ASCII)
.ToList();
Assert.AreEqual(2, result.Count);
Assert.IsTrue(result.All(x => x.IsValid));
Assert.AreEqual("Philipp", result[0].Result.FirstName);
Assert.AreEqual("Wagner", result[0].Result.LastName);
Assert.AreEqual(1986, result[0].Result.BirthDate.Year);
Assert.AreEqual(5, result[0].Result.BirthDate.Month);
Assert.AreEqual(12, result[0].Result.BirthDate.Day);
Assert.AreEqual("Max", result[1].Result.FirstName);
Assert.AreEqual("Mustermann", result[1].Result.LastName);
Assert.AreEqual(2014, result[1].Result.BirthDate.Year);
Assert.AreEqual(1, result[1].Result.BirthDate.Month);
Assert.AreEqual(1, result[1].Result.BirthDate.Day);
}
}
}
User Guide
A full User Guide is available at:
http://bytefish.github.io/TinyCsvParser/
Here is a short and simple solution.
using (TextFieldParser parser = new TextFieldParser(outputLocation))
{
parser.TextFieldType = FieldType.Delimited;
parser.SetDelimiters(",");
string[] headers = parser.ReadLine().Split(',');
foreach (string header in headers)
{
dataTable.Columns.Add(header);
}
while (!parser.EndOfData)
{
string[] fields = parser.ReadFields();
dataTable.Rows.Add(fields);
}
}
Here is my KISS implementation...
using System;
using System.Collections.Generic;
using System.Text;
class CsvParser
{
public static List<string> Parse(string line)
{
const char escapeChar = '"';
const char splitChar = ',';
bool inEscape = false;
bool priorEscape = false;
List<string> result = new List<string>();
StringBuilder sb = new StringBuilder();
for (int i = 0; i < line.Length; i++)
{
char c = line[i];
switch (c)
{
case escapeChar:
if (!inEscape)
inEscape = true;
else
{
if (!priorEscape)
{
if (i + 1 < line.Length && line[i + 1] == escapeChar)
priorEscape = true;
else
inEscape = false;
}
else
{
sb.Append(c);
priorEscape = false;
}
}
break;
case splitChar:
if (inEscape) //if in escape
sb.Append(c);
else
{
result.Add(sb.ToString());
sb.Length = 0;
}
break;
default:
sb.Append(c);
break;
}
}
if (sb.Length > 0)
result.Add(sb.ToString());
return result;
}
}
Some time ago I had wrote simple class for CSV read/write based on Microsoft.VisualBasic library. Using this simple class you will be able to work with CSV like with 2 dimensions array. You can find my class by the following link: https://github.com/ukushu/DataExporter
Simple example of usage:
Csv csv = new Csv("\t");//delimiter symbol
csv.FileOpen("c:\\file1.csv");
var row1Cell6Value = csv.Rows[0][5];
csv.AddRow("asdf","asdffffff","5")
csv.FileSave("c:\\file2.csv");
For reading header only you need is to read csv.Rows[0] cells :)
This code reads csv to DataTable:
public static DataTable ReadCsv(string path)
{
DataTable result = new DataTable("SomeData");
using (TextFieldParser parser = new TextFieldParser(path))
{
parser.TextFieldType = FieldType.Delimited;
parser.SetDelimiters(",");
bool isFirstRow = true;
//IList<string> headers = new List<string>();
while (!parser.EndOfData)
{
string[] fields = parser.ReadFields();
if (isFirstRow)
{
foreach (string field in fields)
{
result.Columns.Add(new DataColumn(field, typeof(string)));
}
isFirstRow = false;
}
else
{
int i = 0;
DataRow row = result.NewRow();
foreach (string field in fields)
{
row[i++] = field;
}
result.Rows.Add(row);
}
}
}
return result;
}
Single source file solution for straightforward parsing needs, useful. Deals with all the nasty edge cases. Such as new line normalization and handling new lines in quoted string literals. Your welcome!
If you CSV file has a header you just read out the column names (and compute column indexes) from the first row. Simple as that.
Note that Dump is a LINQPad method, you might want to remove that if you are not using LINQPad.
void Main()
{
var file1 = "a,b,c\r\nx,y,z";
CSV.ParseText(file1).Dump();
var file2 = "a,\"b\",c\r\nx,\"y,z\"";
CSV.ParseText(file2).Dump();
var file3 = "a,\"b\",c\r\nx,\"y\r\nz\"";
CSV.ParseText(file3).Dump();
var file4 = "\"\"\"\"";
CSV.ParseText(file4).Dump();
}
static class CSV
{
public struct Record
{
public readonly string[] Row;
public string this[int index] => Row[index];
public Record(string[] row)
{
Row = row;
}
}
public static List<Record> ParseText(string text)
{
return Parse(new StringReader(text));
}
public static List<Record> ParseFile(string fn)
{
using (var reader = File.OpenText(fn))
{
return Parse(reader);
}
}
public static List<Record> Parse(TextReader reader)
{
var data = new List<Record>();
var col = new StringBuilder();
var row = new List<string>();
for (; ; )
{
var ln = reader.ReadLine();
if (ln == null) break;
if (Tokenize(ln, col, row))
{
data.Add(new Record(row.ToArray()));
row.Clear();
}
}
return data;
}
public static bool Tokenize(string s, StringBuilder col, List<string> row)
{
int i = 0;
if (col.Length > 0)
{
col.AppendLine(); // continuation
if (!TokenizeQuote(s, ref i, col, row))
{
return false;
}
}
while (i < s.Length)
{
var ch = s[i];
if (ch == ',')
{
row.Add(col.ToString().Trim());
col.Length = 0;
i++;
}
else if (ch == '"')
{
i++;
if (!TokenizeQuote(s, ref i, col, row))
{
return false;
}
}
else
{
col.Append(ch);
i++;
}
}
if (col.Length > 0)
{
row.Add(col.ToString().Trim());
col.Length = 0;
}
return true;
}
public static bool TokenizeQuote(string s, ref int i, StringBuilder col, List<string> row)
{
while (i < s.Length)
{
var ch = s[i];
if (ch == '"')
{
// escape sequence
if (i + 1 < s.Length && s[i + 1] == '"')
{
col.Append('"');
i++;
i++;
continue;
}
i++;
return true;
}
else
{
col.Append(ch);
i++;
}
}
return false;
}
}
Another one to this list, Cinchoo ETL - an open source library to read and write multiple file formats (CSV, flat file, Xml, JSON etc)
Sample below shows how to read CSV file quickly (No POCO object required)
string csv = #"Id, Name
1, Carl
2, Tom
3, Mark";
using (var p = ChoCSVReader.LoadText(csv)
.WithFirstLineHeader()
)
{
foreach (var rec in p)
{
Console.WriteLine($"Id: {rec.Id}");
Console.WriteLine($"Name: {rec.Name}");
}
}
Sample below shows how to read CSV file using POCO object
public partial class EmployeeRec
{
public int Id { get; set; }
public string Name { get; set; }
}
static void CSVTest()
{
string csv = #"Id, Name
1, Carl
2, Tom
3, Mark";
using (var p = ChoCSVReader<EmployeeRec>.LoadText(csv)
.WithFirstLineHeader()
)
{
foreach (var rec in p)
{
Console.WriteLine($"Id: {rec.Id}");
Console.WriteLine($"Name: {rec.Name}");
}
}
}
Please check out articles at CodeProject on how to use it.
This parser supports nested commas and quotes in a column:
static class CSVParser
{
public static string[] ParseLine(string line)
{
List<string> cols = new List<string>();
string value = null;
for(int i = 0; i < line.Length; i++)
{
switch(line[i])
{
case ',':
cols.Add(value);
value = null;
if(i == line.Length - 1)
{// It ends with comma
cols.Add(null);
}
break;
case '"':
cols.Add(ParseEnclosedColumn(line, ref i));
i++;
break;
default:
value += line[i];
if (i == line.Length - 1)
{// Last character
cols.Add(value);
}
break;
}
}
return cols.ToArray();
}//ParseLine
static string ParseEnclosedColumn(string line, ref int index)
{// Example: "b"",bb"
string value = null;
int numberQuotes = 1;
int index2 = index;
for (int i = index + 1; i < line.Length; i++)
{
index2 = i;
switch (line[i])
{
case '"':
numberQuotes++;
if (numberQuotes % 2 == 0)
{
if (i < line.Length - 1 && line[i + 1] == ',')
{
index = i;
return value;
}
}
else if (i > index + 1 && line[i - 1] == '"')
{
value += '"';
}
break;
default:
value += line[i];
break;
}
}
index = index2;
return value;
}//ParseEnclosedColumn
}//class CSVParser
Based on unlimit's post on How to properly split a CSV using C# split() function? :
string[] tokens = System.Text.RegularExpressions.Regex.Split(paramString, ",");
NOTE: this doesn't handle escaped / nested commas, etc., and therefore is only suitable for certain simple CSV lists.
If anyone wants a snippet they can plop into their code without having to bind a library or download a package. Here is a version I wrote:
public static string FormatCSV(List<string> parts)
{
string result = "";
foreach (string s in parts)
{
if (result.Length > 0)
{
result += ",";
if (s.Length == 0)
continue;
}
if (s.Length > 0)
{
result += "\"" + s.Replace("\"", "\"\"") + "\"";
}
else
{
// cannot output double quotes since its considered an escape for a quote
result += ",";
}
}
return result;
}
enum CSVMode
{
CLOSED = 0,
OPENED_RAW = 1,
OPENED_QUOTE = 2
}
public static List<string> ParseCSV(string input)
{
List<string> results;
CSVMode mode;
char[] letters;
string content;
mode = CSVMode.CLOSED;
content = "";
results = new List<string>();
letters = input.ToCharArray();
for (int i = 0; i < letters.Length; i++)
{
char letter = letters[i];
char nextLetter = '\0';
if (i < letters.Length - 1)
nextLetter = letters[i + 1];
// If its a quote character
if (letter == '"')
{
// If that next letter is a quote
if (nextLetter == '"' && mode == CSVMode.OPENED_QUOTE)
{
// Then this quote is escaped and should be added to the content
content += letter;
// Skip the escape character
i++;
continue;
}
else
{
// otherwise its not an escaped quote and is an opening or closing one
// Character is skipped
// If it was open, then close it
if (mode == CSVMode.OPENED_QUOTE)
{
results.Add(content);
// reset the content
content = "";
mode = CSVMode.CLOSED;
// If there is a next letter available
if (nextLetter != '\0')
{
// If it is a comma
if (nextLetter == ',')
{
i++;
continue;
}
else
{
throw new Exception("Expected comma. Found: " + nextLetter);
}
}
}
else if (mode == CSVMode.OPENED_RAW)
{
// If it was opened raw, then just add the quote
content += letter;
}
else if (mode == CSVMode.CLOSED)
{
// Otherwise open it as a quote
mode = CSVMode.OPENED_QUOTE;
}
}
}
// If its a comma seperator
else if (letter == ',')
{
// If in quote mode
if (mode == CSVMode.OPENED_QUOTE)
{
// Just read it
content += letter;
}
// If raw, then close the content
else if (mode == CSVMode.OPENED_RAW)
{
results.Add(content);
content = "";
mode = CSVMode.CLOSED;
}
// If it was closed, then open it raw
else if (mode == CSVMode.CLOSED)
{
mode = CSVMode.OPENED_RAW;
results.Add(content);
content = "";
}
}
else
{
// If opened quote, just read it
if (mode == CSVMode.OPENED_QUOTE)
{
content += letter;
}
// If opened raw, then read it
else if (mode == CSVMode.OPENED_RAW)
{
content += letter;
}
// It closed, then open raw
else if (mode == CSVMode.CLOSED)
{
mode = CSVMode.OPENED_RAW;
content += letter;
}
}
}
// If it was still reading when the buffer finished
if (mode != CSVMode.CLOSED)
{
results.Add(content);
}
return results;
}
For smaller input CSV data LINQ is fully enough.
For example for the following CSV file content:
schema_name,description,utype
"IX_HE","High-Energy data","x"
"III_spectro","Spectrosopic data","d"
"VI_misc","Miscellaneous","f"
"vcds1","Catalogs only available in CDS","d"
"J_other","Publications from other journals","b"
when we read the whole content into single string called data, then
using System;
using System.IO;
using System.Linq;
var data = File.ReadAllText(Path2CSV);
// helper split characters
var newline = Environment.NewLine.ToCharArray();
var comma = ",".ToCharArray();
var quote = "\"".ToCharArray();
// split input string data to lines
var lines = data.Split(newline);
// first line is header, take the header fields
foreach (var col in lines.First().Split(comma)) {
// do something with "col"
}
// we skip the first line, all the rest are real data lines/fields
foreach (var line in lines.Skip(1)) {
// first we split the data line by comma character
// next we remove double qoutes from each splitted element using Trim()
// finally we make an array
var fields = line.Split(comma)
.Select(_ => { _ = _.Trim(quote); return _; })
.ToArray();
// do something with the "fields" array
}
Please help me to fix the piece of code that results in this xml output.
The elements called "Frequency" and "ScheduleType" are all being stuck under the first element of schedule. How do I fix this?
I'm reading data from a CSV type file and then based on the position and value I add the relevant data (or converted values) to the relevant XML values.
You'll see that I use the suggested method where possible but as I have to use an additional CASE statement I get stuck so I revert to using the LAST statement to try and find the correct element to continue from.
<Schedules>
<Schedule Number="1">
<ScheduleName>Cumulative</ScheduleName>
<BackupType>Cinc</BackupType>
<Multiplexing>4</Multiplexing>
<RetentionLevel>infinite</RetentionLevel>
<ResidenceIsSLP>No</ResidenceIsSLP>
<MultipleCopies>No</MultipleCopies>
<ScheduleType />
<Frequency>
<Count>5</Count>
<Unit>Days</Unit>
<Count>2</Count>
<Unit>Weeks</Unit>
<Count>1</Count>
<Unit>Days</Unit>
</Frequency>
<ScheduleType />
<Frequency />
<ScheduleType />
<Frequency />
</Schedule>
<Schedule Number="2">
<ScheduleName>Full</ScheduleName>
<BackupType>Full</BackupType>
<Multiplexing>4</Multiplexing>
<RetentionLevel>infinite</RetentionLevel>
<ResidenceIsSLP>No</ResidenceIsSLP>
<MultipleCopies>No</MultipleCopies>
</Schedule>
<Schedule Number="3">
<ScheduleName>Differential</ScheduleName>
<BackupType>Diff</BackupType>
<Multiplexing>4</Multiplexing>
<RetentionLevel>2 weeks</RetentionLevel>
<ResidenceIsSLP>No</ResidenceIsSLP>
<MultipleCopies>No</MultipleCopies>
</Schedule>
</Schedules>
public XDocument PolicyAttributesSection()
{
XDocument policies = XDocument.Load("C:\\Output Files\\testnbu.xml");
policies.Element("NBUConfig").Add(new XElement("Policies"));
using (TextFieldParser policy = new TextFieldParser("C:\\Input Files\\RAM\\bppllist.txt"))
{
string[] ignoreLines1 = { "NAMES", "KEY", "RCMD", "BCMD", "FOE", "SHAREGROUP", "APPLICATIONDEFINED", "SCHEDFOE" };
policy.TextFieldType = FieldType.Delimited;
policy.Delimiters = new string[] { " ", ",", ";" };
policy.HasFieldsEnclosedInQuotes = true;
policy.CommentTokens = ignoreLines1;
policy.TrimWhiteSpace = true;
string[] policyprops;
int schedulenumber;
while (!policy.EndOfData)
{
schedulenumber = 0;
var pl = policies.Element("NBUConfig").Element("Policies").Elements("Policy");
policyprops = policy.ReadFields();
switch (policyprops[0])
{
case "SCHED":
schedulenumber++;
int scheduletypenumber;
bool schedparsed = int.TryParse((policyprops[2]), out scheduletypenumber);
string scheduletypename = ScheduleType_Lookup[(scheduletypenumber)];
int retlevel;
bool retparsed = int.TryParse((policyprops[5]), out retlevel);
string retentionperiod = (Retention_Lookup[(retlevel)]).ToString();
int isslp;
bool isslpparsed = int.TryParse((policyprops[17]), out isslp);
int copynumber;
bool copynumberparsed = int.TryParse((policyprops[12]), out copynumber);
if (copynumber > 0)
{
copynumber = 1;
}
pl.Last().Element("Schedules").Add(new XElement(("Schedule"), new XAttribute("Number", (schedulenumber)),
(new XElement("ScheduleName", policyprops[1])),
(new XElement("BackupType", (scheduletypename))),
(new XElement("Multiplexing", policyprops[3])),
(new XElement("RetentionLevel", (retentionperiod))),
(new XElement("ResidenceIsSLP", (yesno_lookup[(isslp)]))),
(new XElement("MultipleCopies", (yesno_lookup[(copynumber)])))));
pl.Last().Element("Schedules").Element("Schedule").Add(new XElement(("ScheduleType")));
switch (policyprops[11])
{
case "1":
pl.Last().Element("Schedules").Element("Schedule").Add(new XElement("Calendar", "Enabled"));
pl.Last().Element("Schedules").Element("Schedule").Add
(new XElement("CalendarRetries", "Disabled"));
break;
case "2":
pl.Last().Element("Schedules").Element("Schedule").Add
(new XElement("Calendar", "Enabled"));
pl.Last().Element("Schedules").Element("Schedule").Add
(new XElement("CalendarRetries", "Enabled"));
break;
case "0":
int freqseconds;
bool startparsed = int.TryParse((policyprops[4]), out freqseconds);
var freq = frequency(freqseconds);
pl.Last().Element("Schedules").Element("Schedule").Add
(new XElement("Frequency"));
int count = freq.Item1;
string unit = freq.Item2;
pl.Last().Element("Schedules").Element("Schedule").Element("Frequency").Add
(new XElement("Count", (count).ToString()));
pl.Last().Element("Schedules").Element("Schedule").Element("Frequency").Add
(new XElement("Unit", (unit)));
break;
}
break;
}
}
policies.Save("C:\\Output Files\\testnbu.xml");
return policies;
This statement is the problem:
test.Element("NBUConfig")
.Element("Policies")
.Elements("Policy").Last()
.Elements("Schedules").Last()
.Element("Schedule")
.Add(new XElement(("SchedulesStuff"), (scount)));
You're adding the SchedulesStuff to the first Schedule element within the last Schedules element... instead of adding it to the last Schedule element within the only Schedules element.
All of this code would be much, much simpler - and less error-prone - if you had far less repetitive code. So consider instead:
var policies = new XElement("Policies");
var test = new XDocument(new XElement("NBUConfig", policies));
// Hint: for loops are more idiomatic than while loops here
for (int pcount = 0; pcount < 3; pcount++)
{
var schedules = new XElement("Schedules");
for (int scount = 0; scount < 3; scount++)
{
schedules.Add(new XElement("Schedule",
new XAttribute("Number", scount),
new XElement("ScheduleStuff", scount));
}
policies.Add(new XElement("Policy",
new XElement("PolicyName", pcount),
new XElement("General"),
new XElement("Clients"),
new XElement("IncludeList"),
schedules));
}
I have some XML I am receiving from a server that sometimes has some invalid characters that I would like to remove before deserialization. I have no control over the XML file I receive so I need to check for the invalid characters myself.
Sample XML.....
<PrintStatus>N</PrintStatus>
<CustomerPO> >>>> pearl <<<<< </CustomerPO>
<Description>PO# pearl</Description>
<BranchID>4</BranchID>
<PostDate>
<Date>01/13/2015</Date>
</PostDate>
<ShipDate>
<Date>01/13/2015</Date>
</ShipDate>
As you can see, the customer po section has the invalid characters I need to remove. This sometimes occurs only in certain elements that include user typed data.
Here is my Response code.....
//configure http request
HttpWebRequest httpRequest = WebRequest.Create(url) as HttpWebRequest;
httpRequest.Method = "POST";
//prepare correct encoding for XML serialization
UTF8Encoding encoding = new UTF8Encoding();
//use Xml property to obtain serialized XML data
//convert into bytes using encoding specified above and get length
byte[] bodyBytes = encoding.GetBytes(Xml);
httpRequest.ContentLength = bodyBytes.Length;
//get http request stream for putting XML data into
Stream httpRequestBodyStream = httpRequest.GetRequestStream();
//fill stream with serialized XML data
httpRequestBodyStream.Write(bodyBytes, 0, bodyBytes.Length);
httpRequestBodyStream.Close();
//get http response
HttpWebResponse httpResponse = httpRequest.GetResponse() as HttpWebResponse;
StreamReader httpResponseStream = new StreamReader(httpResponse.GetResponseStream(), System.Text.Encoding.ASCII);
//extract XML from response
string httpResponseBody = httpResponseStream.ReadToEnd();
httpResponseStream.Close();
//ignore everything that isn't XML by removing headers
httpResponseBody = httpResponseBody.Substring(httpResponseBody.IndexOf("<?xml"));
//deserialize XML into ProductInquiryResponse
XmlSerializer serializer = new XmlSerializer(typeof(MyResponseClass));
StringReader responseReader = new StringReader(httpResponseBody);
//return MyResponseClass result
return serializer.Deserialize(responseReader) as MyResponseClass;
Does anyone happen to have any suggestions to check the XML? Should I just check the elements I am concerned with right before the xml string gets deserialized? Or is there a better way?
A general fix for your problem would be to recursively descend the XML, parsing as you go and comparing to the schema for that node. At any point if the input differs from the input expected from the schema, or is malformed in some way, allow an error handler to run to fix the input stream, rolling back to the most recent good state and proceeding forward with the fixed input.
The .Net XmlTextReader class is not flexible enough to do this. However, if you know in advance that from the schema that certain XML Elements cannot have children, then the following will read an XML input stream, and upon encountering an element whose fully qualified name matches the known names of leaf nodes, and "escape" the text of all such nodes:
public enum XmlDoctorStatus
{
NoFixNeeded,
FixMade,
FixFailed
}
public class XmlDoctor
{
internal class XmlFixData
{
public string InitialXml { get; private set; }
public string FixedXml { get; private set; }
public int LineNumber { get; private set; }
public int LinePosition { get; private set; }
public XmlFixData(string initialXml, string fixedXml, int lineNumber, int linePosition)
{
this.InitialXml = initialXml;
this.FixedXml = fixedXml;
this.LineNumber = lineNumber;
this.LinePosition = linePosition;
}
public bool ComesAfter(XmlFixData other)
{
if (LineNumber > other.LineNumber)
return true;
if (LineNumber == other.LineNumber && LinePosition > other.LinePosition)
return true;
return false;
}
}
internal class XmlFixedException : Exception
{
public XmlFixData XmlFixData { get; private set; }
public XmlFixedException(XmlFixData data)
{
this.XmlFixData = data;
}
}
readonly HashSet<XName> childlessNodes;
public string OriginalXml { get; private set; }
public XmlDoctor(string xml, IEnumerable<XName> childlessNodes)
{
if (xml == null)
throw new ArgumentNullException();
this.OriginalXml = xml;
this.childlessNodes = new HashSet<XName>(childlessNodes);
}
List<int> indices = null;
string passXml = string.Empty;
bool inPass = false;
void InitializePass(string xml)
{
if (inPass)
throw new Exception("nested pass");
ClearElementData();
TextHelper.NormalizeLines(xml, out passXml, out indices);
inPass = true;
}
void EndPass()
{
inPass = false;
indices = null;
passXml = string.Empty;
ClearElementData();
}
static int LineNumber(XmlReader reader)
{
return ((IXmlLineInfo)reader).LineNumber;
}
static int LinePosition(XmlReader reader)
{
return ((IXmlLineInfo)reader).LinePosition;
}
// Taken from https://stackoverflow.com/questions/1132494/string-escape-into-xml
public static string XmlEscape(string escaped)
{
var replacements = new KeyValuePair<string, string>[]
{
new KeyValuePair<string,string>("&", "&"),
new KeyValuePair<string,string>("\"", """),
new KeyValuePair<string,string>("'", "'"),
new KeyValuePair<string,string>("<", "<"),
new KeyValuePair<string,string>(">", ">"),
};
foreach (var pair in replacements)
foreach (var index in escaped.IndexesOf(pair.Key, 0).Reverse())
if (!replacements.Any(other => string.Compare(other.Value, 0, escaped, index, other.Value.Length, StringComparison.Ordinal) == 0))
{
escaped = escaped.Substring(0, index) + pair.Value + escaped.Substring(index + 1, escaped.Length - index - 1);
}
return escaped;
}
void HandleNode(XmlReader reader)
{
// Adapted from http://blogs.msdn.com/b/mfussell/archive/2005/02/12/371546.aspx
if (reader == null)
{
throw new ArgumentNullException("reader");
}
switch (reader.NodeType)
{
case XmlNodeType.Element:
HandleStartElement(reader);
if (reader.IsEmptyElement)
{
HandleEndElement(reader);
}
break;
case XmlNodeType.Text:
HandleText(reader);
break;
case XmlNodeType.Whitespace:
case XmlNodeType.SignificantWhitespace:
break;
case XmlNodeType.CDATA:
break;
case XmlNodeType.EntityReference:
break;
case XmlNodeType.XmlDeclaration:
case XmlNodeType.ProcessingInstruction:
break;
case XmlNodeType.DocumentType:
break;
case XmlNodeType.Comment:
break;
case XmlNodeType.EndElement:
HandleEndElement(reader);
break;
}
}
private void HandleText(XmlReader reader)
{
if (string.IsNullOrEmpty(currentElementLocalName) || string.IsNullOrEmpty(currentElementName))
return;
var name = XName.Get(currentElementLocalName, currentElementNameSpace);
if (!childlessNodes.Contains(name))
return;
var lineIndex = LineNumber(reader) - 1;
var charIndex = LinePosition(reader) - 1;
if (lineIndex < 0 || charIndex < 0)
return;
int startIndex = indices[lineIndex] + charIndex;
// Scan forward in the input string until we find either the beginning of a CDATA section or the end of this element.
// Patterns to match: </Name
//
string pattern1 = "</" + currentElementName;
var index1 = FindElementEnd(passXml, startIndex, pattern1);
if (index1 < 0)
return; // BAD XML.
string pattern2 = "<![CDATA[";
var index2 = passXml.IndexOf(pattern2, startIndex);
int endIndex = (index2 < 0 ? index1 : Math.Min(index1, index2));
var text = passXml.Substring(startIndex, endIndex - startIndex);
var escapeText = XmlEscape(text);
if (escapeText != text)
{
if (escapeText != XmlEscape(escapeText))
{
Debug.Assert(escapeText == XmlEscape(escapeText));
throw new InvalidOperationException("Escaping error");
}
string fixedXml = passXml.Substring(0, startIndex) + escapeText + passXml.Substring(endIndex, passXml.Length - endIndex);
throw new XmlFixedException(new XmlFixData(passXml, fixedXml, lineIndex + 1, charIndex + 1));
}
}
static bool IsXmlSpace(char ch)
{
// http://www.w3.org/TR/2000/REC-xml-20001006#NT-S
// [3] S ::= (#x20 | #x9 | #xD | #xA)+
return ch == '\u0020' || ch == '\u0009' || ch == '\u000D' || ch == '\u000A';
}
private static int FindElementEnd(string passXml, int charPos, string tagEnd)
{
while (true)
{
var index = passXml.IndexOf(tagEnd, charPos);
if (index < 0)
return index;
int endPos = index + tagEnd.Length;
if (index + tagEnd.Length >= passXml.Length)
return -1; // Bad xml?
// Now we must have zero or more white space characters and a ">"
while (endPos < passXml.Length && IsXmlSpace(passXml[endPos]))
endPos++;
if (endPos >= passXml.Length)
return -1; // BAD XML;
if (passXml[endPos] == '>')
return index;
index = endPos;
// Spurious ending, keep searching.
}
}
string currentElementName = string.Empty;
string currentElementNameSpace = string.Empty;
string currentElementLocalName = string.Empty;
private void HandleStartElement(XmlReader reader)
{
currentElementName = reader.Name;
currentElementLocalName = reader.LocalName;
currentElementNameSpace = reader.NamespaceURI;
}
private void HandleEndElement(XmlReader reader)
{
ClearElementData();
}
private void ClearElementData()
{
currentElementName = string.Empty;
currentElementNameSpace = string.Empty;
currentElementLocalName = string.Empty;
}
public XmlDoctorStatus TryFix(out string newXml)
{
XmlFixData data = null;
while (true)
{
XmlFixData newData;
var status = TryFixOnePass((data == null ? OriginalXml : data.FixedXml), out newData);
switch (status)
{
case XmlDoctorStatus.FixFailed:
Debug.WriteLine("Could not fix XML");
newXml = OriginalXml;
return XmlDoctorStatus.FixFailed;
case XmlDoctorStatus.FixMade:
if (data != null && !newData.ComesAfter(data))
{
Debug.WriteLine("Warning -- possible infinite loop detected, aborting fix");
newXml = OriginalXml;
return XmlDoctorStatus.FixFailed;
}
data = newData;
break; // Try to fix more
case XmlDoctorStatus.NoFixNeeded:
if (data == null)
{
newXml = OriginalXml;
return XmlDoctorStatus.NoFixNeeded;
}
else
{
newXml = data.FixedXml;
return XmlDoctorStatus.FixMade;
}
}
}
}
XmlDoctorStatus TryFixOnePass(string xml, out XmlFixData data)
{
try
{
InitializePass(xml);
using (var textReader = new StringReader(passXml))
using (XmlReader reader = XmlReader.Create(textReader))
{
while (true)
{
bool read = reader.Read();
if (!read)
break;
HandleNode(reader);
}
}
}
catch (XmlFixedException ex)
{
// Success - a fix was made.
data = ex.XmlFixData;
return XmlDoctorStatus.FixMade;
}
catch (Exception ex)
{
// Failure - the file was not fixed and could not be parsed.
Debug.WriteLine("Fix Failed: " + ex.ToString());
data = null;
return XmlDoctorStatus.FixFailed;
}
finally
{
EndPass();
}
// No fix needed.
data = null;
return XmlDoctorStatus.NoFixNeeded;
}
}
public static class TextHelper
{
public static void NormalizeLines(string text, out string newText, out List<int> lineIndices)
{
var sb = new StringBuilder();
var indices = new List<int>();
using (var sr = new StringReader(text))
{
string line;
while ((line = sr.ReadLine()) != null)
{
indices.Add(sb.Length);
sb.AppendLine(line);
}
}
lineIndices = indices;
newText = sb.ToString();
}
public static IEnumerable<int> IndexesOf(this string str, string value, int startAt)
{
if (str == null)
yield break;
for (int index = startAt, valueLength = value.Length; ; index += valueLength)
{
index = str.IndexOf(value, index);
if (index == -1)
break;
yield return index;
}
}
}
Then use it like:
public static class TestXmlDoctor
{
public static void TestFix()
{
string xml1 = #"<?xml version=""1.0"" encoding=""UTF-8""?>
<MainClass>
<PrintStatus>N</PrintStatus>
<CustomerPO> >>>> pearl <<<<< </CustomerPO>
<Description>PO# pearl</Description>
<BranchID>4</BranchID>
<PostDate>
<Date>01/13/2015</Date>
</PostDate>
<ShipDate>
<Date>01/13/2015</Date>
</ShipDate>
</MainClass>
";
XName[] childlessNodes1 = new XName[]
{
XName.Get("CustomerPO", string.Empty),
};
try
{
TestFix(xml1, childlessNodes1);
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
}
public static string TestFix(string xml, IEnumerable<XName> childlessNodes)
{
string fixedXml;
var status = (new XmlDoctor(xml, childlessNodes).TryFix(out fixedXml));
switch (status)
{
case XmlDoctorStatus.NoFixNeeded:
return xml;
case XmlDoctorStatus.FixFailed:
Debug.WriteLine("Failed to fix xml");
return xml;
case XmlDoctorStatus.FixMade:
Debug.WriteLine("Fixed XML, new XML is as follows:");
Debug.WriteLine(fixedXml);
Debug.WriteLine(string.Empty);
return fixedXml;
default:
Debug.Assert(false, "Unknown fix status " + status.ToString());
return xml;
}
}
}
This with this, your XML fragment can be parsed, and becomes:
<?xml version="1.0" encoding="UTF-8"?>
<MainClass>
<PrintStatus>N</PrintStatus>
<CustomerPO> >>>> pearl <<<<< </CustomerPO>
<Description>PO# pearl</Description>
<BranchID>4</BranchID>
<PostDate>
<Date>01/13/2015</Date>
</PostDate>
<ShipDate>
<Date>01/13/2015</Date>
</ShipDate>
</MainClass>
I am working on a XML reader which shows the result in the labels.
I want to read the node called "Opmerking" which is standing in "Opmerkingen"
A example:
<VertrekkendeTrein>
<RitNummer>4085</RitNummer>
<VertrekTijd>2014-06-13T22:00:00+0200</VertrekTijd>
<EindBestemming>Rotterdam Centraal</EindBestemming>
<TreinSoort>Sprinter</TreinSoort>
<RouteTekst>A'dam Sloterdijk, Amsterdam C., Duivendrecht</RouteTekst>
<Vervoerder>NS</Vervoerder>
<VertrekSpoor wijziging="false">4</VertrekSpoor>
<Opmerkingen>
<Opmerking> Rijdt vandaag niet</Opmerking>
</Opmerkingen>
</VertrekkendeTrein>
"Opmerkingen" is not always there, it is always changing. The code i use now:
XmlNodeList nodeList = xmlDoc.SelectNodes("ActueleVertrekTijden/VertrekkendeTrein/*");
and:
foreach (XmlNode nodelist2 in nodeList)
{
if (i < 17) //4
{
switch (nodelist2.Name)
{
case "VertrekTijd": string kuttijd4 = (nodelist2.InnerText);
var res4 = Regex.Match(kuttijd4, #"\d{1,2}:\d{1,2}").Value;
lblv4.Text = Convert.ToString(res4); break;
case "TreinSoort": lblts4.Text = (nodelist2.InnerText); break;
case "RouteTekst": lblvia4.Text = (nodelist2.InnerText); break;
case "VertrekSpoor": lbls4.Text = (nodelist2.InnerText); i++; break;
}
}
}
How can i read the part "Opmerking" and set it in a case?
I tried it a few times, but it failed.
i also tried:
case "Opmerking": var texeliseeneiland1 = (nodelist2.InnerText); if (texeliseeneiland1 == null) { } else { lblop1.Text = texeliseeneiland1; lblop1.Font = new Font(lblop1.Font.FontFamily, 17); lblop1.Visible = true; picop1.Visible = true; }; break;
Anyone who knows the answer?
Just extend your logic with check whether current node has child nodes and if so, read them and process:
if (nodelist2.HasChildNodes)
{
for (int i=0; i<nodelist2.ChildNodes.Count; i++)
{
var childNode = root.ChildNodes[i];
//do whatever you need to display the contents of the child node.
}
}
Also I have to recommend to consider LinqToXML or at least refactor the code you shared. With LinqToXML is might be as easy as this:
var temp = from remarkNode in nodelist2.Descendants("Opmerking")
select remarkNode.Value;
Somehow load the xml content in an XDocument object and loop through it.
Example: read it from a file
var doc = XDocument.Load("C:/test.xml");
foreach (var xe in doc.Descendants("Opmerking"))
{
var value = xe.Value;
//Do your job with value
}