We Keep Coding

Writing CSV Parser in C#

I recently started work on a project that searches a CSV file for duplicate entries and present the user the option to delete one or both entries.
Simple enough it would seem, however I am having an issue with the function that actually parses the CSV file into memory.
Here is the code in question...
using System;
using System.IO;
using Microsoft.VisualBasic.FileIO;
using System.Collections.Generic;
using System.Windows.Forms;
public List<string[]> parseCSV(string path)
{
List<string[]> parsedData = new List<string[]>();
string[] fields;
TextFieldParser parser = null;
string line = parser.ReadLine();
try
{
/*TextFieldParser*/ parser = new TextFieldParser(#"c:\temp\test.csv");
parser.TextFieldType = FieldType.Delimited;
parser.SetDelimiters(",");
while (!parser.EndOfData)
{
fields = parser.ReadFields();
parsedData.Add(fields);
//Did more stuff here with each field.
}
parser.Close();
}
catch (Exception e)
{
MessageBox.Show(e.Message);
}
return parsedData;
}
For some reason in VS2017 parseCSV is underlined in red in the function declaration.
I can't figure out why this is. I've tried the obvious fixes such as changing the function name from parseCSV to something else but that obviously didn't.

In C# everything is contained in a class, you can't just declare a method within a namespace directly.
using System;
using System.IO;
using Microsoft.VisualBasic.FileIO;
using System.Collections.Generic;
using System.Windows.Forms;
class MyLearningOnlyCsvParser {
public List<Customer_Data> parseCSV(string path)
{
List<Customer_Data> parsedData = new List<Customer_Data>();
string[] fields;
TextFieldParser parser = null;
string line = parser.ReadLine();
try
{
/*TextFieldParser*/ parser = new TextFieldParser(#"c:\temp\test.csv");
parser.TextFieldType = FieldType.Delimited;
parser.SetDelimiters(",");
while (!parser.EndOfData)
{
fields = parser.ReadFields();
// assume the CSV is always with 11 columns
if(fields.length == 11) {
Customer_Data newData = new Customer_Data();
newData.name = fields[0];
newData.company = fields[1];
// assign to the rest of the customer data properties with each fields
parsedData.Add(newData);
}
else {
// error handling of not well formed CSV
}
}
parser.Close();
}
catch (Exception e)
{
MessageBox.Show(e.Message);
}
return parsedData;
}
}

C# Delete line from .txt extension with a changing filename

I am currently trying to make an .exe in c# that I can drag and drop a .txt file onto to remove lines of text that contain the keywords "CM" and/or "Filling". It must be able to overwrite the existing data so there are no new files created. The filename is different every time except for the extension. The data is tab delimited if that has any bearing. I'm aware that there are similar questions to this but I haven't managed to adapt them to suit my needs. Also, I'm very new to this and I've been trying for about a week with no luck.
if (args.Length == 0)
return; // return if no file was dragged onto exe
string text = File.ReadAllText("*.txt");
text = text.Replace("cm", "");
string path = Path.GetDirectoryName(args[0])
+ Path.DirectorySeparatorChar
+ Path.GetFileNameWithoutExtension(args[0])
+ "_unwrapped" + Path.GetExtension(args[0]);
File.WriteAllText("*.txt", text);
\\attempt 1
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Text.RegularExpressions;
namespace ConsoleApp4
{
class Program
{
static void Main(string[] args)
{
string concrete = "CM";
string line;
using (StreamReader reader = new StreamReader(#"C:\\Users\drocc_000\Desktop\1611AN24T99-041805221704.txt"))
{
using (StreamWriter writer = new StreamWriter(#"C:\\Users\drocc_000\Desktop\1611AN24T99-041805221704NEW.txt"))
{
while ((line = reader.ReadLine()) != null)
{
// if (String.Compare(line, yourName) == 0)
// continue;
writer.WriteLine(line.Replace(concrete, ""));
}
}
}
\\attempt 2
Thanks for your time.
Regards,
Danny

You can create a console application with the code below and then drag and drop your text file into the .exe file without opening it.
class Program
{
static void Main(string[] args)
{
if (args.Length > 0 && File.Exists(args[0]))
{
string path = args[0];
EditFile(new List<string>() { "CM", "Filling" }, path);
}
Console.Read();
}
public static void EditFile(List<string> keyWords, string filename)
{
List<string> lines = new List<string>();
using (StreamReader sr = new StreamReader(filename))
{
while (sr.Peek() >= 0)
{
lines.Add(sr.ReadLine());
}
sr.Close();
}
int removedLinesCount = 0;
bool writeline;
using (StreamWriter sw = new StreamWriter(filename))
{
foreach (var line in lines)
{
writeline = true;
foreach (var str in keyWords)
{
if (line.Contains(str))
{
writeline = false;
removedLinesCount++;
break;
}
}
if (writeline)
sw.WriteLine(line);
}
Console.WriteLine(removedLinesCount + " lines removed from the file " + filename);
sw.Close();
}
}
}

Something like this?
using System;
using System.IO;
using System.Linq;
namespace ConsoleApp1
{
internal static class Program
{
private static void Main(string[] args)
{
try
{
// Get the filename from the applications arguments
string filename = args[0];
// Read in all lines in the file.
var linesInFile = File.ReadLines(filename);
// Filter out the lines we don't need.
var linesToKeep = linesInFile.Where(line => !line.Contains("CM") && !line.Contains("Filling")).ToArray();
// Overwrite the file.
File.WriteAllLines(filename, linesToKeep);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
}
}

Reading in first line of csv file and saving to a list without using ',' split

I have a csv file that I need to read in the first line and save it to a List. Only problem is there are commas in some of the text and it is splitting in the middle of a field when I need it not to. Unfortunately I cannot change the data inside so whats there needs to stay. I currently also write the data to csv so I was thinking maybe instead of using a comma I can use a different character. Does anyone know if this is possible? I have been researching but am not coming up with a proper answer. Here is my code below:
using System;
using System.CodeDom;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace TestJSON
{
class Program
{
static void Main()
{
var data = JsonConvert.DeserializeObject<dynamic>(File.ReadAllText(
#"C:\Users\nphillips\workspace\2016R23\UITestAutomation\SeedDataGenerator\src\staticresources\seeddata.resource"));
string fileName = "";
var bundles = data.RecordSetBundles;
foreach (var bundle in bundles)
{
var records = bundle.Records;
foreach (var record in records)
{
var test = record.attributes;
foreach (var testagain in test)
{
// Getting the object Name Ex. Location, Item, etc.
var jprop = testagain as JProperty;
if (jprop != null)
{
fileName = jprop.First.ToString().Split('_')[2]+ ".csv";
}
break;
}
string header = "";
string value = "";
foreach (var child in record)
{
var theChild = child as JProperty;
if (theChild != null && !theChild.Name.Equals("attributes"))
{
header += child.Name + ",";
value += child.Value.ToString() + ",";
}
}
value += "+" + Environment.NewLine;
if (!File.Exists(fileName))
{
header += "+" + Environment.NewLine;
File.WriteAllText(fileName, header);
}
else
{
// Need to read in here
var readCSV = new StreamReader(fileName);
var splits = readCSV.ReadLine();
}
File.AppendAllText(fileName, value);
}
}
}
}
}

You need to know how the file is delimited. I would guess that this file is tab delimited, so split on that instead.
Assuming your line is called myCSVLine... I.E
string seperator = "\t";
string[] splitLine = myCSVLine.Split(seperator.ToCharArray());
splitLine would now have all of your strings, including ones with commas

How to create a generic text file parser for any find of text file?

Want to create a generic text file parser in c# for any find of text file.Actually i have 4 application all 4 getting input data from txt file format but text files are not homogeneous in nature.i have tried fixedwithdelemition.
private static DataTable FixedWidthDiliminatedTxtRead()
{
string[] fields;
StringBuilder sb = new StringBuilder();
List<StringBuilder> lst = new List<StringBuilder>();
DataTable dtable = new DataTable();
ArrayList aList;
using (TextFieldParser tfp = new TextFieldParser(testOCC))
{
tfp.TextFieldType = FieldType.FixedWidth;
tfp.SetFieldWidths(new int[12] { 2,25,8,12,13,5,6,3,10,11,10,24 });
for (int col = 1; col < 13; ++col)
dtable.Columns.Add("COL" + col);
while (!tfp.EndOfData)
{
fields = tfp.ReadFields();
aList = new ArrayList();
for (int i = 0; i < fields.Length; ++i)
aList.Add(fields[i] as string);
if (dtable.Columns.Count == aList.Count)
dtable.Rows.Add(aList.ToArray());
}
}
return dtable;
}
but i feel its very rigid one and really varies application to application making it configgurable .any better way ..
tfp.SetFieldWidths(new int[12] { 2,25,8,12,13,5,6,3,10,11,10,24 });
File nature :
Its a report kind of file .
position of columns are very similar
row data of file id different .
I get this as a reference
http://www.codeproject.com/Articles/11698/A-Portable-and-Efficient-Generic-Parser-for-Flat-F
any other thoughts ?

If the only thing different is the field widths, you could just try sending the field widths in as a parameter:
private static DataTable FixedWidthDiliminatedTxtRead(int[] fieldWidthArray)
{
string[] fields;
StringBuilder sb = new StringBuilder();
List<StringBuilder> lst = new List<StringBuilder>();
DataTable dtable = new DataTable();
ArrayList aList;
using (TextFieldParser tfp = new TextFieldParser(testOCC))
{
tfp.TextFieldType = FieldType.FixedWidth;
tfp.SetFieldWidths(fieldWidthArray);
for (int col = 1; col < 13; ++col)
dtable.Columns.Add("COL" + col);
while (!tfp.EndOfData)
{
fields = tfp.ReadFields();
aList = new ArrayList();
for (int i = 0; i < fields.Length; ++i)
aList.Add(fields[i] as string);
if (dtable.Columns.Count == aList.Count)
dtable.Rows.Add(aList.ToArray());
}
}
return dtable;
}
If you will have more logic to grab the data, you might want to consider defining an interface or abstract class for a GenericTextParser and create concrete implementations for each other file.

Hey I made one of these last week.
I did not write it with the intentions of other people using it so I appologize in advance if its not documented well but I cleaned it up for you. ALSO I grabbed several segments of code from stack overflow so I am not the original author of several pieces of this.
The places you need to edit are the path and pathout and the seperators of text.
char[] delimiters = new char[]
So it searches for part of a word and then grabs the whole word. I used a c# console application for this.
Here you go:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
namespace UniqueListofStringFinder
{
class Program
{
static void Main(string[] args)
{
string path = #"c:\Your Path\in.txt";
string pathOut = #"c:\Your Path\out.txt";
string data = "!";
Console.WriteLine("Current Path In is set to: " + path);
Console.WriteLine("Current Path Out is set to: " + pathOut);
Console.WriteLine(Environment.NewLine + Environment.NewLine + "Input String to Search For:");
Console.Read();
string input = Console.ReadLine();
// Delete the file if it exists.
if (!File.Exists(path))
{
// Create the file.
using (FileStream fs = File.Create(path))
{
Byte[] info =
new UTF8Encoding(true).GetBytes("This is some text in the file.");
// Add some information to the file.
fs.Write(info, 0, info.Length);
}
}
System.IO.StreamReader file = new System.IO.StreamReader(path);
List<string> Spec = new List<string>();
using (StreamReader sr = File.OpenText(path))
{
while (!file.EndOfStream)
{
string s = file.ReadLine();
if (s.Contains(input))
{
char[] delimiters = new char[] { '\r', '\n', '\t', ')', '(', ',', '=', '"', '\'', '<', '>', '$', ' ', '#', '[', ']' };
string[] parts = s.Split(delimiters,
StringSplitOptions.RemoveEmptyEntries);
foreach (string word in parts)
{
if (word.Contains(input))
{
if( word.IndexOf(input) == 0)
{
Spec.Add(word);
}
}
}
}
}
Spec.Sort();
// Open the stream and read it back.
//while ((s = sr.ReadLine()) != null)
//{
// Console.WriteLine(s);
//}
}
Console.WriteLine();
StringBuilder builder = new StringBuilder();
foreach (string s in Spec) // Loop through all strings
{
builder.Append(s).Append(Environment.NewLine); // Append string to StringBuilder
}
string result = builder.ToString(); // Get string from StringBuilder
Program a = new Program();
data = a.uniqueness(result);
int i = a.writeFile(data,pathOut);
}
public string uniqueness(string rawData )
{
if (rawData == "")
{
return "Empty Data Set";
}
List<string> dataVar = new List<string>();
List<string> holdData = new List<string>();
bool testBool = false;
using (StringReader reader = new StringReader(rawData))
{
string line;
while ((line = reader.ReadLine()) != null)
{
foreach (string s in holdData)
{
if (line == s)
{
testBool = true;
}
}
if (testBool == false)
{
holdData.Add(line);
}
testBool = false;
// Do something with the line
}
}
int i = 0;
string dataOut = "";
foreach (string s in holdData)
{
dataOut += s + "\r\n";
i++;
}
// Write the string to a file.
return dataOut;
}
public int writeFile(string dataOut, string pathOut)
{
try
{
System.IO.StreamWriter file = new System.IO.StreamWriter(pathOut);
file.WriteLine(dataOut);
file.Close();
}
catch (Exception ex)
{
dataOut += ex.ToString();
return 1;
}
return 0;
}
}
}

private static DataTable FixedWidthTxtRead(string filename, int[] fieldWidths)
{
string[] fields;
DataTable dtable = new DataTable();
ArrayList aList;
using (TextFieldParser tfp = new TextFieldParser(filename))
{
tfp.TextFieldType = FieldType.FixedWidth;
tfp.SetFieldWidths(fieldWidths);
for (int col = 1; col <= fieldWidths.length; ++col)
dtable.Columns.Add("COL" + col);
while (!tfp.EndOfData)
{
fields = tfp.ReadFields();
aList = new ArrayList();
for (int i = 0; i < fields.Length; ++i)
aList.Add(fields[i] as string);
if (dtable.Columns.Count == aList.Count) dtable.Rows.Add(aList.ToArray());
}
}
return dtable;
}

Here's what I did:
I built a factory for the type of processor needed (based on file type/format), which abstracted the file reader.
I then built a collection object that contained a set of triggers for each field I was interested in (also contained the property name for which this field is destined). This settings collection is loaded in via an XML configuration file, so all I need to change are the settings, and the base parsing process can react to how the settings are configured. Finally I built a reflection wrapper wherein once a field is parsed, the corresponding property on the model object is set.
As the file flowed through, the triggers for each setting evaluated each lines value. When it found what it was set to find (via pattern matching, or column length values) it fired and event that bubbled up and set a property on the model object. I can show some pseudo code if you're interested. It needs some work for efficiency's sake, but I like the concept.

Writing into .txt file without erasing previous data C#

I am trying to split a string in a .txt-file by commas (,) into a string[] and then replacing every item of the string[] to another formula, for example:
"Marko Kostic, Faculty of Technical Sciences, University of Novi Sad,
Trg D. Obradovica 6, 21125 Novi Sad, Serbia"
I want to split this string by commas in between the words and then I want to put every value in separate line like a list and then changing every value with another like "Marko Kostic" to be
<addr-line>Marko Kostic<\addr-line>
The problem is the writer wrote only the last value of string[] and erase the previous values.
Any suggestions?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Text.RegularExpressions;
using Microsoft.Office.Interop;
using Microsoft.Office.Interop.Word;
using System.Diagnostics;
using System.Reflection;
using System.Collections;
using System.Runtime.InteropServices;
namespace AffiliationParser
{
class Program
{
static void Main(string[] args)
{
Microsoft.Office.Interop.Word.Application oWord = new Microsoft.Office.Interop.Word.Application();
object missing = System.Reflection.Missing.Value;
object isVisible = false;
using (StreamReader batch = new StreamReader(#"D:\Developing\REF\AffiliationParser\AffiliationParser\AffiliationParser\bin\Debug\Run.bat"))
{
string bat;
while (!batch.EndOfStream)
{
bat = batch.ReadLine();
// do your processing with batch command
if (bat == "pause")
{
continue;
}
string fpath = bat.Substring(bat.IndexOf(" \""));
string path = fpath.Replace("\"", "").Replace(" ","");
string[] name = Directory.GetFiles(path, "*.txt");
string words = name.Min();
string word = words.Substring(words.LastIndexOf("\\")).Replace("\\", "");
Console.WriteLine("Processing........");
Console.WriteLine(word);
string Npath = path + #"\Arr" + word;
if (File.Exists(Npath))
{
System.Windows.Forms.MessageBox.Show("The file Arr" + word + " alredy exist in " + path);
continue;
}
else
{
File.Copy(words, Npath);
StreamReader temp = new StreamReader(Npath, Encoding.UTF8);
string tempstring = temp.ReadToEnd();
string[] temp3 = tempstring.Split(',');
temp.Close();
foreach (string item in temp3)
{
string Nitem = item.TrimStart().TrimEnd();
//Match MatchCont = Regex.Match(Nitem, #"Afganistan|Albania|Algeria|American\s+Samoa|Andorra|Angola|Anguilla|Antarctica|Antigua\s+and\s+Barbuda|Argentina|Armenia|Aruba|Australia|Austria|Azerbaijan|Bahamas|Bahrain|Bangladesh|Barbados|Belarus|Belgium|Belize|Benin|Bermuda|Bhutan|Bolivia|Bosnia\s+and\s+Herzegovina|Botswana|Bouvet\s+Island|Brazil|British\s+Indian\s+Ocean\s+Territory|Brunei\s+Darussalam|Bulgaria|Burkina\s+Faso|Burundi|Cambodia|Cameroon|Canada|Cape\s+Verde|Cayman\s+Islands|Central\s+African\s+Republic|Chad|Chile|China|Christmas\s+Island|Cocos\s+\(Keeling\)\s+Islands|Colombia|Comoros|Democratic\s+People's\s+Republic\s+of\s+Korea|Democratic\s+Republic\s+of\s+Congo|Cook\s+Islands|Costa\s+Rica|Cote\s+D'Ivoire|Croatia|Cuba|Cyprus|Czech\s+Republic|Republic\s+of\s+Korea|Denmark|Djibouti|Dominica|Dominican\s+Republic|East\s+Timor|Ecuador|Egypt|El\s+Salvador|Equatorial\s+Guinea|Eritrea|Estonia|Ethiopia|Falkland\s+Islands\s+\(Malvinas\)|Faroe\s+Islands|Fiji|Finland|France\s+Metropolitan|France|French\s+Guiana|French\s+Polynesia|French\s+Southern\s+Territories|Gabon|Gambia|Georgia|Germany|Ghana|Gibraltar|Greece|Greenland|Grenadaf|Guadeloupe|Guam|Guatemala|Guinea|Guinea\-Bissau|Guyana|Haiti|Heard\s+Island\s+and\s+McDonald\s+Island|Honduras|Hong\s+Kong|Hungary|Iceland|India|Indonesia|Iran|Iraq|Ireland|Northern\s+Ireland|Isle\s+Of\s+Man|Israel|Italy|Jamaica|Japan|Jordan|Kazakhstan|Kenya|Kiribati|Kuwait|Kyrgyzstan|Lao\s+People'S\s+Democratic\s+Republic|Latvia|Lebanon|Lesotho|Liberia|Libya|Liechtenstein|Lithuania|Luxembourg|Macau|Macedonia|Madagascar|Malawi|Malaysia|Maldives|Mali|Malta|Marshall\s+Islands|Martinique|Mauritania|Mauritius|Mayotte|Mexico|Micronesia|Moldova|Monaco|Mongolia|Montserrat|Morocco|Mozambique|Myanmar|Namibia|Nauru|Nepal|Netherlands\s+Antilles|New\s+Caledonia|New\s+Zealand|Nicaragua|Nigeria|Niger|Niue|Norfolk\s+Island|Northern\s+Mariana\s+Islands|Norway|Oman|Pakistan|Palau|Palestine|Panama|Papua\s+New\s+Guinea|Paraguay|Peru|Philippines|Pitcairn|Poland|Portugal|Puerto\s+Rico|Qatar|Reunion|Romania|Russia|Rwanda|Saint\s+Kitts\s+and\s+Nevis|Saint\s+Lucia|Saint\s+Vincent\s+and\s+The\s+Grenadines|Samoa|San\s+Marino|Sao\s+Tome\s+and\s+Principe|Saudi\s+Arabia|Scotland|Senegal|Serbia|Kosovo|Montenegro|Seychelles|Sierra\s+Leone|Singapore|Slovakia|Slovenia|Solomon\s+Islands|Somalia|South\s+Africa|South\s+Georgia\s+and\s+The\s+South\s+Sandwich\s+Islands|Spain|Sri\s+Lanka|St.\s+Helena|St.\s+Pierre\s+and\s+Miquelon|Sudan|Suriname|Svalbard\s+and\s+Jan\s+Mayen\s+Islands|Swaziland|Sweden|Switzerland|Syria|Taiwan|Tajikistan|Tanzania|Thailand|The\s+Netherlands|Togo|Tokelau|Tonga|Trinidad\s+and\s+Tobago|Tunisia|Turkey|Turkmenistan|Turks\s+and\s+Caicos\s+Islands|Tuvalu|Uganda|Ukraine|United\s+Arab\s+Emirates|UAE|UK|United\s+States\s+Minor\s+Outlying\s+Islands|Uruguay|USA|Uzbekistan|Vanuatu|Vatican\s+City\s+State\s+\(Holy\s+See\)|Venezuela|Vietnam|British\s+Virgin\s+Islands|USA\s+Virgin\s+Islands|Wallis\s+and\s+Futuna\s+Islands|Western\s+Sahara|West\s+Indies|Yemen|Zambia|Zimbabwe|Abkhazia|Afghanistan|Akrotiri\s+and\s+Dhekelia|Aland|Ascension\s+Island|The\s+Bahamas|Brunei|Central\s+Africa|Cocos|Congo|Cote\s+d'lvoire|Czech|Dominican|Falkland\s+Islands|Cambia,\s+The|Grenada|Guemsey|Isle\s+of\s+Man|Jersey|Korea|Laos|Macao|Nagorno\-Karabakh|Netherlands|Northern\s+Cyprus|Pitcaim\s+Islands|Sahrawi\s+Arab\s+Democratic|Saint\-Barthelemy|Saint\s+Helena|Saint\s+Martin|Saint\s+Pierre\s+and\s+Miquelon|Saint\s+Vincent\s+and\s+Grenadines|Samos|Somaliland|South\s+Ossetia|Svalbard|Transnistria|Tristan\s+da\s+Cunha|United\s+Kingdom|Vatican\s+City|Virgin\s+Islands|Wallis\s+and\s+Futuna|Espa�a|Witsch|United\s+States|Prague\s+Czech\s+Republic", RegexOptions.Singleline | RegexOptions.Compiled | RegexOptions.ExplicitCapture | RegexOptions.IgnoreCase);
//if (MatchCont.Success==true)
//{
// MatchCont.Result(#"<country>" + Nitem + #"<\country>");
//}
}
}
}
}
}
}
}

Try to include code in you question, it's not a best practice to simply hand out answers. That being said, you'll want to look at the String.Split method, String.Trim and the File.AppendText method.
Simple ways to do this:
string[] stuff = data.Split(',');
StreamWriter sW = File.AppendText(pathToFile);
foreach(string parts in stuff)
{
sW.WriteLine(parts.Trim());
}
Very, very basic, and not giving you the answer without some work on your part. Good luck!
Here's some references: File.AppendText and String.Trim

string input="a,b,c,d";
string [] parts=input.Split(",",StringSplitOptions.RemoveEmptyEntries);
List<string> output=new List<string>();
foreach(string s in parts)
{
// do sth you like;
var newStr="<abc>"+s+"</abc>";
output.Add(newStr);
}
return output.ToArray();