We Keep Coding

How to import csv file which contains qoute signs and the delimiter is coma?

So i am having a problem importing a csv file, i want to make an object from the columns but i cant read in properly.
So the header line looks like this: Title,Year,Genre,Rating,Votes,Directors
The data line looks like this: The Last of Us: Mass Effect 2,2010,"Action, Adventure, Drama",9.5,19961,Casey Hudson
The problem is that, i get the exception "Input string is not in correct form"
I am using coma as delimiter, is there a way to make quotes as delimiters too?
Also, what are in the qoutes belongs to the Genre attribute.
I am using this code as the CsvParser right now:
using Games.Models;
using System.Globalization;
using System.Text;
namespace Games.Utils
{
public class CsvParser
{
private readonly string _path;
public char Delimiter { get; set; } = ',';
public bool SkipFirst { get; set; } = true;
public bool Verbose { get; set; } = true;
public NumberFormatInfo NumberFormatInfo { get; private set; } = new NumberFormatInfo();
public Encoding Encoding { get; set; } = Encoding.Default;
public CsvParser(string path) => _path = path;
public IEnumerable<Game> StreamParseGames() => GenerateGames(Enumerables.EnumerateStreamReaderLines(new(_path, Encoding)));
public IEnumerable<Game> TextParseGames() => GenerateGames(File.ReadAllLines(_path, Encoding));
private IEnumerable<Game> GenerateGames(IEnumerable<string> lineProvider)
{
if (SkipFirst) lineProvider = lineProvider.Skip(1);
int lineNum = SkipFirst ? 1 : 0;
foreach (var line in lineProvider)
{
string[] parts = line.Split(Delimiter);
Game game;
try
{
game = new()
{
Title = parts[0],
Year = Convert.ToInt32(parts[1], NumberFormatInfo),
Genre = parts[2],
Rating = Convert.ToDouble(parts[3], NumberFormatInfo),
Votes = Convert.ToDouble(parts[4], NumberFormatInfo),
Directors = parts[5],
};
}
catch (FormatException e)
{
if (Verbose) Console.WriteLine($"Line {lineNum + 1:000000} omitted due: {e.Message}");
continue;
}
catch (IndexOutOfRangeException e)
{
if (Verbose) Console.WriteLine($"Line {lineNum + 1:000000} omitted due: {e.Message}");
continue;
}
finally
{
++lineNum;
}
yield return game;
}
}
}
}

I'd suggest you use CsvHelper which can deal with that instead of rolling your own CSV parser.
using CsvHelper;
using CsvHelper.Configuration;
var config = new CsvConfiguration(CultureInfo.InvariantCulture)
{
Delimiter = ",",
};
using (var reader = new StreamReader("path\\to\\file.csv"))
using (var csv = new CsvReader(reader, config))
{
var records = csv.GetRecords<Foo>();
}

How to fix the coding error 'input string was not in a correct format'

Here are the full details of my code:
public partial class Form1 : Form
{
List<Sales> sales = new List<Sales>();
BindingSource bs = new BindingSource();
public Form1()
{
InitializeComponent();
LoadCSV();
bs.DataSource = sales;
dgvSales.DataSource = bs;
}
private void Form1_Load(object sender, EventArgs e)
{
}
private void LoadCSV()
{
string filePath = #"c:\Users\demo\Task3_shop_data.csv";
List<string> lines = new List<string>();
lines = File.ReadAllLines(filePath).ToList();
foreach (string line in lines)
{
List<string> items = line.Split(',').ToList();
Sales s = new Sales();
s.TextBook = items[0];
s.Subject = items[1];
s.Seller = items[2];
s.Purchaser = items[3];
s.purchasedPrice = float.Parse(items[4]);
s.SalePrice = items[6];
s.Rating = items[7];
sales.Add(s);
}
}
}
}
my sales class:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace MichaelSACU301task3
{
internal class Sales
{
public string TextBook { get; set; }
public string Subject { get; set; }
public string Seller { get; set; }
public string Purchaser { get; set; }
public float purchasedPrice { get; set; }
public string SalePrice { get; set; }
public string Rating { get; set; }
}
}
I tried launching it but the error message keeps appearing can someone please help me fix this problem.

Use float.TryParse prior to assigning to purchasedPrice property, if the value can not be converted remember it in a list. In the example below the code to read file data is in a separate class which returns a list of sales and a list of int which is used to remember invalid lines where purchasedPrice data is invalid. You should also consider validating other data and also ensure proper amount of data after performing the line split.
public class FileOperations
{
public static (List<Sales>, List<int>) LoadSalesFromFile()
{
List<Sales> sales = new List<Sales>();
List<int> InvalidLine = new List<int>();
string filePath = #"c:\Users\demo\Task3_shop_data.csv";
List<string> lines = File.ReadAllLines(filePath).ToList();
for (int index = 0; index < lines.Count; index++)
{
var parts = lines[0].Split(',');
// validate purchase price
if (float.TryParse(parts[4], out var purchasePrice))
{
Sales s = new Sales();
s.TextBook = parts[0];
s.Subject = parts[1];
s.Seller = parts[2];
s.Purchaser = parts[3];
s.purchasedPrice = purchasePrice;
s.SalePrice = parts[6];
s.Rating = parts[7];
sales.Add(s);
}
else
{
// failed to convert purchase price
InvalidLine.Add(index);
}
}
return (sales, InvalidLine);
}
}
Call the above code in your form
var (salesList, invalidLines) = FileOperations.LoadSalesFromFile();
if (invalidLines.Count > 0)
{
// use to examine bad lines in file
}
else
{
// uses sales list
}

the error sis probably due the impossiability of float.Parse() parse the items[4] in float
you may track value of items[4] using brake point in VS

Console doesn't write any value

I'm trying to scrape books.toscrape.com everything seems perfect, but it doesn't output anything to console.
I'm sure about the XPaths are correct, and syntax is right.
I don't see any errors, or warnings.
Don't have any clue what i can try for this problem.
using System;
using System.Windows;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using System.Data;
using System.Collections.Generic;
namespace book_scraping
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
DataTable table = new DataTable();
public MainWindow()
{
InitializeComponent();
}
string user_url;
class Book
{
public string Titlex { get; set; }
public string Price { get; set; }
public string Rate { get; set; }
}
public void Scrape()
{
var books = new List<Book>();
IWebDriver driver = new FirefoxDriver();
user_url = Textbox1.Text;
int.TryParse(Textbox2.Text, out var x);
for (int i = 1; i < x; i++)
{
driver.Url = "http://" + user_url + "/catalogue/" + "page-" + i + ".html";
var element = driver.FindElements(By.XPath("//article[#class='product_pod']"));
foreach (var elements in element) {
var book = new Book
{
Titlex = driver.FindElement(By.XPath("//h3/a")).Text,
Price = driver.FindElement(By.XPath("//p[#class='price_color']")).Text,
Rate = driver.FindElement(By.XPath("//article/p")).GetAttribute("class")?.Replace("star-rating ", ""),
};
foreach (var a in books)
{
Console.WriteLine($"{a.Titlex} {a.Price} {a.Rate}");
}
}
}
}
private void Button_Click(object sender, RoutedEventArgs e)
{
Scrape();
}
}
}
I expect the output of title,price,rate as text like
hello world 50 three, or something similar

Your Books list is empty, try this
books.Add(new Book
{
Titlex = driver.FindElement(By.XPath("//h3/a")).Text,
Price = driver.FindElement(By.XPath("//p[#class='price_color']")).Text,
Rate = driver.FindElement(By.XPath("//article/p")).GetAttribute("class")?.Replace("star-rating ", ""),
});
instead of:
var book = new Book
{
Titlex = driver.FindElement(By.XPath("//h3/a")).Text,
Price = driver.FindElement(By.XPath("//p[#class='price_color']")).Text,
Rate = driver.FindElement(By.XPath("//article/p")).GetAttribute("class")?.Replace("star-rating ", ""),
};

How do i properly initialize a variable with a custom type

This is the code in C#:
public bool IsNation(string country)
{
for (int i = 0; i < Nations.Count; i++)
{
if (Nations[i].Name == country)
{
return true;
}
else { return false; }
}return true;
}
In C# you must initialize variables. But what if you made one of your own like this below?
public class WorldMarket
{
public WorldMarket()
{
Nations = new List<NationBuilder>();
}
internal List<NationBuilder> Nations { get; set; }
public void AddToWorldMarket(NationBuilder nation)
{
Nations.Add(nation);
}
The main idea is that from this structure:
- wm
- Nations
- [0]
- Name "USA"
- stockpile
- [0]
- Name "Coal"
- Quantity "quantity"
- Value "value"
- [1] //Same as above
Find the country Name "USA" or whatever name inside this structure with a function that by only inserting a string with a name it outputs {1 or 0}
or True or False (if type == bool).
My attempt is the first code presented in this question. It tries to "travel" the structure and find the Name Tag you input as country using this call.
IsNation(string country); where country can be whatever string input.
Question
If C# wants me to declare every variable with an initial value, how do i do it with this custom or whatever custom type i may do?

Initialize the variable in the constructor public WorkdMarket(). See code below :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
WorldMarket wm = new WorldMarket();
}
}
public class WorldMarket
{
internal List<NationBuilder> Nations { get; set; }
public WorldMarket()
{
Nations = new List<NationBuilder>() {
new NationBuilder() {
name = "USA",
stockPiles = new List<StockPile>() {
new StockPile() { name = "Coal", quantity = 2, value = "value"},
new StockPile() { name = "Coal", quantity = 2, value = "value"}
}
}
};
}
public void AddToWorldMarket(NationBuilder nation)
{
Nations.Add(nation);
}
}
public class NationBuilder
{
public string name { get; set; }
public List<StockPile> stockPiles { get; set; }
}
public class StockPile
{
public string name { get; set; }
public int quantity { get; set; }
public string value { get; set; }
}
}

The line you requested would be:
WorldMarket con = new WorldMarket();
However, this would initialize as a new WorldMarket object, which has no pre-populated values yet. If the nations are meant to be static, you could initialize all the nations within the WorldMarket class
public class WorldMarket
{
public WorldMarket()
{
Nations = new List<NationBuilder>() {
new NationBuilder() {
name = "USA",
...
},
new NationBuilder() {
name = "AUS",
...
}
}
}
}
Or alternatively if you could have your isNation method within WorldMarket, that might work better such that.
public class WorldMarket()
{
// various class constructor methods
public int IsNation(string country)
{
// you could access Nations directly here
for (int i = 0; i < Nations.Count; i++)
{
if (Nations[i].Name == country)
{
return 1;
}
// else { return 0; } -- this would exit the loop unnecessarily
}
return 0;
}
}
and the usage in your main program would be something like
program static void Main(string[] args)
{
WorldMarket con = new WorldMarket();
con.AddToWorldMarket(new NationBuilder() {
name = "USA",
...
}
Console.WriteLine(con.IsNation("USA"));
}
}

Parsing text file using C#

Looking for a good way to parse out of this text file, the values highlighted with the yellow boxes using C#. Each section is delineated by a TERM # which I forgot to highlight. Tried this:
string fileName = "ATMTerminalTotals.txt";
StreamReader sr = new StreamReader(fileName);
string[] delimiter = new string[] { " " };
while (!sr.EndOfStream)
{
string[] lines = sr.ReadLine().Split(delimiter, StringSplitOptions.RemoveEmptyEntries);
foreach (string line in lines)
{
Console.WriteLine(line);
}
}
Console.ReadLine();
Safe to say I am reading lines correctly and removing "white spaces." Although, as an amateur to programming, not sure of a valid way to accurately "know" that I am getting the values from this report that I need. Any advice?

i've tested this with a very simple program to parse the given file,
basically i've created two basic classes, a page class holding a collection of terminal report class (the tran type rows)
these rows maybe even can be represented as transaction and a billing class too
first parsed the data, setting the parameters needed and lastly just accessing the properties
just rushed it to be as simple as possible, no error handling etc... its just to give you a sense of how id start solving these kind of tasks, hope it helps
Adam
namespace TerminalTest
{
class Program
{
public class TerminalReport
{
public string Word { get; set; }
public int Denials { get; set; }
public int Approvals { get; set; }
public int Reversals { get; set; }
public double Amount { get; set; }
public int ON_US { get; set; }
public int Alphalink { get; set; }
public int Interchange { get; set; }
public int Surcharged { get; set; }
public static TerminalReport FromLine(string line)
{
TerminalReport report = new TerminalReport();
report.Word = line.Substring(0, 11);
line = line.Replace(report.Word, string.Empty).Trim();
string[] split = line.Split(' ');
int i = 0;
// transaction summary
report.Denials = int.Parse(split[i++]);
report.Approvals = int.Parse(split[i++]);
report.Reversals = int.Parse(split[i++]);
report.Amount = double.Parse(split[i++]);
// billing counts
report.ON_US = int.Parse(split[i++]);
report.Alphalink = int.Parse(split[i++]);
report.Interchange = int.Parse(split[i++]);
report.Surcharged = int.Parse(split[i++]);
return report;
}
}
public class TerminalPage
{
public int PageNumber { get; set; }
public double TotalSurcharges { get; set; }
public List<TerminalReport> Rows { get; set; }
public TerminalPage(int num)
{
PageNumber = num;
Rows = new List<TerminalReport>();
}
public int TotalDenials
{
get
{
return rows.Sum(r => r.Denials);
}
}
public int TotalApprovals
{
get
{
return Rows.Sum(r => r.Approvals;
}
}
public int TotalReversals
{
get
{
return Rows.Sum(r => r.Reversals;
}
}
public double TotalAmount
{
get
{
return Rows.Sum(r => r.Amount);
}
}
public int TotalON_US
{
get
{
return Rows.Sum(r => r.ON_US);
}
}
public int TotalAlphalink
{
get
{
return Rows.Sum(r => r.Alphalink);
}
}
public int TotalInterchange
{
get
{
return Rows.Sum(r => r.Interchange);
}
}
public int TotalSurcharged
{
get
{
return Rows.Sum(r => r.Surcharged);
}
}
}
private static string CleanString(string text)
{
return Regex.Replace(text, #"\s+", " ").Replace(",", string.Empty).Trim();
}
private static List<TerminalPage> ParseData(string filename)
{
using (StreamReader sr = new StreamReader(File.OpenRead(filename)))
{
List<TerminalPage> pages = new List<TerminalPage>();
int pageNumber = 1;
TerminalPage page = null;
bool parse = false;
while (!sr.EndOfStream)
{
string line = sr.ReadLine();
line = CleanString(line);
if (line.StartsWith("TRAN TYPE"))
{
// get rid of the ----- line
sr.ReadLine();
parse = true;
if (page != null)
{
pages.Add(page);
}
page = new TerminalPage(pageNumber++);
}
else if (line.StartsWith("="))
{
parse = false;
}
else if (line.StartsWith("TOTAL SURCHARGES:"))
{
line = line.Replace("TOTAL SURCHARGES:", string.Empty).Trim();
page.TotalSurcharges = double.Parse(line);
}
else if (parse)
{
TerminalReport r = TerminalReport.FromLine(line);
page.Rows.Add(r);
}
}
if (page != null)
{
pages.Add(page);
}
return pages;
}
}
static void Main(string[] args)
{
string filename = #"C:\bftransactionsp.txt";
List<TerminalPage> pages = ParseData(filename);
foreach (TerminalPage page in pages)
{
Console.WriteLine("TotalSurcharges: {0}", page.TotalSurcharges);
foreach (TerminalReport r in page.Rows)
Console.WriteLine(r.Approvals);
}
}
}
}

I'm not sure I'd split it by spaces actually.. the textfile looks like its split into columns. You might want to read like 10 chars (or whatever the width of the column is) at a time... and I'd parse the whole file into a dictionary so you get entries like
dict["WDL FRM CHK"]["# DENIALS"] = 236
then you can easily retrieve the values you want from there, and if you ever need more values in the future, you've got them.
Alternatively, you can use regexs. You can grab the first value with a regex like
^WDL FRM CHK\s+(?<denials>[0-9,]+)\s+(?<approvals>[0-9,]+)$
using
m.Groups["approvals"]

anyway I recommend you to wrap your StreamReader with using block:
using (StreamReader sr = new StreamReader(fileName))
{
// do stuff
}
Read more on MSDN

Given that it seems to have a standard, regular format, I would use regular expressions. You can check the starting code to figure out what row you're on, then an expression that will parse out the numbers and ignore whitespace will, very likely, be easier than handling it manually.

using System;
using System.Text.RegularExpressions;
namespace ConsoleApplication3
{
class Program
{
static void Main(string[] args)
{
Regex exp = new Regex(#"WDL FRM CHK(\s)+[1-9,]+(\s)+(?<approvals>[1-9,]+)(\s)+");
string str = "WDL FRM CHK 236 1,854 45,465 123 3";
Match match = exp.Match(str);
if (match.Success)
{
Console.WriteLine("Approvals: " + match.Groups["approvals"].Value);
}
Console.ReadLine();
}
}
}
Apdated from the following article to parse one of your numbers:
How to match a pattern by using regular expressions and Visual C#