Related
I am reading a tab-delimited file in C#. I am passing each value to an INSERT statement by using StringBuilder.
My code is as follows:
StringBuilder sCommand = new StringBuilder("INSERT INTO `interactions` (`id`, `id_interaction`) VALUES ");
var lines = File.ReadLines(#"C:\Users\Edges.txt").Take(50).ToArray().Distinct();
List<string> rows = new List<string>();
foreach (var item in lines)
{
string[] values = item.Split("\t");
rows.Add(string.Format("('{0}', '{1}')", MySqlHelper.EscapeString(values[0]), MySqlHelper.EscapeString(values[1])));
var sep = string.Join(",", rows);
sCommand.Append(sep);
}
On breakpoint mode, I have sep holding values: "('6416', '2318'),('84665', '88'),('90', '2339'),('2624', '5371'),('6118', '6774')" which are good.
But when I append the sep to the sCommand string, the string turns out to be: INSERT INTOinteractions(id,id_interaction) VALUES ('6416', '2318')('6416', '2318'),('84665', '88')('6416', '2318'),('84665', '88'),('90', '2339')('6416', '2318'),('84665', '88'),('90', '2339')...
Can someone please help me with this ? Why does this ('6416', '2318')('6416', '2318'),('84665', '88') sequence occur?
To understand the behavior, it would be worth examining the values of rows,sep, and sCommand in each iteration.
Iteration 1
Iteration 2
Iteration 3
Iteration 4
As you can observe, in each iteration, you are adding an string to the List.However, when you append to sCommand, you are joining the entire list and appending. This would mean, you are appending strings, which are already part of the sCommand repeatedly with each iteration.
To fix it, you would need to make following moving following lines out of the loop.
var sep = string.Join(",", rows);
sCommand.Append(sep);
What this does is that you would first create the complete collection within the loop and then, only at the end, appends the values to the StringBuilder.
Code would look like
foreach (var item in lines)
{
string[] values = item.Split('\t');
rows.Add(string.Format("('{0}', '{1}')", values[0], values[1]));
}
var sep = string.Join(",", rows);
sCommand.Append(sep);
sCommand.Append(sep) is being called for each iteration of the foreach loop.
Your string is likely being correctly appended to the end, and the snippet you showed (VALUES ('6416', '2318')('6416', '2318'),('84665', '88')('6416', '2318'),('84665', '88'),('90', '2339')('6416', '2318'),('84665', '88'),('90', '2339')...) may be the result of appending done in earlier iterations of the foreach loop.
It might be worth inspecting your Edges.txt file?
Edit: Anu has the correct explanation - see Anu's answer.
I created a little class for the values in the file. I separated reading the file from the database code; just returning a list to the database code.
class Values
{
public string Val1 { get; set; }
public string Val2 { get; set; }
public Values(string v1,string v2)
{
Val1 = v1;
Val2 = v2;
}
}
private void InsertValues()
{
List<Values> lst = ReadFile();
using (MySqlConnection cn = new MySqlConnection("Your connection string"))
using (MySqlCommand cmd = new MySqlCommand("INSERT INTO `interactions` (`id`, `id_interaction`) VALUES (#id, #idInteraction);", cn))
{
cmd.Parameters.Add("#id", MySqlDbType.VarChar, 50);
cmd.Parameters.Add("#idInteraction", MySqlDbType.VarChar, 100);
cn.Open();
foreach (Values v in lst)
{
cmd.Parameters["#id"].Value = v.Val1;
cmd.Parameters["#idInteraction"].Value = v.Val2;
cmd.ExecuteNonQuery();
}
}
}
private List<Values> ReadFile()
{
List<Values> lst = new List<Values>();
string[] lines = File.ReadAllLines(#"C:\Users\Edges.txt");
foreach (string line in lines)
{
string[] vals = line.Split(',');
Values v = new Values(vals[0].Trim(), vals[1].Trim());
lst.Add(v);
}
return lst;
}
Move the last two lines out of the foreach loop.
rows.Add is in your foreach loop, this adds as many rows as there are lines. So this is the full set you need.
Your append is in the foreach so it is appending the rows it has at that moment:
e.g. I have the numbers 1,2,3,4,5,6,7, in the foreach I add it as rows, the first time I will have 1, this is appended, my string contains "1".
The second time around we will have 1 & 2, this is appended. my string contains now 1,1,2 (1 from last time and 1,2 now.
the third time the rows have 1,2 & 3. the string is 1,1,2,1,2,3
The var sep is adding in
I need to write a CSV Parser I am now trying to separat the fields to manipulate them.
Sample CSV:
mitarbeiter^tagesdatum^lohnart^kostenstelle^kostentraeger^menge^betrag^belegnummer
11005^23.01.2018^1^^31810020^5,00^^
11081^23.01.2018^1^^31810020^5,00^^
As you can see, there a several empty cells.
I am doing the following:
using (CsvFileReader reader = new CsvFileReader(path))
{
CsvRow row = new CsvRow();
while (reader.ReadRow(row))
{
foreach (string s in row)
{
csvROW.Add(new aCSVROW());
string[] items = s.Split(new char[] { '^' }, StringSplitOptions.None);
csvROW[0].mitarbeiter = items[0];
csvROW[0].tagesdatum = items[1];
csvROW[0].lohnart = items[2];
csvROW[0].kostenstelle = items[3];
csvROW[0].kostentraeger = items[4];
csvROW[0].menge = items[5];
csvROW[0].betrag = items[6];
csvROW[0].belegnummer = items[7];
}
}
}
Problem:
It seems that Split stops after the comma (5,00). The separator is ^ ... is there a reason why?
I tried several things without success...
Thank you so much!
CsvFileReader reads rows from a CSV file and then strings within that row. What else do you expect the CsvFileReader to do than separating the row?
After reading the second line, row will have the contents
11005^23.01.2018^1^^31810020^5
and
00^^
When you split the first row by ^, the last entry of the resulting array will be "5". Anyway, your code will throw, because you are trying to access items exceeding the bounds of the array.
I don't know CsvFileReader. Maybe you can pass ^ as a separator and spare the splitting of the string. Anyway, you could use a StreamReader, too. This will work much more like you expected.
using (StreamReader reader = new StreamReader(path))
{
while (!reader.EndOfStream)
{
var csvLine = reader.ReadLine();
csvROW.Add(new aCSVROW());
string[] items = csvLine.Split(new char[] { '^' }, StringSplitOptions.None);
csvROW[0].mitarbeiter = items[0];
csvROW[0].tagesdatum = items[1];
csvROW[0].lohnart = items[2];
csvROW[0].kostenstelle = items[3];
csvROW[0].kostentraeger = items[4];
csvROW[0].menge = items[5];
csvROW[0].betrag = items[6];
csvROW[0].belegnummer = items[7];
}
}
Is CsvRow meant to be the data of all rows, or of one row? Because as it is, you keep adding a new aCSVROW object into csvROW for each read line, but you keep replacing the data on just csvROW[0], the first inserted aCSVROW. This means that in the end, you will have a lot of rows that all have no data in them, except for the one on index 0, that had its properties overwritten on each iteration, and ends up containing the data of the last read row.
Also, despite using a CsvReader class, you are using plain normal String.Split to actually separate the fields. Surely that's what the CsvReader class is for?
Personally, I always use the TextFieldParser, from the Microsoft.VisualBasic.FileIO namespace. It has the advantage it's completely native in the .Net framework, and you can simply tell it which separator to use.
This function can get the data out of it as simple List<String[]>:
A:
Using C# to search a CSV file and pull the value in the column next to it
Once you have your data, you can paste it into objects however you want.
List<String[]> lines = SplitFile(path, textEncoding, "^");
// I assume "CsvRow" is some kind of container for multiple rows?
// Looks like pretty bad naming to me...
CsvRow allRows = new CsvRow();
foreach (String items in lines)
{
// Create new object, and add it to list.
aCSVROW row = new aCSVROW();
csvROW.Add(row);
// Fill the actual newly created object, not the first object in allRows.
// conside adding index checks here though to avoid index out of range exceptions.
row.mitarbeiter = items[0];
row.tagesdatum = items[1];
row.lohnart = items[2];
row.kostenstelle = items[3];
row.kostentraeger = items[4];
row.menge = items[5];
row.betrag = items[6];
row.belegnummer = items[7];
}
// Done. All rows added to allRows.
CsvRow row = new CsvRow();
while (reader.ReadRow(row))
{
foreach (string s in row)
{
csvROW.Add(new aCSVROW());
s.Split("^","");
csvROW[0].mitarbeiter = items[0];
csvROW[0].tagesdatum = items[1];
csvROW[0].lohnart = items[2];
csvROW[0].kostenstelle = items[3];
csvROW[0].kostentraeger = items[4];
csvROW[0].menge = items[5];
csvROW[0].betrag = items[6];
csvROW[0].belegnummer = items[7];
}
}
}
I need to send some pdf files as email attachments stored in a database.
I need convert comma separated string into a List<string> and i have tried with this c# code:
if (reader.HasRows)
{
while (reader.Read())
{
RecoveryAtt = reader["Att"].ToString();
string Commaseplist;
string[] itemList = { RecoveryAtt.ToString() };
Commaseplist = String.Join("; ", itemList);
Response.Write(Commaseplist.ToString());
}
}
But in output I have:
D:\inetpub\wwwroot\app\public\015.pdfD:\inetpub\wwwroot\app\public\016.pdfD:\inetpub\wwwroot\app\public\017.pdfD:\inetpub\wwwroot\app\public\018.pdfD:\inetpub\wwwroot\app\public\019.pdf
And in email attachment is first only Pdf file...
This doesn't work cause returns a string[] without separator.
Please help.
I think this will do the trick
string[] itemList = Regex.Split(RecoveryAtt, #"(?=D:)").Where(x => !string.IsNullOrEmpty(x)).ToArray();
Commaseplist = String.Join("; ", itemList);
Response.Write(Commaseplist);
You are re-creating the itemList array and the Commaseplist string every time you read a new row in the database and so loosing previous data. The output is being outputted once a row and appended - you are not adding to the array.
Perhaps a better approach would be along the lines of:
if (reader.HasRows)
{
StringBuilder commaList = new StringBuilder();
while (reader.Read())
{
commaList.Append(reader["Att"].ToString());
commaList.Append(",");
}
commaList.Remove(commaList.Lenth-1,1);
Response.Write(commaList.ToString());
}
I'm trying to generate Item IDs using StreamReader on my .CSV file (It has to be a .csv file). The Item ID should start at 1000 and go up (1001, 1002, etc.)
Right now, if the user presses "Generate ID", it will search the entire file for the value "1000", if it doesn't exist, it will write "1000" in the textbox.
Here's what I need help with: If the file contains "1000", I want it to read the LAST line, increase it by 1, then write the value in the textbox.. So, if my last value is 1005 in the .csv file, I want it to write 1006 in the textbox.
private void GenerateID_Click(object sender, EventArgs e)
{
try
{
string searchString = "1000";
using (StreamReader sr = new StreamReader("file.csv"))
{
string line;
while ((line = sr.ReadLine()) != null)
{
if (line.Contains(searchString))
{
/* If file contains 1000, read the LAST line
* (Whatever number that may be: 1001, 1002, 1003, etc.)
* and increase that number by 1, then write to textbox. */
}
else
{
invItemIDField.Text = Convert.ToString("1000");
}
}
}
}
catch (Exception)
{
MessageBox.Show("The file could not be read");
}
}
I suggest you use FileHelpers. It's the most suitable library for reading CSV files.
To install this, you need to install first NuGet. Once installed, go to Tools > Library Package Manager > Package Manager Console:
Then, type in: Install-Package Filehelpers
You're good to go!
Import FileHelpers to your code
using FileHelpers;
Create a class that describes the structure of your CSV:
DelimitedRecord("'")]
public class MyCsv
{
public int Column1; // Your ID column
public string SomeOtherColumn;
}
Create a List<MyCsv>:
List<MyCsv> myList;
Then, to load your CSV:
FileHelperEngine<MyCsv> engine = new FileHelperEngine<MyCsv>();
myList = new List<MyCsv>(engine.ReadFile("my.csv")); // replace with your filename or variable containing the filename
You can now read your CSV by accessing the list myList:
foreach(MyCsv line in myList) {
// Do something;
}
Each object inside that list corresponds to each row in your CSV. In order to access the first column of a row (given the foreach loop above):
line.Column1
So, if you need to compare values, you can either use LINQ or do the traditional loop-search:
foreach(MyCsv line in myList) {
if (txtId.Text == line.Column1.ToString()) {
found = true;
break;
}
}
Then, to get the id of the last row:
myList.[myList.Count - 1].Column1
You can do the rest. Cheers!
Here's my go at it, it's slighlty different from yours, but it works. Granted there are things you must consider, such as are the elements surrounded in quotes, are the line breaks \r\n, and the like:
int TextBoxValue = 1000;
var reader = new StreamReader(File.OpenRead(#"C:\Users\J\Desktop\New Text Document (4).txt"));
var contents = reader.ReadToEnd().Split(new string[] {"\r\n"}, StringSplitOptions.None);
var iValueExists = (from String sLine in contents
where sLine.Contains("1000")
select sLine).Count();
if (iValueExists > 0)
{
TextBoxValue = int.Parse(contents.Last().Split(new string[] {","}, StringSplitOptions.None).First()) + 1;
}
invItemIDField.Text = TextBoxValue;
reader.Close();
As of now, I am using this code to open a file and read it into a list and parse that list into a string[]:
string CP4DataBase =
"C:\\Program\\Line Balancer\\FUJI DB\\KTS\\KTS - CP4 - Part Data Base.txt";
CP4DataBaseRTB.LoadFile(CP4DataBase, RichTextBoxStreamType.PlainText);
string[] splitCP4DataBaseLines = CP4DataBaseRTB.Text.Split('\n');
List<string> tempCP4List = new List<string>();
string[] line1CP4Components;
foreach (var line in splitCP4DataBaseLines)
tempCP4List.Add(line + Environment.NewLine);
string concattedUnitPart = "";
foreach (var line in tempCP4List)
{
concattedUnitPart = concattedUnitPart + line;
line1CP4PartLines++;
}
line1CP4Components = new Regex("\"UNIT\",\"PARTS\"", RegexOptions.Multiline)
.Split(concattedUnitPart)
.Where(c => !string.IsNullOrEmpty(c)).ToArray();
I am wondering if there is a quicker way to do this. This is just one of the files I am opening, so this is repeated a minimum of 5 times to open and properly load the lists.
The minimum file size being imported right now is 257 KB. The largest file is 1,803 KB. These files will only get larger as time goes on as they are being used to simulate a database and the user will continually add to them.
So my question is, is there a quicker way to do all of the above code?
EDIT:
***CP4***
"UNIT","PARTS"
"BLOCK","HEADER-"
"NAME","106536"
"REVISION","0000"
"DATE","11/09/03"
"TIME","11:10:11"
"PMABAR",""
"COMMENT",""
"PTPNAME","R160805"
"CMPNAME","R160805"
"BLOCK","PRTIDDT-"
"PMAPP",1
"PMADC",0
"ComponentQty",180
"BLOCK","PRTFORM-"
"PTPSZBX",1.60
"PTPSZBY",0.80
"PTPMNH",0.25
"NeedGlue",0
"BLOCK","TOLEINF-"
"PTPTLBX",0.50
"PTPTLBY",0.40
"PTPTLCL",10
"PTPTLPX",0.30
"PTPTLPY",0.30
"PTPTLPQ",30
"BLOCK","ELDT+" "PGDELSN","PGDELX","PGDELY","PGDELPP","PGDELQ","PGDELP","PGDELW","PGDELL","PGDELWT","PGDELLT","PGDELCT","PGDELR"
0,0.000,0.000,0,0,0.000,0.000,0.000,0.000,0.000,0.000,0
"BLOCK","VISION-"
"PTPVIPL",0
"PTPVILCA",0
"PTPVILB",0
"PTPVICVT",10
"PENVILIT",0
"BLOCK","ENVDT"
"ELEMENT","CP43ENVDT-"
"PENNMI",1.0
"PENNMA",1.0
"PENNZN",""
"PENNZT",1.0
"PENBLM",12
"PENCRTS",0
"PENSPD1",100
"PTPCRDCT",0
"PENVICT",1
"PCCCRFT",1
"BLOCK","CARRING-"
"PTPCRAPO",0
"PTPCRPCK",0
"PTPCRPUX",0.00
"PTPCRPUY",0.00
"PTPCRRCV",0
"BLOCK","PACKCLS-"
"FDRTYPE","Emboss"
"TAPEWIDTH","8mm"
"FEEDPITCH",4
"REELDIAMETER",0
"TAPEDEPTH",0.0
"DOADVVACUUM",0
"CHKBEFOREFEED",0
"TAPEARMLENGTH",0
"PPCFDPP",0
"PPCFDEC",4
"PPCMNPT",30
"UNIT","PARTS"
"BLOCK","HEADER-"
"NAME","106653"
"REVISION","0000"
"DATE","11/09/03"
"TIME","11:10:42"
"PMABAR",""
"COMMENT",""
"PTPNAME","0603R"
"CMPNAME","0603R"
"BLOCK","PRTIDDT-"
"PMAPP",1
"PMADC",0
"ComponentQty",18
"BLOCK","PRTFORM-"
"PTPSZBX",1.60
"PTPSZBY",0.80
"PTPMNH",0.23
"NeedGlue",0
"BLOCK","TOLEINF-"
"PTPTLBX",0.50
"PTPTLBY",0.34
"PTPTLCL",0
"PTPTLPX",0.60
"PTPTLPY",0.40
"PTPTLPQ",30
"BLOCK","ELDT+" "PGDELSN","PGDELX","PGDELY","PGDELPP","PGDELQ","PGDELP","PGDELW","PGDELL","PGDELWT","PGDELLT","PGDELCT","PGDELR"
0,0.000,0.000,0,0,0.000,0.000,0.000,0.000,0.000,0.000,0
"BLOCK","VISION-"
"PTPVIPL",0
"PTPVILCA",0
"PTPVILB",0
"PTPVICVT",10
"PENVILIT",0
"BLOCK","ENVDT"
"ELEMENT","CP43ENVDT-"
"PENNMI",1.0
"PENNMA",1.0
"PENNZN",""
"PENNZT",1.0
"PENBLM",12
"PENCRTS",0
"PENSPD1",80
"PTPCRDCT",0
"PENVICT",1
"PCCCRFT",1
"BLOCK","CARRING-"
"PTPCRAPO",0
"PTPCRPCK",0
"PTPCRPUX",0.00
"PTPCRPUY",0.00
"PTPCRRCV",0
"BLOCK","PACKCLS-"
"FDRTYPE","Emboss"
"TAPEWIDTH","8mm"
"FEEDPITCH",4
"REELDIAMETER",0
"TAPEDEPTH",0.0
"DOADVVACUUM",0
"CHKBEFOREFEED",0
"TAPEARMLENGTH",0
"PPCFDPP",0
"PPCFDEC",4
"PPCMNPT",30
... the file goes on and on and on.. and will only get larger.
The REGEX is placing each "UNIT PARTS" and the following code until the NEXT "UNIT PARTS" into a string[].
After this, I am checking each string[] to see if the "NAME" section exists in a different list. If it does exist, I am outputting that "UNIT PARTS" at the end of a textfile.
This bit is a potential performance killer:
string concattedUnitPart = "";
foreach (var line in tempCP4List)
{
concattedUnitPart = concattedUnitPart + line;
line1CP4PartLines++;
}
(See this article for why.) Use a StringBuilder for repeated concatenation:
// No need to use tempCP4List at all
StringBuilder builder = new StringBuilder();
foreach (var line in splitCP4DataBaseLines)
{
concattedUnitPart.AppendLine(line);
line1CP4PartLines++;
}
Or even just:
string concattedUnitPart = string.Join(Environment.NewLine,
splitCP4DataBaseLines);
Now the regex part may well also be slow - I'm not sure. It's not obvious what you're trying to achieve, whether you need regular expressions at all, or whether you really need to do the whole thing in one go. Can you definitely not just process it line by line?
You could achieve the same output list 'line1CP4Components' using the following:
Regex StripEmptyLines = new Regex(#"^\s*$", RegexOptions.Multiline);
Regex UnitPartsMatch = new Regex(#"(?<=\n)""UNIT"",""PARTS"".*?(?=(?:\n""UNIT"",""PARTS"")|$)", RegexOptions.Singleline);
string CP4DataBase =
"C:\\Program\\Line Balancer\\FUJI DB\\KTS\\KTS - CP4 - Part Data Base.txt";
CP4DataBaseRTB.LoadFile(CP4DataBase, RichTextBoxStreamType.PlainText);
List<string> line1CP4Components = new List<string>(
UnitPartsMatch.Matches(StripEmptyLines.Replace(CP4DataBaseRTB.Text, ""))
.OfType<Match>()
.Select(m => m.Value)
);
return line1CP4Components.ToArray();
You may be able to ignore the use of StripEmptyLines, but your original code is doing this via the Where(c => !string.IsNullOrEmpty(c)). Also your original code is causing the '\r' part of the "\r\n" newline/linefeed pair to be duplicated. I assumed this was an accident and not intentional?
Also you don't seem to be using the value in 'line1CP4PartLines' so I omitted the creation of the value. It was seemingly inconsistent with the omission of empty lines later so I guess you're not depending on it. If you need this value a simple regex can tell you how many new lines are in the string:
int linecount = new Regex("^", RegexOptions.Multiline).Matches(CP4DataBaseRTB.Text).Count;
// example of what your code will look like
string CP4DataBase = "C:\\Program\\Line Balancer\\FUJI DB\\KTS\\KTS - CP4 - Part Data Base.txt";
List<string> Cp4DataList = new List<string>(File.ReadAllLines(CP4DataBase);
//or create a Dictionary<int,string[]> object
string strData = string.Empty;//hold the line item data which is read in line by line
string[] strStockListRecord = null;//string array that holds information from the TFE_Stock.txt file
Dictionary<int, string[]> dctStockListRecords = null; //dictionary object that will hold the KeyValuePair of text file contents in a DictList
List<string> lstStockListRecord = null;//Generic list that will store all the lines from the .prnfile being processed
if (File.Exists(strExtraLoadFileLoc + strFileName))
{
try
{
lstStockListRecord = new List<string>();
List<string> lstStrLinesStockRecord = new List<string>(File.ReadAllLines(strExtraLoadFileLoc + strFileName));
dctStockListRecords = new Dictionary<int, string[]>(lstStrLinesStockRecord.Count());
int intLineCount = 0;
foreach (string strLineSplit in lstStrLinesStockRecord)
{
lstStockListRecord.Add(strLineSplit);
dctStockListRecords.Add(intLineCount, lstStockListRecord.ToArray());
lstStockListRecord.Clear();
intLineCount++;
}//foreach (string strlineSplit in lstStrLinesStockRecord)
lstStrLinesStockRecord.Clear();
lstStrLinesStockRecord = null;
lstStockListRecord.Clear();
lstStockListRecord = null;
//Alter the code to fit what you are doing..