How can I read big txt file? - c#

i have a 1000 text file and i want read single to single and Each file has a 4700000 record,for example one of line in file is:
43266200 6819 43295200 1393/05/23 14:28:45 113 1
and i want save into sql server for example:
field1:43266200
field2:6819
how can i do this?

var seperators = " ".ToCharArray();
foreach(var line in File.ReadLines(path))
{
var fields = line.Split(seperators, StringSplitOptions.RemoveEmptyEntries);
//now you have fields[0] and fields[1], save them in your database
}

This may help you
var message ="43266200 6819 43295200 1393/05/23 14:28:45 113 1";
//Split your data into pieces
var messages=message.Split(' ').Where( o => !string.IsNullOrEmpty(o));
var i=0;
foreach(var item in messages)
{
// do whatever you wanna to do with pieces
Console.Write( "field {0}:{1}",++i,item);
}

If you're reading the text from a file, and you can reasonably assume that the space character will be your only delimiter, you should use the String.Split() method to tokenize each line:
// instantiate FileInfo of your file as yourFile
foreach (string line in yourFile.ReadLines())
{
string[] lineTokens = line.Split(new char[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);
}
String.Split() allows you to separate any string into a string[] of substrings based on the char delimiters you provide in the first argument. The second argument in the code above is one of the values in the StringSplitOptions enumeration, which has values of either None (provide all strings) or RemoveEmptyEntries (do not return any substrings that consist solely of delimiter characters).
Then, from there, you can iterate through lineTokens and assemble an object from each token, or you can assemble an SQL query where any given index corresponds to a column in the row you intend to add.

Related

how to get before and after text in the string which contains 'Or' in it

I want a code that takes a string with 'Or' in it and takes text before and after 'Or' and stores it in seperate variable
I tried the substring function
var text = "Actor or Actress";
var result= text.Substring(0, text.LastIndexOf("or"));
but with this getting only actor I want actor as well as actress but in seperate variables as a whole word so it can be anything in place of 'actor or actress'
You need to use one of the flavors of String.Split that accepts an array of string delimiters:
string text = "Actor or Actress";
string[] delim = new string[] { " or " }; // add spaces around to avoid spliting `Actor` due to the `or` in the end
string[] elements = text.Split(delim, StringSplitOptions.None);
foreach (string elem in elements)
{
Console.WriteLine(elem);
}
Output:
Actor
Actress
Note: I am using .NET framework 4.8, but .NET 6 also has an overload of String.Split that accepts a single string delimiter so there's no need to create delim as an array of strings, unless you want to be able to split based on variations like " Or "," or ".
Spli() should do the job
text.Split(“or”);
use the Split() method
var text = "Actor or Actress";
Console.WriteLine(text.Split("or")[0]);
Console.WriteLine(text.Split("or")[1]);
Output
Actor
Actress
try this
string[] array = text.Split(' ');
foreach (string item in array)
{
if (item != "or")
{
Console.WriteLine(item);
}
}
Output
Actor
Actress

How to delimit strings in each line of a text file and compare them to a user input

My goal is to match the users input into a field with data in a text file.
1000|I-002096.02.02|EL|MISCMI
1000|I-002097.02.02|EL|ESYEED
1000|I-002098.02.02|EL|MISCCA
1000|I-002099.02.02|EL|MISCCA
1000|I-002100.02.02|EL|MISCCA
1000|I-002101.02.02|EL|USQUIC00
1000|I-002102.02.02|EL|MISCMI
The portion after the first "|" delimiter is what I need to check against the users input. (users input is stored in TxtWBS.Text in the code below)
This is what I have tried but this only works when each line has nothing to delimit.
string[] wbslist = File.ReadAllLines(filePath);
bool wbsExists = Array.Exists(wbslist, element => element == TxtWBS.Text);
if (wbsExists)
/*leave empty*/;
else
errMessage += "This WBS does not exist" + Environment.NewLine;
I expect to be able to check if the users input exists in the text file.
This could be done with a single line in Linq.
Change your test to:
bool wbsExists = wbslist.Any(x => x.Contains(TxtWBS.Text));
And if you are not sure about the case of the input you can have
bool wbsExists = wbslist.Any(x => -1 != x.IndexOf(TxtWBS.Text, StringComparison.CurrentCultureIgnoreCase));
More, if you want to check an exact match against the second item in the line then
bool wbsExists = wbslist.Select(x => x.Split('|')[1]).Any(k => k == TxtWBS.Text);
Consider also to change the loading of your text data to
var wbslist = File.ReadLines(filePath);
File.ReadLines doesn't read all the lines in memory immediately but returns an IEnumerable<String> that is more suited in Linq expressions
You can use the following code. Read file and iterate line by line, splitting each line into array of strings by token '|'.
string[] wbslist = File.ReadAllLines(filePath);
foreach(string line in wbslist)
{
string [] splittedLine = line.Split('|');
// I assume you need the second element in the delimited line
if(string.Equals(splittedLine[1], TxtWBS.Text, StringComparison.OrdinalIgnoreCase))
Console.WriteLine("Website found");
}
Well, if you're absolutely sure about the format of the file then you can split each line on the delimiter and check the user input against the second element in the resulting array.
bool wbsExists = Array.Exists(wbslist, element => element.Split('|')[1] == TxtWBS.Text);

split string with comma and newspace in 2d or jagged array and access items in c#

Hi there I have a text file like this, many lines 2 columns
CF7CED1BF035345269118A15EF2D45A06, product1
CF7CED1BF035345269118A15EF2D45A09, product2
....
...
...
...
I need to split this and access each field, more precise I need to make a loop that creates many files like product1.txt product2.txt etc and will enclose the codes on its left.
So I need to create files with filenames of columns [2] of all lines and enclose the column[1] as value of each line
I know how to do basic stuff in arrays, like read all lines and store them, but i don't know how to make a loop that will read both field 1 then 2 of LINE 1, create the file and store (I know how to read and save to file) and go on on next LINE 2 and go on field 1 and field 2 again and so on.
Someone suggested using jagged arrays would be faster than 2d arrays
"When you use ReadLines, you can start enumerating the collection of
strings before the whole collection is returned; when you use
ReadAllLines, you must wait for the whole array of strings be
returned before you can access the array. Therefore, when you are
working with very large files, ReadLines can be more efficient."
string path_read = #"c:\read\file.txt";
//Path to save resulting files.
string path = #"c:\temp\";
char[] comma = new char[1]{','};
//ASSUMPTION: Your every row has comma separated 2 values.
//Do a for loop.
//Code now use File.ReadLines
foreach (var currentLine in File.ReadLines(path_read))
{
string[] itemArray = currentLine.Split(comma, StringSplitOptions.RemoveEmptyEntries);
// Your item array now has 2 values from 2 columns in the same row.
// Do whatever with it.
File.WriteAllText(path+itemArray[1]+".txt", itemArray[0], Encoding.UTF8);
}
Do you need to keep the contents for any further use? if the intent is to read and the save the contents to separate files then there is no need for a separate array.
using (var reader = new StreamReader(#"input.txt"))
{
while (!reader.EndOfStream)
{
var inputText = reader.ReadLine();
var splitText = inputText.Split(',');
File.AppendAllLines(splitText[1] + ".txt", new List<string> {splitText[0]});
}
}

split a string from a text file into another list

Hi i know the Title might sound a little confusing but im reading in a text file with many lines of data
Example
12345 Test
34567 Test2
i read in the text 1 line at a time and add to a list
using (StreamReader reader = new StreamReader("Test.txt"))
{
string line;
while ((line = reader.ReadLine()) != null)
{
list.Add(line);
}
}
how do i then separate the 1234 from the test so i can pull only the first column of data if i need like list(1).pars[1] would be 12345 and list(2).pars[2] would be test2
i know this sounds foggy but i hope someone out there understands
Maybe something like this:
string test="12345 Test";
var ls= test.Split(' ');
This will get you a array of string. You can get them with ls[0] and ls[1].
If you just what the 12345 then ls[0] is the one to choose.
If you're ok with having a list of string[]'s you can simply do this:
var list = new List<string[]>();
using (StreamReader reader = new StreamReader("Test.txt"))
{
string line;
while ((line = reader.ReadLine()) != null)
{
list.Add(line.Split(' '));
}
}
string firstWord = list[0][0]; //12345
string secondWord = list[0][1]; //Test
When you have a string of text you can use the Split() method to split it in many parts. If you're sure every word (separated by one or more spaces) is a column you can simply write:
string[] columns = line.Split(' ');
There are several overloads of that function, you can specify if blank fields are skipped (you may have, for example columns[1] empty in a line composed by 2 words but separated by two spaces). If you're sure about the number of columns you can fix that limit too (so if any text after the last column will be treated as a single field).
In your case (add to the list only the first column) you may write:
if (String.IsNullOrWhiteSpace(line))
continue;
string[] columns = line.TrimLeft().Split(new char[] { ' ' }, 2);
list.Add(columns[0]);
First check is to skip empty or lines composed just of spaces. The TrimLeft() is to remove spaces from beginning of the line (if any). The first column can't be empty (because the TrimLeft() so yo do not even need to use StringSplitOptions.RemoveEmptyEntries with an additional if (columns.Length > 1). Finally, if the file is small enough you can read it in memory with a single call to File.ReadAllLines() and simplify everything with a little of LINQ:
list.Add(
File.ReadAllLines("test.txt")
.Where(x => !String.IsNullOrWhiteSpace(x))
.Select(x => x.TrimLeft().Split(new char[] { ' ' }, 2)[0]));
Note that with the first parameter you can specify more than one valid separator.
When you have multiple spaces
Regex r = new Regex(" +");
string [] splitString = r.Split(stringWithMultipleSpaces);
var splitted = System.IO.File.ReadAllLines("Test.txt")
.Select(line => line.Split(' ')).ToArray();
var list1 = splitted.Select(split_line => split_line[0]).ToArray();
var list2 = splitted.Select(split_line => split_line[1]).ToArray();

remove blank values from array and suggest a better method to read text file data delimited by '|'

Am trying to get values from a text file in which entries are delimited using '|'.am getting the values using string .Split method..but in some places the delimiter appears multiple times in succession like '||||||||',so empty space gets inserted in the array how should i remove those empty elements from array or is there any efficient technique to read values from text file delimited by '|".below is my code and the screen shot of array values
var reader = new StreamReader(File.OpenRead(#"d:\er.txt"));
while (!reader.EndOfStream)
{
var line = reader.ReadLine().Trim();
var values = line.Split('|');
string[] ee = values;
}
can any one suggest a better method for reading data from text file delimited by '|'
Split has an overload that takes a StringSplitOptions enumeration value:
var values = line.Split(new char[]{'|'}, StringSplitOptions.RemoveEmptyEntries);
This will remove empty entries.
You can use linq on your array?
var values = line.Split("|").Where(v => !string.IsNullOrEmpty(v));
See, the below line of codes, you can do something like that.
var values = line.Split('|');
List<string> FilteredValues = new List<string>();
foreach (var value in values)
{
if (value != "")
{
FilteredValues.Add(value);
}
}
further to Oded's answer this is the correct syntax
var reader = new StreamReader(File.OpenRead(#"d:\er.txt"));
while (!reader.EndOfStream)
{
var line = reader.ReadLine().Trim();
var values = line.Split(new char[]{'|'}, StringSplitOptions.RemoveEmptyEntries);
string[] ee = values;
}
Or, maybe, you can use the trim() function !?

Categories