I have got around 800 files of maximum 55KB-100KB each where the data is in this format
Date,Time,Float1,Float2,Float3,Float4,Integer
Date is in DD/MM/YYYY format and Time is in the format of HH:MM
Here the date ranges from say 1st May to 1June and each day, the Time varies from 09:00 to 15:30.
I want to run a program so that, for each file, it extracts the data pertaining to a particular given date and writes to a file.
I am trying to get around, to form a to do a search and extract operation. I dont know, how to do it, would like to have some idea.
I have written the code below:
static void Main(string[] args)
{
string destpath = Directory.GetCurrentDirectory();
destpath += "\\DIR";
DirectoryInfo Dest = Directory.CreateDirectory(destpath);
DirectoryInfo Source = new DirectoryInfo(Directory.GetCurrentDirectory() + "\\IEOD");
FileInfo[] fiArr = Source.GetFiles("*.csv");
Console.WriteLine("Search Date:");
string srchdate = Console.ReadLine();
String FileNewLine;
String FileNewdt;
FileInfo r;
foreach (FileInfo f in fiArr)
{
r = new FileInfo(destpath + "\\" + f.Name);
r.Create();
StreamWriter Sw = r.AppendText();
StreamReader Sr = new StreamReader(f.FullName);
while (Sr.Peek() >= 0)
{
FileNewLine = Sr.ReadLine();
FileNewdt = FileNewLine.Substring(0,10);
if (String.Compare(FileNewdt, srchdate, true) == 0)
{
//write it to a file;
Console.WriteLine(FileNewLine);
}
}
}
Console.ReadKey();
}
As of now, it should write into the Console. The writing with the help of StreamWriter will be done later, but I am facing a runtime error. It says, " 'C:\Documents and Settings\Soham Das\Desktop\Test\DIR\ABAN.csv' because it is being used by another process."
Here ABAN is a newly created file, by the code. The problem is faced at StreamWriter Sw = r.AppendText()
Help appreciated.
Thanks
Soham
Now that you have edited the question to show that the delimiter is actually a comma instead of a slash (which would have conflicted with the date format) this becomes a lot easier. I've re-posted the answer from last night below.
// This would come from Stream.ReadLine() or something
string line = "02/06/2010,10:05,1.0,2.0,3.0,4.0,5";
string[] parts = line.Split(',');
DateTime date = DateTime.ParseExact(parts[0], "dd/MM/yyyy", null);
TimeSpan time = TimeSpan.Parse(parts[1]);
date = date.Add(time); // adds the time to the date
float float1 = Single.Parse(parts[2]);
float float2 = Single.Parse(parts[3]);
float float3 = Single.Parse(parts[4]);
float float4 = Single.Parse(parts[5]);
int integer = Int32.Parse(parts[6]);
Console.WriteLine("Date: {0:d}", date);
Console.WriteLine("Time: {0:t}", date);
Console.WriteLine("Float1: {0}", float1);
Console.WriteLine("Float2: {0}", float2);
Console.WriteLine("Float3: {0}", float3);
Console.WriteLine("Float4: {0}", float4);
Console.WriteLine("Integer: {0}", integer);
Obviously you can make it more resilient by adding error handling, using TryParse, etc. But this should give you a basic idea of how to manipulate strings in .NET.
So 800 files with around 100KB sums up to 80 KBytes. So why don't built up a little class like
public class Entry
{
public DateTime Date {get; set;}
public float Float1 {get; set;}
public int Integer1 {get; set;}
public Entry(string values)
{
//ToDo: Parse single line into properties
// e.g. use String.Split, RegEx, etc.
}
}
Also you should take care about implementing GetHashCode() and Equals() (there is a good explanation in the book Essential C#). And you should add the interface IComparable to that class which just makes somethine like
public int CompareTo(Entry rhs)
{
return this.Date.CompareTo(rhs.Date);
}
If you got this you can easily do the following:
var allEntries = new SortedList<Entry>();
string currentLine = null;
using (var streamReader = new StreamReader("C:\\MyFile.txt"))
while ((currentLine = streamReader.ReadLine()) != null)
{
try
{
var entry = new Entry(currentLine);
allEntries.Add(entry);
}
catch (Exception ex)
{
//Do whatever you like
//maybe just
continue;
//or
throw;
}
}
So what's missing is to read in all the files (instead of a single one). But this can be done by another loop on Directory.GetFiles() which maybe itself is looped through a Directory.GetDirectories().
After reading all the files into your List you can do whatever LINQ query comes to your mind.
Related
I have developed a file reading routine using c#.net that will read the entire file contents into memory using a suitable Data Class or structure.
I have a text file of 600MB that has RoadId and many other entries. I have to read that file using query method so I used Stream Reader in c#.net that reads line by line. But I want to know is there any other method in c#.net that will be memory efficient and less time taking or by converting the text to binary and then reading.
Not sure please guide me through this.
I am putting my code for reading whole file line by line...
public static void read_time()
{
DateTime end;
StreamReader file =
new StreamReader(#"C:\Users\Reva-Asus1\Desktop\DTF Test\F_Network_out.txt");
DateTime start = DateTime.Now;
while ((file.ReadLine()) != null) ;
end = DateTime.Now;
Console.WriteLine();
Console.WriteLine("Full File Read Time: " + (end - start));
Console.WriteLine();
file.Close();
Console.WriteLine("Data is read");
Console.ReadLine();
return;
}
// This querying method is to take roadId from user from console and display the record....
public static void querying_method()
{
Console.WriteLine("Give a RoadId to search record\n");
DateTime start, end;
string id =Console.ReadLine().Trim();
try
{
System.IO.StreamReader file =
new System.IO.StreamReader(#"C:\Users\Reva-Asus1\Desktop\DTF Test\F_Network_out.txt");
string line1;
int count = 1;
start = DateTime.Now;
while ((line1 = file.ReadLine()) != null)
{
if(line1 == id)
{
string line2 = " ";
while (count != 14)
{
Console.WriteLine(line2 = file.ReadLine());
count++;
}
int n = Convert.ToInt16(line2);
while (n != 0)
{
Console.WriteLine(line2 = file.ReadLine());
n--;
}
break;
}
}
end = DateTime.Now;
Console.WriteLine("Read Time for the data record: " + (end - start));
Console.ReadLine();
return;
}
catch (Exception)
{
Console.WriteLine("No ID match found in the file entered by user");
Console.ReadLine();
return;
}
}
You could maybe use this:
foreach (var line in File.ReadLines(path))
{
// TODO: Parse the line and convert to your object...
}
File.ReadLines(YourPath) is using using StreamReader in the background, so you can continue using it. Here reference code. So if you already using StreamReader and reading only by one line, you don't need to change anything.
using (StreamReader sr = new StreamReader(path, encoding))
{
while ((line = sr.ReadLine()) != null)
{
//You are reading the file line by line and you load only the current line in the memory, not the whole file.
//do stuff which you want with the current line.
}
}
So i'm in my 1st year of college, C# in Visual Studio is one of six modules.
Basically my problem is, i need to read in a value that's in a .txt file and calculate commission from that value.
The .txt file consists of:
1,Pat Ryan,280
2,Mary Smith,300
3,Tom Lynch,20
The 3rd value on each line is what i need to calculate the commission but i can't wrap my head around getting that value since you can't just pick out a value with the code we are currently using, you need to go through each line to get to the next.
This is what i've done so far. I tried doing the calculations this way:
if (columns [0] < 1000) {commission = column[0] * .05}
But get an error:
"Operator '<' cannot be applied to operands of type 'string[]' and 'int'"
Any help would be greatly appreciated!
static void salesReport()
{
string path = "sales.txt";
FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read);
StreamReader salesReport = new StreamReader(fs);
string inputText = salesReport.ReadLine();
Console.WriteLine("{0,-15}{1,-30}{2,-20}\n", "Number","Name","Sales");
while (inputText != null)
{
string[] columns = new string [3];
columns = inputText.Split(',');
Console.WriteLine("{0,-15}{1,-30}{2,-10}\n", columns[0], columns[1], columns[2]);
inputText = salesReport.ReadLine();
}
}
You cannot perform a comparison operation between a string and int as specified in your error. You will need to cast the value you get from the text file to int and then do a comparison.
if (Convert.ToInt32(columns[2]) < 1000)
{
commission = Convert.ToInt32(columns[2]) / .05;
}
Looks like you want the 3rd column, I have changed the index to 2.
here is a quick example of trying to parse a file and do what you want. This has a lot of bad practices, such has the way I am concatenating the output string, but you should get the idea.
static void Main(string[] args)
{
using (StreamReader reader = new StreamReader(#"C:\Path\To\File.txt"))
{
string line;
while ((line = reader.ReadLine()) != null)
{
string[] stuff = line.Split(',');
int id = Convert.ToInt32(stuff[0]);
string name = stuff[1];
int val = Convert.ToInt32(stuff[2]);
double commission = (double)val * 0.05;
Console.WriteLine(name + "'s Commission: " + commission.ToString());
}
}
}
Your issue is that you are not evaluating an integer. You are attempting to apply your comparison operator to the string representation after the split operation.
I added a method safeToInt which will prevent pesky exceptions if the string is not an int. Of course, if you want to be aware of those errors, you should just use Int32.TryParse directly and evaluate the boolean result.
I did not change your code to use the method I added for you :-) You should be able to figure that out.
static void salesReport() {
string path = "sales.txt";
FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read);
StreamReader salesReport = new StreamReader(fs);
string inputText = salesReport.ReadLine();
Console.WriteLine("{0,-15}{1,-30}{2,-20}\n", "Number","Name","Sales");
while (inputText != null) {
string[] columns = new string [3];
columns = inputText.Split(',');
Console.WriteLine("{0,-15}{1,-30}{2,-10}\n", columns[0], columns[1], columns[2]);
inputText = salesReport.ReadLine();
}
}
static int safeToInt(string input, int defaultValue = 0){
int result = 0;
if(Int32.TryParse(input, out result)){
return result;
}
return defaultValue;
}
Try this
if (int.Parse(columns[0]) < 1000) {commission = int.Parse(columns[0]) * .05}
In my console application, I'm writing to text file with the hour it executes in the following hour format:
DateTime.Now.ToString("t")
However, this format doesn't contain leading zero. After the text file is created I read it and then send an email with the content.
string x = DateTime.Now.ToString("hh");
Logs.writeSummary(DateTime.Now.ToString("t"), "Succeeded",x);
For example, after it generates the text file this is how it will look like:
6:09 AM:Succeeded
7:09 AM:Succeeded
8:09 AM:Succeeded
9:09 AM:Succeeded
10:09 AM:Succeeded
11:09 AM:Succeeded
12:09 PM:Succeeded
1:09 PM:Succeeded
2:09 PM:Succeeded
3:18 PM:Succeeded
4:09 PM:Succeeded
5:09 PM:Succeeded
6:09 PM:Succeeded
What I'm trying to fix is all Succeeded should be aligned.
The first approach I took is to find a way to add a leading zero and it would fix my issue. But it wasn't successful. So I took a different approach.
I pass current hour in the format of hh as a parameter to the function and if the current hour is not 10, 11, 12 then I add a space before the hour.
public static void writeSummary(string dt, string msg, string f)
{
string filePath = #"C:\Logs\summary.txt";
string _dt = dt.ToString();
string _msg = msg;
string _f = f;
using (StreamWriter writer = new StreamWriter(filePath, true))
{
if (_f == "10" || _f == "11" || _f == "12")
{
writer.WriteLine(_dt + ":" + _msg);
}
else
{
writer.WriteLine(" "+ _dt + ":" + _msg);
}
}
}
This fixes the issue in the text file. Everything is aligned, however, when I read it and send an email, I lose the space and end up with unaligned text posted above.
public static void readSummary()
{
try
{
using (StreamReader sr = new StreamReader(#"C:\Logs\summary.txt"))
{
String line = sr.ReadToEnd();
Console.WriteLine(line);
//send email using line
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
Any suggestion on how I can fix this?
Thanks.
Instead of DateTime.Now.ToString("t") use a custom format like:
DateTime.Now.ToString("hh:mm tt");
See: Custom Date and Time Format Strings
I am rookie in C#, but I need solve one Problem.
I have several text files in Folder and each text files has this structure:
IdNr 000000100
Name Name
Lastname Lastname
Sex M
.... etc...
Load all files from Folder, this is no Problem ,but i need delete "zero" in IdNr, so delete 000000 and 100 leave there. After this file save. Each files had other IdNr, Therefore, it is harder :(
Yes, it is possible each files manual edit, but when i have 3000 files, this is not good :)
Can C# one algorithm, which could this 000000 delete and leave only number 100?
Thank you All.
Vaclav
So, thank you ALL !
But in the End I have this Code :-) :
using System.IO;
namespace name
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void Browse_Click(object sender, EventArgs e)
{
DialogResult dialog = folderBrowserDialog1.ShowDialog();
if (dialog == DialogResult.OK)
TP_zdroj.Text = folderBrowserDialog1.SelectedPath;
}
private void start_Click(object sender, EventArgs e)
{
try
{
foreach (string file in Directory.GetFiles(TP_zdroj.Text, "*.txt"))
{
string text = File.ReadAllText(file, Encoding.Default);
text = System.Text.RegularExpressions.Regex.Replace(text, "IdNr 000*", "IdNr ");
File.WriteAllText(file, text, Encoding.Default);
}
}
catch
{
MessageBox.Show("Warning...!");
return;
}
{
MessageBox.Show("Done");
}
}
}
}
Thank you ALL ! ;)
You can use int.Parse:
int number = int.Parse("000000100");
String withoutzeros = number.ToString();
According to your read/save file issue, do the files contain more than one record, is that the header or does each record is a list of key and value like "IdNr 000000100"? It's difficult to answer without these informations.
Edit: Here's a simple but efficient approach which should work if the format is strict:
var files = Directory.EnumerateFiles(path, "*.txt", SearchOption.TopDirectoryOnly);
foreach (var fPath in files)
{
String[] oldLines = File.ReadAllLines(fPath); // load into memory is faster when the files are not really huge
String key = "IdNr ";
if (oldLines.Length != 0)
{
IList<String> newLines = new List<String>();
foreach (String line in oldLines)
{
String newLine = line;
if (line.Contains(key))
{
int numberRangeStart = line.IndexOf(key) + key.Length;
int numberRangeEnd = line.IndexOf(" ", numberRangeStart);
String numberStr = line.Substring(numberRangeStart, numberRangeEnd - numberRangeStart);
int number = int.Parse(numberStr);
String withoutZeros = number.ToString();
newLine = line.Replace(key + numberStr, key + withoutZeros);
newLines.Add(line);
}
newLines.Add(newLine);
}
File.WriteAllLines(fPath, newLines);
}
}
Use TrimStart
var trimmedText = number.TrimStart('0');
This should do it. It assumes your files have a .txt extension, and it removes all occurrences of "000000" from each file.
foreach (string fileName in Directory.GetFiles("*.txt"))
{
File.WriteAllText(fileName, File.ReadAllText(fileName).Replace("000000", ""));
}
These are the steps you would want to take:
Loop each file
Read file line by line
for each line split on " " and remove leading zeros from 2nd element
write the new line back to a temp file
after all lines processed, delete original file and rename temp file
do next file
(you can avoid the temp file part by reading each file in full into memory, but depending on your file sizes this may not be practical)
You can remove the leading zeros with something like this:
string s = "000000100";
s = s.TrimStart('0');
Simply, read every token from the file and use this method:
var token = "000000100";
var result = token.TrimStart('0');
You can write a function similar to this one:
static IEnumerable<string> ModifiedLines(string file) {
string line;
using(var reader = File.OpenText(file)) {
while((line = reader.ReadLine()) != null) {
string[] tokens = line.Split(new char[] { ' ' });
line = string.Empty;
foreach (var token in tokens)
{
line += token.TrimStart('0') + " ";
}
yield return line;
}
}
}
Usage:
File.WriteAllLines(file, ModifiedLines(file));
Basically I need someone to help me or show me the code that will allow me to read a name and a price from a file i have called c1.txt.
This is what i already have.
TextReader c1 = new StreamReader("c1.txt");
if (cse == "c1")
{
string compc1;
compc1 = c1.ReadLine();
Console.WriteLine(compc1);
Console.WriteLine();
compcase = compc1;
compcasecost = 89.99;
}
also how to select a line to read from a text document would be great.
You haven't told us the format of the text file. I am going to assume the following:
Milk|2.69
Eggs|1.79
Yogurt|2.99
Soy milk|3.79
You also didn't specify the output. I am going to assume the following:
Name = Milk, Price = 2.69
Name = Eggs, Price = 1.79
Name = Yogurt, Price = 2.99
Name = Soy milk, Price = 3.79
Then the following will read such a file and produce the desired output.
using(TextReader tr = new StreamReader("c1.txt")) {
string line;
while((line = tr.ReadLine()) != null) {
string[] fields = line.Split('|');
string name = fields[0];
decimal price = Decimal.Parse(fields[1]);
Console.WriteLine(
String.Format("Name = {0}, Price = {1}", name, price)
);
}
}
If your separator is different then you need to change the parameter '|' to the method String.Split (invoked on the instance of String named line as line.Split('|')).
If your format needs to be different then you need to play with the line
String.Format("Name = {0}, Price = {1}", name, price)
Let me know if you have any questions.
You can also try using a parsing helper class as a starting point, such as the one described at http://www.blackbeltcoder.com/Articles/strings/a-text-parsing-helper-class.
static void ReadText()
{
//open the file, read it, put each line into an array of strings
//and then close the file
string[] text = File.ReadAllLines("c1.txt");
//use StringBuilder instead of string to optimize performance
StringBuilder name = null;
StringBuilder price = null;
foreach (string line in text)
{
//get the name of the product (the string before the separator "," )
name = new StringBuilder((line.Split(','))[0]);
//get the Price (the string after the separator "," )
price = new StringBuilder((line.Split(','))[1]);
//finally format and display the result in the Console
Console.WriteLine("Name = {0}, Price = {1}", name, price);
}
It gives the same results as #Jason's method, but I think this is an optimized version.