So this is what I would like to do. I am kind of all over the place with this but I hope you can bear with me. This is a very new concept to me.
1) In my program I wish create an array of 50 integers to hold the data that comes from the file.
My program must get the path to the user's Documents folder.
2) The name of the file will be "grades.txt". Code this file name right in your program. No user input is required to get the file name.
3) Create a StreamReader object, using this path. This will open the file.
Write a loop that reads data from the file, until it discovers the end of the file.
4) As each integer value is read in, I display it, and store it in the array.
5) Using the concepts of partially filled arrays, write a method that takes the array as a parameter and calculates and returns the average value of the integers stored in the array
Output the average.
So right now I am having a very hard time figuring out how to get the numbers saved in the grades.txt file, save them to an array, and display them. I try to split the integers and save them as that but it doesn't seem to work.
This is the code that I have so far:
class Program
{
const int SIZE = 50;
static void Main()
{
// This line of code gets the path to the My Documents Folder
int zero = 0;
int counter = 0;
int n, m;
StreamReader myFile;
myFile = new StreamReader("C:/grades.txt");
string inputNum = myFile.ReadLine();
do
{
Console.Write("The test scores are listed as follows:");
string[] splitNum = myFile.Split();
n = int.Parse(splitNum[0]);
{
if (n != zero)
{
Console.Write("{0}", n);
counter++;
}
}
} while (counter < SIZE && inputNum != null);
// now we can use the full path to get the document
Console.ReadLine();
}
}
This is the grades.Txt file:
88
90
78
65
50
83
75
23
60
94
For reading the file you need something like this:
var scores = new List<int>();
StreamReader reader = new StreamReader("C:/grades.txt");
while (!reader.EndOfStream)
{
int score;
if (int.TryParse(reader.ReadLine(), out score) && score != 0)
scores.Add(score);
}
and you can have count of scores with scores.Count property.
1) In my program I wish create an array of 50 integers to hold the data that comes from the file.
See Arrays Tutorial (C#).
2) My program must get the path to the user's Documents folder. The name of the file will be "grades.txt". Code this file name right in your program. No user input is required to get the file name.
Use these two:
Environment.GetFolderPath Method (Environment.SpecialFolder)
Path.Combine()
3) Create a StreamReader object, using this path. This will open the file. Write a loop that reads data from the file, until it discovers the end of the file.
See StreamReader.EndOfStream().
4) As each integer value is read in, I display it, and store it in the array.
If there is only one score per line, you don't need to do any Split() calls. Use your counter variable to know where in the Array to store the value.
5) Using the concepts of partially filled arrays, write a method that takes the array as a parameter and calculates and returns the average value of the integers stored in the array Output the average.
See Methods (C# Programming Guide).
You'd pass the Array and how many values are stored in it (the counter variable).
Related
How to sort a large csv file with 10 columns?
The sorting should be based on data type for example, string, Date, integer etc
Assuming Based on 5th column (Period Column) we need to sort.
As it is large CSV file, Without loading the same in memory we have to do.
I tried using logparser, but beyond certain size it throws error saying
"log parser tool has stopped working"
So please suggest any algorithm which i can implement in c#. Or if there is any other component or code which can help me.
Thanks in advance
Do know that running a program without memory is hard, specially if you have an algorithm that by its nature requires memory allocation.
I've looked at the External sort method mentioned by Jim Menschel and this is my implementation.
I didn't implement sorting on the fifth field but left some hints in the code so you can add that yourself.
This code reads a file, line by line and creates, in a temporary directory for each line a new file. Then we open two of those files and create a new target file. After reading a line from the two open files, we can compare them (or their fields). Based on their comparison we write the smallest one to the target file and read the next line from the file we used.
Although this doesn't keep much strings in memory it is hard on the diskdrive. I checked the NTFS limits and 50,000,000 files is within the specs.
Here are the main methods of the class:
main entry point
This take the file to be sorted
public void Sort(string file)
{
Directory.CreateDirectory(sortdir);
Split(file);
var sortedFile = SortAndCombine();
// if you feel confident you can override the original file
File.Move(sortedFile, file + ".sorted");
Directory.Delete(sortdir);
}
Split file
Split the file in a new file for each line
Yes, that will be a lot of files but it guarantees the least amount of memory used. It is easy to optimize though, read a couple of lines, sort those and write to a file.
void Split(string file)
{
using (var sr = new StreamReader(file, Encoding.UTF8))
{
var line = sr.ReadLine();
while (!String.IsNullOrEmpty(line))
{
// whatever you do, make sure this file your writed
// is ordered, just writing a single line is the easiest
using (var sw = new StreamWriter(CreateUniqueFilename()))
{
sw.WriteLine(line);
}
line = sr.ReadLine();
}
}
}
Combine the files
Iterate over all files and take one and the next one, merge those files
string SortAndCombine()
{
long processed; // keep track of how much we processed
do
{
// iterate the folder
var files = Directory.EnumerateFiles(sortdir).GetEnumerator();
bool hasnext = files.MoveNext();
processed = 0;
while (hasnext)
{
processed++;
// have one
string fileOne = files.Current;
hasnext = files.MoveNext();
if (hasnext)
{
// we have number two
string fileTwo = files.Current;
// do the work
MergeSort(fileOne, fileTwo);
hasnext = files.MoveNext();
}
}
} while (processed > 1);
var lastfile = Directory.EnumerateFiles(sortdir).GetEnumerator();
lastfile.MoveNext();
return lastfile.Current; // by magic is the name of the last file
}
Merge and Sort
Open two files and create one target file. Read a line from both of these and write sthe mallest of the two to the target file.
Keep doing that until both lines are null
void MergeSort(string fileOne, string fileTwo)
{
string result = CreateUniqueFilename();
using(var srOne = new StreamReader(fileOne, Encoding.UTF8))
{
using(var srTwo = new StreamReader(fileTwo, Encoding.UTF8))
{
// I left the actual field parsing as an excersise for the reader
string lineOne, lineTwo; // fieldOne, fieldTwo;
using(var target = new StreamWriter(result))
{
lineOne = srOne.ReadLine();
lineTwo = srTwo.ReadLine();
// naive field parsing
// fieldOne = lineOne.Split(';')[4];
// fieldTwo = lineTwo.Split(';')[4];
while(
!String.IsNullOrEmpty(lineOne) ||
!String.IsNullOrEmpty(lineTwo))
{
// use your parsed fieldValues here
if (lineOne != null && (lineOne.CompareTo(lineTwo) < 0 || lineTwo==null))
{
target.WriteLine(lineOne);
lineOne = srOne.ReadLine();
// fieldOne = lineOne.Split(';')[4];
}
else
{
if (lineTwo!=null)
{
target.WriteLine(lineTwo);
lineTwo = srTwo.ReadLine();
// fieldTwo = lineTwo.Split(';')[4];
}
}
}
}
}
}
// all is perocessed, remove the input files.
File.Delete(fileOne);
File.Delete(fileTwo);
}
Helper variable and method
There is one shared member for the temporary directory and a method for generating temporary unique filenames.
private string sortdir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N"));
string CreateUniqueFilename()
{
return Path.Combine(sortdir, Guid.NewGuid().ToString("N"));
}
Memory analysis
I've created a small file with 5000 lines in it with the following code:
using(var sw= new StreamWriter("c:\\temp\\test1.txt"))
{
for(int line=0; line<5000; line++)
{
sw.WriteLine(Guid.NewGuid().ToString());
}
}
I then ran the sorting code with the memory profiler. This is what the summary looked like on my box with Windows 10, 4GB RAM and a spinning disk:
The object lifetime shows as expected a lot of String, char[] and byte[] allocations, but none of those have survived a Gen 0 collection, which means they are all short lived and I don't expect this to be a problem if the number of lines to sort increases.
This is the simplest solution that works for me. From here easy alterations and improvements are possible, either leading to even less memory consumption, reduce allocations or a higher speed. Make sure to measure, select the area where you can make the biggest impact and compare successive results. That should give you the optimum between memory usage and performance.
Instead of reading CSV completely you can simply index it:
Read unsorted CSV line by line and remember 5th element (column) value and something to identify this line later: line number or offset of this line from beginning of the file and size.
You will have some kind of List<Tuple<string, ...>>. Sort that
var sortedList = unsortedList.OrderBy(item => item.Item1);
Now you can create sorted CSV by enumerating sorted list, reading line from source file and appending it to new CSV:
using (var sortedCSV = File.AppendText(newCSVFileName))
foreach(var item in sortedList)
{
... // read line from unsorted csv using item.Item2, etc.
sortedCSV.WriteLine(...);
}
I am trying to split the data in this array, but it keeps giving me this error:
index out of bounds.
The size of the array is 10. The line of code that is creating the error is
int score = int.Parse(scoreInfo[1]);
This is the code that I have:
static void Main(string[] args)
{
const int SIZE = 10;
for (int j = 0; j < SIZE; j++)
{
// prompt the user
Console.WriteLine("Enter player name and score on one line");
Console.WriteLine("seperated by a space.");
// Read one line of data from the file and save it in inputStr
string inputStr = Console.ReadLine();
// The split method creates an array of two strings
string[] scoreInfo = inputStr.Split();
// Parse each element of the array into the correct data type.
string name = (scoreInfo[0]);
int score = int.Parse(scoreInfo[1]);
if (inputStr == "")
{
Console.WriteLine(scoreInfo[j]);
}
}
Console.ReadLine();
}
A few things of note. You are using SIZE to denote how many iterations your loop should make. It is not the size of your array. Your array is only going to be large as the split makes it. If you split with a space as the delimiter, you're going to have 2 objects in the array at indices 0 and 1.
That last section where you're saying to write scoreInfo[j] will fail as soon as j is larger than 1.
I tried your code using a constant, something like "Guy 19". The .Split() method will break up your input string based on spaces. If you're not entering a space followed by a number, then your scoreInfo[1] will be empty and parsing will fail.
Try adding a break point after your .Split() to determine if scoreInfo is correctly split into a size 2 array, with [0] being the name of the player and [1] being an integer score. If you're entering a name like "FirstName LastName 20" then the information at position [1] is not going to be an integer and the parsing will also fail.
Your for-loop is sort of puzzling, are you looping through 10 times for a team of 10 players? You could put your dialog in a while loop and break out when parsing fails or the user types something like "exit".
Very simple and straight forward, I want to read from a file; convert the string value to an int, iterate using "for statement" and write the file into another file. When writing, it should write each digit on a new line. I want to use the WriteAllLines static method of File class. It only accepts a string array, how do I get this done? My code snippet is this:
static void Main(string[] args)
{
String Readfiles = File.ReadAllText(#"C:\Users\ken4ward\Desktop\Tidy\WriteLines.txt");
Int32 myInt = Int32.Parse(Readfiles);
for (int i = 0; i < myInt; ++i)
{
Console.WriteLine(i);
Console.ReadLine();
String[] start = new String[i];
File.WriteAllLines(#"C:\Users\ken4ward\Desktop\Tidy\writing.txt", start);
}
}
It is very simple. With a bunch of codes, an output of an iteration is written to a .txt file. The iteration only counts how many times a method is called. This part is perfectly done. If the method is called 10 times, it simply writes 10. The second class file reads this file and writes it to another .txt file. What I want to do is that since the first file writes only a digit. As an example - 10, what is written in the second file should like this:
1
2
3
4
5
6
7
8
9
10
meaning that it writes each digit on a new line. The problem is that it does not write to the txt file.
The problem is that you are declaring your string array inside the loop and never populating it with anything. Move that string array outside the loop instead. Also, I don't think you want to write the file every time through the loop, so move file write outside the loop too.
static void Main(string[] args)
{
String Readfiles = File.ReadAllText(#"C:\Users\ken4ward\Desktop\Tidy\WriteLines.txt");
Int32 myInt = Int32.Parse(Readfiles);
//Declare array outside the loop
String[] start = new String[myInt];
for (int i = 0; i < myInt; ++i)
{
//Populate the array with the value (add one so it starts with 1 instead of 0)
start[i] = (i + 1).ToString();
Console.WriteLine(i);
Console.ReadLine();
}
//Write to the file once the array is populated
File.WriteAllLines(#"C:\Users\ken4ward\Desktop\Tidy\writing.txt", start);
}
You could do this:
File
.WriteAllLines(#"C:\Users\ken4ward\Desktop\Tidy\writing.txt",
File
.ReadAllLines(#"C:\Users\ken4ward\Desktop\Tidy\WriteLines.txt")
.Select(x => int.Parse(x))
.Select(x => x.ToString())
.ToArray());
But is just the same as a file copy, but with a fragile int validation of each line.
I am trying to write a bubble sort on a large text file that has 1 column of numbers that are all between 1 and 99. I want to sort it without bringing the contents or the data in memory at the same time. I used a loop of 20 passes to get the logic down before I do this on the whole file. I am using streamreader and streamwriter to receive and send the numbers back and forth so only 2 numbers are processed at any one time. the numbers in the text file are coded int32.I converted them in the writing process. I have been unable to populate the "sorted" file with the data. Here is my code:
static void BubbleSort()
{
int bubbleOut, bubbleIn;
StreamReader readToSort = new StreamReader(#"C:Random # File.txt");
StreamWriter writeSorted = new StreamWriter(#"C:Sorted_File.txt");
bubbleIn = readToSort.Read();
bubbleOut = readToSort.Read();
for (bubbleIn = 1; bubbleIn <= 20; bubbleIn++)
{
for (bubbleOut = bubbleIn; bubbleIn < bubbleOut; bubbleOut++ )
{
if (bubbleOut > bubbleIn)
{
int temp = bubbleIn;
bubbleIn = bubbleOut;
bubbleOut = temp;
writeSorted.WriteLine(bubbleIn);
writeSorted.WriteLine(bubbleOut);
}
}
}
}
First of all, for large data sets you better use MergeSort because it is much faster (O(n log n)) than BubbleSort (O(n * n)).
Besides that, you are only reading the first two characters of the inputstream and after reading the data, you override the value in the for-loop.
I think you first have to copy the entire content of in input file, to the output file. Then you need a way to swap 2 characters in the output stream. After that, you can run the bubblesort algorithm on the inputfile and apply the swap on the output file.
Your for-loop should look something like:
// TODO: Copy input to output
for (int sortedElement = 0; sortedElement < StreamLength; sortedElement++)
{
// You don't have to loop till the end of the stream. Last
// character is sorted on each iteration.
for (int index = 0; index < StreamLength - sortedElements - 1; index++)
{
// read character from input stream at position 'index' as leftValue
// read character from input stream at position 'index+1' as rightValue
if (rightValue < leftValue)
{
// Perform swap values in outputfile
}
}
}
That should be it.
Ok, so hopefully I can explain this in enough detail for somebody to be able to help me.. I am writing a program in C# that is supposed to take a text file and replace specific text, which happen to be names of files, and print a new text file for every single combination of the given filenames. The specific places to change the text of filenames have their own set of possible filenames, listed as an array described below. The program should run regardless of how many filenames are available for each location as well as how many total locations for the filenames. If you really wanted to make it awesome, it can be slightly optimized knowing that no filenames should be duplicated throughout any single text file.
text is an array of lines that make up the base of the total file.
lineNum holds an array of the line locations of the filename entries.
previousFiles is an array of previously used filenames, starting with what is already in the file.
files is a jagged 2-dimensional array of possible filenames where files[1] would be an array of all the possible filenames for the 2nd location
Here is an example of how it would work with 3 separate filename locations, the first one given 3 possible filenames, the second given 8 possible filenames, and the third given 3 possible filenames.
Oh and assume buildNewFile works.
int iterator = 0;
for (int a = 0; a < 3; a++)
{
for (int b = 0; b < 8; b++)
{
for (int c = 0; c < 3; c++)
{
iterator++;
text[lineNums[0]] = text[lineNums[0]].Replace(previousFiles[0], files[0][a]);
text[lineNums[1]] = text[lineNums[1]].Replace(previousFiles[0], files[0][a]);
text[lineNums[2]] = text[lineNums[2]].Replace(previousFiles[1], files[1][b]);
text[lineNums[3]] = text[lineNums[3]].Replace(previousFiles[1], files[1][b]);
text[lineNums[4]] = text[lineNums[4]].Replace(previousFiles[2], files[2][c]);
text[lineNums[5]] = text[lineNums[5]].Replace(previousFiles[2], files[2][c]);
previousFiles = new string[] { files[0][a], files[1][b], files[2][c] };
buildNewFile(text, Info.baseFolder + "networks\\" + Info.dsnFilename + iterator + ".dsn");
}
}
}
If you guys can help me, thank you so much, I just can't figure out how to do it recursively or anything. If you have any questions I'll answer them and edit up here to reflect that.
It took me a little while to figure out what you really wanted to do. This problem can be solved without recursion, the trick is to look at the data you have and get it into a more usable format.
Your "files" array is the one that is the most inconvenient. The trick is to transform the data into usable permutations. To do that, I suggest taking advantage of yield and using a method that returns IEnumerable. The code for it is here:
public IEnumerable<string[]> GenerateFileNameStream(string[][] files)
{
int[] current_indices = new int[files.Length];
current_indices.Initialize();
List<string> file_names = new List<string>();
while (current_indices[0] < files[0].Length)
{
file_names.Clear();
for (var index_index = 0; index_index < current_indices.Length; index_index++)
{
file_names.Add(files[index_index][current_indices[index_index]]);
}
yield return file_names.ToArray();
// increment the indices, trickle down as needed
for (var check_index = 0; check_index < current_indices.Length; check_index++)
{
current_indices[check_index]++;
// if the index hasn't rolled over, we're done here
if (current_indices[check_index] < files[check_index].Length) break;
// if the last location rolls over, then we are totally done
if (check_index == current_indices.Length - 1) yield break;
// reset this index, increment the next one in the next iteration
current_indices[check_index] = 0;
}
}
}
Basically, it keeps track of the current index for each row of the files 2D array and returns the file name at each current index. Then it increments the first index. If the first index rolls over, then it resets to 0 and increments the next index instead. This way we can iterate through every permutation of the file names.
Now, looking at the relationship between lineNum and files, I assume that each location in the file is copied to two lines. The rest of the code is here:
public void MakeItWork(string[][] files, int[] lineNum, string[] text, string[] previousFiles)
{
var iterator = 0;
var filenames = GenerateFileNameStream(files);
// work a copy of the text, assume the "previousFiles" are in this text
var text_copy = new string[text.Length];
foreach (var filenameset in filenames)
{
iterator++;
Array.Copy(text, text_copy, text.Length);
for (var line_index = 0; line_index < lineNum.Length; line_index++)
{
var line_number = lineNum[line_index];
text[line_number] = text[line_number].Replace(previousFiles[line_index], filenameset[line_index / 2]);
}
buildNewFile(text_copy, Info.baseFolder + "networks\\" + Info.dsnFilename + iterator + ".dsn");
}
}
This code just takes the results from the enumerator and generates the files for you. The assumption based on your sample code is that each filename location is used twice per file (since the lineNum array was twice as long as the files location count.
I haven't fully tested all the code, but the crux of the algorithm is there. The key is to transform your data into a more usable form, then process it. The other suggestion I have when asking a question here is to describe the problem more as a "problem" and not in the terms of your current solution. If you detailed the goal you are trying to achieve instead of showing code, you can get more insights into the problem.