I have a multi-megabyte .sql file, there are GO statements on newlines every 10k or so. I am trying to come up with a way to read the file, line-by-line, until I hit a new line that only has "go" and a Newline character, then return what was read to the caller, and then read the next bunch of text until I hit GO again.
Peek only lets me read one character, what's a smart way to make this work in C# on Framework 4.0?
Thanks.
This should do exactly what you were saying.
Call this function with a string of the filename of the SQL file, and it will return an IEnumerable<string> (a bunch of strings) that each hold a SQL batch (each up until a GO statement) which you can then loop through with foreach or anything else.
public static IEnumerable<string> GetSqlBatches(string filename)
{
using(StreamReader sr = new StreamReader(filename))
{
StringBuilder ReadSoFar = new StringBuilder();
while (!sr.EndOfStream)
{
string line = sr.ReadLine();
ReadSoFar.AppendLine(line);
if (line.Trim() == "GO")
{
yield return ReadSoFar.ToString();
ReadSoFar = new StringBuilder();
}
}
sr.Close();
}
}
If you use SQL Server there is no need to parse file manually. Use Server.ConnectionContext.ExecuteNonQuery method instead. It is part of SMO.
See Handling "GO" Separators in SQL Scripts - the easy way by Jon Galloway.
If you read the file with a BufferedStream then you can seek back to the beginning of the line once you read a "GO" token.
If the file will always be on disk (instead of in memory or coming from a network connection), you can also just use a FileStream because that is also seekable.
Related
When modifying a file, is it always necessary to rewrite the entire file or is it possible to find and change some small part of a file without having to rewrite the whole thing?
If you're not inserting or deleting data, then you don't need to rewrite the file. You will be replacing existing bytes with new values, or appending to the file.
If you need to insert or delete, you only need to rewrite the file from that point onwards. The only time you would need to rewrite the entire file is if you are inserting bytes at the beginning.
It's okay to open a file with both read and write permissions. That way you can search the file for whatever you're looking for, and once you have the position you can seek to it (from memory, the write pointer is separate from the read pointer) and overwrite data to your heart's content =)
If you're not changing the length of the data, you can always just seek to the appropriate position in the file, and write a new set of bytes. This replaces whatever bytes where originally there.
There are two possible ways:
when you use StreamWriter sw = new StreamWriter it will delete the data from file.
To change only, you can store the file data in a List<string>. This works nice for me:
List<string> lines_list = new List<string>();
int file_l = 0
StreamReader sr_temp = new StreamReader(_path);
string line;
while ((line = sr_temp.ReadLine()) != null)
{
lines_list.Add(line);
file_l++;
}
sr_temp.Close();
StreamWriter sw = new StreamWriter(_path);
for (int i = 0; i < file_l; i++)
{
sw.WriteLine(lines_list[i]);
}
//here you add some data
sw.Close();
I have an application that dumps text to a text file. I think there might be an issue with the text not containing the proper carriage returns, so I'm in the process of writing a test that will compare the contents of of this file to a string variable that I declare in the code.
Ex:
1) Code creates a text file that contains the text:
This is line 1
This is line 2
This is line 3
2) I have the following string that I want to compare it to:
string testString = "This is line 1\nThis is line 2\nThis is line3"
I understand that I could open a file stream reader and read the text file line by line and store that in a mutable string variable while appending "\n" after each line, but wondering if this is re-inventing the wheel (other words, .NET has a built in class for something like this). Thanks in advance.
you can either use StreamReader's ReadToEnd() method to read contents in a single string like
using System.IO;
using(StreamReader streamReader = new StreamReader(filePath))
{
string text = streamReader.ReadToEnd();
}
Note: you have to make sure that you release the resources (above code uses "using" to do that) and ReadToEnd() method assumes that stream knows when it has reached an end. For interactive protocols in which the server sends data only when you ask for it and does not close the connection, ReadToEnd might block indefinitely because it does not reach an end, and should be avoided and also you should take care that current position in the string should be at the start.
You can also use ReadAllText like
// Open the file to read from.
string readText = File.ReadAllText(path);
which is simple it opens a file, reads all lines and takes care of closing as well.
No, there is nothing built in for this. The easiest way, assuming that your file is small, is to just read the whole thing and compare them:
var fileContents = File.ReadAllText(fileName);
return testString == filecontents;
If the file is fairly long, you may want to compare the file line by line, since finding a difference early on would allow you to reduce IO.
A faster way to implement reading all the text in a file is
System.IO.File.ReadAllText()
but theres no way to do the string level comparison shorter
if(System.IO.File.ReadAllText(filename) == "This is line 1\nThis is line 2\nThis is line3") {
// it matches
}
This should work:
StreamReader streamReader = new StreamReader(filePath);
string originalString = streamReader.ReadToEnd();
streamReader.Close();
I don't think there is a quicker way of doing it in C#.
You can read the entire file into a string variable this way:
FileStream stream;
StreamReader reader;
stream = new FileStream(yourFileName, FileMode.Open, FileAccess.Read, FileShare.Read);
reader = new StreamReader(stream);
string stringContainingFilesContent = reader.ReadToEnd();
// and check for your condition
if (testString.Equals(stringContainingFilesContent, StringComparison.OrdinalIgnoreCase))
This question seems to have been asked a million times around the web, but I cannot find an answer which will work for me.
Basically, I have a CSV file which has a number of columns (say two). The program goes through each row in the CSV file, taking the first column value, then asks the user for the value to be placed in the second column. This is done on a handheld running Windows 6. I am developing using C#.
It seems a simple thing to do. But I cant seem to add text to a line.
I cant use OleDb, as System.Data.Oledb isnt in the .Net version I am using. I could use another CSV file, and when they complete each line, it writes it to another CSV file. But the problems with that are - The file thats produced at the end needs to contain EVERY line (so what if they pull the batterys out half way). And what if they go back, to continue doing this another time, how will the program know where to start back from.
For every row, open the output file, append the new row to it, and then close the output file. To restart, count the number of rows in the existing output file from the previous run, which will give you your starting in the input file (i.e., skip that number of rows in the input file).
Edit: right at the start, use System.IO.File.Copy to copy the input file to the output file, so you have all the file in case of failure. Now open the input file, read a line, convert it, use File.ReadAllLines to read ALL of the output file into an array, replace the line you have changed at the right index in the array, then use File.WriteAllLines to write out the new output file.
Something like this:
string inputFileName = ""; // Use a sensible file name.
string outputFileName = ""; // Use a sensible file name.
File.Copy(inputFileName, outputFileName, true);
using (StreamReader reader = new StreamReader(inputFileName))
{
string line = null;
int inputLinesIndex = 0;
while ((line = reader.ReadLine()) != null)
{
string convertedLine = ConvertLine(line);
string[] outputFileLines = File.ReadAllLines(outputFileName);
if (inputLinesIndex < outputFileLines.Length)
{
outputFileLines[inputLinesIndex] = convertedLine;
File.WriteAllLines(outputFileName, outputFileLines);
}
inputLinesIndex++;
}
}
I want to insert the data at some positions in the text file without actually overwriting on the existing data. I have two text file. "one.txt" file have 1000 lines, "two.txt" file have 10000 lines. I want to read "one.txt" file content and insert into first 1000 lines of "two.txt" file content(Append the content of "one.txt" to the beginning of "two.txt").
Criteria:
Minimum code .
Less Memory consumption(irrespective of programming language )
Performance (will be considered based on size of the file).
just open up a streamreader for the first file, and a stream writer (in append mode) for the second file. As your reading the first 1000 lines from the first file, insert them into the second.
Something like this:
StreamReader sr = new StreamReader("one.txt");
StreamWriter sw = new StreamWriter("two.txt", true); //true for append
index i = 0;
while (i < 1000) {
sw.WriteLine(sr.ReadLine());
i++;
}
You may want to check for end of file on the StreamReader, but this will give you the general idea....
Based on the new information in OP:
You can use this same type of method, but just create a brand new file, reading the data from the first file, followed by the data from the second file. Once it's inside the new file, replace the original "two.txt".
If you're not limited to c# you can just do the following from a windows command line:
copy one.txt + two.txt three.txt
This would create the file you want, but it would be called three.txt. If you must have it in two.txt, you could simply rename two.txt to something else first and then do the copy append with two.txt as the third parm.
If you only have to do this once, here is some code that will do what you want. I did not compile this, but I believe there are no issues.
string[] linesOne = File.ReadAllLines(pathToFile1);
string[] linesTwo = File.ReadAllLines(pathToFile2);
List<string> result = new List<string>();
for(int i=0;i<1000;i++)
{
result.Add(linesOne[i]);
}
result.AddRange(linesTwo);
File.WriteAllLines(pathToFile2, result);
Hope this gets you started.
Bob
I have a large file(60mb) and I am reading the file into a string and Iam returning that string to another method.
Now when I am reading the file into a string its giving System out of memory exception.
Is there a way to read file in parts and append it to the string?
If not is there a way around this?
static public string Serialize()
{
string returnValue;
System.IO.FileInfo file1 = new FileInfo(#"c:\file.txt");
returnValue = System.IO.File.ReadAllText(file1.ToString());
}
How do you read the file right now ?
You could use the StreamReader class, and read the file line by line (ReadLine method).
You could also read a specified amount of bytes from the file on each read operation (Read method)
Yes- it's called streaming. Have a look at the StreamReader Class. Though I'm not sure why you want 1 60MB in one string. Probably best to deal with it a little at a time if possible (possibly in your scenario on a line by line basis?).
Instead of ReadAllText look at OpenRead and passing the returned FileStream into the constructor of a StreamReader, have a look at doing something along these lines if possible:
using (FileStream fs = File.OpenRead("c:\theFile.text"))
using (StreamReader sr = new StreamReader(fs))
{
string oneLine = sr.ReadLine();
}
even if you read it line by line (or in parts by streaming), you will run out of memory as you are appending it to a single string. is compressing it along the way an option? if not, i'd probably up the maxHeap for the JVM to 512MB or similar.