I'm reading a file to my C# application and decompressing a tile_data BLOB using a gzip stream. I'm currently accessing the BLOB data through this method:
SQLiteCommand command = new SQLiteCommand(query, DbConn);
SQLiteDataReader reader = command.ExecuteReader();
bool isMet = false;
while (reader.Read())
{
using (var file = reader.GetStream(0))
using (var unzip = new GZipStream(file, CompressionMode.Decompress))
using (var fileReader = new StreamReader(unzip))
{
var line = fileReader.ReadLine();
while (!fileReader.EndOfStream)
{
}
Console.WriteLine("End of tile_data");
}
}
reader.Close();
Console.WriteLine("Reader closed");
Console.ReadKey();
}
catch (Exception e)
{
Console.Write(e.StackTrace);
Console.ReadKey();
}
I'm looking to wait until the fileReader detects "tertiary" (string) and then prints all data afterwards. I attempted to use a bool and a nested while loop but it came back as an infinite loop, hence the question.
The code that I used (and failed with):
if (line.Contains("tertiary"))
{
isMet = true;
}
while (!fileReader.EndOfStream && isMet)
{
Console.WriteLine(line);
}
How can I perform an operation only with my fileReader once a condition has been met?
Your fileReader.EndOfStream loop will only work if your stream has only a single line. The problem is that you're only reading from the stream once - so unless you've already read the whole thing, you're in an endless loop.
Instead, do something like this:
string line;
while ((line = fileReader.ReadLine()) != null)
{
if (line.Contains("...")) break; // Or whatever else you want to do
}
You can use this code. It find any character in the string. It is working for me.
string matchStr = "tertiary";
if (line.Any(matchStr.Contains)
{
isMet = true;
}
while (!fileReader.EndOfStream && isMet)
{
Console.WriteLine(line);
}
The line var line = fileReader.ReadLine(); be inside the while loop?
while (!fileReader.EndOfStream)
{
var line = fileReader.ReadLine();
// do some other stuff
}
Try something like this
var yourval;
var secval = fileReader.ReadLine()
while ((yourval = secval ) != null)
{
if (line.Contains("your string here"))
{
break;
}
}
Related
I have below code which is reading a .json stream line by line. since it will be a lengthy process, I have decided that I will take 100 lines at a time before I call my main function. and so the below code works fine. but this also gives me an issue if number of lines is less than 100, in that case my main function will not be called. how can I optimize my below code to handle both the scenario i.e. read maximum 100 lines at a time and pass it to main function or read all the lines if it is below 100
public async void ReadJsonStream()
{
JsonSerializer serializer = new JsonSerializer();
using (Stream data = await manager.DownloadBlob(null, "TestMultipleLines.json", null))
{
using (StreamReader streamReader = new StreamReader(data, Encoding.UTF8))
{
int counter = 1;
List<string> lines = new List<string>();
while (streamReader.Peek() >= 0)
{
lines.Add(streamReader.ReadLine());
if (counter == 100)
{
counter = 1;
// call main function with line
lines.Clear();
}
counter++;
}
}
}
}
I feel what you are trying to do is wrong. How will you parse 100 lines? Do you want to rebuild from scratch a Json deserializer? And what will happen if some piece of json is split between the line 100 and the line 101?
But in the end, you asked for something and I'll give you what you asked.
public async void ReadJsonStream()
{
JsonSerializer serializer = new JsonSerializer();
using (Stream data = await manager.DownloadBlob(null, "TestMultipleLines.json", null))
{
using (StreamReader streamReader = new StreamReader(data, Encoding.UTF8))
{
List<string> lines = new List<string>();
string line;
while ((line = await streamReader.ReadLineAsync()) != null)
{
lines.Add(line);
if (lines.Count == 100)
{
// call main function with lines
lines.Clear();
}
}
if (lines.Count != 0)
{
// call main function with lines
lines.Clear(); // useless
}
}
}
}
As others noted, you forgot the "additional" call to // call main function with lines at the end of the cycle. I've even modified the code. You don't need to .Peek(), .ReadLine() returns null at the end of the input stream. You made your method async... You could make it fully async by using .ReadLineAsync().
Note that the JsonSerializer of Json.NET already has a Deserialize method that accept a TextReader (and a StreamReader is a TextReader), and that method will read the file "a piece at a time", and won't preload it before parsing it.
Add a check after the while loop. If the lines list is not empty, call main.
public async void ReadJsonStream()
{
JsonSerializer serializer = new JsonSerializer();
using (Stream data = await manager.DownloadBlob(null, "TestMultipleLines.json", null))
{
using (StreamReader streamReader = new StreamReader(data, Encoding.UTF8))
{
int counter = 1;
List<string> lines = new List<string>();
while (streamReader.Peek() >= 0)
{
lines.Add(streamReader.ReadLine());
if (counter == 100)
{
counter = 1;
// call main function with line
lines.Clear();
}
counter++;
}
if (lines.Count > 0)
// call main function with line
}
}
}
``
I'm relatively new to C# and I'm trying to get my head around a problem that I believe should be pretty simple in concept, but I just cant get it.
I am currently, trying to display a message to the console when the program is run from the command line with two arguments, if a sequence ID does not exist inside a text file full of sequence ID's and DNA sequences against a query text file full of Sequence ID's. For example args[0] is a text file that contains 41534 lines of sequences which means I cannot load the entire file into memory.:
NR_118889.1 Amycolatopsis azurea strain NRRL 11412 16S ribosomal RNA, partial sequence
GGTCTNATACCGGATATAACAACTCATGGCATGGTTGGTAGTGGAAAGCTCCGGCGT
NR_118899.1 Actinomyces bovis strain DSM 43014 16S ribosomal RNA, partial sequence
GGGTGAGTAACACGTGAGTAACCTGCCCCNNACTTCTGGATAACCGCTTGAAAGGGTNGCTAATACGGGATATTTTGGCCTGCT
NR_074334.1 Archaeoglobus fulgidus DSM 4304 16S ribosomal RNA, complete sequence >NR_118873.1 Archaeoglobus fulgidus DSM 4304 strain VC-16 16S ribosomal RNA, complete sequence >NR_119237.1 Archaeoglobus fulgidus DSM 4304 strain VC-16 16S ribosomal RNA, complete sequence
ATTCTGGTTGATCCTGCCAGAGGCCGCTGCTATCCGGCTGGGACTAAGCCATGCGAGTCAAGGGGCTT
args[1] is a query text file with some sequence ID's:
NR_118889.1
NR_999999.1
NR_118899.1
NR_888888.1
So when the program is run, all I want are the sequence ID's that were not found in args[0] from args[1] to be displayed.
NR_999999.1 could not be found
NR_888888.1 could not be found
I know this probably super simple, and I have spent far too long on trying to figure this out by myself to the point where I want to ask for help.
Thank you in advance for any assistance.
You can try this.
It loads each file content and compare with each other.
static void Main(string[] args)
{
if ( args.Length != 2 )
{
Console.WriteLine("Usage: {exename}.exe [filename 1] [filename 2]");
Console.ReadKey();
return;
}
string filename1 = args[0];
string filename2 = args[1];
bool checkFiles = true;
if ( !File.Exists(filename1) )
{
Console.WriteLine($"{filename1} not found.");
checkFiles = false;
}
if ( !File.Exists(filename2) )
{
Console.WriteLine($"{filename2} not found.");
checkFiles = false;
}
if ( !checkFiles )
{
Console.ReadKey();
return;
}
var lines1 = System.IO.File.ReadAllLines(args[0]).Where(l => l != "");
var lines2 = System.IO.File.ReadAllLines(args[1]).Where(l => l != "");
foreach ( var line in lines2 )
if ( !lines1.StartsWith(line) )
{
Console.WriteLine($"{line} could not be found");
checkFiles = false;
}
if (checkFiles)
Console.WriteLine("There is no difference.");
Console.ReadKey();
}
This works, but it only processes the first line of the files...
using( System.IO.StreamReader sr1 = new System.IO.StreamReader(args[1]))
{
using( System.IO.StreamReader sr2 = new System.IO.StreamReader(args[2]))
{
string line1,line2;
while ((line1 = sr1.ReadLine()) != null)
{
while ((line2 = sr2.ReadLine()) != null)
{
if(line1.Contains(line2))
{
found = true;
WriteLine("{0} exists!",line2);
}
if(found == false)
{
WriteLine("{0} does not exist!",line2);
}
}
}
}
}
var saved_ids = new List<String>();
foreach (String args1line in File.ReadLines(args[1]))
{
foreach (String args2line in File.ReadLines(args[2]))
{
if (args1line.Contains(args2line))
{
saved_ids.Add(args2line);
}
}
}
using (System.IO.StreamReader sr1 = new System.IO.StreamReader(args[1]))
{
using (System.IO.StreamReader sr2 = new System.IO.StreamReader(args[2]))
{
string line1, line2;
while ((line1 = sr1.ReadLine()) != null)
{
while ((line2 = sr2.ReadLine()) != null)
{
if (line1.Contains(line2))
{
saved_ids.Add(line2);
break;
}
if (!line1.StartsWith(">"))
{
break;
}
if (saved_ids.Contains(line1))
{
break;
}
if (saved_ids.Contains(line2))
{
break;
}
if (!line1.Contains(line2))
{
saved_ids.Add(line2);
WriteLine("The sequence ID {0} does not exist", line2);
}
}
if (line2 == null)
{
sr2.DiscardBufferedData();
sr2.BaseStream.Seek(0, System.IO.SeekOrigin.Begin);
continue;
}
}
}
}
I am getting null reference exception when reading data from my txt file.
public class Appointments : List<Appointment>
{
Appointment appointment;
public Appointments()
{
}
public bool Load(string fileName)
{
string appointmentData = string.Empty;
using (StreamReader reader = new StreamReader(fileName))
{
while((appointmentData = reader.ReadLine()) != null)
{
appointmentData = reader.ReadLine();
//**this is where null ref. exception is thrown** (line below)
if(appointmentData[0] == 'R')
{
appointment = new RecurringAppointment(appointmentData);
}
else
{
appointment = new Appointment(appointmentData);
}
this.Add(appointment);
}
return true;
}
}
RecurringAppointment inherits from Appointments. File exists, file location is correct. Funny thing is that program was working 30 min ago I've only changed Load method from below to what u can see above :
public bool Load(string fileName)
{
string appointmentData = string.Empty;
using (StreamReader reader = new StreamReader(fileName))
{
while((appointmentData = reader.ReadLine()) != null)
{
appointmentData = reader.ReadLine();
if(appointmentData[0] == 'R')
{
this.Add(appointment = new RecurringAppointment(appointmentData));
}
else
{
this.Add(appointment = new Appointment(appointmentData));
}
}
return true;
}
}
Now it does not work in either case.
Your code reads two times at each loop. This means that, if your file has an odd number of rows when you read the last line of the file, the check against null inside the while statement allows your code to enter the loop but the following ReadLine returns a null string. Of course trying to read the char at index zero of a null string will throw the NRE exception.
There is also the problem of empty lines in your file. If there is an empty line then, again reading at index zero will throw an Index out of range exception
You could fix your code in this way
public bool Load(string fileName)
{
string appointmentData = string.Empty;
using (StreamReader reader = new StreamReader(fileName))
{
while((appointmentData = reader.ReadLine()) != null)
{
if(!string.IsNullOrWhiteSpace(appointmentData))
{
if(appointmentData[0] == 'R')
this.Add(appointment = new RecurringAppointment(appointmentData));
else
this.Add(appointment = new Appointment(appointmentData));
}
}
return true;
}
}
I've made a program and I want to save the data. Saving is working, but "Loading" doesn't work.
public void Save(StreamWriter sw)
{
for (int i = 0; i < buecher.Count; i++)
{
Buch b = (Buch)buecher[i];
if (i == 0)
sw.WriteLine("ISDN ; Autor ; Titel");
sw.WriteLine(b.ISDN + ";" + b.Autor + ";" + b.Titel);
}
}
public void Load(StreamReader sr)
{
int isd;
string aut;
string tit;
while (sr.ReadLine() != "")
{
string[] teile = sr.ReadLine().Split(';');
try
{
isd = Convert.ToInt32(teile[0]);
aut = teile[1];
tit = teile[2];
}
catch
{
throw new Exception("umwandlung fehlgeschlagen");
}
Buch b = new Buch(isd, aut, tit);
buecher.Add(b);
}
}
If I'm doing that with an break after buecher.Add(b); than its everything fine but it obviously shows me only 1 book... if I'm not using the break he gives me an error "nullreference.."
Would be awesome if someone could help me
best regards
Ramon
The problem is that you are reading two lines for each iteration in the loop (and throwing away the first one). If there are an odd number of lines in the file, the second call to Read will return null.
Read the line into a variable in the condition, and use that variable in the loop:
public void Load(StreamReader sr) {
int isd;
string aut;
string tit;
// skip header
sr.ReadLine();
string line;
while ((line = sr.ReadLine()) != null) {
if (line.Length > 0) {
string[] teile = line.Split(';');
try {
isd = Convert.ToInt32(teile[0]);
aut = teile[1];
tit = teile[2];
} catch {
throw new Exception("umwandlung fehlgeschlagen");
}
Buch b = new Buch(isd, aut, tit);
buecher.Add(b);
}
}
}
You are calling sr.ReadLine() twice for every line, once in the while() and once right after. You are hitting the end of the file, which returns a null.
Different approach to this but I suggest it because it's simpler;
Load(string filepath)
{
try
{
List<Buch> buches = File.ReadAllLines(filepath)
.Select(x => new Buch(int.Parse(x.Split(';')[0]), x.Split(';')[1], x.Split(';')[2]));
{
catch
{
throw new Exception("umwandlung fehlgeschlagen");
}
}
You could do it in more lines if you find it to be more readable but I've come to prefer File.ReadAllText and File.ReadAllLines to StreamReader approach of reading files.
Instead of using the LINQ statement you could also do;
Load(string filepath)
{
try
{
string[] lines = File.ReadAllLines(filepath);
foreach (string line in lines)
{
string[] tokens = line.Split(';');
if (tokens.Length != 3)
// error
int isd;
if (!int.TryParse(tokens[0], out isd))
//error, wasn't an int
buetcher.Add(new Buch(isd, tokens[1], tokens[2]);
}
{
catch
{
throw new Exception("umwandlung fehlgeschlagen");
}
}
I'm designing an API. Currently I'm trying to safely handle a condition where we run out of disk space. Basically, we have a series of files holding some data. When the disk is full, when we go to write another data file, it will of course throw an error. At this point, we delete a single file(loop through file list from oldest to newest and retry after we successfully delete a file). Then, we retry writing the file. Repeat that process until the file is written without error.
Now the fun part. All of this happens concurrently. Like, at some point there are 8 threads doing this at once. This makes things extra interesting, and has lead to an odd error.
Here is the code
public void Save(string text, string id)
{
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
var existing = store.GetFileNames(string.Format(Prefix + "/*-{0}.dat", id));
if (existing.Any()) return; //it already is saved
string name = string.Format(Prefix + "/{0}-{1}.dat", DateTime.UtcNow.ToString("yyyyMMddHHmmssfffffff"), id);
tryagain:
bool doover=false;
try
{
AttemptFileWrite(store, name, text);
}
catch (IOException)
{
doover = true;
}
catch (IsolatedStorageException) //THIS LINE
{
doover = true;
}
if (doover)
{
Attempt(() => store.DeleteFile(name)); //because apparently this can also fail.
var files = store.GetFileNames(Path.Combine(Prefix, "*.dat"));
foreach (var file in files.OrderBy(x=>x))
{
try
{
store.DeleteFile(Path.Combine(Prefix, file));
}
catch
{
continue;
}
break;
}
goto tryagain; //prepare the velociraptor shield!
}
}
}
void AttemptFileWrite(IsolatedStorageFile store, string name, string text)
{
using (var file = store.OpenFile(
name,
FileMode.Create,
FileAccess.ReadWrite,
FileShare.None | FileShare.Delete
))
{
using (var writer = new StreamWriter(file))
{
writer.Write(text);
writer.Flush();
writer.Close();
}
file.Close();
}
}
static void Attempt(Action func)
{
try
{
func();
}
catch
{
}
}
static T Attempt<T>(Func<T> func)
{
try
{
return func();
}
catch
{
}
return default(T);
}
public string GetSaved()
{
string content=null;
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
var files = store.GetFileNames(Path.Combine(Prefix,"*.dat")).OrderBy(x => x);
if (!files.Any()) return new MessageBatch();
foreach (var filename in files)
{
IsolatedStorageFileStream file=null;
try
{
file = Attempt(() =>
store.OpenFile(Path.Combine(Prefix, filename), FileMode.Open, FileAccess.ReadWrite, FileShare.None | FileShare.Delete));
if (file == null)
{
continue; //couldn't open. assume locked or some such
}
file.Seek(0L, SeekOrigin.Begin);
using (var reader = new StreamReader(file))
{
content = reader.ReadToEnd();
}
//take note here. We delete the file, while we still have it open!
//This is done because having the file open prevents other readers, but if we close it first,
//then there is a race condition that right after closing the stream, another reader could pick it up and
//open exclusively. It looks weird, but it's right. Trust me.
store.DeleteFile(Path.Combine(Prefix, filename));
if (!string.IsNullOrEmpty(content))
{
break;
}
}
finally
{
if (file != null) file.Close();
}
}
}
return content;
}
At the line marked THIS LINE, is what I'm talking about. When doing AttemptFileWrite, I can look over at store.AvailableSpace and see that there is enough room to fit the data into it, but upon trying to open the file, it throws this IsolatedStorageException with the description of Operation Not Permitted. Aside from this weird case, in all other cases it's just an IOException thrown with a message about the disk being full
I'm trying to figure out if I have some odd race condition, or if this is an error I just have to deal with or what?
Why does this error occur?