Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have a minor project that i've been working on for a while for an Independent study class. I am supposed to import around a 140,000 data points for a IDW based math analysis, currently it takes the program around 10~14 minutes to import all my points.
What I'm doing is reading off of a .txt file, splitting based on new line, and then splitting future based on spaces betweeen the fields. Then turning them into Datapt objects i designed for easy OOD style manipulation.
All i want to know is that I'll probably have to do a live demo with the program and i don't want people to have to sit there for 14 minutes while it preloads, if push comes to shove i could find a laptop from a friend (My main comp is a dekstop) and preload it on there before my presentation but the whole issue is making me wonder why is it taking so long to load only 100k datapts? I'd figure it would take a shorter amount of time? If there a quicker way than anyone knows of, it would be greatly appreciated if you could share it!
private void openPointsToolStripMenuItem_Click(object sender, EventArgs e)
{
openFileDialog1.Filter = "Text files|*.txt|All files|*.*";
openFileDialog1.Title = "Open the Captured Packets";
openFileDialog1.ShowDialog();
//Check to see if a filename was given
if (openFileDialog1.FileName != "")
{
readOut = System.IO.File.ReadAllText(openFileDialog1.FileName);
//textBox1.Text = System.IO.File.ReadAllText(openFileDialog1.FileName);
dataChain = readOut.Split(new String[] { "\r\n", "\n" }, StringSplitOptions.None);
//Read out Code
string[] link; //dataChain[0].Split(null);
for(int i = 0; i < 100000; i++)
{
link = dataChain[i].Split(null);
textBox1.AppendText(link[0] + " " + link[1] + " " + link[2] + " "+ link[3] + "\r\n");
dataPt Temp = new dataPt(Convert.ToDouble(link[0]), Convert.ToDouble(link[1]), Convert.ToDouble(link[2]), Convert.ToDouble(link[3]));
dataList.Add(Temp);
ptDisplay.Items.Add(Temp.ToString());
}
}
}
One improvement that comes to mind is that you don't need to load the whole file in memory. You can process it line by line using the ReadLines method which returns an Enumerable<string> on which you can further filter down the results using the Take extension method:
private void openPointsToolStripMenuItem_Click(object sender, EventArgs e)
{
openFileDialog1.Filter = "Text files|*.txt|All files|*.*";
openFileDialog1.Title = "Open the Captured Packets";
openFileDialog1.ShowDialog();
if (openFileDialog1.FileName != "")
{
foreach (string line in System.IO.File.ReadLines(openFileDialog1.FileName).Take(100000))
{
var link = line.Split(null);
textBox1.AppendText(link[0] + " " + link[1] + " " + link[2] + " "+ link[3] + "\r\n");
dataPt Temp = new dataPt(Convert.ToDouble(link[0]), Convert.ToDouble(link[1]), Convert.ToDouble(link[2]), Convert.ToDouble(link[3]));
dataList.Add(Temp);
ptDisplay.Items.Add(Temp.ToString());
}
}
}
Do not load the text all in memory at once. Instead use the File.ReadLines in your inner loop to enumerate the lines and process them one by one, but it is also very important to not change the TextBox.Text at each line. This is very expensive both in terms of execution and in terms of memory footprint. (Strings are immutable, so at each loop a new string is allocated in memory and the previous one is discarder wreaking havoc with memory fragmentation)
StringBuilder sb = new StringBuilder();
foreach(string line in File.ReadLines(openFileDialog1.FileName))
{
link = line.Split();
sb.AppendLine(link[0] + " " + link[1] + " " + link[2] + " "+ link[3]);
dataPt Temp = new dataPt(Convert.ToDouble(link[0]), Convert.ToDouble(link[1]), Convert.ToDouble(link[2]), Convert.ToDouble(link[3]));
dataList.Add(Temp);
ptDisplay.Items.Add(Temp.ToString());
}
textBox1.AppendText(sb.ToString());
Instead add every line to a StringBuilder class that handles a lot better the string concatenated together than a TextBox.Text property. Then when exiting the loop, change the TextBox.Text just one time.
Related
Sorry for the bad title, i didn't know how to explain it better. I just started in Csharp so its probably a dumb mistake.
private void timer1_Tick(object sender, EventArgs e)
{
label1.Text = "CPU Usage" + " " + (int)cpuCounter.NextValue() + "%";
string[] usageCPUay = { label1.Text };
System.IO.File.WriteAllLines(#"C:\Users\Filip\Desktop\CPUOutput.txt", usageCPUay);
}
So this is the code I have issues with and can't understand how to fix it. I tried with streamwriter but I got the same issues. I want to make the output the cpuCounter gives to store in a txt file. But every time it just writes the latest CPU Usage and not every one. Like this wrong output. I want it to type all the cpu usages it got and store them like that but every new one in a separate line.
Let's analyze your code:
you have a method, in the body you have label1.text which is text I assume, a string.
then you have got an array of strings, but this array contains only a single element, which is label1.text, then you write in the text file the array, which contains only a single element.
3 errors I see:
1.- the array contains a single element, therefore you are only storing a single line.
2.- at the end of the string that you are meant to record in the text file you should add "\n" for a new line.
3.- I understand from https://learn.microsoft.com/en-us/dotnet/api/system.io.file.writealllines?view=netcore-3.1#System_IO_File_WriteAllLines_System_String_System_String___ that it creates the file as new, therefore you may be deleting what you already have.
Solution:
Like in the example that was presented to you in the link, I would do the following:
label1.Text = "CPU Usage" + " " + (int)cpuCounter.NextValue() + "%\n";
string[] usageCPUay = { label1.Text };
if (!File.Exists(path))
{
// Create a file to write to.
File.WriteAllLines(path, usageCPUay);
}
else{
File.AppendAllText(path, usageCPUay);}
So, I am running SSIS (through VS) and I have two segments that hang me up when my clients don't send in the exact files every day. I have a task that deletes old files, and then renames the current files to the filename with _OLD at the end of it.
The issue is: If the files that are in there aren't the exact same, it crashes, failing the entire thing.
An example:
A client sends in on Monday files: Names, Addresses, Grades, Schools
The same client, on Tuesday sends in: Names, Addresses, Schools
Since the Grades file doesn't exist, it still gets renamed to Grades_OLD but the SSIS fails.
The scripts are:
del Names_OLD.csv
bye
This will then go to the Rename Script:
ren Names.csv Names_OLD.csv
bye
and will then go on to Addresses, to do the same thing. It is super frustrating that these fail when a single file doesn't exist the next day, and there doesn't seem to be a need for it.
We have two scripts that generate the archive data to process:
public void Main()
{
Dts.Variables["ARCHIVEFILE"].Value = Path.GetFileNameWithoutExtension(Dts.Variables["FTPFILE"].Value.ToString()) + "_OLD" + Path.GetExtension(Dts.Variables["FTPFILE"].Value.ToString());
Dts.TaskResult = (int)ScriptResults.Success;
}
and
public void Main()
{
/*PSFTP_DEL_script.txt
del %1
bye
PSFTP_REN_script.txt
ren %1 %2
bye
*/
var lineOut = String.Empty;
var File1 = Dts.Variables["User::FTPWORKINGDIR"].Value.ToString() + "\\SSIS_PSFTP_DEL_script.txt";
var File2 = Dts.Variables["User::FTPWORKINGDIR"].Value.ToString() + "\\SSIS_PSFTP_REN_script.txt";
lineOut = "del " + Dts.Variables["User::ARCHIVEFILE"].Value.ToString() + Environment.NewLine + "bye";
System.IO.File.WriteAllText(File1, lineOut);
lineOut = "ren " + Dts.Variables["User::FTPFILE"].Value.ToString() + " " + Dts.Variables["User::ARCHIVEFILE"].Value.ToString() + Environment.NewLine + "bye";
System.IO.File.WriteAllText(File2, lineOut);
Dts.TaskResult = (int)ScriptResults.Success;
}
Researching it doesn't really give anything helpful, and kind of just leads me back to where I am right now.
Try using a foreach loop on files for each file that can be processed and put all the processing of the file inside it. And do not put any precendence constraints between the foreach loops.
This will process the files that are there an not fail when the others aren't there.
The foreach loop essentially works as a check if the file exists.
This assumes you do not need all the files to properly process them.
Why not checking if the file exists before writing the script:
if (System.IO.File.Exists(Dts.Variables["User::ARCHIVEFILE"].Value.ToString())){
lineOut = "del " + Dts.Variables["User::ARCHIVEFILE"].Value.ToString() + Environment.NewLine + "bye";
System.IO.File.WriteAllText(File1, lineOut);
}
if (Dts.Variables["User::FTPFILE"].Value.ToString())){
lineOut = "ren " + Dts.Variables["User::FTPFILE"].Value.ToString() + " " + Dts.Variables["User::ARCHIVEFILE"].Value.ToString() + Environment.NewLine + "bye";
System.IO.File.WriteAllText(File2, lineOut);
}
So I am currently trying to do this question for tafe but am having trouble getting it to add text to the file after the first time.
"Create an application that at start up asks the user to choose an output file using a Save File Dialog Window. The application to have a “Write to File” button that when clicked will take a name and age that the user has entered into two text boxes and write them directly to the file. The user can repeat the button click action as often as they wish."
This is my current code for it:
private void Form4_Load(object sender, EventArgs e)
{
saveFileDialog1.Title = ("Choose Save Location");
saveFileDialog1.ShowDialog();
}
private void btnSendToFile_Click(object sender, EventArgs e)
{
string strName = txtName.Text;
string strAge = txtAge.Text;
string strTitles = ("Name \t\t Age");
string strCombined = strTitles + "\n" + (strName + "\t\t" + strAge);
System.IO.StreamWriter OutFile;
MessageBox.Show("The Name and Age of the Person Entered Will be Written to a File");
OutFile = System.IO.File.CreateText(saveFileDialog1.FileName);
OutFile.WriteLine(strCombined);
OutFile.Close();
MessageBox.Show("The Details Have Been Written to File" + saveFileDialog1.FileName);
StreamWriter AddFile = File.AppendText(saveFileDialog1.FileName);
AddFile.Write(strCombined);
}
I'm not sure if I should be doing a loop or not and this is the form itself, https://gyazo.com/e2c4170d46295d6f92a35026e1f2304b
Any help is appreciated, Thanks in advance
For what you're doing you absolutely don't need to work with streams or anything that low level (you don't need to call the create methods on files and use a stream returned by it). You can directly use the methods that write or append to the file and handle all of this for you.
For example using AppendAllText will do everything for you (managing the streams, flushing and closing them, appending at the end, creating the file if it doesn't exist etc etc).
so you can replace all of this code :
System.IO.StreamWriter OutFile;
MessageBox.Show("The Name and Age of the Person Entered Will be Written to a File");
OutFile = System.IO.File.CreateText(saveFileDialog1.FileName);
OutFile.WriteLine(strCombined);
OutFile.Close();
MessageBox.Show("The Details Have Been Written to File" + saveFileDialog1.FileName);
with
File.AppendAllText(saveFileDialog1.FileName, strCombined);
I have no idea what you're doing in the code that follows it (calling appendtext, i assume you're thinking you need to append after creating?) in any case you don't need it. So if you just want to append all the text of "strcombined" at the end of the file each time you click the whole function should look like :
private void btnSendToFile_Click(object sender, EventArgs e)
{
string strName = txtName.Text;
string strAge = txtAge.Text;
string strTitles = ("Name \t\t Age");
string strCombined = strTitles + "\n" + (strName + "\t\t" + strAge);
MessageBox.Show("The Name and Age of the Person Entered Will be Written to a File");
File.AppendAllText(saveFileDialog1.FileName, strCombined);
MessageBox.Show("The Details Have Been Written to File" + saveFileDialog1.FileName);
}
Also as a side comment i would rewrite all of it as such for readability, these are suggestions and not hard rules:
1) Removed hungarian notation (Don't add a prefix to your variable to indicate it's type, it's generally frowned upon)
2) Removed a lot of the temporary variables that aren't really needed (used once in the same local function)
This makes for a much smaller and very readable function:
private void btnSendToFile_Click(object sender, EventArgs e)
{
MessageBox.Show("The Name and Age of the Person Entered Will be Written to a File");
File.AppendAllText(saveFileDialog1.FileName,"Name \t\t Age"+ "\n" + txtName.Text + "\t\t" + txtAge.Text);
MessageBox.Show("The Details Have Been Written to File" + saveFileDialog1.FileName);
}
C#, Winforms:
I have a log file I need to parse. This file contains transactions requests from a program, but the program writes the transaction across multiple lines.
I need to get the ID# and if the request was processed or denied for whatever reason. The problem is that these requests are on multiple lines. My only saving grace is that they contain the same time stamp from the logger. The (##) is not usable since it is a temporary placeholder, thus (19) may repeat multiple times throughout the log.
I was thinking of scanning for a PR_Request, substringing the ID# and the time stamp, but I dont know how to make a streamreader move down to the next 4 lines and write it out to be one single line in a file.
Examples:
06/10/16 08:09:33.031 (1) PR_Request: IID=caa23b14,
06/10/16 08:09:33.031 (1) PR_Mon: IID=caa23b14,
06/10/16 08:09:33.031 (1) RESUME|BEGIN
06/10/16 08:09:33.031 (1) RESUME_TRIG|SC-TI
06/10/16 08:19:04.384 (19) PR_Request: IID=90dg01b,
06/10/16 08:19:04.384 (19) PR_Mon: IID=90dg01b,
06/10/16 08:19:04.384 (19) RESUME|DENIED: Access not granted.
I need output to be in a single line for a file. That way, I can just parse it with another program and feed the data into a database.
06/10/16 08:09:33.031 PR_Request: IID=caa23b14 | RESUME | BEGIN | RESUME_TRIG | SC-TI
06/10/16 08:19:04.384 PR_Request: IID=90dg01b | RESUME | DENIED: Access not granted.
EDIT:
Okay I think I have a base code here. It works, kind of. It takes such a long time because I had to open another file streamer when it found a match to PR_Request, then scan the file again with the same fullstamp (date + process number). It will then look for RESUME|BEGIN or RESUME|DENIED and then write out that it succeeded or failed.
Is there any way to perhaps speed this up by getting the streamreader line where it originally found the PR_Request, have it start on another line, count maybe to 5 more lines, then stop it? This would help speed up the program considerably.
string inputfolder = inputloctxt.Text;
string outputfolder = saveloctxt.Text;
string outputfile = #"ParsedFile.txt";
try
{
string[] readfromdir = Directory.GetFiles(outputfolder);
foreach (string readnow in readfromdir)
{
using (StreamReader fileread = new StreamReader(readnow))
{
string fileisreading;
while ((fileisreading = fileread.ReadLine()) != null)
{
if (fileisreading.Contains("PR_Request"))
{
string resumed = null;
string fullstamp = fileisreading.Substring(1, 26);
string datestamp = fileisreading.Substring(1, 21);
string requesttype = fileisreading.Substring(27, 22);
string iidnum = fileisreading.Substring(53, 8);
using (StreamReader grabnext01 = new StreamReader(readnow))
{
string grabnow01;
while ((grabnow01 = grabnext01.ReadLine()) != null)
{
if (grabnow01.Contains(fullstamp))
{
if (grabnow01.Contains("RESUME|BEGIN"))
{
resumed = "TRUE";
break;
}
else if (grabnow01.Contains("RESUME|DENIED"))
{
resumed = "FALSE";
break;
}
}
}
}
File.AppendAllText(outputfolder + outputfile,
datestamp + " " + requesttype + " " + iidnum + " " + resumed + Environment.NewLine);
resumed = null;
}
}
}
}
}
This sounds like you need to use Regular Expressions. There is a namespace System.Text.RegularExpressions you can use and reference the capture groups that I made for you in the example.
Use these sites for reference:
https://regex101.com/
https://msdn.microsoft.com/en-us/library/bs2twtah(v=vs.110).aspx
I started off the Regex for you, it is not pretty but it should get the job done.
(?:\d{2}\/\d{2}\/\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{3}\s\(\d+\)\s)(PR_Request: IID=[^,\n]+)(?:\,\n\d{2}\/\d{2}\/\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{3}\s\(\d+\)\sPR_Mon: IID=[^,\n]*\,\n\d{2}\/\d{2}\/\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{3}\s\(\d+\)\s)((RESUME|BEGIN|\||DENIED: Access not granted.)*)(?:\n\d{2}\/\d{2}\/\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{3}\s\(\d+\)\s)*((RESUME_TRIG|SC\-TI|\|)*)
I'm wondering if it is possible to search and read from a file and display what's in the file in a message box.
I'm wanting to search for a file by its ID, which ID is known by the user. When the user enters the ID my program opens the file which shares the same ID; eg.ID.txt in the preset folder.
when it's selected it is then read and put in a MessageBox which will then display what is in the file.
Can anyone show me how to do this?
Thanks.
//Declare variables
int TID;
private void TIDFileCreate_Click(object sender, EventArgs e)
{
StreamWriter outputFile;
outputFile = File.CreateText (TID.ToString()+".txt");
outputFile.WriteLine("Investor :" +" " + InvestorNameLabel.Text);
outputFile.WriteLine("Initial Amount" + " " +AmountLabel.Text);
outputFile.WriteLine("Date Invested" +" " +DateLabel.Text);
outputFile.WriteLine("Period Chosen" + " "+DaysInvestedLabel.Text);
outputFile.WriteLine("Rate Chosen" + " " + RateLabel.Text);
outputFile.WriteLine("Total Interest" + " " +InterestAmountLabel.Text);
outputFile.WriteLine("Transaction Number :" + " " + TransactionIDLabel.Text);
outputFile.Close();
MessageBox.Show("Transaction file for Transaction: " + TransactionIDLabel.Text + "Was Created", "Transaction File");
}
private void SearchButton_Click(object sender, EventArgs e)
{
SearchID = int.Parse(searchTextBox.Text);
string[] lines = File.ReadAllLines(#"C:\Users\Public\TestFolder\"+SearchID+".txt");
}
Can't you just use MessageBox.Show() in your SearchButton_Click method?
private void SearchButton_Click(object sender, EventArgs e)
{
SearchID = int.Parse(searchTextBox.Text);
string[] lines = File.ReadAllLines(#"C:\Users\Public\TestFolder\"+SearchID+".txt");
System.Windows.Forms.MessageBox.Show(string.Join("\r\n", lines));
}
I think that for what you want to do you should read this:
http://msdn.microsoft.com/en-us/library/ms233843.aspx
or some similar article about serialization.
This said, if the number of IDs grow in number, you should considering the adoption of a lightweight DB, i.e. SQLite, just to name one.
Take a look at the DirectoryInfo class. Something like the following should do what you want;
DirecectoryInfo dir = new DirectoryInfo(pathToRoot);
FileInfo[] files = dir.GetFiles("Id.txt"); //if using a specific name should return 1 item
TextBox.Text = File.ReadAllText(files.FirstOrDefault().FullName);
Keep in mind I have no error handling or anything here. You'll have to add that yourself, this just shows you the key classes/methods to get the files and read their contents. There are many other options like doing GetFiles(), keeping that collection in memory and then operating on them later on.