We Keep Coding

Load huge txt file for winform quickly

I am going to make a sinhala english dictionary. SO i have a file that contains sinhala meaning for every english word. So i thought to load it while form is loading. So i added following command to get all file content to a string variable. SO i used following command in FormLoad method,
private string DictionaryWords = "";
private string ss = null;
...
private void Form1_Load(object sender, EventArgs e)
{
this.BackColor = ColorTranslator.FromHtml("#AFC3E0");
string fileName = #"SI-utf8.Txt";
using (StreamReader sr = File.OpenText(fileName))
{
while ((ss = sr.ReadLine()) != null)
{
DictionaryWords += ss;
}
}
}
But unfortunately that txt file has 130000+ line and it size it more than 5MB. SO my winform not loading.
see the image
I need to load this faster for winform to use REGEX form getting right meaning for every english word..
Could anybody tell me a method to do this. I tried everything.
Load this huge file to my project within 15 more less and need to use Regex for finding each english words..

Well, there are too little code to analyze. I suspect that
DictionaryWords += ss;
is the felon: appending string 130000 times which means re-creating quite long string over and over again can well put the system on the knees, but I have not rigorous proof (I've asked about DictionaryWords in the comment). Another possible candidate to be blamed is the unknown for me your regular expression.
That's why let me try to solve the problem from scratch.
We a have a (long) dictionary in SI-utf8.Txt.
We should load the dictionary without freezing the UI.
We should use the dictionary loaded to translate the English texts.
I have got something like this:
using System.IO;
using System.Linq;
using System.Threading.Tasks;
...
// Loading dictionary (async, since dictionary can be quite long)
// static: we want just one dictionary for all the instances
private static readonly Task<IReadOnlyDictionary<string, string>> s_Dictionary =
Task<IReadOnlyDictionary<string, string>>.Run(() => {
char[] delimiters = { ' ', '\t' };
IReadOnlyDictionary<string, string> result = File
.ReadLines(#"SI-utf8.Txt")
.Where(line => !string.IsNullOrWhiteSpace(line))
.Select(line => line.Split(delimiters, StringSplitOptions.RemoveEmptyEntries))
.Where(items => items.Length == 2)
.ToDictionary(items => items[0],
items => items[1],
StringComparer.OrdinalIgnoreCase);
return result;
});
Then we need a translation part:
// Let it be the simplest regex: English letters and apostrophes;
// you can improve it if you like
private static readonly Regex s_EnglishWords = new Regex("[A-Za-z']+");
// Tanslation is async, since we have to wait for dictionary to be loaded
private static async Task<string> Translate(string englishText) {
if (string.IsNullOrWhiteSpace(englishText))
return englishText;
var dictionary = await s_Dictionary;
return s_EnglishWords.Replace(englishText,
match => dictionary.TryGetValue(match.Value, out var translation)
? translation // if we know the translation
: match.Value); // if we don't know the translation
}
Usage:
// Note, that button event should be async as well
private async void button1_Click(object sender, EventArgs e) {
TranslationTextBox.Text = await Translate(OriginalTextBox.Text);
}
Edit: So, DictionaryWords is a string and thus
DictionaryWords += ss;
is a felon. Please, don't append string in a (deep) loop: each append re-creates the string which is slow. If you insist on the looping, use StringBuilder:
// Let's pre-allocate a buffer for 6 million chars
StringBuilder sb = new StringBuilder(6 * 1024 * 1024);
using (StreamReader sr = File.OpenText(fileName))
{
while ((ss = sr.ReadLine()) != null)
{
sb.Append(ss);
}
}
DictionaryWords = sb.ToString();
Or, why should you loop at all? Let .net do the work for you:
DictionaryWords = File.ReadAllText(#"SI-utf8.Txt");
Edit 2: If actual file size is not that huge (it is DictionaryWords += ss; alone who spoils the fun) you can stick to a simple synchronous solution:
private static readonly Regex s_EnglishWords = new Regex("[A-Za-z']+");
private static readonly IReadOnlyDictionary<string, string> s_Dictionary = File
.ReadLines(#"SI-utf8.Txt")
.Where(line => !string.IsNullOrWhiteSpace(line))
.Select(line => line.Split(new char[] { ' ', '\t' },
StringSplitOptions.RemoveEmptyEntries))
.Where(items => items.Length == 2)
.ToDictionary(items => items[0],
items => items[1],
StringComparer.OrdinalIgnoreCase);
private static string Translate(string englishText) {
if (string.IsNullOrWhiteSpace(englishText))
return englishText;
return s_EnglishWords.Replace(englishText,
match => s_Dictionary.TryGetValue(match.Value, out var translation)
? translation
: match.Value);
}
An then the usage is quite simple:
// Note, that button event should be async as well
private void button1_Click(object sender, EventArgs e) {
TranslationTextBox.Text = Translate(OriginalTextBox.Text);
}

Read Text File, Update Fields C# and WPF

I am trying to basically create config files. A text file will hold something like:
Name::Adam
Location::Washington
I am trying to grab the first part as the field name (i.e. Name.Text would update the TextBox) then put the second part to that Text. Just not sure where to go or what the best way to build this is. The code below is incomplete because I can't figure out how to update the textboxes.
Thanks for the help!
private void clickImportConfig_ItemClick(object sender, DevExpress.Xpf.Bars.ItemClickEventArgs e)
{
Stream myStream = null;
string fieldUpdate = string.Empty;
string fieldUpdateTo = string.Empty;
try
{
using (myStream)
{
string[] lines = File.ReadAllLines(#"c:\\config.txt");
foreach (string s in lines)
{
var splitted = Regex.Split(s, "::");
fieldUpdate = splitted[0].ToString();
fieldUpdateTo = splitted[1].ToString();
}
}
}
catch (Exception ex)
{
MessageBox.Show("Error: Could not read file from disk. Original error: " + ex.Message);
}
}

I think this is what you're looking for:
private void clickImportConfig_ItemClick(object sender, DevExpress.Xpf.Bars.ItemClickEventArgs e)
{
Stream myStream = null;
string fieldUpdate = string.Empty;
string fieldUpdateTo = string.Empty;
try
{
using (myStream)
{
string[] lines = File.ReadAllLines(#"c:\\config.txt");
foreach (string s in lines)
{
string[] splitted = s.Split(new string[] { "::" }, StringSplitOptions.RemoveEmptyEntries);
fieldUpdate = splitted[0].ToString();
fieldUpdateTo = splitted[1].ToString();
// TextBox textBox = (TextBox)this.FindName(fieldUpdate);
// Or
TextBox textBox = this.FindName(fieldUpdate) as TextBox;
// See below for an explanation
if (textBox != null) // FindName returns null if nothing is found with that name
{
textBox.Text = fieldUpdateTo;
}
}
}
}
catch (Exception ex)
{
MessageBox.Show("Error: Could not read file from disk. Original error: " + ex.Message);
}
}
As insane_developer pointed out, you will be better off using the String.Split method (s being the string in this case so written as s.Split()) instead of Regex.Split. It will give you the benefit of removing any empty results from the array. It may also have better performance as Regex is capable of a lot more complicated things, but I haven't tested that so I could be wrong.
You can use the FindName(string name) method to find an element with the specified name. This method returns null if nothing is found and an object if the element is found. This object will need to be cast to the type you are expecting (I.e. TextBox). You can do this in one of the following ways:
TextBox textBox = (TextBox)this.FindName(fieldUpdate);
or
TextBox textBox = this.FindName(fieldUpdate) as TextBox;
The first option will throw an InvalidCastException if FindName returns an object which is not a TextBox. The second option will instead just set the value of textBox to null which will be checked by the if statement and the exception will be avoided. As you are only catching all generic exceptions in this code, an InvalidCastException would show your "Could not read file from disk" message which is not true. So you may want to add an additional catch block to handle any invalid casting.
If you're wondering why you don't just stick to the second option as it solves this problem, then consider this scenario as an example. Lets say in the future you decide for some reason that you want to change all of your TextBox to TextBlock or something else, but forget to come back to change this code, or accidently end up with the name of another type of control in your text file. The second option will set the value of textBox to null and your field(s) won't be updated. But there will be absolutely no errors, leaving you scratching your head and having to debug the problem. The first option would throw an InvalidCastException showing you exactly where the problem is. You could then choose how to handle this problem by either showing another message box or silently writing the error to a log file etc.

You don't need a regular expression, just:
var splitted = s.Split("::", StringSplitOptions.RemoveEmptyEntries);
fieldUpdate = splitted[0];
fieldUpdateTo = splitted[1];
For the rest you have to be more explicit

Reading text and variables from text file c#

I have the following code which tries to read data from a text file (so users can modify easily) and auto format a paragraph based on a the words in the text document plus variables in the form. I have the file "body" going into a field. my body text file has the following data in it
"contents: " + contents
I was hoping based on that to get
contents: Item 1, 2, etc.
based on my input. I only get exactly whats in the text doc despite putting "". What am I doing wrong? I was hoping to get variables in addition to my text.
string readSettings(string name)
{
string path = System.Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) + "/Yuneec_Repair_Inv";
try
{
// Create an instance of StreamReader to read from a file.
// The using statement also closes the StreamReader.
using (StreamReader sr = new StreamReader(path + "/" + name + ".txt"))
{
string data = sr.ReadToEnd();
return data;
}
}
catch (Exception e)
{
// Let the user know what went wrong.
Console.WriteLine("The settings file for " + name + " could not be read:");
Console.WriteLine(e.Message);
string content = "error";
return content;
}
}
private void Form1_Load(object sender, EventArgs e)
{
createSettings("Email");
createSettings("Subject");
createSettings("Body");
yuneecEmail = readSettings("Email");
subject = readSettings("Subject");
body = readSettings("Body");
}
private void button2_Click(object sender, EventArgs e)
{
bodyTextBox.Text = body;
}

If you want to provide the ability for your users to customize certain parts of the text you should use some "indicator" that you know before hand, that can be searched and parsed out, something like everything in between # and # is something you will read as a string.
Hello #Mr Douglas#,
Today is #DayOfTheWeek#.....
At that point your user can replace whatever they need in between the # and # symbols and you read that (for example using Regular Expressions) and use that as your "variable" text.
Let me know if this is what you are after and I can provide some C# code as an example.
Ok, this is the example code for that:
StreamReader sr = new StreamReader(#"C:\temp\settings.txt");
var set = sr.ReadToEnd();
var settings = new Regex(#"(?<=\[)(.*?)(?=\])").Matches(set);
foreach (var setting in settings)
{
Console.WriteLine("Parameter read from settings file is " + setting);
}
Console.WriteLine("Press any key to finish program...");
Console.ReadKey();
And this is the source of the text file:
Hello [MrReceiver],
This is [User] from [Company] something else, not very versatile using this as an example :)
[Signature]
Hope this helps!

When you read text from a file as a string, you get a string of text, nothing more.
There's no part of the system which assumes it's C#, parses, compiles and executes it in the current scope, casts the result to text and gives you the result of that.
That would be mostly not what people want, and would be a big security risk - the last thing you want is to execute arbitrary code from outside your program with no checks.
If you need a templating engine, you need to build one - e.g. read in the string, process the string looking for keywords, e.g. %content%, then add the data in where they are - or find a template processing library and integrate it.

Writing from a list to a text file C#

How do I code the whole list into the text file with commas in between each bit of data? Currently it is creating the file newData, but it is not putting in the variables from the list. Here is what I have so far.
public partial class Form1 : Form {
List<string> newData = new List<string>();
}
Above is where I create my list. Below is where I am reading it from.
private void saveToolStripMenuItem_Click(object sender, EventArgs e) {
TextWriter tw = new StreamWriter("NewData.txt");
tw.WriteLine(newData);
buttonSave.Enabled = true;
textBoxLatitude.Enabled = false;
textBoxLongtitude.Enabled = false;
textBoxElevation.Enabled = false;
}
And below is where the variables are coming from.
private void buttonSave_Click(object sender, EventArgs e) {
newData.Add (textBoxLatitude.Text);
newData.Add (textBoxLongtitude.Text);
newData.Add (textBoxElevation.Text);
textBoxLatitude.Text = null;
textBoxLongtitude.Text = null;
textBoxElevation.Text = null;
}

While you can use String.Join as others have mentioned they're ignoring three important things:
The fact that what you're really trying to do is write a comma-separated values file
The input that you're receiving and whether or not it will have commas in it
If you sanitize your input, what the current culture on the thread is when you write it out to the file
You want to write a comma-delimited file. There's no standardized format for this, but you do have to be careful of string content, especially in your case, where you're getting user input. Consider the following input:
latitude = "39,41"
longitude = "41,20"
There are a number of countries where the comma is used as a decimal separator, so this kind of input is very possible, depending on how distributed your application is (I'd be even more concerned if this was a website, personally).
And when getting the elevation, it's absolutely possible in most other places that use a comma as the thousands separator:
elevation = 20,000
In all of the other answers, your output for the line in the file will be:
39,41,41,20,20,000
Which when parsed (assuming it will be parsed, you're creating a machine-readable format) will fail.
What you want to do is parse the content first into a decimal and then output that.
Assuming you sanitize your input like so:
decimal latitude = Decimal.Parse(textBoxLatitude.Text);
decimal longitude = Decimal.Parse(textBoxLongitude.Text);
decimal elevation = Decimal.Parse(textBoxElevation.Text);
You would then format the values so that there are no commas (if you want).
To that end, I really recommend that you want to use a dedicated CSV writer/parser (try ServiceStack's serializer on NuGet, or others, if you prefer), which accounts for commas within the content you want separated by commas.

private void saveToolStripMenuItem_Click(object sender, EventArgs e)
{
TextWriter tw = new StreamWriter("NewData.txt");
tw.WriteLine(String.Join(", ", newData));
// Add appropriate error detection
}
In response to the discussion in both main answer threads, here is an example from my older code of a more robust way to handle CSV output:
The above not checked for syntax, but the key concept is String.Join.
public const string Quote = "\"";
public static void EmitCsvLine(TextWriter report, IList<string> values)
{
List<string> csv = new List<string>(values.Count);
for (var z = 0; z < values.Count; z += 1)
{
csv.Add(Quote + values[z].Replace(Quote, Quote + Quote) + Quote);
}
string line = String.Join(",", csv);
report.WriteLine(line);
}
This could be made slightly more general with an IEnumerable<object> but in the code I took this form, I didn't have the need to.

You cannot output the list just by calling tw.WriteLine(newData);
But something like this will achieve that:
tw.WriteLine(string.Join(", ", newData));

you could:
StringBuilder b = new StringBuilder();
foreach (string s in yourList)
{
b.Append(s);
b.Append(", ");
}
string dir = "c:\mypath";
File.WriteAllText(dir, b.ToString());

You have to iterate the List (not tested) or use string.Join, as the other users suggested (you need to convert your list to an array then)
private void saveToolStripMenuItem_Click(object sender, EventArgs e)
{
TextWriter tw = new StreamWriter("NewData.txt");
for (int i = 0; i < newData.Count; i++)
{
tw.Write(newData[i]);
if(i < newData.Count-1)
{
tw.Write(",");
}
}
tw.close();
buttonSave.Enabled = true;
textBoxLatitude.Enabled = false;
textBoxLongtitude.Enabled = false;
textBoxElevation.Enabled = false;
}

Getting bibliographic data from text in a PDF and exporting to a window form

I use iText5 for .NET to extract text from a PDF, by using below code.
private void button1_Click(object sender, EventArgs e)
{
PdfReader reader2 = new PdfReader("Scharfetter1969.pdf");
int pagen = reader2.NumberOfPages;
reader2.Close();
ITextExtractionStrategy its = new iTextSharp.text.pdf.parser.SimpleTextExtractionStrategy();
for (int i = 1; i < 2; i++)
{
textBox1.Text = "";
PdfReader reader = new PdfReader("Scharfetter1969.pdf");
String s = PdfTextExtractor.GetTextFromPage(reader, i, its);
s = Encoding.UTF8.GetString(ASCIIEncoding.Convert(Encoding.Default, Encoding.UTF8, Encoding.Default.GetBytes(s)));
textBox1.Text = s;
reader.Close();
}
}
But I want to get bibliographic data from research paper pdf.
Here is example of data which is extrected from this pdf (in endnote format), Here's a link!
%0 Journal Article
%T Repeated temperature modulation epitaxy for p-type doping and light-emitting diode based on ZnO
%A Tsukazaki, A.
%A Ohtomo, A.
%A Onuma, T.
%A Ohtani, M.
%A Makino, T.
%A Sumiya, M.
%A Ohtani, K.
%A Chichibu, S.F.
%A Fuke, S.
%A Segawa, Y.
%J Nature Materials
%V 4
%N 1
%P 42-46
%# 1476-1122
%D 2004
%I Nature Publishing Group
But remember that this is bibliographic information, it is not available in metadata of this pdf. I want to access Article Type (%O), Title (%T), Authors (%A), Date (%D) and (%I) and show it to different assigned textbox in window form.
I am using C# if any one have any code for this, or guide me how to do this.

PDF is a one-way format. You put data in so that it renders consistently on all devices (monitors, printers, etc) but the format was never intended to pull data back out. Any and all attempts to do that will be pure guess work. iText's PdfTextExtractor works but you are going to have to piece things together based on your own arbitrary set of rules, and these rules will probably change from PDF to PDF. The supplied PDF was created by InDesign which does such a great job of making text look good that it actually makes it even harder to parse the data back out.
That said, if your PDFs are all visually consistent, you could try to pull the data out while retaining formatting and use the formatting rules to guess what is what. That post will get you some HTML formatting that you could guess at. (If this actually works I'd recommend returning something more specific than HTML but I'll leave that up to you.)
Running it against your supplied PDF shows that the title is using the font HelveticaNeue-LightExt at about 17pts so you could write a rule to look for all lines that use that font at that size and combine them together. Authors are done in HelveticaNeue-Condensed at about 10pts so that's another rule.
The below code is a modified version of the one linked to above. Its a full working C# 2010 WinForms app targeting iTextSharp 5.1.1.0. It pulls out the title and authors for the supplied PDF but you'll need to tweak it for other PDFs and meta data. See the comments in the code for specific implementation details.
using System;
using System.Collections.Generic;
using System.Text;
using System.Windows.Forms;
using iTextSharp.text.pdf.parser;
using iTextSharp.text.pdf;
namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
PdfReader reader = new PdfReader(System.IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "nmat4-42.pdf"));
TextWithFontExtractionStategy S = new TextWithFontExtractionStategy();
string F = iTextSharp.text.pdf.parser.PdfTextExtractor.GetTextFromPage(reader, 1, S);
//Buffers to hold various parts from the PDF
List<string> titles = new List<string>();
List<string> authors = new List<string>();
//Array of lines of text
string[] lines = F.Split(new string[] { Environment.NewLine }, StringSplitOptions.None);
//Temporary string
string t;
//Loop through each line in the array
foreach (string line in lines)
{
//See if the line looks like a "title"
if (line.Contains("HelveticaNeue-LightExt") && line.Contains("font-size:17.28003"))
{
//Remove the HTML tags
titles.Add(System.Text.RegularExpressions.Regex.Replace(line, "</?span.*?>", "").Trim());
}
//See if the line looks like an "author"
else if (line.Contains("HelveticaNeue-Condensed") && line.Contains("font-size:9.995972"))
{
//Remove the HTML tags and trim extra characters
t = System.Text.RegularExpressions.Regex.Replace(line, "</?span.*?>", "").Trim(new char[] { ' ', ',', '*' });
//Make sure we have a valid name, probably need some more exceptions here, too
if (!string.IsNullOrWhiteSpace(t) && t != "AND")
{
authors.Add(t);
}
}
}
//Write out the title to the console
Console.WriteLine("Title : {0}", string.Join(" ", titles.ToArray()));
//Write out each author
foreach (string author in authors)
{
Console.WriteLine("Author : {0}", author);
}
Console.WriteLine(F);
this.Close();
}
public class TextWithFontExtractionStategy : iTextSharp.text.pdf.parser.ITextExtractionStrategy
{
//HTML buffer
private StringBuilder result = new StringBuilder();
//Store last used properties
private Vector lastBaseLine;
private string lastFont;
private float lastFontSize;
//http://api.itextpdf.com/itext/com/itextpdf/text/pdf/parser/TextRenderInfo.html
private enum TextRenderMode
{
FillText = 0,
StrokeText = 1,
FillThenStrokeText = 2,
Invisible = 3,
FillTextAndAddToPathForClipping = 4,
StrokeTextAndAddToPathForClipping = 5,
FillThenStrokeTextAndAddToPathForClipping = 6,
AddTextToPaddForClipping = 7
}
public void RenderText(iTextSharp.text.pdf.parser.TextRenderInfo renderInfo)
{
string curFont = renderInfo.GetFont().PostscriptFontName;
//Check if faux bold is used
if ((renderInfo.GetTextRenderMode() == (int)TextRenderMode.FillThenStrokeText))
{
curFont += "-Bold";
}
//This code assumes that if the baseline changes then we're on a newline
Vector curBaseline = renderInfo.GetBaseline().GetStartPoint();
Vector topRight = renderInfo.GetAscentLine().GetEndPoint();
iTextSharp.text.Rectangle rect = new iTextSharp.text.Rectangle(curBaseline[Vector.I1], curBaseline[Vector.I2], topRight[Vector.I1], topRight[Vector.I2]);
Single curFontSize = rect.Height;
//See if something has changed, either the baseline, the font or the font size
if ((this.lastBaseLine == null) || (curBaseline[Vector.I2] != lastBaseLine[Vector.I2]) || (curFontSize != lastFontSize) || (curFont != lastFont))
{
//if we've put down at least one span tag close it
if ((this.lastBaseLine != null))
{
this.result.AppendLine("</span>");
}
//If the baseline has changed then insert a line break
if ((this.lastBaseLine != null) && curBaseline[Vector.I2] != lastBaseLine[Vector.I2])
{
this.result.AppendLine("<br />");
}
//Create an HTML tag with appropriate styles
this.result.AppendFormat("<span style=\"font-family:{0};font-size:{1}\">", curFont, curFontSize);
}
//Append the current text
this.result.Append(renderInfo.GetText());
//Set currently used properties
this.lastBaseLine = curBaseline;
this.lastFontSize = curFontSize;
this.lastFont = curFont;
}
public string GetResultantText()
{
//If we wrote anything then we'll always have a missing closing tag so close it here
if (result.Length > 0)
{
result.Append("</span>");
}
return result.ToString();
}
//Not needed
public void BeginTextBlock() { }
public void EndTextBlock() { }
public void RenderImage(ImageRenderInfo renderInfo) { }
}
}
}