I've a list of paragraphs. Each paragagraph can contain Text. I'm trying to search for a string that may be as whole within a single paragraph, or spread across multiple paragraphs with as bad case where each letter is different paragraph.
public List<WordParagraph> FindText(string text) {
List<WordParagraph> list = new List<WordParagraph>();
var found = false;
Paragraph currentParagraph = null;
foreach (var paragraph in this.Paragraphs) {
//if (currentParagraph == null) {
// currentParagraph = paragraph._paragraph;
//} else {
// if (currentParagraph != paragraph._paragraph) {
// found = false;
// }
//}
// paragraph.Text
// logic missing to find text that can start within some paragraph.Text, but
// can span across multiple paragraphs
// for example searching for text "This Is MyTest" within 4 paragraphs that
// may be written like
// paragraph.Text = "Thi"
// paragraph.Text = "s Is"
// paragraph.Text = " MyTes"
// paragraph.Text = "t"
}
return list;
}
I've tried some logic around foreach char in text, and nested loop over text from the paragraph.text but the logic was failing me.
To give you a bit of background. Consider a Word Document that has a single sentence - one long sentence but each word, or even letter is formatted differently - different font size, bold, underline or whatever. It looks like this:
Now what Word actually saved in the file is a single paragraph, but each paragraph has multiple "runs". The run contains a Text element. Each text element contains the text that you see in Word, but due to formatting of possibly even each word it can be split into many many small Text properties.
Now in my example, I've simplified the logic and for me, each "run" is a paragraph with a text. So List of WordParagraphs is a list of runs within Screenshot you see.
Now I need to find a string "I have that" from the whole sentence you see in word. That means I need to go thru all paragraphs, find the first letter that matches and then check if next letter matches as well, if not I need to start again.
My brain is having hard time to grasp this logic in code.
Basically what I'm trying to do is find the first string that starts with "/Game/Mods" but the problem is how do i tell the program where to end the string? here's an example what a string can look like: string example
As you can see the string starts with "/Game/Mods", i want it to end after the word "TamingSedative", the problem is that the ending word (TamingSedative)is different for every file it has to check, for example: example 2
There you can see that the ending word is now "WeapObsidianSword" (instead of TamingSedative) so basically the string has to end when it comes across the "NUL" but how do i specify that in c# code?
This a simple example using Regex.
Dim yourString As String = "/Game/Mods/TamingSedative/PrimalItemConsumable_TamingSedative"
Dim M As System.Text.RegularExpressions.Match = System.Text.RegularExpressions.Regex.Match(yourString, "/Game/Mods/(.+?)/")
MessageBox.Show(M.Groups(0).Value) 'This should show /Game/Mods/TamingSedative/
MessageBox.Show(M.Groups(1).Value) 'This should show TamingSedative
Since you need only the first occurance, this is the simplest solution I could think of:
(In case you cannot see the image, click on it to open in new tab)
EDIT:
In case the existence of a path like this is not guaranteed in the string, you can do an additional check before proceeding to use Substring, like this:
int exists = fullString.IndexOf("/Game/Mods");
if (exists == -1) return null;
Note: I have included "ENDED" in order to see in case any NULL chars have been included (white spaces)
From your comments: "the string just has to start at /Game/Mods and end when it reaches the whitespace".
In that case, you can easily get the matches using Linq, like this (assuming filePath is a string that has the path to your file):
var text = File.ReadAllText(filePath);
var matches = text.Split(null).Where(s => s.StartsWith("/Game/Mods"));
And, if you only need the first occurrence, it would be:
var firstMatch = matches.Any() ? matches.First() : null;
Check this post.
I'm trying to remove new lines from a text file. Opening the text file in notepad doesn't reveal the line breaks I'm trying to remove (it looks like one big wall of text), however when I open the file in sublime, I can see them.
In sublime, I can remove the pattern '\n\n' and then the pattern '\n(?!AAD)' no problem. However, when I run the following code, the resulting text file is unchanged:
public void Format(string fileloc)
{
string str = File.ReadAllText(fileloc);
File.WriteAllText(fileloc + "formatted", Regex.Replace(Regex.Replace(str, "\n\n", ""), "\n(?!AAD)", ""));
}
What am I doing wrong?
If you do not want to spend hours trying to re-adjust the code for various types of linebreaks, here is a generic solution:
string str = File.ReadAllText(fileloc);
File.WriteAllText(fileloc + "formatted",
Regex.Replace(Regex.Replace(str, "(?:\r?\n|\r){2}", ""), "(?:\r?\n|\r)(?!AAD)", "")
);
Details:
A linebreak can be matched with (?:\r?\n|\r): an optional CR followed with a single obligatory LF. To match 2 consecutive linebreaks, a limiting quantifier can be appended - (?:\r?\n|\r){2}.
An empirical solution. Opening your sample file in binary mode revealed that it contains 0x0D characters, which are carriage returns \r. So I came up with this (multiple lines for easier debugging):
public void Format(string fileloc)
{
var str = File.ReadAllText(fileloc);
var firstround = Regex.Replace(str, #"\r\r", "");
var secondround = Regex.Replace(firstround, #"\r(?!AAD)", "");
File.WriteAllText(fileloc + "formatted", secondround);
}
Is this possibly a windows/linux mismatch? Try replacing '\r\n' instead.
I know that the data should be correct. I have no control over the data and my boss is just going to tell me that I need to figure out a way to deal with someone else's mistake. So please don't tell me it's not my problem that the data is bad, because it is.
Anywho, this is what I'm looking at:
"Words","email#email.com","","4253","57574","FirstName","","LastName, MD","","","576JFJD","","1971","","Words","Address","SUITE "A"","City","State","Zip","Phone","",""
Data has been scrubbed for confidentiality reasons.
So as you see, the data contains quotation marks and there are commas inside some of these quoted fields. So I cannot remove them. But the "Suite A""" is throwing off the parser. There are too many quotation marks. >.<
I'm using the TextFieldParser in the Microsoft.VisualBasic.FileIO namespace with these settings:
parser.HasFieldsEnclosedInQuotes = true;
parser.SetDelimiters(",");
parser.TextFieldType = FieldType.Delimited;
The error is
MalformedLineException: Line 9871 cannot be parsed using the current
delimiters.
I would like to scrub the data somehow to account for this but I'm not sure how to do it. Or maybe there's a way to just skip this line? Although I suspect my higher ups will not approve of me just skipping data that we might need.
If you are only trying to get rid of the stray " marks in your csv, you can use the following regex to find them and replace them with '
String sourcestring = "source string to match with pattern";
String matchpattern = #"(?<!^|,)""(?!(,|$))";
String replacementpattern = #"$1'";
Console.WriteLine(Regex.Replace(sourcestring,matchpattern,replacementpattern,RegexOptions.Multiline));
Explanation:
#"(?<!^|,)""(?!(,|$))"; will find will find any " that is not preceded by the beginning of the string, or a , and that is not followed by the end of the string or a ,
I am not familiar with TextFieldParser. However with CsvHelper, you can add a custom handler for invalid data:
var config = new CsvConfiguration();
config.IgnoreReadingExceptions = true;
config.ReadingExceptionCallback += (e, row) =>
{
// you can add some custom patching here if possible
// or, save the line numbers and add/edit them manually later.
};
using(var file = File.OpenRead(".csv"))
using(var reader = new CsvReader(reader, config))
{
reader.GetRecords<YourDtoClass>();
}
My only addition to what everyone is saying (because we've all been there) is to try to attempt to rectify each new issue you encounter with code. There are some decent REGEX strings out there https://www.google.com/?ion=1&espv=2#q=c-sharp+regex+csv+clean or you could manually fix things using String.Replace (String.Replace("\"\"\"","").Replace("\"\","").Replace("\",,","\",") or such). Eventually, as you detect and find ways of correcting more and more mistakes, your manual recovery rate will be minimized substantially (most of your bad data will likely come from similar mistakes). Cheers!
PS - Idea-ish (it's been a while - the logic may neeed some tweaking as I'm writing from memory), but you'll get the gist:
public string[] parseCSVWithQuotes(string csvLine,int expectedNumberOfDataPoints)
{
string ret = "";
string thisChar = "";
string lastChar = "";
bool needleDown = true;
for(int i = 0; i < csvLine.Length; i++)
{
thisChar = csvLine.Substring(i, 1);
if (thisChar == "'"&&lastChar!="'")
needleDown = needleDown == true ? false : true;//when needleDown = true, characters are treated literally
if (thisChar == ","&&lastChar!=",") {
if (needleDown)
{
ret += "|";//convert literal comma to pipe so it doesn't cause another break on split
}else
{
ret += ",";//break on split is intended because the comma is outside the single quote
}
}
if (!needleDown && (thisChar == "\"" || thisChar == "*")) {//repeat for any undesired character or use RegEx
//do not add -- this eliminates any undesired characters outside single quotes
}
else
{
if ((lastChar == "'" || lastChar == "\"" || lastChar == ",") && thisChar == lastChar)
{
//do not add - this eliminates double characters
}else
{
ret += thisChar;
lastChar = thisChar;
//this character is not an undesired character, is no a double, is valid.
}
}
}
//we've cleaned as best we can
string[] parts = ret.Split(',');
if(parts.Length==expectedNumberOfDataPoints){
for(int i = 0; i < parts.Length; i++)
{
//go back and replace the temporary pipe with the literal comma AFTER split
parts[i] = parts[i].Replace("|", ",");
}
return parts;
}else{
//save ret to bad CSV log
return null;
}
}
I've had to do this before,
The first step is to parse the data using string.split(',')
The next step is to combine the segments that belong together.
What I essentially did was
make a new list representing the combined strings
if a string begins with a quote, push it onto your new list
if it does not begin with a quote, append it to the last string in your list
Bonus: throw exceptions when a string ends with a quote but the next one does not begin with a quote
Depending on what the rules are regarding what can actually appear in your data, you might have to change your code to account for that.
At the core of CSV's file format, each line is a row, each cell in that row is separated by a comma. In your case, your format also contains the (very unfortunate) stipulation that commas inside a pair of quotation marks do not count as separators and are instead part of the data. I say very unfortunate because a misplaced quotation mark affects the entire rest of the line, and since quotation marks in standard ASCII do not distinguish between open and closed, there really is nothing you can do to recover from this without knowing the original intent.
That is when you log a message in a way that the person who does know the original intent (the person that provided the data) can look at the file and correct the error:
if (parse_line(line, &data)) {
// save the data
} else {
// log the error
fprintf(&stderr, "Bad line: %s", line);
}
And since your quotation marks aren't escaping newlines, you can keep on going with the next line after running into this error.
ADDENDUM: And if your company has a choice (i.e. your data is being serialized by a company tool) don't use CSV. Use something like XML or JSON with a much more clearly defined parsing mechanism.
I had to do this once aswell. My approach was to go through a line and keep track on what I was reading.
Basicly, I coded my own scanner chopping off tokens from the input line which gave me full control over my faulty .csv data.
This is what I did:
For each character on a line of input.
1. when outside of a string meeting a comma => all of the previous string (which can be empty) is a valid token.
2. when outside of a sting meeting anything but a comma or a quote => now you have a real problem, unquoted tekst => handle as you see fit.
3. when outside of a string meeing a quote => found a start of string.
4. when inside of a string meeting a comma => accept the comma as part of the string.
5. when inside of the string meeting a qoute => trouble starts here, mark this point.
6. continue and when meeting a comma (skipping white space if desired) close the string, 'unread' the comma and continue. (than will bring you to point 1.)
7. or continue and when meeting a quote -> obviously, what was read must be part of the string, add it to the string, 'unread' the quote and continue. (that will you bring to point 5)
8. or continue and find an whitespace, then End Of Line ('\n') -> the last qoute must be the closing quote. accept the string as a value.
9. or continue and fine non-whitespace, then End Of Line. -> now you have a real problem, you have the start of a string but it is not closed -> handle the error as you see fit.
If the number of fields in your .csv file is fixed you can count the comma's you recognise as field seperators and when you see a End Of Line you know you have another problem or not.
With the stream of strings received from the input line you can build a 'clean' .csv line and this way build a buffer of accepted and cleaned input that you can use in your already existing code.
I am trying to make my program display the text above the input text which matches a pattern I set.
For example, if user input 'FastModeIdleImmediateCount"=dword:00000000', I should get the closest HKEY above, which is [HKEY_CURRENT_CONFIG\System\CurrentControlSet\Enum\SCSI\Disk&Ven_ATA&Prod_TOSHIBA_MQ01ABD0\4&6a0976b&0&000000] for this case.
[HKEY_CURRENT_CONFIG\System\CurrentControlSet\Enum\SCSI\Disk&Ven_ATA&Prod_TOSHIBA_MQ01ABD0\4&6a0976b&0&000000]
"StandardModeIdleImmediateCount"=dword:00000000
"FastModeIdleImmediateCount"=dword:00000000
[HKEY_CURRENT_CONFIG\System\CurrentControlSet\SERVICES]
[HKEY_CURRENT_CONFIG\System\CurrentControlSet\SERVICES\TSDDD]
[HKEY_CURRENT_CONFIG\System\CurrentControlSet\SERVICES\TSDDD\DEVICE0]
"Attach.ToDesktop"=dword:00000001
Could anyone please show me how I can code something like that? I tried playing around with regular expressions to match text with bracket, but I am not sure how to make it to only search for the text above my input.
I'm assuming your file is a .txt file, although it's most probably not. But the logic is the same.
It is not hard at all, a simple for() loop would do the trick.
Code with the needed description:
string[] lines = File.ReadAllLines(#"d:\test.txt");//replace your directory. We're getting all lines from a text file.
string inputToSearchFor = "\"FastModeIdleImmediateCount\"=dword:00000000"; //that's the string to search for
int indexOfMatchingLine = Array.FindIndex(lines, line => line == inputToSearchFor); //getting the index of the line, which equals the matchcode
string nearestHotKey = String.Empty;
for(int i = indexOfMatchingLine; i >=0; i--) //looping for lines above the matched one to find the hotkey
{
if(lines[i].IndexOf("[HKEY_") == 0) //if we find a line which begins with "[HKEY_" (that means it's a hotkey, right?)
{
nearestHotKey = lines[i]; //we get the line into our hotkey string
break; //breaking the loop
}
}
if(nearestHotKey != String.Empty) //we have actually found a hotkey, so our string is not empty
{
//add code...
}
You could try to split the text into lines, find the index of the line that contains your text (whether exact match or regex is used doesn't matter) and then backsearch for the first key. Reverse sorting the lines first might help.