I've been trying to solve these problem for last two hours but seems like I can't find any solution.
I need to extract links from an HTML file. There are 100+ links, but only 25 of them are valid.
Valid links are placed inside
<td><a href=" (link) ">
First I had (and still have) a problem with double quotes inside verbatim strings. So, I have replaced verbatim with "normal" strings so I can use \" for " but the problem is that this Regex I have written doesn't work
Match LinksTemp = Regex.Match(
htmlCode,
"<td><a href=\"(.*)\">",
RegexOptions.IgnoreCase);
as I get "<td><a href="http://www.google.com"> as output instead of http://www.google.com
Anyone know how can I solve this problem and how can I use double quotes inside of verbatim strings (example #" <>"das"sa ")
Escaped double quotes sample: #"some""test"
Regex sample: "<a href=\"(.*?)\">"
var match = Regex.Match(html, "<td><a href=\"(.*?)\">",
RegexOptions.Singleline); //spelling error
var url = match.Groups[1].Value;
Also you may want to use Regex.Matches(...) instead of Regex.Match(...)
If you want to take every elements use code simply like this:
string htmlCode = "<td><a href=\" www.aa.pl \"><td> <a href=\" www.cos.com \"><td>";
Regex r = new Regex( "<a href=\"(.*?)\">", RegexOptions.IgnoreCase );
MatchCollection mc = r.Matches(htmlCode);
foreach ( Match m1 in mc ) {
MessageBox.Show( m1.Groups[1].ToString() );
}
Why not parse this with an HTML-parsing is good and fast HTML-Parsing.
example:
string HTML = "<td><a href='http://www.google.com'>";
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(HTML);
HtmlNodeCollection a = doc.DocumentNode.SelectNodes("//a[#href]");
string url = a[0].GetAttributeValue("href", null);
Console.WriteLine(url);
Console.ReadLine();
you need import using HtmlAgilityPack;
Related
I would like to replace words in a html string with another word, but it must only replace the exact word and not if it is part of the spelling of part of a word. The problem that I am having is that the html open or closing tags or other html elements are affecting what words are matched in the regex or it is replacing parts of words.
PostTxt = “<div>The <b>cat</b> sat on the mat, what a catastrophe.
The <span>cat</span> is not allowed on the mat. This makes things complicated; the cat  must go!
</div><p>cat cat cat</p>”;
string pattern = "cat";
//replacement string to use
string replacement = "******";
//Replace words
PostTxt = Regex.Replace(PostTxt, pattern, replacement, RegexOptions.IgnoreCase);
}
I would like it to return.
<div>The <b>***</b> sat on the mat, what a catastrophe. The <span>***</span> is not allowed on the mat. This makes things complicated; the ***  must go! </div><p>*** *** ***</p>
Any suggestions and help will be greatly appreciated.
This is the simplified solution of the code I implemented using html-agility-pack.net. Regex is not a solution to this problem as noted by See: Why it's not possible to use regex to parse HTML/XML: a formal explanation in layman's terms. – Olivier Jacot-Descombes
PostTxt = "<div>The <b>cat</b> sat on the mat, what a catastrophe.
The <span>cat</span> is not allowed on the mat. This makes things complicated; the cat must go!
</div><p>Cat cat cat</p>";
HtmlDocument mainDoc = new HtmlDocument();
mainDoc.LoadHtml(PostTxt);
//replacement string to use
string replacement = “*****”;
string pattern = #"\b" + Regex.Escape("cat") + #"\b";
var nodes = mainDoc.DocumentNode.SelectNodes("//*") ?? new HtmlNodeCollection(null);
foreach (var node in nodes)
{
node.InnerHtml = Regex.Replace(node.InnerHtml, pattern, replacement, RegexOptions.IgnoreCase);
}
PostTxt = mainDoc.DocumentNode.OuterHtml;
I have been trying to get the id from the following text
<body id=\"body\" runat=\"server\">
In C# using substring or even Regex, but nothing seems to be working. No matter what regex i use, i always get the whole line back. I have been trying to use ^id, ^id.*, ^id=\\\\\\\\.* and id=.* but they don't either work or give me the desired output. Is there any way i can get the id portion from this text which is enclosed between the characters \" "\?
Try this:
string htmlString = "<body id=\"body\" runat=\"server\">";
Regex regex = new Regex("id=\"(.*?)\"");
Match m = regex.Match(htmlString);
Group g = m.Groups[1];
string id = g.ToString();
Console.WriteLine(id); //body
Test here:
http://rextester.com/BQSF93427
I am using this regex to math all contents of href's on a page:
(?:href)=[\"|']?(.*?)[\"|'|>]+
It works fine. But i want to match only links that are not media like (png|jpg|avi|wav|gif) etc.
I tried something like adding
((?!png).)
to my regex, but this did not work. I read this question
but could not get any working solution.
I know this question was already answered.
I'd like to offer a different approach using CsQuery instead of HtmlAgilityPack
I think the syntax is more compact and is very similar to other structures since it's based on LINQ
//input is your input HTML string
var links = CQ.Create(input).Find("a").Select(x=>x.Cq().Attr("href"));
For example
var links = CQ.Create("<div><a href='blah'></a><a href='blah2'></a></div>").Find("a").Select(x=>x.Cq().Attr("href"));
Console.Write(string.Join(",",dom)); //prints blah,blah2
Hope this helps anyone :)
using HtmlAgilityPack;
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
List<string> href = new List<string>();
private void addHREF()
{
//put your input to check
string input = "";
doc.LoadHtml(input);
//Which files ignore?
string[] stringArray = { ".png", ".jpg" };
foreach (var item in doc.DocumentNode.SelectNodes("//a"))
{
string value = item.Attributes["href"].Value;
if (stringArray.Any(value.Contains) == false)
href.Add(value);
}
}
I tested with my input works great... if you have any problem let me know..
Even though I recommend against this approach, you may find this regex helpful:
(?<=href\s*=\s*['"]?)(?>(https?://)?([\da-z\.-]+)\.([a-z\.]{2,6})([/\w\.-]*)*/?)(?<!png|gif|etc)
(Based on URL regex from 8 Regular Expressions You Should Know)
Note that this expression will not allow spaces in the URL. This is because HREF's without quotes will match the following attribute (for example, "domain.com/resource.txt title")
EXAMPLE:
static void Main( string[] args )
{
string l_input =
"<a href=\n" +
" \"HTTPS://example.com/page.html\" title=\"match\" />\n" +
"<a href='http://site.com/pic.png' title='do not match'> <a href=domain.com/resource.txt title=match>\n" +
" <script src=scripts.com/script.js>";
foreach ( Match l_match in Regex.Matches( l_input, #"(?<=href\s*=\s*['""]?)(?>(https?://)?([\da-z\.-]+)\.([a-z\.]{2,6})([/\w\.-]*)*/?)(?<!png|gif|etc)", RegexOptions.IgnoreCase ) )
Console.WriteLine( "'" + l_match.Value + "'" );
/*
* Returns:
*
* HTTPS://example.com/page.html
* domain.com/resource.txt
*
*/
Console.ReadKey( true );
}
My effort
#"(?<=\shref\s*=\s*[""']?)(?![""']|\S+\.(?:png|jpg|avi|wav|gif)[""']?[\s>])\S+?(?=[""']?[\s>])";
It uses a positive look-behind to locate the content, and a negative lookahead to make sure it doesn't contain a dot followed by either of png jpg avi wav gif followed by an optional quote mark and a space or >. It then matches up until an optional quote mark followed by a space or >. The content does not have to be quoted but it must not contain whitespace.
I need a working Regex code in C# that detects plain text urls (http/https/ftp/ftps) in a string and make them clickable by putting an anchor tag around it with same url. I have already made a Regex pattern and the code is attached below.
However, if there is already any clickable url is present in the input string then the above code puts another anchor tag over it. For example the existing substring in the below code: string sContent: "ftp://www.abc.com'>ftp://www.abc.com" has another anchor tag over it when the code below is run. Is there any way to fix it?
string sContent = "ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc ftp://www.abc.com abbbbb http://www.abc2.com";
Regex regx = new Regex("(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
MatchCollection mactches = regx.Matches(sContent);
foreach (Match match in mactches)
{
sContent = sContent.Replace(match.Value, "<a href='" + match.Value + "'>" + match.Value + "</a>");
}
Also, I want a Regex code to make emails as clickable with "mailto" tag. I can do it myself but the above mentioned issue of double anchor tag will also appear in it.
I noticed in your example test string that if a duplicate link e.g. ftp://www.abc.com is in the string and is already linked then the result will be to double anchor that link. The Regular Expression that you already have and that #stema has supplied will work, but you need to approach how you replace the matches in the sContent variable differently.
The following code example should give you what you want:
string sContent = "ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc ftp://www.abc.com abbbbb http://www.abc2.com";
Regex regx = new Regex("(?<!(?:href='|<a[^>]*>))(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
MatchCollection matches = regx.Matches(sContent);
for (int i = matches.Count - 1; i >= 0 ; i--)
{
string newURL = "<a href='" + matches[i].Value + "'>" + matches[i].Value + "</a>";
sContent = sContent.Remove(matches[i].Index, matches[i].Length).Insert(matches[i].Index, newURL);
}
Try this
Regex regx = new Regex("(?<!(?:href='|>))(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
It should work for your example.
(?<!(?:href='|>)) is a negative lookbehind, that means the pattern matches only if it is not preceeded by "href='" or ">".
See lookarounds on regular-expressions.info
and the especially the zero-width negative lookbehind assertion on msdn
See something similar on Regexr. I had to remove the alternation from the look behind, but .net should be able to handle it.
Update
To ensure that there are also (maybe possible) cases like "<p>ftp://www.def.com</p>" correctly handled, I improved the regex
Regex regx = new Regex("(?<!(?:href='|<a[^>]*>))(http|https|ftp|ftps)://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);
The lookbehind (?<!(?:href='|<a[^>]*>)) is now checking that there is not a "href='" nor a tag starting with "
The output of the teststring
ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc <p>ftp://www.def.com</p> abbbbb http://www.ghi.com
is with this expression
ttt <a href='ftp://www.abc.com'>ftp://www.abc.com</a> abc <p><a href='ftp://www.def.com'>ftp://www.def.com</a></p> abbbbb <a href='http://www.ghi.com'>http://www.ghi.com</a>
I know I arrived late to this party, but there are several problems with the regex that the existing answers don't address. First and most annoying, there's that forest of backslashes. If you use C#'s verbatim strings, you don't have to do all that double escaping. And anyway, most of the backslashes weren't needed in the first place.
Second, there's this bit: ([\\w+?\\.\\w+])+. The square brackets form a character class, and everything inside them is treated either as a literal character or a class shorthand like \w. But getting rid of the square brackets isn't enough to make it work. I suspect this is what you were trying for: \w+(?:\.\w+)+.
Third, the quantifiers at the end of the regex - ]*)? - are mismatched. * can match zero or more characters, so there's no point making the enclosing group optional. Also, that kind of arrangement can result in severe performance degradation. See this page for details.
There are other, minor problems, but I won't go into them right now. Here's the new and improved regex:
#"(?n)(https?|ftps?)://\w+(\.\w+)+([-a-zA-Z0-9~!##$%^&*()_=+/?.:;',\\]*)(?![^<>]*+(>|</a>))"
The negative lookahead - (?![^<>]*+(>|</a>)) is what prevents matches inside tags or in the content of an anchor element. It's still very crude, though. There are several areas, like inside <script> elements, where you don't want it to match but it does. But trying to cover all the possibilities would result in a mile-long regex.
Check out: Detect email in text using regex and Regex URL Replace, ignore Images and existing Links, just replace the regex for links, it will never replace a link inside a tag, only in contents.
http://html-agility-pack.net/?z=codeplex
Something like:
string textToBeLinkified = "... your text here ...";
const string regex = #"((www\.|(http|https|ftp|news|file)+\:\/\/)[_.a-z0-9-]+\.[a-z0-9\/_:#=.+?,##%&~-]*[^.|\'|\# |!|\(|?|,| |>|<|;|\)])";
Regex urlExpression = new Regex(regex, RegexOptions.IgnoreCase | RegexOptions.ExplicitCapture);
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(textToBeLinkified);
var nodes = doc.DocumentNode.SelectNodes("//text()[not(ancestor::a)]") ?? new HtmlNodeCollection();
foreach (var node in nodes)
{
node.InnerHtml = urlExpression.Replace(node.InnerHtml, #"$0");
}
string linkifiedText = doc.DocumentNode.OuterHtml;
I am trying to get an ID from a url parameter inside an href that looks like this:
MyItemName
I want the 71312 only and at the momment I am trying to do it using regex (but if you have a better approch I would be glad to try):
string html,itemID;
using (var client = new WebClient())
{
html = client.DownloadString("http://www.mysite.com/search.php?search_text=" + myItemName);
}
string pattern = "" + myItemName + "";
Match m = Regex.Match(html, pattern, RegexOptions.IgnoreCase);
if (m.Success)
{
itemID = m.Groups[1].Value;
MessageBox.Show(itemID);
}
Example of the html:
more html body
<h1>Items - List</h1>
<p>MyItemNameTest, MyItemNameTestB, MYItemNameOther</p>
</div>
more html body
To show where your regex went wrong:
. and ? are special characters in regular expressions. . means "any character" and ? means "zero or one occurences of the previous expression". Therefore your regex fails to match. Also, you need to use verbatim strings in C# (unless you want to escape every backslash):
#"" + myItemName + "";
will probably work.
That said, unless all the links you're examining follow exactly this format, you might run into problems. It's kind of a running gag here on SO that parsing HTML with regular expressions will earn you the wrath of Cthulhu.
Use:
Uri u = new Uri("http://www.mysite.com/myitem.php?id=12313");
string s = u.Query;
HttpUtility.ParseQueryString(s).Get("id");
In variable id you have the number. Figure out the rest of the function :)