Related
I'm trying to generate a HTML document with RazorEngine (http://razorengine.codeplex.com/). Everything is mostly working, but the problem I have now is some of the HTML is being rendered correctly, and HTML that I have nested in that is being rendered as literal HTML, so rather than the browser displaying the tables & divs as expected, it's displaying e.g.
"<table></table><div></div>"
I kick off this process by calling the following:
string completeHTML = RazorEngine.Razor.Parse("InstallationTemplate.cshtml", new { Data = viewModels });
The completeHTML is then written to a file.
"InstallationTemplate.cshtml" is defined as:
#{
var installationReport = new InstallationReport(Model.Data);
}
<!DOCTYPE html>
<html>
<head></head>
<body>
<div>
<!-- I would expect this to write the rendered HTML
in place of "#installationReport.DigiChannels()" -->
#installationReport.DigiChannels()
</div>
</body>
</html>
Where InstallationReport and DigiChannels are defined as follows:
public static class InstallationReportExtensions
{
public static string DigiChannels(this InstallationReport installationReport)
{
return installationReport.GetDigiChannelsHtml();
}
}
public class InstallationReport
{
public string GetDigiChannelsHtml()
{
// the following renders the template correctly
string renderedHtml = RazorReport.GetHtml("DigiChannels.cshtml", GetDigiChannelData());
return renderedHtml;
}
}
public static string GetHtml(string templateName, object data)
{
var templateString = GetTemplateString(templateName);
return RazorEngine.Razor.Parse(templateString, data);
}
After GetDigiChannelsHtml() runs and returns renderedHtml, the line of execution returns to TemplateBase.cs into the method ITemplate.Run(ExecuteContext context), which is defined as:
string ITemplate.Run(ExecuteContext context)
{
_context = context;
var builder = new StringBuilder();
using (var writer = new StringWriter(builder))
{
_context.CurrentWriter = writer;
Execute(); // this is where my stuff gets called
_context.CurrentWriter = null;
}
if (Layout != null)
{
// Get the layout template.
var layout = ResolveLayout(Layout);
// Push the current body instance onto the stack for later execution.
var body = new TemplateWriter(tw => tw.Write(builder.ToString()));
context.PushBody(body);
return layout.Run(context);
}
return builder.ToString();
}
When I inspect builder.ToString(), I can see it contains proper HTML for the InstallationTemplate.cshtml stuff, and escaped HTML for the DigiChannels.cshtml stuff. For example:
How can I get #installationReport.DigiChannels() to include the correct HTML instead of the escaped HTML that it's currently doing?
Have you tried:
#Raw(installationReport.DigiChannels())
Edit : I could use it in following way (MVC3)
#Html.Raw(installationReport.DigiChannels())
The alternative to #Raw is to change your API to return HtmlStrings in appropriate places
Represents an HTML-encoded string that should not be encoded again.
The default razor behaviour is to encode strings.
I'm looking for C# code to convert an HTML document to plain text.
I'm not looking for simple tag stripping , but something that will output plain text with a reasonable preservation of the original layout.
The output should look like this:
Html2Txt at W3C
I've looked at the HTML Agility Pack, but I don't think that's what I need. Does anyone have any other suggestions?
EDIT: I just download the HTML Agility Pack from CodePlex, and ran the Html2Txt project. What a disappointment (at least the module that does html to text conversion)! All it did was strip the tags, flatten the tables, etc. The output didn't look anything like the Html2Txt # W3C produced. Too bad that source doesn't seem to be available.
I was looking to see if there is a more "canned" solution available.
EDIT 2: Thank you everybody for your suggestions. FlySwat tipped me in the direction i wanted to go. I can use the System.Diagnostics.Process class to run lynx.exe with the "-dump" switch to send the text to standard output, and capture the stdout with ProcessStartInfo.UseShellExecute = false and ProcessStartInfo.RedirectStandardOutput = true. I'll wrap all this in a C# class. This code will be called only occassionly, so i'm not too concerned about spawning a new process vs. doing it in code. Plus, Lynx is FAST!!
Just a note about the HtmlAgilityPack for posterity. The project contains an example of parsing text to html, which, as noted by the OP, does not handle whitespace at all like anyone writing HTML would envisage. There are full-text rendering solutions out there, noted by others to this question, which this is not (it cannot even handle tables in its current form), but it is lightweight and fast, which is all I wanted for creating a simple text version of HTML emails.
using System.IO;
using System.Text.RegularExpressions;
using HtmlAgilityPack;
//small but important modification to class https://github.com/zzzprojects/html-agility-pack/blob/master/src/Samples/Html2Txt/HtmlConvert.cs
public static class HtmlToText
{
public static string Convert(string path)
{
HtmlDocument doc = new HtmlDocument();
doc.Load(path);
return ConvertDoc(doc);
}
public static string ConvertHtml(string html)
{
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(html);
return ConvertDoc(doc);
}
public static string ConvertDoc (HtmlDocument doc)
{
using (StringWriter sw = new StringWriter())
{
ConvertTo(doc.DocumentNode, sw);
sw.Flush();
return sw.ToString();
}
}
internal static void ConvertContentTo(HtmlNode node, TextWriter outText, PreceedingDomTextInfo textInfo)
{
foreach (HtmlNode subnode in node.ChildNodes)
{
ConvertTo(subnode, outText, textInfo);
}
}
public static void ConvertTo(HtmlNode node, TextWriter outText)
{
ConvertTo(node, outText, new PreceedingDomTextInfo(false));
}
internal static void ConvertTo(HtmlNode node, TextWriter outText, PreceedingDomTextInfo textInfo)
{
string html;
switch (node.NodeType)
{
case HtmlNodeType.Comment:
// don't output comments
break;
case HtmlNodeType.Document:
ConvertContentTo(node, outText, textInfo);
break;
case HtmlNodeType.Text:
// script and style must not be output
string parentName = node.ParentNode.Name;
if ((parentName == "script") || (parentName == "style"))
{
break;
}
// get text
html = ((HtmlTextNode)node).Text;
// is it in fact a special closing node output as text?
if (HtmlNode.IsOverlappedClosingElement(html))
{
break;
}
// check the text is meaningful and not a bunch of whitespaces
if (html.Length == 0)
{
break;
}
if (!textInfo.WritePrecedingWhiteSpace || textInfo.LastCharWasSpace)
{
html= html.TrimStart();
if (html.Length == 0) { break; }
textInfo.IsFirstTextOfDocWritten.Value = textInfo.WritePrecedingWhiteSpace = true;
}
outText.Write(HtmlEntity.DeEntitize(Regex.Replace(html.TrimEnd(), #"\s{2,}", " ")));
if (textInfo.LastCharWasSpace = char.IsWhiteSpace(html[html.Length - 1]))
{
outText.Write(' ');
}
break;
case HtmlNodeType.Element:
string endElementString = null;
bool isInline;
bool skip = false;
int listIndex = 0;
switch (node.Name)
{
case "nav":
skip = true;
isInline = false;
break;
case "body":
case "section":
case "article":
case "aside":
case "h1":
case "h2":
case "header":
case "footer":
case "address":
case "main":
case "div":
case "p": // stylistic - adjust as you tend to use
if (textInfo.IsFirstTextOfDocWritten)
{
outText.Write("\r\n");
}
endElementString = "\r\n";
isInline = false;
break;
case "br":
outText.Write("\r\n");
skip = true;
textInfo.WritePrecedingWhiteSpace = false;
isInline = true;
break;
case "a":
if (node.Attributes.Contains("href"))
{
string href = node.Attributes["href"].Value.Trim();
if (node.InnerText.IndexOf(href, StringComparison.InvariantCultureIgnoreCase)==-1)
{
endElementString = "<" + href + ">";
}
}
isInline = true;
break;
case "li":
if(textInfo.ListIndex>0)
{
outText.Write("\r\n{0}.\t", textInfo.ListIndex++);
}
else
{
outText.Write("\r\n*\t"); //using '*' as bullet char, with tab after, but whatever you want eg "\t->", if utf-8 0x2022
}
isInline = false;
break;
case "ol":
listIndex = 1;
goto case "ul";
case "ul": //not handling nested lists any differently at this stage - that is getting close to rendering problems
endElementString = "\r\n";
isInline = false;
break;
case "img": //inline-block in reality
if (node.Attributes.Contains("alt"))
{
outText.Write('[' + node.Attributes["alt"].Value);
endElementString = "]";
}
if (node.Attributes.Contains("src"))
{
outText.Write('<' + node.Attributes["src"].Value + '>');
}
isInline = true;
break;
default:
isInline = true;
break;
}
if (!skip && node.HasChildNodes)
{
ConvertContentTo(node, outText, isInline ? textInfo : new PreceedingDomTextInfo(textInfo.IsFirstTextOfDocWritten){ ListIndex = listIndex });
}
if (endElementString != null)
{
outText.Write(endElementString);
}
break;
}
}
}
internal class PreceedingDomTextInfo
{
public PreceedingDomTextInfo(BoolWrapper isFirstTextOfDocWritten)
{
IsFirstTextOfDocWritten = isFirstTextOfDocWritten;
}
public bool WritePrecedingWhiteSpace {get;set;}
public bool LastCharWasSpace { get; set; }
public readonly BoolWrapper IsFirstTextOfDocWritten;
public int ListIndex { get; set; }
}
internal class BoolWrapper
{
public BoolWrapper() { }
public bool Value { get; set; }
public static implicit operator bool(BoolWrapper boolWrapper)
{
return boolWrapper.Value;
}
public static implicit operator BoolWrapper(bool boolWrapper)
{
return new BoolWrapper{ Value = boolWrapper };
}
}
As an example, the following HTML code...
<!DOCTYPE HTML>
<html>
<head>
</head>
<body>
<header>
Whatever Inc.
</header>
<main>
<p>
Thanks for your enquiry. As this is the 1<sup>st</sup> time you have contacted us, we would like to clarify a few things:
</p>
<ol>
<li>
Please confirm this is your email by replying.
</li>
<li>
Then perform this step.
</li>
</ol>
<p>
Please solve this <img alt="complex equation" src="http://upload.wikimedia.org/wikipedia/commons/8/8d/First_Equation_Ever.png"/>. Then, in any order, could you please:
</p>
<ul>
<li>
a point.
</li>
<li>
another point, with a hyperlink.
</li>
</ul>
<p>
Sincerely,
</p>
<p>
The whatever.com team
</p>
</main>
<footer>
Ph: 000 000 000<br/>
mail: whatever st
</footer>
</body>
</html>
...will be transformed into:
Whatever Inc.
Thanks for your enquiry. As this is the 1st time you have contacted us, we would like to clarify a few things:
1. Please confirm this is your email by replying.
2. Then perform this step.
Please solve this [complex equation<http://upload.wikimedia.org/wikipedia/commons/8/8d/First_Equation_Ever.png>]. Then, in any order, could you please:
* a point.
* another point, with a hyperlink<http://en.wikipedia.org/wiki/Hyperlink>.
Sincerely,
The whatever.com team
Ph: 000 000 000
mail: whatever st
...as opposed to:
Whatever Inc.
Thanks for your enquiry. As this is the 1st time you have contacted us, we would like to clarify a few things:
Please confirm this is your email by replying.
Then perform this step.
Please solve this . Then, in any order, could you please:
a point.
another point, with a hyperlink.
Sincerely,
The whatever.com team
Ph: 000 000 000
mail: whatever st
You could use this:
public static string StripHTML(string HTMLText, bool decode = true)
{
Regex reg = new Regex("<[^>]+>", RegexOptions.IgnoreCase);
var stripped = reg.Replace(HTMLText, "");
return decode ? HttpUtility.HtmlDecode(stripped) : stripped;
}
Updated
Thanks for the comments I have updated to improve this function
I've heard from a reliable source that, if you're doing HTML parsing in .Net, you should look at the HTML agility pack again..
http://www.codeplex.com/htmlagilitypack
Some sample on SO..
HTML Agility pack - parsing tables
What you are looking for is a text-mode DOM renderer that outputs text, much like Lynx or other Text browsers...This is much harder to do than you would expect.
I had some decoding issues with HtmlAgility and I didn't want to invest time investigating it.
Instead I used that utility from the Microsoft Team Foundation API:
var text = HtmlFilter.ConvertToPlainText(htmlContent);
Have you tried http://www.aaronsw.com/2002/html2text/ it's Python, but open source.
Assuming you have well formed html, you could also maybe try an XSL transform.
Here's an example:
using System;
using System.IO;
using System.Xml.Linq;
using System.Xml.XPath;
using System.Xml.Xsl;
class Html2TextExample
{
public static string Html2Text(XDocument source)
{
var writer = new StringWriter();
Html2Text(source, writer);
return writer.ToString();
}
public static void Html2Text(XDocument source, TextWriter output)
{
Transformer.Transform(source.CreateReader(), null, output);
}
public static XslCompiledTransform _transformer;
public static XslCompiledTransform Transformer
{
get
{
if (_transformer == null)
{
_transformer = new XslCompiledTransform();
var xsl = XDocument.Parse(#"<?xml version='1.0'?><xsl:stylesheet version=""1.0"" xmlns:xsl=""http://www.w3.org/1999/XSL/Transform"" exclude-result-prefixes=""xsl""><xsl:output method=""html"" indent=""yes"" version=""4.0"" omit-xml-declaration=""yes"" encoding=""UTF-8"" /><xsl:template match=""/""><xsl:value-of select=""."" /></xsl:template></xsl:stylesheet>");
_transformer.Load(xsl.CreateNavigator());
}
return _transformer;
}
}
static void Main(string[] args)
{
var html = XDocument.Parse("<html><body><div>Hello world!</div></body></html>");
var text = Html2Text(html);
Console.WriteLine(text);
}
}
Because I wanted conversion to plain text with LF and bullets, I found this pretty solution on codeproject, which covers many conversion usecases:
Convert HTML to Plain Text
Yep, looks so big, but works fine.
The easiest would probably be tag stripping combined with replacement of some tags with text layout elements like dashes for list elements (li) and line breaks for br's and p's.
It shouldn't be too hard to extend this to tables.
Here is the short sweet answer using HtmlAgilityPack. You can run this in LinqPad.
var html = "<div>..whatever html</div>";
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
var plainText = doc.DocumentNode.InnerText;
I simply use HtmlAgilityPack in any .NET project that needs HTML parsing. It's simple, reliable, and fast.
Update - you are correct that the above removes tags but does not decode the escaped characters. This will do it:
var a = "This & that";
var result = System.Web.HttpUtility.HtmlDecode(a);
result.Dump();
Using the two together you can get the plain text from the HTML.
Another post suggests the HTML agility pack:
This is an agile HTML parser that
builds a read/write DOM and supports
plain XPATH or XSLT (you actually
don't HAVE to understand XPATH nor
XSLT to use it, don't worry...). It is
a .NET code library that allows you to
parse "out of the web" HTML files. The
parser is very tolerant with "real
world" malformed HTML. The object
model is very similar to what proposes
System.Xml, but for HTML documents (or
streams).
I have used Detagger in the past. It does a pretty good job of formatting the HTML as text and is more than just a tag remover.
This function convert "What You See in the browser" to plain text with line breaks. (If you want to see result in the browser just use commented return value)
public string HtmlFileToText(string filePath)
{
using (var browser = new WebBrowser())
{
string text = File.ReadAllText(filePath);
browser.ScriptErrorsSuppressed = true;
browser.Navigate("about:blank");
browser?.Document?.OpenNew(false);
browser?.Document?.Write(text);
return browser.Document?.Body?.InnerText;
//return browser.Document?.Body?.InnerText.Replace(Environment.NewLine, "<br />");
}
}
I don't know C#, but there is a fairly small & easy to read python html2txt script here: http://www.aaronsw.com/2002/html2text/
I have recently blogged on a solution that worked for me by using a Markdown XSLT file to transform the HTML Source. The HTML source will of course need to be valid XML first
Try the easy and usable way: just call StripHTML(WebBrowserControl_name);
public string StripHTML(WebBrowser webp)
{
try
{
doc.execCommand("SelectAll", true, null);
IHTMLSelectionObject currentSelection = doc.selection;
if (currentSelection != null)
{
IHTMLTxtRange range = currentSelection.createRange() as IHTMLTxtRange;
if (range != null)
{
currentSelection.empty();
return range.text;
}
}
}
catch (Exception ep)
{
//MessageBox.Show(ep.Message);
}
return "";
}
In Genexus You can made with Regex
&pattern = '<[^>]+>'
&TSTRPNOT=&TSTRPNOT.ReplaceRegEx(&pattern,"")
In Genexus possiamo gestirlo con Regex,
If you are using .NET framework 4.5 you can use System.Net.WebUtility.HtmlDecode() which takes a HTML encoded string and returns a decoded string.
Documented on MSDN at: http://msdn.microsoft.com/en-us/library/system.net.webutility.htmldecode(v=vs.110).aspx
You can use this in a Windows Store app as well.
You can use WebBrowser control to render in memory your html content. After LoadCompleted event fired...
IHTMLDocument2 htmlDoc = (IHTMLDocument2)webBrowser.Document;
string innerHTML = htmlDoc.body.innerHTML;
string innerText = htmlDoc.body.innerText;
This is another solution to convert HTML to Text or RTF in C#:
SautinSoft.HtmlToRtf h = new SautinSoft.HtmlToRtf();
h.OutputFormat = HtmlToRtf.eOutputFormat.TextUnicode;
string text = h.ConvertString(htmlString);
This library is not free, this is commercial product and it is my own product.
I've been having problems writing XML and reading it in. I have a handwritten XML that gets read in fine, but after I write the XML it acts funny.
The output of the WriteXML: http://www.craigmouser.com/random/test.xml
It works if you hit enter after the (specials) tag. I.E. make (specials)(special) look like
(specials)
(special)
If I step through it, when reading it, it goes to the start node of specials, then the next iteration reads it as an EndElement with name Shots. I have no idea where to go from here. Thanks in advance.
Code: Writing
public void SaveXMLFile(string filename, Bar b, Boolean saveOldData)
{
XmlWriter xml;
if(filename.Contains(".xml"))
{
xml = XmlWriter.Create(filename);
}
else
{
xml = XmlWriter.Create(filename + ".xml");
}
xml.WriteStartElement("AggievilleBar");
xml.WriteElementString("name", b.Name);
xml.WriteStartElement("picture");
xml.WriteAttributeString("version", b.PictureVersion.ToString());
xml.WriteEndElement();
xml.WriteElementString("location", b.Location.Replace(Environment.NewLine, "\n"));
xml.WriteElementString("news", b.News.Replace(Environment.NewLine, "\n"));
xml.WriteElementString("description", b.Description.Replace(Environment.NewLine, "\n"));
xml.WriteStartElement("specials");
xml.WriteString("\n"); //This line fixes the problem... ?!?!
foreach (Special s in b.Specials)
{
if (s.DayOfWeek > 0 || (s.DayOfWeek == -1
&& ((s.Date.CompareTo(DateTime.Today) < 0 && saveOldData )
|| s.Date.CompareTo(DateTime.Today) >= 0)))
{
xml.WriteStartElement("special");
xml.WriteAttributeString("dayofweek", s.DayOfWeek.ToString());
if (s.DayOfWeek == -1)
xml.WriteAttributeString("date", s.Date.ToString("yyyy-MM-dd"));
xml.WriteAttributeString("price", s.Price.ToString());
xml.WriteString(s.Name);
xml.WriteEndElement();
}
}
xml.WriteEndElement();
xml.WriteEndElement();
xml.Close();
}
Code: Reading
public Bar LoadXMLFile(string filename)
{
List<Special> specials = new List<Special>();
XmlReader xml;
try
{
xml = XmlReader.Create(filename);
}
catch (Exception)
{
MessageBox.Show("Unable to open file. If you get this error upon opening the program, we failed to pull down your current data. You will most likely be unable to save, but you are free to try. If this problem persists please contact us at pulsarproductionssupport#gmail.com",
"Error Opening File", MessageBoxButtons.OK, MessageBoxIcon.Error);
return null;
}
Bar current = new Bar();
Special s = new Special();
while (xml.Read())
{
if (xml.IsStartElement())
{
switch (xml.Name)
{
case "AggievilleBar":
current = new Bar();
break;
case "name":
if (xml.Read())
current.Name = xml.Value.Trim();
break;
case "picture":
if (xml.HasAttributes)
{
try
{
current.PictureVersion = Int32.Parse(xml.GetAttribute("version"));
}
catch (Exception)
{
MessageBox.Show("Error reading in the Picture Version Number.","Error",MessageBoxButtons.OK,MessageBoxIcon.Error);
}
}
break;
case "location":
if (xml.Read())
current.Location = xml.Value.Trim();
break;
case "news":
if (xml.Read())
current.News = xml.Value.Trim();
break;
case "description":
if (xml.Read())
current.Description = xml.Value.Trim();
break;
case "specials":
if (xml.Read())
specials = new List<Special>();
break;
case "special":
s = new Special();
if (xml.HasAttributes)
{
try
{
s.DayOfWeek = Int32.Parse(xml.GetAttribute(0));
if (s.DayOfWeek == -1)
{
s.Date = DateTime.Parse(xml.GetAttribute(1));
s.Price = Int32.Parse(xml.GetAttribute(2));
}
else
s.Price = Int32.Parse(xml.GetAttribute(1));
}
catch (Exception)
{
MessageBox.Show("Error reading in a special.", "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
if (xml.Read())
s.Name = xml.Value.Trim();
break;
}
}
else
{
switch (xml.Name)
{
case "AggievilleBar":
xml.Close();
break;
case "special":
specials.Add(s);
break;
case "specials":
current.Specials = specials;
break;
}
}
}
return current;
}
Without seeing your code it's hard to really give a straight answer to that question. However, I can suggest using Linq-to-XML instead of XMLReader/XMLWriter -- it's so much easier to work with when you don't have to read each node one at a time and determine what node you're working with, which sounds like the problem you're having.
For example, code like:
using (var reader = new XmlReader(...))
{
while reader.Read()
{
if (reader.Name = "book" && reader.IsStartElement)
{
// endless, confusing nesting!!!
}
}
}
Becomes:
var elem = doc.Descendants("book").Descendants("title")
.Where(c => c.Attribute("name").Value == "C# Basics")
.FirstOrDefault();
For an introduction to LINQ-to-XML, check out http://www.c-sharpcorner.com/UploadFile/shakthee/2868/, or just search for "Linq-to-XML". Plenty of examples out there.
Edit: I tried your code and I was able to reproduce your problem. It seems that without a newline before the special tag, the first special element is read in as IsStartElement() == false. I wasn't sure why this is; even skimmed through the XML Specifications and didn't see any requirements about newlines before elements.
I rewrote your code in Linq-to-XML and it worked fine without any newlines:
var xdoc = XDocument.Load(filename);
var barElement = xdoc.Element("AggievilleBar");
var specialElements = barElement.Descendants("special").ToList();
var specials = new List<Special>();
specialElements.ForEach(s =>
{
var dayOfWeek = Convert.ToInt32(s.Attribute("dayofweek").Value);
var price = Convert.ToInt32(s.Attribute("price").Value);
var date = s.Attribute("date");
specials.Add(new Special
{
Name = s.Value,
DayOfWeek = dayOfWeek,
Price = price,
Date = date != null ? DateTime.Parse(date.Value) : DateTime.MinValue
});
});
var bar = new Bar() {
Name = barElement.Element("name").Value,
PictureVersion = Convert.ToInt32(barElement.Elements("picture").Single()
.Attribute("version").Value),
Location = barElement.Element("location").Value,
Description = barElement.Element("description").Value,
News = barElement.Element("news").Value,
Specials = specials
};
return bar;
Would you consider using Linq-to-XML instead of XMLReader? I've had my share of trouble with XMLReader in the past and once I switched to Linq-to-XML haven't looked back!
EDIT: I know this question is rather old now, but I just came across an article that reminded me of this question and might explain why this is happening: --> http://www.codeproject.com/KB/dotnet/pitfalls_xml_4_0.aspx
The author states:
In this light, a nasty difference between XmlReaders/Writers and XDocument is the way whitespace is treated. (See http://msdn.microsoft.com/en-us/library/bb387014.aspx.)
From msdn:
In most cases, if the method takes LoadOptions as an argument, you can optionally preserve insignificant white space as text nodes in the XML tree. However, if the method is loading the XML from an XmlReader, then the XmlReader determines whether white space will be preserved or not. Setting PreserveWhitespace will have no effect.
So perhaps, since you're loading using an XmlReader, the XmlReader is making the determination as to whether or not it should preserve white space. Most likely it IS preserving the white space which is why the newline (or lack thereof) makes a difference. And it doesn't seem like you can do anything to change it, so long as you're using an XmlReader! Very peculiar.
I'd recommend you use the XmlDocument class and its Load and Save methods, and then work with the XML tree instead of messing around with XmlReader and XmlWriter. In my experience using XmlDocument has fewer weird formatting problems.
Okay so, I have read that .INI files have become obsolete now and the .NET Framework creators want us to use .XML files. However, I feel that it would be difficult for some of my users to use .XML files so I thought of creating a custom config file.
I have a list string which has 3 parameters (it is actually the snippet function in Scintilla), like this:
Snippet.Insert("Name", "Code", 'char');
Now I want to insert all snippets from a files which the user can add himself, including the name, code and character but I have no clue about how I would do this. Maybe something like:
[Snippet1] [Snippet1Code] [Snippet1Char]
[Snippet2] [Snippet2Code] [Snippet2Char]
[Snippet3] [Snippet3Code] [Snippet3Char]
However I don't know how I would read the values that way. If someone can tell me an efficient way to read snippets from a file I would be really grateful.
As others have suggested, and similar to #gmcalab's approach, here is a quick example using Linq To XML.
public class Snippet
{
public string Name { get; set; }
public string Code { get; set; }
public char Character { get; set; }
}
XDocument doc = XDocument.Parse(#"<snippets>
<snippet name='snippet1' character='a'>
<![CDATA[
// Code goes here
]]>
</snippet>
<snippet name='snippet2' character='b'>
<![CDATA[
// Code goes here
]]>
</snippet>
</snippets>");
List<Snippet> snippetsList = (from snippets in doc.Descendants("snippet")
select new Snippet
{
Name = snippets.Attribute("name").Value,
Character = Convert.ToChar(snippets.Attribute("character").Value),
Code = snippets.Value
}).ToList();
snippetsList.ForEach(s => Console.WriteLine(s.Code));
If you prefer ini files, I've read good things about Nini
Why not just set up your snippets in the xml you talked about then read it with XMLTextReader?
Pretty straight forward...
<snippets>
<snippet name="Name1" char="a">
<snippetCode>
for(int i = 0; i < 10; i++) { // do work }
</snippetCode>
</snippet>
<snippet name="Name2" char="b">
<snippetCode>
foreach(var item in items) { // do work }
</snippetCode>
</snippet>
</snippets>
C#
public class Snippet
{
public string Name {get;set;}
public char Char { get; set;}
public string Value { get; set;}
}
List<Snippet> Snippets = new List<Snippet>();
XmlTextReader reader = new XmlTextReader ("snippets.xml");
Snippet snippet = new Snippet();
while (reader.Read())
{
// Do some work here on the data.
switch (reader.NodeType)
{
case XmlNodeType.Element:
if(reader.Name == "snippet"){
while (reader.MoveToNextAttribute())
{
if(reader.Name == "Name"){
// get name attribute value
snippet.Name = reader.Value;
}
else if(reader.Name == "Char"){
// get char attribute value
snippet.Char = Char.Parse(reader.Value);
}
}
}
break;
case XmlNodeType.Text: //Display the text in each element.
//Here we get the actual snippet code
snippet.Value = reader.Value;
break;
case XmlNodeType. EndElement:
if(reader.Name == "snippet"){
// Add snippet to list
Snippets.Add(snippet);
// Create a new Snippet object
snippet = new Snippet();
}
break;
}
}
I'm looking for C# code to convert an HTML document to plain text.
I'm not looking for simple tag stripping , but something that will output plain text with a reasonable preservation of the original layout.
The output should look like this:
Html2Txt at W3C
I've looked at the HTML Agility Pack, but I don't think that's what I need. Does anyone have any other suggestions?
EDIT: I just download the HTML Agility Pack from CodePlex, and ran the Html2Txt project. What a disappointment (at least the module that does html to text conversion)! All it did was strip the tags, flatten the tables, etc. The output didn't look anything like the Html2Txt # W3C produced. Too bad that source doesn't seem to be available.
I was looking to see if there is a more "canned" solution available.
EDIT 2: Thank you everybody for your suggestions. FlySwat tipped me in the direction i wanted to go. I can use the System.Diagnostics.Process class to run lynx.exe with the "-dump" switch to send the text to standard output, and capture the stdout with ProcessStartInfo.UseShellExecute = false and ProcessStartInfo.RedirectStandardOutput = true. I'll wrap all this in a C# class. This code will be called only occassionly, so i'm not too concerned about spawning a new process vs. doing it in code. Plus, Lynx is FAST!!
Just a note about the HtmlAgilityPack for posterity. The project contains an example of parsing text to html, which, as noted by the OP, does not handle whitespace at all like anyone writing HTML would envisage. There are full-text rendering solutions out there, noted by others to this question, which this is not (it cannot even handle tables in its current form), but it is lightweight and fast, which is all I wanted for creating a simple text version of HTML emails.
using System.IO;
using System.Text.RegularExpressions;
using HtmlAgilityPack;
//small but important modification to class https://github.com/zzzprojects/html-agility-pack/blob/master/src/Samples/Html2Txt/HtmlConvert.cs
public static class HtmlToText
{
public static string Convert(string path)
{
HtmlDocument doc = new HtmlDocument();
doc.Load(path);
return ConvertDoc(doc);
}
public static string ConvertHtml(string html)
{
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(html);
return ConvertDoc(doc);
}
public static string ConvertDoc (HtmlDocument doc)
{
using (StringWriter sw = new StringWriter())
{
ConvertTo(doc.DocumentNode, sw);
sw.Flush();
return sw.ToString();
}
}
internal static void ConvertContentTo(HtmlNode node, TextWriter outText, PreceedingDomTextInfo textInfo)
{
foreach (HtmlNode subnode in node.ChildNodes)
{
ConvertTo(subnode, outText, textInfo);
}
}
public static void ConvertTo(HtmlNode node, TextWriter outText)
{
ConvertTo(node, outText, new PreceedingDomTextInfo(false));
}
internal static void ConvertTo(HtmlNode node, TextWriter outText, PreceedingDomTextInfo textInfo)
{
string html;
switch (node.NodeType)
{
case HtmlNodeType.Comment:
// don't output comments
break;
case HtmlNodeType.Document:
ConvertContentTo(node, outText, textInfo);
break;
case HtmlNodeType.Text:
// script and style must not be output
string parentName = node.ParentNode.Name;
if ((parentName == "script") || (parentName == "style"))
{
break;
}
// get text
html = ((HtmlTextNode)node).Text;
// is it in fact a special closing node output as text?
if (HtmlNode.IsOverlappedClosingElement(html))
{
break;
}
// check the text is meaningful and not a bunch of whitespaces
if (html.Length == 0)
{
break;
}
if (!textInfo.WritePrecedingWhiteSpace || textInfo.LastCharWasSpace)
{
html= html.TrimStart();
if (html.Length == 0) { break; }
textInfo.IsFirstTextOfDocWritten.Value = textInfo.WritePrecedingWhiteSpace = true;
}
outText.Write(HtmlEntity.DeEntitize(Regex.Replace(html.TrimEnd(), #"\s{2,}", " ")));
if (textInfo.LastCharWasSpace = char.IsWhiteSpace(html[html.Length - 1]))
{
outText.Write(' ');
}
break;
case HtmlNodeType.Element:
string endElementString = null;
bool isInline;
bool skip = false;
int listIndex = 0;
switch (node.Name)
{
case "nav":
skip = true;
isInline = false;
break;
case "body":
case "section":
case "article":
case "aside":
case "h1":
case "h2":
case "header":
case "footer":
case "address":
case "main":
case "div":
case "p": // stylistic - adjust as you tend to use
if (textInfo.IsFirstTextOfDocWritten)
{
outText.Write("\r\n");
}
endElementString = "\r\n";
isInline = false;
break;
case "br":
outText.Write("\r\n");
skip = true;
textInfo.WritePrecedingWhiteSpace = false;
isInline = true;
break;
case "a":
if (node.Attributes.Contains("href"))
{
string href = node.Attributes["href"].Value.Trim();
if (node.InnerText.IndexOf(href, StringComparison.InvariantCultureIgnoreCase)==-1)
{
endElementString = "<" + href + ">";
}
}
isInline = true;
break;
case "li":
if(textInfo.ListIndex>0)
{
outText.Write("\r\n{0}.\t", textInfo.ListIndex++);
}
else
{
outText.Write("\r\n*\t"); //using '*' as bullet char, with tab after, but whatever you want eg "\t->", if utf-8 0x2022
}
isInline = false;
break;
case "ol":
listIndex = 1;
goto case "ul";
case "ul": //not handling nested lists any differently at this stage - that is getting close to rendering problems
endElementString = "\r\n";
isInline = false;
break;
case "img": //inline-block in reality
if (node.Attributes.Contains("alt"))
{
outText.Write('[' + node.Attributes["alt"].Value);
endElementString = "]";
}
if (node.Attributes.Contains("src"))
{
outText.Write('<' + node.Attributes["src"].Value + '>');
}
isInline = true;
break;
default:
isInline = true;
break;
}
if (!skip && node.HasChildNodes)
{
ConvertContentTo(node, outText, isInline ? textInfo : new PreceedingDomTextInfo(textInfo.IsFirstTextOfDocWritten){ ListIndex = listIndex });
}
if (endElementString != null)
{
outText.Write(endElementString);
}
break;
}
}
}
internal class PreceedingDomTextInfo
{
public PreceedingDomTextInfo(BoolWrapper isFirstTextOfDocWritten)
{
IsFirstTextOfDocWritten = isFirstTextOfDocWritten;
}
public bool WritePrecedingWhiteSpace {get;set;}
public bool LastCharWasSpace { get; set; }
public readonly BoolWrapper IsFirstTextOfDocWritten;
public int ListIndex { get; set; }
}
internal class BoolWrapper
{
public BoolWrapper() { }
public bool Value { get; set; }
public static implicit operator bool(BoolWrapper boolWrapper)
{
return boolWrapper.Value;
}
public static implicit operator BoolWrapper(bool boolWrapper)
{
return new BoolWrapper{ Value = boolWrapper };
}
}
As an example, the following HTML code...
<!DOCTYPE HTML>
<html>
<head>
</head>
<body>
<header>
Whatever Inc.
</header>
<main>
<p>
Thanks for your enquiry. As this is the 1<sup>st</sup> time you have contacted us, we would like to clarify a few things:
</p>
<ol>
<li>
Please confirm this is your email by replying.
</li>
<li>
Then perform this step.
</li>
</ol>
<p>
Please solve this <img alt="complex equation" src="http://upload.wikimedia.org/wikipedia/commons/8/8d/First_Equation_Ever.png"/>. Then, in any order, could you please:
</p>
<ul>
<li>
a point.
</li>
<li>
another point, with a hyperlink.
</li>
</ul>
<p>
Sincerely,
</p>
<p>
The whatever.com team
</p>
</main>
<footer>
Ph: 000 000 000<br/>
mail: whatever st
</footer>
</body>
</html>
...will be transformed into:
Whatever Inc.
Thanks for your enquiry. As this is the 1st time you have contacted us, we would like to clarify a few things:
1. Please confirm this is your email by replying.
2. Then perform this step.
Please solve this [complex equation<http://upload.wikimedia.org/wikipedia/commons/8/8d/First_Equation_Ever.png>]. Then, in any order, could you please:
* a point.
* another point, with a hyperlink<http://en.wikipedia.org/wiki/Hyperlink>.
Sincerely,
The whatever.com team
Ph: 000 000 000
mail: whatever st
...as opposed to:
Whatever Inc.
Thanks for your enquiry. As this is the 1st time you have contacted us, we would like to clarify a few things:
Please confirm this is your email by replying.
Then perform this step.
Please solve this . Then, in any order, could you please:
a point.
another point, with a hyperlink.
Sincerely,
The whatever.com team
Ph: 000 000 000
mail: whatever st
You could use this:
public static string StripHTML(string HTMLText, bool decode = true)
{
Regex reg = new Regex("<[^>]+>", RegexOptions.IgnoreCase);
var stripped = reg.Replace(HTMLText, "");
return decode ? HttpUtility.HtmlDecode(stripped) : stripped;
}
Updated
Thanks for the comments I have updated to improve this function
I've heard from a reliable source that, if you're doing HTML parsing in .Net, you should look at the HTML agility pack again..
http://www.codeplex.com/htmlagilitypack
Some sample on SO..
HTML Agility pack - parsing tables
What you are looking for is a text-mode DOM renderer that outputs text, much like Lynx or other Text browsers...This is much harder to do than you would expect.
I had some decoding issues with HtmlAgility and I didn't want to invest time investigating it.
Instead I used that utility from the Microsoft Team Foundation API:
var text = HtmlFilter.ConvertToPlainText(htmlContent);
Have you tried http://www.aaronsw.com/2002/html2text/ it's Python, but open source.
Assuming you have well formed html, you could also maybe try an XSL transform.
Here's an example:
using System;
using System.IO;
using System.Xml.Linq;
using System.Xml.XPath;
using System.Xml.Xsl;
class Html2TextExample
{
public static string Html2Text(XDocument source)
{
var writer = new StringWriter();
Html2Text(source, writer);
return writer.ToString();
}
public static void Html2Text(XDocument source, TextWriter output)
{
Transformer.Transform(source.CreateReader(), null, output);
}
public static XslCompiledTransform _transformer;
public static XslCompiledTransform Transformer
{
get
{
if (_transformer == null)
{
_transformer = new XslCompiledTransform();
var xsl = XDocument.Parse(#"<?xml version='1.0'?><xsl:stylesheet version=""1.0"" xmlns:xsl=""http://www.w3.org/1999/XSL/Transform"" exclude-result-prefixes=""xsl""><xsl:output method=""html"" indent=""yes"" version=""4.0"" omit-xml-declaration=""yes"" encoding=""UTF-8"" /><xsl:template match=""/""><xsl:value-of select=""."" /></xsl:template></xsl:stylesheet>");
_transformer.Load(xsl.CreateNavigator());
}
return _transformer;
}
}
static void Main(string[] args)
{
var html = XDocument.Parse("<html><body><div>Hello world!</div></body></html>");
var text = Html2Text(html);
Console.WriteLine(text);
}
}
Because I wanted conversion to plain text with LF and bullets, I found this pretty solution on codeproject, which covers many conversion usecases:
Convert HTML to Plain Text
Yep, looks so big, but works fine.
The easiest would probably be tag stripping combined with replacement of some tags with text layout elements like dashes for list elements (li) and line breaks for br's and p's.
It shouldn't be too hard to extend this to tables.
Here is the short sweet answer using HtmlAgilityPack. You can run this in LinqPad.
var html = "<div>..whatever html</div>";
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
var plainText = doc.DocumentNode.InnerText;
I simply use HtmlAgilityPack in any .NET project that needs HTML parsing. It's simple, reliable, and fast.
Update - you are correct that the above removes tags but does not decode the escaped characters. This will do it:
var a = "This & that";
var result = System.Web.HttpUtility.HtmlDecode(a);
result.Dump();
Using the two together you can get the plain text from the HTML.
Another post suggests the HTML agility pack:
This is an agile HTML parser that
builds a read/write DOM and supports
plain XPATH or XSLT (you actually
don't HAVE to understand XPATH nor
XSLT to use it, don't worry...). It is
a .NET code library that allows you to
parse "out of the web" HTML files. The
parser is very tolerant with "real
world" malformed HTML. The object
model is very similar to what proposes
System.Xml, but for HTML documents (or
streams).
I have used Detagger in the past. It does a pretty good job of formatting the HTML as text and is more than just a tag remover.
This function convert "What You See in the browser" to plain text with line breaks. (If you want to see result in the browser just use commented return value)
public string HtmlFileToText(string filePath)
{
using (var browser = new WebBrowser())
{
string text = File.ReadAllText(filePath);
browser.ScriptErrorsSuppressed = true;
browser.Navigate("about:blank");
browser?.Document?.OpenNew(false);
browser?.Document?.Write(text);
return browser.Document?.Body?.InnerText;
//return browser.Document?.Body?.InnerText.Replace(Environment.NewLine, "<br />");
}
}
I don't know C#, but there is a fairly small & easy to read python html2txt script here: http://www.aaronsw.com/2002/html2text/
I have recently blogged on a solution that worked for me by using a Markdown XSLT file to transform the HTML Source. The HTML source will of course need to be valid XML first
Try the easy and usable way: just call StripHTML(WebBrowserControl_name);
public string StripHTML(WebBrowser webp)
{
try
{
doc.execCommand("SelectAll", true, null);
IHTMLSelectionObject currentSelection = doc.selection;
if (currentSelection != null)
{
IHTMLTxtRange range = currentSelection.createRange() as IHTMLTxtRange;
if (range != null)
{
currentSelection.empty();
return range.text;
}
}
}
catch (Exception ep)
{
//MessageBox.Show(ep.Message);
}
return "";
}
In Genexus You can made with Regex
&pattern = '<[^>]+>'
&TSTRPNOT=&TSTRPNOT.ReplaceRegEx(&pattern,"")
In Genexus possiamo gestirlo con Regex,
If you are using .NET framework 4.5 you can use System.Net.WebUtility.HtmlDecode() which takes a HTML encoded string and returns a decoded string.
Documented on MSDN at: http://msdn.microsoft.com/en-us/library/system.net.webutility.htmldecode(v=vs.110).aspx
You can use this in a Windows Store app as well.
You can use WebBrowser control to render in memory your html content. After LoadCompleted event fired...
IHTMLDocument2 htmlDoc = (IHTMLDocument2)webBrowser.Document;
string innerHTML = htmlDoc.body.innerHTML;
string innerText = htmlDoc.body.innerText;
This is another solution to convert HTML to Text or RTF in C#:
SautinSoft.HtmlToRtf h = new SautinSoft.HtmlToRtf();
h.OutputFormat = HtmlToRtf.eOutputFormat.TextUnicode;
string text = h.ConvertString(htmlString);
This library is not free, this is commercial product and it is my own product.