\
**Category:** blog\
**Created:** [November 15, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131 "2009-11-15T00:00:00Z")\
**Posts on this page:** 20\
**Page:** 2
**Author:**  [@phreakre](https://discourse.codinghorror.com/u/phreakre)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/21 "2009-11-16T00:00:00Z")
The only crime committed by most novice developers is not knowing what the alternatives are. This post, on the other hand, does nothing to help promote what those alternatives are. No cherry picked recommendations for a few common web languages? PHP, RoR, etc?
Hpricot and Rubyful Soup help me a good bit in RoR.
---
**Author:**  [@Wayne](https://discourse.codinghorror.com/u/Wayne)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/22 "2009-11-16T00:00:00Z")
Now, I have that song in my head! Go Metallica!
---
**Author:**  [@Chris\_F](https://discourse.codinghorror.com/u/Chris_F)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/23 "2009-11-16T00:00:00Z")
“Even Jon Skeet cannot parse HTML using regular expressions.”
Them’s fightin’ words.
---
**Author:**  [@PadraigB](https://discourse.codinghorror.com/u/PadraigB)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/24 "2009-11-16T00:00:00Z")
ESR disagrees 🙂
[http://www.jgc.org/blog/2009/11/parsing-html-in-python-with.html](http://www.jgc.org/blog/2009/11/parsing-html-in-python-with.html)
---
**Author:**  [@R\_\_Bemrose2](https://discourse.codinghorror.com/u/R__Bemrose2)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/25 "2009-11-16T00:00:00Z")
Is it just me or does this blog post try to argue both sides of the same issue?
---
**Author:**  [@Patrick](https://discourse.codinghorror.com/u/Patrick)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/26 "2009-11-16T00:00:00Z")
“Even Jon Skeet cannot parse HTML using regular expressions.”
I lol’d, what a great way to put this in perspective.
---
**Author:**  [@RobertC](https://discourse.codinghorror.com/u/RobertC)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/27 "2009-11-16T00:00:00Z")
But the link to CPAN HTML::Sanitizer is broken.
---
**Author:**  [@DennisG](https://discourse.codinghorror.com/u/DennisG)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/28 "2009-11-16T00:00:00Z")
Link behind “HTML::Sanitizer” is dead.
---
**Author:**  [@DominicP](https://discourse.codinghorror.com/u/DominicP)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/29 "2009-11-16T00:00:00Z")
So what is the preferred method for dealing with XSS (Cross Site Scripting) issues then, particularly if you’re using a Rich Text Editor that saves formatting as HTML?
---
**Author:**  [@DanielS](https://discourse.codinghorror.com/u/DanielS)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/30 "2009-11-16T00:00:00Z")
The last time I went for an HTML library to parse some HTML, the HTML was so broken I had to resort to regex.
The regex broke afterwards, after the generated HTML was slightly changed. It was trivially fixed.
So, while I agree that HTML (and, particularly, XML) should be parsed appropriately, YMMV. I get the feeling a lot of people who complain about regex have never bothered to LEARN it as the complex language it is.
---
**Author:**  [@Arethuza](https://discourse.codinghorror.com/u/Arethuza)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/31 "2009-11-16T00:00:00Z")
What about HTMLTidy? [http://tidy.sourceforge.net/](http://tidy.sourceforge.net/) Convert stuff to proper XHTML and then use your XML processing mechanism of choice to parse the data.
---
**Author:**  [@Joe](https://discourse.codinghorror.com/u/Joe)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/32 "2009-11-16T00:00:00Z")
This would be a lot more helpful if some specific libs besides the Perl solution were posted. I had a non-trivial time trying to find ready-to-use stable libraries on various platforms (e.g. PHP). Any suggestions?
---
**Author:**  [@Goran](https://discourse.codinghorror.com/u/Goran)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/33 "2009-11-16T00:00:00Z")
Can someone get me one of those T-shirts with “I parse HTML with RE” on front?
---
**Author:**  [@Anonymous](https://discourse.codinghorror.com/u/Anonymous)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/34 "2009-11-16T00:00:00Z")
> I think that’s just as wrongheaded as demanding every trivial HTML processing task be handled by a full-blown parsing engine.
Maybe HTML processing isn’t trivial, Jeff.
---
**Author:**  [@sylvainulg](https://discourse.codinghorror.com/u/sylvainulg)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/35 "2009-11-16T00:00:00Z")
[http://search.cpan.org/~nesting/HTML-Sanitizer-0.04/Sanitizer.pm](http://search.cpan.org/~nesting/HTML-Sanitizer-0.04/Sanitizer.pm) tells me “not found”, btw. Pretty surprising, as we can still find the code in Nesting’s archives, and that it is still refered to at e.g. [http://search.cpan.org/~podmaster/HTML-Scrubber-0.08/Scrubber.pm](http://search.cpan.org/~podmaster/HTML-Scrubber-0.08/Scrubber.pm)
---
**Author:**  [@Rev\_Matt](https://discourse.codinghorror.com/u/Rev_Matt)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/36 "2009-11-16T00:00:00Z")
I’m torn here. I mean there’s jwz’s famous quote about
Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.
On the other hand, CPAN isn’t particularly useful if you aren’t using Perl. And “Hell is other people’s Perl”.
---
**Author:**  [@chaos](https://discourse.codinghorror.com/u/chaos)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/37 "2009-11-16T00:00:00Z")
I have no joke here, I just like saying [http://discordianquotes.com/quote/8008](http://discordianquotes.com/quote/8008)
---
**Author:**  [@JarrettM](https://discourse.codinghorror.com/u/JarrettM)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/38 "2009-11-16T00:00:00Z")
“The only crime being perpetrated is not knowing what the alternatives are.”
Yup - that just about covers the whole darn thing! Who hasn’t worked with developers who rolled their own XYZ, when that program is already out there and supported by some other community.
---
**Author:**  [@LinusT](https://discourse.codinghorror.com/u/LinusT)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/39 "2009-11-16T00:00:00Z")
This is the most useful thing you said in a long time. Glad to have you bad Jeff.
---
**Author:**  [@Paul\_N](https://discourse.codinghorror.com/u/Paul_N)\
**Post date:** [November 16, 2009, 12:00am UTC](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131/40 "2009-11-16T00:00:00Z")
I have a daily job that scrapes a page. Unfortunately, it looks like the author of that page is emitting with Word or something similarly horrible. Tables within tables within tables without sanity(or regularity). The faint whisper of “ia, ia” curdles around the mind when contemplating the source.
A DOM parser would gibber insanely to itself, quietly screaming at the brokenness of the form.
Since I didn’t particularly want to do that to a parse, I regexed away and was able to do it. Only minor loss of sanity points…
[Previous page](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131.md?page=1)
[Next page](https://discourse.codinghorror.com/t/parsing-html-the-cthulhu-way/131.md?page=3)