The only crime committed by most novice developers is not knowing what the alternatives are. This post, on the other hand, does nothing to help promote what those alternatives are. No cherry picked recommendations for a few common web languages? PHP, RoR, etc?
Hpricot and Rubyful Soup help me a good bit in RoR.
Now, I have that song in my head! Go Metallica!
āEven Jon Skeet cannot parse HTML using regular expressions.ā
Themās fightinā words.
Is it just me or does this blog post try to argue both sides of the same issue?
āEven Jon Skeet cannot parse HTML using regular expressions.ā
I lolād, what a great way to put this in perspective.
But the link to CPAN HTML::Sanitizer is broken.
Link behind āHTML::Sanitizerā is dead.
So what is the preferred method for dealing with XSS (Cross Site Scripting) issues then, particularly if youāre using a Rich Text Editor that saves formatting as HTML?
The last time I went for an HTML library to parse some HTML, the HTML was so broken I had to resort to regex.
The regex broke afterwards, after the generated HTML was slightly changed. It was trivially fixed.
So, while I agree that HTML (and, particularly, XML) should be parsed appropriately, YMMV. I get the feeling a lot of people who complain about regex have never bothered to LEARN it as the complex language it is.
What about HTMLTidy? http://tidy.sourceforge.net/ Convert stuff to proper XHTML and then use your XML processing mechanism of choice to parse the data.
This would be a lot more helpful if some specific libs besides the Perl solution were posted. I had a non-trivial time trying to find ready-to-use stable libraries on various platforms (e.g. PHP). Any suggestions?
Can someone get me one of those T-shirts with āI parse HTML with REā on front?
I think thatās just as wrongheaded as demanding every trivial HTML processing task be handled by a full-blown parsing engine.
Maybe HTML processing isnāt trivial, Jeff.
http://search.cpan.org/~nesting/HTML-Sanitizer-0.04/Sanitizer.pm tells me ānot foundā, btw. Pretty surprising, as we can still find the code in Nestingās archives, and that it is still refered to at e.g. http://search.cpan.org/~podmaster/HTML-Scrubber-0.08/Scrubber.pm
Iām torn here. I mean thereās jwzās famous quote about
Some people, when confronted with a problem, think āI know, Iāll use regular expressions.ā Now they have two problems.
On the other hand, CPAN isnāt particularly useful if you arenāt using Perl. And āHell is other peopleās Perlā.
āThe only crime being perpetrated is not knowing what the alternatives are.ā
Yup - that just about covers the whole darn thing! Who hasnāt worked with developers who rolled their own XYZ, when that program is already out there and supported by some other community.
This is the most useful thing you said in a long time. Glad to have you bad Jeff.
I have a daily job that scrapes a page. Unfortunately, it looks like the author of that page is emitting with Word or something similarly horrible. Tables within tables within tables without sanity(or regularity). The faint whisper of āia, iaā curdles around the mind when contemplating the source.
A DOM parser would gibber insanely to itself, quietly screaming at the brokenness of the form.
Since I didnāt particularly want to do that to a parse, I regexed away and was able to do it. Only minor loss of sanity pointsā¦
