An Exercise Program for the Fat Web

When I wrote about App-pocalypse Now in 2014, I implied the future still belonged to the web. And it does. But it's also true that the web has changed a lot in the last 10 years, much less the last 20 or 30.


This is a companion discussion topic for the original entry at https://blog.codinghorror.com/an-exercise-program-for-the-fat-web/
1 Like

Also! Support the project, I did!

image

Patreon https://www.patreon.com/pihole/

Donations at https://pi-hole.net/donate/

2 Likes

When blocking everything at this level of your network, how do you deal with sites that require you to disable adblocking? With browser extensions, it’s generally a matter of disabling blocking for a page and then reloading.

3 Likes

I really like the Pi-Hole, I have a Ubiquity Unifi network and have a pair of Pi-Holes set up as DNS servers as the Router requires 2 DNS servers. It works really well, but lately I have been very frustrated by the number of sites and systems my family use that are blocking logins and content when they are unable to connect to ad servers.

These are NOT companies that are dependent upon ads for revenue, these are companies that we pay monthly fees to for their services, but they have decided to use ad companies trackers to collect data on their paying customers. So I have had to whitelist a lot of ad servers to be able to use those services and it really bothers me. Almost enough to cause me to cancel my subscriptions to those services.

I am not looking for a good way to block ad servers based on the device’s IP address without having to setup a DNS server for each device.

1 Like

Why switch DHCP to pi-hole though? Most routers allow to specify which DNS servers they propagate to connected clients, and you can set pi-hole’s IP address as DNS server clients get from the router.

Moreover, it’s better to set it as first choice, but leave old DNS server as second and 1.1.1.1 or 8.8.8.8 as the 3rd if there is an option — pi’s are prone to sudden death unless SD card is chosen really carefully. Network without both DHCP and DNS servers is a really fun thing to debug.

2 Likes

PiHole is great, but it has some definite limitations.

  • It can only block at the domain/sub-domain level, so it will never be as effective as an in-browser ad-blocker.

  • Setting up ad blocking for https is (or at least has been) more involved than the project suggests. Unless something has recently changed, this involves setting up a local web server on the Pi specifically for TLS indirection. With how much of the web is behind TLS, and the push to put everything behind TLS, this is a huge limitation.

1 Like

You can’t always count on your OS resolving multiple DNS servers as “first, then second, then third”. It’s implementation dependent and sometimes the “fallback” servers will get queried instead. The beauty of using one device for DHCP + DNS is that if the whole device dies, you just turn the router’s DHCP service back on and you’re back in business – as leases expire, everybody gets the updated DNS server addresses as part of the renewal.

1 Like

This is my biggest hangup. The very first time that a page breaks and my wife asks how to fix it, and I have to tell her “well you log into this web page and turn this thing off, then wait a little bit and force-refresh the page, and if that didn’t work then wait a little longer and try again, then when it works you can log in again and turn the thing back on”… she’s going to find that Pi and feed it into a wood chipper.

I’ve switched my browsers to Brave (though I know it’s still Chromium-based and subject to future concern perhaps) but my home internet has my cable modem separated from my LAN/WIFI using the F/OSS edition of Netgate’s pfSense on a home-made box with a couple of NICs (NICs for inbound WAN, LAN, WIFI, and Dodgy Chinese Security cameras). I serve DNS from this box and use DNS blocking, caching, and payload monitoring for dodgy content as part of the repertoire of amazing pfSense packages that are available. It’s an awesome, but somewhat complex solution if your not at a basic geek level! It’s also opensource, and can be installed in a VM or a modest older machine and the basic install would probably equal that of a hardware-based router.

Of course, as your other followers have noted, this causes untold hell on certain “popular” sites and I invariably have to whitelist some sites to maintain family harmony!

1 Like

Give your wife a little credit, a quick bookmark and clicking a button isn’t so hard. I’ve saved way more pain with the global ad block than I’ve created by having to disable it occasionally.

FWIW I’ve been running it for a couple months and have only needed to do it probably 5 times max. It’s pretty reliable too, after clicking the button I’ve never had to wait any amount of time. Just click and refresh.

1 Like

Well, that’s encouraging. I can at least pitch the idea with that description and see what happens. Of course, 90% of my kids’ web usage is via some kind of app anyway, so it probably is really just down to me and her, but what about guests? I guess you can always hand-jam your DNS settings to avoid the 'hole…

That said, I’m also not 100% convinced that client-side blocking is on the way out yet. I read the whole (massive!) thread on the chrome-dev mailing list and it sounds like at least the folks on the Chrome team are genuinely trying to not wreck the blocking extensions. They’ve proposed some workarounds and it’s going to take time to see if they’re viable or not.

2 Likes

There’s no need to use pihole for DHCP. Just set your router to use the pihole IP address for DNS, and it will tell all your clients to use it too. The pihole DHCP service basically exists for people with crappy ISP-provided routers that don’t allow you to do that.

Also as several people have noted, while Pihole is awesome for mobile devices and I use it myself for blocking ads inside apps, it is much less effective than something like uBlock Origin.

Pihole is actually less effective than even the castrated ad blocking possible with Chrome’s proposed blocklists, which is essentially also how Safari content blocking works. Those blocklists can handle patterns while Pihole can only block entire domains.

I’ve been using my hosts file for years to block crappy ad sites. Just add a simple line like:

127.0.0.1 intellitxt.com
127.0.0.1 doubleclick.net

And never see ads from that domain ever again.

Using my old WRT54GL had the Tomato firmware and I used MVPS list to send many IP requests to localhost.

I suggest for hardware using Asus Tinker Board. RasPi is has good community support but not well made hardware. I’ve been using this thing for months crunching BOINC tasks 24/7 at 100% 4c/4t, so it can handle the workload of a DNS server: https://www.asus.com/us/Single-Board-Computer/Tinker-Board/

1 Like

While I tire of the Web moving ever more toward $$$$$$$$$$$, the gnats that irritate me daily are about usability - horribly illegible gray type, and hipster-designed sites with big, fat horizontal blocks that waste my time. Thank heaven for Reader View in Firefox.

Pi-hole is in every way worse even than alternatives like AdBlock Plus and 1Blocker.

Pi-hole is an URL-based (“rules-based”) content blocker, like ABP, but since Pi-hole blocks at the DNS level, there’s no way to block individual URLs on a domain, and no way to temporarily disable content blocking.

uBlock Origin is more sophisticated than that; it can inspect the details of individual requests and the content of responses. Chrome is deprecating the API that uBO uses, but still allows rules-based content blockers. uBO’s complaint is that Chrome’s approach will make it as stupid as ABP, though not as stupid as Pi-hole.

Rules-based content blockers are the only kind of blockers Apple supports on Safari. (I recommend 1Blocker X for iOS, a rules-based blocker with 350,000 rules.)

3 Likes

You can temporarily disable pihole through its web UI, but it’s a pain in the butt compared to turning off adblocking in a browser addon.

I love pihole and use it myself to block ads inside apps. That’s what it’s good for.

1 Like

Note that the article doesn’t make a clear distinction between the two, but uBlock & uBlock Origin are two different addons. uBlock Origin does not participate in the Acceptable Ads program, nor is it owned by Eye/o.

2 Likes

The DNS method seems to avoid this, at least I haven’t seen it yet. I saw it a ton with browser based adblockers.

That is a bummer. Can you indicate which services these are out of curiosity?

Mine does not. I do think that the DHCP switch is cleaner because it’s trivial to turn it off and switch back to the router DHCP which has the old DNS settings. You also said “Network without both DHCP and DNS servers is really fun to debug” but if you have the router fallback, it’s easy?

Yeah, don’t buy a cheap SD card! I recommend the Samsung Evo or Pro per these excellent benchmarks. A quality 32GB is only like $10, so why skimp?

Only truly massive websites serve ads from their own domain, or inline, though. We’ll see how this changes over time if DNS based blocking becomes more popular…

Why would this matter? the DNS requests are the thing being dealt with not the HTTP/HTTPS content.

You still need client side blocking for mobile devices that are actually used … y’know while mobile … and that’s a LOT of devices. I don’t think client side is going away any time soon on smartphones.

I don’t find this level of sophistication to be necessary, the vast majority of ads are served from distinct DNS domain names. Also, the performance impact of these client side rules is kinda serious, on the order of 25-33%. Try running https://browserbench.org/Speedometer2.0/ with client side ad blocking on and off. You’ll see what I mean.

Also having to configure each device as its own special snowflake with its own special client adblocker is kind of a pain. Very clean to do it at the DNS level for all devices globally.

That said as per above, client ad blockers will still be needed for mobile devices without a doubt. They aren’t going away or anything.

1 Like

Lately it has been the CBS app on Roku and the Peloton bike. They seem to add more ad services with every update.

1 Like