Rate Limiting and Velocity Checking

Lately, I've been seeing these odd little signs pop up in storefronts around town.

All the signs have various forms of this printed on them:


This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2009/02/rate-limiting-and-velocity-checking.html

Yes, however students in this context are minors and minors have a limited subset of rights that adults have. More importantly, they can’t vote and therefore can’t make something like this illegal in the future when they can vote (either because by then they’ll agree with it or they’ll have forgotten about it)

Restricted rights for minors are intended to protect the minors, not to protect adults from them. As for legality, that reminds me of when Afghanistan legalized voting for women. Tribal leaders were legitimately confused as to how women were supposed to vote when they could not leave the house.

Looking at it from the many years of experience as a convenience store manager:

Shoplifting is pretty much right out. Theft correlates more strongly with socioeconomic status than age, and while students have a low socioeconomic status per se, they also share their parent’s status to a large degree.

I wouldn’t even look very hard for shoplifting, because it’s not very efficient - less than ten percent of inventory shortage, and virtually zero cash shortage, is caused by the customer. The greatest source of inventory shortage, and almost the sum total of cash shortage, is your own employees (roughly half simple counting error, roughly half intentional theft)

There are a couple of better reasons, namely the aforementioned large groups and obnoxiousness. Efficiency drops with high customer count, and let’s face it, while most teens are decent and reasonable people, more than a few are obnoxious little bastards who will drive off other customers.

But limiting the number of students is still the wrong way to do handle the situation. Even better, the store should have more staff on hand for customer rushes (in this case, lunch and after school) - all registers open with good fast cashiers, one or two people stocking (really, keeping an eye on people), and the manager present to handle any problems as quickly as possible. Limiting their numbers accomplishes nothing but losing their custom, both in the immediate sense (they would buy something if allowed to) and in the long run (if you deny their custom as teenagers, you’ll never recapture it later)

If I had a simple site that required authentication to be able to post questions and submit answers, I’d:

  • Perform a captcha and email address verification on sign-up
  • Have a report spam/inappropriate link on each user-submission
  • Perform back-off algorithms similar to resends of packets for asking for captchas. e.g. Captcha every authenticaton, and then on the submission of the first 50 questions or answers. If none of their submissions were reported by other users as spam, start captcha-ing slightly randomly submissions for another 50, and then remove captchas permanently for that account unless they are reported for spam.
  • If more than one of their submissions is reported as spam by multiple users, reset the account’s captcha counters and warn the user. If they are reported a second time by multiple users, lock/ban the account.

I know it’s complicated, but I think it’s a decent compromise. Yes/no?

speaking about kids getting a tough time from store owners. We have a store near us that has the following sign:

Students are not allowed into the store with their school bags. Bags must be left outside.

Of course, kids being kids, the bags tend to go missing from outside the store :stuck_out_tongue:

In all seriousness, I’m sure there is some sort of breach of basic rights here.

I just read Neil (SM)'s post.

I think the progressive timer idea would work well there, too. Let the user perform up to x searches in a minute, then x searches in 10 mins, then x searches in an hour. If x is 3, that’s only 9 searches per hour. But, following his behavior, he’ll probably make the bulk of them up front while revising his search terms. Then, he’ll find a good set of thread results and spend the next 30 mins reading through them all (no searching). You still prevent large amounts of big queries on your system, but you’re more smartly tailoring it to the behavior of your users.

THREE students? They could wreak havoc. One could cause a diversion while the others filled their pockets.

In my hometown, Wayland Mass, there was/is a small convenience store right next to a pizza place, where kids would hang out. The store had a limit of ONE student at a time.

Re: students – I don’t think it’s to stop shoplifting per se (though that probably also happens), but just to prevent loitering. If just one or two people walk in, it’s probably because they need to buy something. If ten high school students walk in, 9 of them (or maybe all) are just causing a ruckus and aren’t real customers.

So it’s not really intending to be discriminatory (like the ethnic examples people gave), at least not against a group. It’s just trying to ensure that only legit customers come in. Arguably, no loitering would be less imflammatory, though perhaps the historical artifact is that students is a good proxy (maybe even more effective, if students tended to heed the n students sign more reliably than no loitering).

@Neil (SM) and others with forum search problems - Just use Google search with the site: modifier… For example, to search this site, type programming font site:codinghorror.com to find all Jeffs (great) posts on programming fonts.

Only 3 students at a time in the store please

You read it wrong. The sign says only 3 at a time. That’s because one or two students don’t have enough money to buy anything. You need at least three students to shake up enough change for a purchase.

That’s why you’ll often see one or two students waiting in front of the store. They are just waiting for another student so they have enough to go in.

Rather than annoy genuine users with more CAPTCHAs or timeouts, I would filter the questions using a Bayesian style spam filter. These have been proven to be easy to implement and highly effective.

An excellent read: http://www.paulgraham.com/spam.html

The spammers are only a small minority, so the rest of us shouldn’t be treated as criminals.

I think that entry predates its appearance.

Nah, it has been around for a while. I remember running into it at school last year (last school year, 07) in networking class, some time during first semester. Apparently a lot of people at the school were searching Google at once.

They really just limit the size of the groups of ‘students’, because they don’t buy much, and they really irritate the hell out of the grownups.

If the sign said Only 3 black people at a time in the store,
you would probably respond very differently.

That’s ridiculous. A teen will one day became an adult. That’s not discrimination. It’s like the voting age or the drinking age.

A black person will always be black…unless he’s Michael Jackson.

Take a long, hard look your own website – how would it deal with a roving band of bored, morally ambiguous schoolkids?

Sounds like /b/.

Fascinating…Zzzzzzz

Jeff, can you recommend some rate throttlers for ASP.net?

Some techniques are mentioned here: http://msmvps.com/blogs/omar/archive/2007/03/24/prevent-denial-of-service-dos-attacks-in-your-web-application.aspx

I’ve hit that screen so many times in regular usage at home, at work, at a friends’, and at school. There’s no single thing that’s more likely to make me finally switch from Google to an alternative (probably Live Search) than all those *#%(#@% false positives.

I haven’t switched yet, and to be fair it’s been a long while since I’ve seen them. There was a time though where I was getting this at least twice a month and lasting for most of the day when it hit (and yes, ethereal and other network monitor tools confirmed that I wasn’t infected and spamming Google).

Jon, that is incredibly discriminatory.

Yes, completely. But what’s your point? It’s neither illegal nor (for most people) unethical. And it serves a material purpose.

but nobody would dare ban old people. Young people are the last group
it’s acceptable to discriminate against. Offtopic, but it burns me
being a former young person and all.

I don’t care for it much either, but it doesn’t seem to be a grand sin, really.

The reason you don’t ban old people is because they have much more power. But there are other ways to discourage them from mobbing the premises…

Neil (SM):

In the case of forum searches I think a good compromise would be a threshold. For instance, instead of 1 search every 30 seconds, you could have 10 free searches from your IP and then 1 search every 60 seconds. You converge at 20 searches in 10 minutes per IP and after that the algorithm is really even more conservative than it was before, but still less irritating.