a companion discussion area for blog.codinghorror.com

Searching all Craigslist.org Cities


#1

If you've ever used Craigslist.org-- a fantastic and rather odd resource-- you may have noticed that it's heavily biased towards per-city searches. This is a pain if you want to do a national search across all cities that Craigslist.org operates sites for. A while back, I found a web page offering an all-city search, but I wasn't happy with the performance. And now it's been perma-banned by the Craigslist.org brass, so.. time to roll up my sleeves and implement my own improved Craigslist.org "all cities" jobs search.


This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2005/03/searching-all-craigslistorg-cities.html

#2

Also I’m 99.999% sure that posting is a cleverly constructed joke. And what a great one!


#3

One thing you have to admire about ads like this one, assuming it really is for real, is that some people really know what they want and are not afraid to ask for it. :slight_smile:

Is non-progressive rendering really an ASP.NET issue or a general issue with serving Web pages? AFAIK, you can’t get progressive rendering of straight HTML pages. What about other dynamic rendering platforms?

Just curious, I know very little about this.


#4

AFAIK, you can’t get progressive rendering of straight HTML pages

Er… yeah you can. That’s the way HTML works; it renders what it can as soon as it arrives, except for pathological cases. Here’s an overview:

http://blogs.msdn.com/ie/archive/2005/02/10/370721.aspx

and here’s a demo:

http://msdn.microsoft.com/workshop/samples/author/dhtml/refs/tables3.htm

The slower your connection (and/or the larger the page) the bigger the difference will be.


#5

Welp, there ya go … I said I know very little about this … :slight_smile:


#6

I wanted to tell you thank you very much for developing the Craigslist Search All-Cities program! I work from home as an independent contractor, and I have found your program very valuable in searching for telecommuting jobs.

Thank you!!


#7

I did the same thing you did (funny how craigslist doesnt do what so many people request) but i made it a little different, i did city and state wide searches for For Sale items on craigslist.

so like everyone elses… my server got blocked automatically

but… i found a way around it. and i am in the process of completing the site/program so everyone can benefit what craigslist ignores.

stay tuned. if youd like… email me and ill give you the site/info


#8

Maybe you can help

There are 1000s of postings on Craig’s List for offers of housing to Katrina survivors. But these folks have little or no access to the web, email and craig’s list etc. I have been trying to find out how I can get a file or report that includes housing offers out of their database accross all cities that can then be printed and distributed. The people who need housing need a piece of paper with phone numbers (and money to make the call). They don’t have computers!

I have had ZERO luck getting anyone from craig’s list to contact me. Do you have knowledge on how to get in touch with someone from Craig’s list who has power to make this happen or a way to pull this data from their system?

PLEASE reply to this post or contact me directly @ glen@colemancomputers.com

Thanks…

Glen


#9

Hi Glen,

Your best bet is the craigslist forums:

http://forums.craigslist.org/?i=


#10

can you get me the email address of that guy “Eric” who posted the below?

i would very much like to see his site and the code used to do what he claims…
contact me jionin@optonline.com
or aim: ionizer5
thanks


#11

I realize that craigslist has blocked this service, but I’m more interested in how you did it from an academic standpoint.

How did you get,parse, and then cache the data from the craigslist posts? I’m looking at the rss feeds on the site, but they generally only contain sparse information on the last 15 posts in that section.

Would you mind sharing your overarching design? or possibly source code snippets?


#12

a href="http://www.crxsearch"http://www.crxsearch/a dot com

Thats my multi city, multi state, multi region craigslist search tool. i wont go into details on how I did it, but i will say that it uses RSS feeds.

See the biggest problem in this kind of search, is getting the user to be able to search craigslist via rss feeds… without getting banned by the CL servers. The CL servers will ban an IP if it hits the CL rss feeds too often. So my task was to find a way for offer the search tool, but have it so the CL servers will not be able to block the IP. I feel like I just went in a big circle with that… But yes… as of right now…

As of right now… my program has
New Users per day: 1.18
We have 348 registered users
Total Craigslist searches: 4541
Total hits: 2375

Which I know is not a “lot” compared to chovys original site, but hey… at least mine works :slight_smile:

And yes, I do know that if RSS feeds gets taken off, my search will not work. BUT… there is another way, that I believe I can make this work. But hopefully I wont have to try that method because it will take even longer than this did.


#13

Eric, it works, eh?


#14

Here’s another craigslist search tool. I’ve taken a look at what else is out there, and I’m pretty sure this is the best.

It’s all web based, so no need to install anything. It won’t be blocked, because it simply opens the results in iFrames. But it’s set up to let you easily and QUICKLY scroll through the various cities. Anyway, just go give it a try and you’ll see what I mean.

http://www.craigshelper.com


#15

I’ll through my tool onto the pile.

http://www.craig2mail.com/craigs.html

Ok, it doesn’t search multiple cities. What it does do is email you when an ad that matches your search is posted. No more searching over and over just to find that someone else beat you to it. Just sit back and watch your email. I think it is pretty cool.

Craig (no not that Craig)


#16

Beyond blocking IP’s is there legal action that CL has taken in the past? What are these cases?


#17

Here’s a Craigslist search you might digg:
http://cl.radiux.com


#18

Anyone tried CraigSpy? Standalone prog so potential IP bans are user-to-user, but I’ve been able to search about as much as 10,000 pages on CL with simple search terms. Good for getting free stuff for the house…lot of potential, really. :slight_smile:


#19

Anyone tried CraigSpy? Standalone prog so potential IP bans are user-to-user, but I’ve been able to search about as much as 10,000 pages on CL with simple search terms. Good for getting free stuff for the house…lot of potential, really. :slight_smile:

http://www.craig-spy.com


#20

I discovered that if you put this in google “not ok to contact this poster” and then type in some keywords, it will bring up craigslist ads from all over the country. sample google search:

“not ok to contact this poster” katrina

look at all those results! I tried to find some wording that was in all craig’s list postings, and this was as close as I could get.