Craigslist is an amazingly useful site. I have bought many things from its listings and even got my current job through it. I hope also to find my next job through it, but my field has many nebulous names for itself: library, catalog, information science, archive, museum. Also, I don’t want to work anywhere in the Bay Area, but wouldn’t mind being in the South Bay or the Peninsula. In situations such as this, which are likely quite frequent, Craigslist is not so helpful. There is no way to truncate searches, such as “librar*” to include librarian, library, libraries, etc. There is no way to perform Boolean AND, OR, NOT searches. There is no way to remove frequently occuring irrelevant items. There is no way to search two sub-regions at once. So, unless I want to perform 20 searches a day and receive MANY completely irrelevant hits, I basically have to browse.

Until Yahoo Pipes that is. This site makes Craigslist searching powerful and incredibly useful. It allows you to combine any number of Craigslist searches via Craigslist’s RSS feed (see the link at the bottom of your search results) and then manipulate them in untold ways using the built-in RSS field values, such as title, text, date, etc. I designed my search by including every possible variation I could think of for the relevant terms, filtering out non-unique links (you could do this to titles too, but I didn’t want to risk losing a job with the same title), filtering out frequent false positives I saw, sorting by date, and then dumping to an RSS feed.

The site works with a fairly simple module system. On the left hand side of the pipes workflow you have a variety of categories of modules and within them various modules, the titles of which are usually self-explanatory. There is, however, decent documentation on-site if you can’t figure it out. For Craigslist, as I said, it’s simplest to use RSS feeds and for that you use the fetch feed module, seen to the left. You simply enter the URL for the feed you want and click the plus sign to add as many as you want. By including several search feeds, you are in essence building your OR Boolean search.

Then you can manipulate the information gathered from the feed in a variety of ways. Most of these fall under the category “operations”. One that basically any good Craigslist search will want to use is “Unique”. As you can see from the drop down in the image to the left, the fields that you make unique are actually derived from the fields of the RSS feed. There are a few generic fields as well, but clearly the power of pipes comes from that integration. Note, this can only happen once you connect the feed and unique modules with the blue “pipe.”

Filter is obviously going to be important as well. It likwise bases the fields on the RSS feed that went into it, as you can see in the dropdown to the left. You can set any of the various settings to include, exclude, any or all of the matching hits. If you would like an include list and an exclude list, you simply add a second module and connect them. This is the NOT of your Boolean search.

Once you have performed all your selecting operations, you will likely want to sort the results. This is simple with the module to the left, which once again includes the fields in the RSS feed itself.

Throughout the process, you can evaluate how your operations are effecting the results by using the debugger at the bottom of the screen. From it you can select any of the modules and see its output and see if maybe you messed something up and have no results or find that you are leaving out an important filter. You can also change the order the data passes through each module. Once you’ve saved the pipe and are taken to the results, you can get a link to it in any of the various ways you might want the feed sent to you.

The end result: my RSS reader dumps several relevant hits a day, along with a few bad ones of course, but a lot fewer and while spending no time performing the same searches over and over. To start with a completed pipe and modify it, see my job search pipe.

How to Actually Search Craigslist - Witigonen

Blogged with Flock

Tags: craigslistyahoo pipes searches

Popularity: 4% [?]