Bounty: PHP Function that Scrapes Google

TOP del.icio.us digg

I need a function to plug into a bigger piece of software that I wrote, that scrapes Google.

It should be pretty simple for an experienced PHP programmer. I initially wrote a function that scrapes Scroogle, but they ban IPs after 100 queries, and I’d like to cut to the source since the function has to be rewritten anyway.

Project requirements:
– Create a PHP function that receives $search_term, scrapes Google, and returns the resulting hostnames as an array. If there are zero results, the function should return FALSE.
– The function must be usable for at least 5,000 queries per day. Something more like 20,000 would be ideal, of course.
– This is not a requirement, but in order to be able to do 5,000 queries per day, it may need to support proxies. See the links below.

Resources:
This is the function that I wrote, that you will be replacing. It’s perfectly functional, but isn’t usable in the long-term due to Scroogle’s IP banning.
This page outlines what you’re up against, and offers some solutions.
Another example PHP snippet.
Yes another example of working PHP code that scrapes Google.
– It has been suggested (but I have Not confirmed) that it’s easier to scrape Google while passing accessibility flags: http://labs.google.com/accessible/

Initial bounty offer: $250. Let me know if you want to bid higher.

Contact me if you’re interested. Good luck!

Leave a Reply