Comments on: Scrape A Webpage – Save To CSV

By: Steve

Steve — Mon, 11 Jan 2010 23:51:30 +0000

Me too … I am interested in the incrementatl navigation aspect of web crawling and web scraping (page1.htm, page2.htm, page3.htm … page99.htm) … I suppose I could create a .txt file with 1 to 99 URLs and manually increment the pages but I’d prefer an automated solution that knows to STOP at page 66 if there are no more results … looks promising as I used to write this stuff in either Java or C for a large data compiler …

By: Jeff

Jeff — Wed, 16 Dec 2009 14:54:52 +0000

Hey Aaron, great tutorial. I want to do the same type of scraping to a CSV, but instead of a single URL, I want to scrape from an auto incrementing URL…ex:

http://www.domain.com/index.php?id=1
http://www.domain.com/index.php?id=2
http://www.domain.com/index.php?id=3

The page elements will all be the same that I’m scraping

How would you do this? Or can you post an updated bot from your scraper and I can look at it?

Thanks!

By: Scott

Scott — Tue, 15 Dec 2009 03:19:05 +0000

Hi Aaron,

Thank you very much for this tutorial. I haven’t actually seen it yet but I can’t wait. Just been burning the midnight oil trying to finish a big project. I’ll donate too when I come back to visit, hopefully tomorrow!

Thanks again, very cool indeed!

By: Manuel Merz

Manuel Merz — Mon, 14 Dec 2009 11:50:51 +0000

Hey Kirsteen, nice to see you here

By: Aaron

Aaron — Sun, 13 Dec 2009 23:39:20 +0000

Thanks for the wonderful feedback!

Good luck on your scraper bot. If you need any help, just let me know.

By: Kirsteen

Kirsteen — Sun, 13 Dec 2009 22:40:32 +0000

Fantastic Tutorial, I like how simply you explained everything, I’ve already been using subs but you make it look neater and I like your suggestion of ‘Initializer’ and working from the bottom up. I’m away to start my first scraper bot. Thanks a lot!

By: Aaron

Aaron — Sun, 13 Dec 2009 13:10:54 +0000

Thanks for the donation!

Keeps me motivated to make more videos.

By: Xdreamer

Xdreamer — Sun, 13 Dec 2009 09:44:26 +0000

Hey Aaron,

just donated some bucks, not much but hey its a beginning Great tutorial, exactly what I was looking for for a loooooooooooong time now Covers pretty everything I always wanted to learn with ubot. Also thanks for sharing your technique with the subs, looks quite logical to me.

Hopefully there is more coming like this. Thanks again.

By: Ray

Ray — Sat, 12 Dec 2009 20:30:49 +0000

GREAT tutorial!

The software creator ought to post a link to this tutorial on their website.

None of the tutorials they have on their site covers this vital subject.

Thanks. I appreciate your work.

Ray