How to add a url to scrap?

[tttp]

Hi,

I'm scrapping a page, that contains links to other pages I'd like to scrap too (that have the next format, eg for paginated results).

Was wondering if there is a way to inject more urls to parse in the current scrapper?

X+

P.S. jQuery and node.js is by far the more pleasant way to parse (still feel the pain of regex and other string operations ;)

[chltjdgh86]

I tried to do the same asynchronously, but did not work for me.

[tttp]

Hi,

Ended up writing code based on async. Worked fine... until I upgraded node. Will check this week-end and report

X+

[chltjdgh86]

Hey,

What I ended up doing is using jQuery & Jsdom as secondary scraper inside scraper function. This worked with the latest version of node, I guess you can use this for now. Refer to this article here:

http://blog.nodejitsu.com/jsdom-jquery-in-5-lines-on-nodejs