最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

javascript - Node.js Twitter API cursors - Stack Overflow

programmeradmin3浏览0评论

I'm using npm-twit to get followers of a specific account.

The Twitter API returns up to 5000 results from a single GET request.

If the user I'm querying has over 5000 followers a "next_cursor" value is returned with the data.

To get the next 5000 results, I need to re-run the GET function, passing it the "next_cursor" value as an argument. I just can't seem to work out how to do it.

I was thinking a while loop, but I can't reset the global variable, I think because of scope:

var cursor = -1

while ( cursor != 0 ) { 

  T.get('followers/ids', { screen_name: 'twitter' },  function (err, data, response) {

  // Do stuff here to write data to a file

  cursor = data["next_cursor"];

  })

}

Obviously I'm not a JS genius, so any help would be much appreciated.

I'm using npm-twit to get followers of a specific account.

The Twitter API returns up to 5000 results from a single GET request.

If the user I'm querying has over 5000 followers a "next_cursor" value is returned with the data.

To get the next 5000 results, I need to re-run the GET function, passing it the "next_cursor" value as an argument. I just can't seem to work out how to do it.

I was thinking a while loop, but I can't reset the global variable, I think because of scope:

var cursor = -1

while ( cursor != 0 ) { 

  T.get('followers/ids', { screen_name: 'twitter' },  function (err, data, response) {

  // Do stuff here to write data to a file

  cursor = data["next_cursor"];

  })

}

Obviously I'm not a JS genius, so any help would be much appreciated.

Share Improve this question asked Jan 18, 2015 at 10:56 user1385827user1385827 3351 gold badge7 silver badges23 bronze badges
Add a ment  | 

3 Answers 3

Reset to default 8

The issue you are having is due to Node.js being asynchronous.

T.get('followers/ids', { screen_name: 'twitter' },  function getData(err, data, response) {

  // Do stuff here to write data to a file

  if(data['next_cursor'] > 0) T.get('followers/ids', { screen_name: 'twitter', next_cursor: data['next_cursor'] }, getData);

  })

}

Please note:

  1. I gave a name to the internal callback function. That is so that we can recursively call it from the inside.
  2. The loop is replaced with a recursive callback.
  3. If there is a next_cursor data, then we call T.get using the same function getData.

Be aware that Do stuff here code will be executed many times (as many as there are next cursors). Since it is recursive callback - the order is guaranteed.


If you do not like the idea of recursive callbacks, you can avoid it by:

  1. Finding out beforehand all the next_cursor's if possible, and generate requests using for loop.
  2. Alternatively, use asynchronous-helper modules like Async (though for learning purposes, I would avoid modules unless you are fluent in the concept already).

Consider testing with some 5K+ account.

    const T = new Twit(tokens)

    function getFollowers (screenName, followers = [], cur = -1) {
      return new Promise((resolve, reject) => {
        T.get('followers/ids', { screen_name: screenName, cursor: cur, count: 5000 }, (err, data, response) => {
          if (err) {
            cur = -1
            reject(err)
          } else {
            cur = data.next_cursor
            followers.push(data.ids)
            if (cur > 0) {
              return resolve(getFollowers(screenName, followers, cur))
            } else {
              return resolve([].concat(...followers))
            }
          }
        })
      })
    }

    async function getXaqron () {
      let result = await getFollowers('xaqron')
      return result
    }

 console.log(getXaqron().catch((err) => {
  console.log(err) // Rate limit exceeded
}))

Struggled with this one.. Everything seemed to work, but data['next_cursor'] didn't change, EVER!

Code should be like this:

T.get('followers/ids', { screen_name: 'twitter' },  function getData(err, data, response) {

  // Do stuff here to write data to a file

  if(data['next_cursor'] > 0) T.get('followers/ids', { screen_name: 'twitter', cursor: data['next_cursor'] }, getData);

  })

}

Parameter for Twit isn't "next_cursor", it's just "cursor" ;)

发布评论

评论列表(0)

  1. 暂无评论