javascript google-chrome-devtools devtools

Collect the full list of buttons to follow without having to scroll the page (DevTools Google Chrome)

DevTools Google Chrome:

On this site (https://booyah.live/users/41874362/followers), to load the complete list of followers it is necessary to keep scrolling down the page to reload more profiles, but there comes a time when the page weighs so much that the browser crashes and it ends up needing to be closed.

Is there any way to be able to collect the follow buttons without this happening?

The current script I use is:

setInterval(function(){
document.getElementById("layout-content").scrollTo(0, 50000000000000000000000000000000000000);

document.querySelectorAll('.components-button.components-button-size-mini.components-button-type-orange.desktop.components-button-inline').forEach(btn => btn.click());
}, 10)

I use setInterval to create a loop of:

1 - Scrolling the page
2 - Loading more profiles
3 - Clicking the follow buttons

My need:

For the study I'm doing for learning, the idea is that my profile follows all profiles followers of a single most famous profile in order to analyze how many people follow back on this social media.

Additional:

In this answer provided by Leftium, it is possible to follow only one profile:

https://stackoverflow.com/a/67882688/11462274

In this answer given by KCGD, it is possible to collect the entire list of followers but during this collection the profiles are not followed, it is possible to create a list and save the data, but not follow the profiles:

https://stackoverflow.com/a/67865968/11462274

I tried to contact them both, but they haven't returned yet. It was a good way but I couldn't combine the two answers so I can follow all the profiles, I thought about the possibility according to which I would collect the profiles of the KCGD response, I would follow the profiles too, but not only the first one but also the answer of the Leftium.

Would it be possible to take advantage of the loop created by the response from KCGD and from each response, already follow all profiles instead of just the first one as in Leftium's response?

I tried to create but was unsuccessful.

Solution

This is a fully working solution that I have tested in my own Chrome browser with a fresh account, successfully following all the follower accounts of the account you are targeting.

UPDATE (2021-06-18)

I've updated my solution to a drastically improved and faster function, rewritten with async/await. This new function reduces the estimated runtime from ~45min to ~10min. 10min is still a long while, but that's to be expected considering the large number of followers the user you are targeting has.

After a few iterations, the latest function not only improves speed, performance, and error reporting, but it also extends what is possible with the function. I provide several example below my solutions of how to use the function completely.

For the sake of de-cluttering my answer, I am removing my older function from this solution altogether, but you can still reference it in my solution's edit history if you like.

TL;DR

Here is the final, fastest, working solution. Make sure to replace PUT_YOUR_CSRF_TOKEN_HERE with your own CSRF token value. Detailed instructions on how to find your CSRF token are below.

You must run this in your console on the Booyah website in order to avoid CORS issues.

const csrf = 'PUT_YOUR_CSRF_TOKEN_HERE';
async function booyahGetAccounts(uid, type = 'followers', follow = 1) {
    if (typeof uid !== 'undefined' && !isNaN(uid)) {
        const loggedInUserID = window.localStorage?.loggedUID;
        if (uid === 0) uid = loggedInUserID;
        const unfollow = follow === -1;
        if (unfollow) follow = 1;
        if (loggedInUserID) {
            if (csrf) {
                async function getUserData(uid) {
                    const response = await fetch(`https://booyah.live/api/v3/users/${uid}`),
                          data = await response.json();
                    return data.user;
                }
                const loggedInUserData = await getUserData(loggedInUserID),
                      targetUserData = await getUserData(uid),
                      followUser = uid => fetch(`https://booyah.live/api/v3/users/${loggedInUserID}/followings`, { method: (unfollow ? 'DELETE' : 'POST'), headers: { 'X-CSRF-Token': csrf }, body: JSON.stringify({ followee_uid: uid, source: 43 }) }),
                      logSep = (data = '', usePad = 0) => typeof data === 'string' && usePad ? console.log((data ? data + ' ' : '').padEnd(50, '━')) : console.log('━'.repeat(50),data,'━'.repeat(50));
                async function getList(uid, type, follow) {
                    const isLoggedInUser = uid === loggedInUserID;
                    if (isLoggedInUser && follow && !unfollow && type === 'followings') {
                        follow = 0;
                        console.warn('You alredy follow your followings. `follow` mode switched to `false`. Followings will be retrieved instead of followed.');
                    }
                    const userData = await getUserData(uid),
                          totalCount = userData[type.slice(0,-1)+'_count'] || 0,
                          totalCountStrLength = totalCount.toString().length;
                    if (totalCount) {
                        let userIDsLength = 0;
                        const userIDs = [],
                              nickname = userData.nickname,
                              nicknameStr = `${nickname ? ` of ${nickname}'s ${type}` : ''}`,
                              alreadyFollowedStr = uid => `User ID ${uid} already followed by ${loggedInUserData.nickname} (Account #${loggedInUserID})`;
                        async function followerFetch(cursor = 0) {
                            const fetched = [];
                            await fetch(`https://booyah.live/api/v3/users/${uid}/${type}?cursor=${cursor}&count=100`).then(res => res.json()).then(data => {
                                const list = data[type.slice(0,-1)+'_list'];
                                if (list?.length) fetched.push(...list.map(e => e.uid));
                                if (fetched.length) {
                                    userIDs.push(...fetched);
                                    userIDsLength += fetched.length;
                                    if (follow) followUser(uid);
                                    console.log(`${userIDsLength.toString().padStart(totalCountStrLength)} (${(userIDsLength / totalCount * 100).toFixed(4)}%)${nicknameStr} ${follow ? 'followed' : 'retrieved'}`);
                                    if (fetched.length === 100) {
                                        followerFetch(data.cursor);
                                    } else {
                                        console.log(`END REACHED. ${userIDsLength} accounts ${follow ? 'followed' : 'retrieved'}.`);
                                        if (!follow) logSep(targetList);
                                    }
                                }
                            });
                        }
                        await followerFetch();
                        return userIDs;
                    } else {
                        console.log(`This account has no ${type}.`);
                    }
                }
                logSep(`${follow ? 'Following' : 'Retrieving'} ${targetUserData.nickname}'s ${type}`, 1);
                const targetList = await getList(uid, type, follow);
            } else {
                console.error('Missing CSRF token. Retrieve your CSRF token from the Network tab in your inspector by clicking into the Network tab item named "bug-report-claims" and then scrolling down in the associated details window to where you see "x-csrf-token". Copy its value and store it into a variable named "csrf" which this function will reference when you execute it.');
            }
        } else {
            console.error('You do not appear to be logged in. Please log in and try again.');
        }
    } else {
        console.error('UID not passed. Pass the UID of the profile you are targeting to this function.');
    }
}
booyahGetAccounts(41874362);

Detailed explanation of the process

As the function runs, it logs the progress to the console, both how many users have been followed so far, and how much progress has been made percentage-wise, based on the total number of followers the profile you are targeting has.

Retrieving your CSRF token

The only manual portion of this process is retrieving your CSRF token. This is rather simple though. Once you log into Booyah, navigate to the Network tab of your Chrome console and click on the item named bug-report-claims, then scroll all the way down the details window which appears on the right. There should see x-csrf-token. Store this value as a string variable in your console as csrf, which my function will reference when it runs. This is necessary in order to use the POST method to follow users.

Here is what it will look like:

The solution

The function will loop through all users the account you are targeting follows in batches of 100 (the max amount allowed per GET request) and follow them all. When the end of each batch is met, the next batch is automatically triggered recursively.

🚀 Version 3 (Fastest and most flexible, using `async`/`await` and `fetch()`)

My previous two solution versions (🐇 …🐢) can be referenced in this answer's edit history.

Make sure to replace PUT_YOUR_CSRF_TOKEN_HERE with your own CSRF token value. Detailed instructions on how to find your CSRF token are below.

You must run this in your console on the Booyah website in order to avoid CORS issues.

const csrf = 'PUT_YOUR_CSRF_TOKEN_HERE';
async function booyahGetAccounts(uid, type = 'followers', follow = 1) {
    if (typeof uid !== 'undefined' && !isNaN(uid)) {
        const loggedInUserID = window.localStorage?.loggedUID;
        if (uid === 0) uid = loggedInUserID;
        const unfollow = follow === -1;
        if (unfollow) follow = 1;
        if (loggedInUserID) {
            if (csrf) {
                async function getUserData(uid) {
                    const response = await fetch(`https://booyah.live/api/v3/users/${uid}`),
                          data = await response.json();
                    return data.user;
                }
                const loggedInUserData = await getUserData(loggedInUserID),
                      targetUserData = await getUserData(uid),
                      followUser = uid => fetch(`https://booyah.live/api/v3/users/${loggedInUserID}/followings`, { method: (unfollow ? 'DELETE' : 'POST'), headers: { 'X-CSRF-Token': csrf }, body: JSON.stringify({ followee_uid: uid, source: 43 }) }),
                      logSep = (data = '', usePad = 0) => typeof data === 'string' && usePad ? console.log((data ? data + ' ' : '').padEnd(50, '━')) : console.log('━'.repeat(50),data,'━'.repeat(50));
                async function getList(uid, type, follow) {
                    const isLoggedInUser = uid === loggedInUserID;
                    if (isLoggedInUser && follow && !unfollow && type === 'followings') {
                        follow = 0;
                        console.warn('You alredy follow your followings. `follow` mode switched to `false`. Followings will be retrieved instead of followed.');
                    }
                    const userData = await getUserData(uid),
                          totalCount = userData[type.slice(0,-1)+'_count'] || 0,
                          totalCountStrLength = totalCount.toString().length;
                    if (totalCount) {
                        let userIDsLength = 0;
                        const userIDs = [],
                              nickname = userData.nickname,
                              nicknameStr = `${nickname ? ` of ${nickname}'s ${type}` : ''}`,
                              alreadyFollowedStr = uid => `User ID ${uid} already followed by ${loggedInUserData.nickname} (Account #${loggedInUserID})`;
                        async function followerFetch(cursor = 0) {
                            const fetched = [];
                            await fetch(`https://booyah.live/api/v3/users/${uid}/${type}?cursor=${cursor}&count=100`).then(res => res.json()).then(data => {
                                const list = data[type.slice(0,-1)+'_list'];
                                if (list?.length) fetched.push(...list.map(e => e.uid));
                                if (fetched.length) {
                                    userIDs.push(...fetched);
                                    userIDsLength += fetched.length;
                                    if (follow) followUser(uid);
                                    console.log(`${userIDsLength.toString().padStart(totalCountStrLength)} (${(userIDsLength / totalCount * 100).toFixed(4)}%)${nicknameStr} ${follow ? 'followed' : 'retrieved'}`);
                                    if (fetched.length === 100) {
                                        followerFetch(data.cursor);
                                    } else {
                                        console.log(`END REACHED. ${userIDsLength} accounts ${follow ? 'followed' : 'retrieved'}.`);
                                        if (!follow) logSep(targetList);
                                    }
                                }
                            });
                        }
                        await followerFetch();
                        return userIDs;
                    } else {
                        console.log(`This account has no ${type}.`);
                    }
                }
                logSep(`${follow ? 'Following' : 'Retrieving'} ${targetUserData.nickname}'s ${type}`, 1);
                const targetList = await getList(uid, type, follow);
            } else {
                console.error('Missing CSRF token. Retrieve your CSRF token from the Network tab in your inspector by clicking into the Network tab item named "bug-report-claims" and then scrolling down in the associated details window to where you see "x-csrf-token". Copy its value and store it into a variable named "csrf" which this function will reference when you execute it.');
            }
        } else {
            console.error('You do not appear to be logged in. Please log in and try again.');
        }
    } else {
        console.error('UID not passed. Pass the UID of the profile you are targeting to this function.');
    }
}

Usage

To run the function (for either of the above solutions), just call the function name with the desired User ID name as an argument, in your example case, 41874362. The function call would look like this:

booyahGetAccounts(41874362);

The function is quite flexible in its abilities though. booyahGetAccounts() accepts three parameters, but only the first is required.

booyahGetAccounts(
    uid, // required, no default
    type = 'followers', // optional, must be 'followers' or 'followings' -> default: 'followers'
    follow = 1 // optional, must be 0, 1, or -1, -> default: 1 (boolean true)
)

The second parameter, type, allows you to choose whether you would like to process the targeted user's followers or followings (the users which that user follows).

The third parameter allows you to choose whether you would like to follow/unfollow the returned users or only retrieve their User IDs. This defaults to 1 (boolean true) which will follow the users returned, but if you only want to test the function and not actually follow the returned users, set this to a falsy value such as 0 or false. Using -1 will unfollow the users returned.

This function intelligently retrieves your own User ID for you from the window.localStorage object, so you don't need to retrieve that yourself. If you would like to process your own followers or followings, simply pass 0 as the main uid parameter value, and the function will default the uid to your own User ID.

Because you can't re-follow users you already follow, if you try to follow your followings, the function will produce the warning You already follow your followings. 'follow' mode switched to 'false'. Followings will be retrieved instead of followed. and instead return them as if you had set the follow parameter to false.

However, it can be very useful to process your own list. For example, if you want to follow all of your own followers back, you could do so like this:

booyahGetAccounts(0); // `type` and `follow` parameters already default to the correct values here

On the other hand, if you were strategically using a follow/unfollow technique in order to increase your number of followers and needed to unfollow all of your followers, you could do so like this:

booyahGetAccounts(0, 'followers', -1);

By setting the follow parameter value to -1, you instruct the function to run its followUser function on all returned User IDs using the DELETE method instead of the POST method, thereby unfollowing those users returned instead of following them.

Desired outcome	Function call
Follow all your own followers	`booyahGetAccounts(0, 'followers');`
Unfollow all your own followers	`booyahGetAccounts(0, 'followers', -1);`
Unfollow all your own followings	`booyahGetAccounts(0, 'followings', -1);`
Follow users that follow User ID #12345	`booyahGetAccounts(12345, 'followers');`
Follow users followed by User ID #12345	`booyahGetAccounts(12345, 'followings');`
Retrieve User IDs of accounts following User ID #12345	`booyahGetAccounts(12345, 'followers', 0);`
Retrieve User IDs of accounts followed by User ID #12345	`booyahGetAccounts(12345, 'followings', 0);`

Other notes

To improve the performance of this function, as it's very heavy, I've replaced all calls to userIDs.length with a dedicated userIDsLength variable which I add to using += with each iteration rather than calling length each time. Similarly, I store the length of the stringified followerCount in the variable followerCountStrLength rather than calling followerCount.toString().length with each iteration. Because this is a rather heavy function, it is possible for your browser window to crash. However, it should eventually complete.
If the page appears to crash by flickering and auto-closing the console, FIRST try to re-open the console without refreshing the page at all. In my case, the inspector occasionally closed on its own, likely due to the exhaustion from the function, but when I opened the inspector's console again, the function was still running.