Scraping Tutorials
Learn how to scrape social media data using various programming languages and tools
TikTok Tutorials
Profile
Scrapes a public TikTok profile
User's Audience Demographics
Get the audience demographics of a TikTok user. Right now you can only get the audience countries. 🚨 Costs 26 credits per request.
Profile Videos
Scrapes videos from a TikTok profile. Pass cursor to get more videos.
Profile Videos (We handle pagination)
Scrapes videos from a TikTok profile. Just pass handle and amount you want.
Video Info
Scrapes data from a TikTok video
Transcript
Scrapes transcript from a TikTok video
Is a TikTok Shop?
Checks if a TikTok user has a shop
TikTok Live
Scrapes a TikTok user's live stream
Comments
Scrapes comments from a TikTok video
Following
Scrapes accounts that a TikTok user follows
Followers
Scrapes followers of a TikTok account
Search Users
Scrapes TikTok users matching a search query
Search by Hashtag
Scrapes TikTok videos by hashtag
Search by Keyword
Scrapes TikTok videos matching a keyword
Get popular songs
This endpoint can take up to 30 seconds. Get popular songs from TikTok. If you want to check out the page this is actually scraping, check out: https://ads.tiktok.com/business/creativecenter/inspiration/popular/music/pc/en
Get Song Details
Scraping details from the song detail
Instagram Tutorials
Profile
Gets public Instagram profile data, recent posts, and related accounts
Posts
Get a public profile's public posts.
Post/Reel Info
Get the public detailed information about a specific post or reel
Transcript
Get the transcript of an Instagram post or reel. This is a little slow since I run it through an AI. You should get results in 10-30 seconds. If no one is spreaking it should return null. Since some posts are carousel, it will return a transcript for each item in the carousel.
Reels
Get all public reels from a profile. Can provide a user_id or handle, but for faster response times, use user_id. This won't include pinned reels right now. And I'm not sure why, but it looks like IG doesn't return the description of the reel on this endpoint :( You'll have to use the post detail endpoint to get that.
Reels (We handle pagination)
Get the public reels from a user's profile. Can provide a user_id or handle, but for faster response times, use user_id. This won't include pinned reels right now. And I'm not sure why, but it looks like IG doesn't return the description of the reel on this endpoint :( You'll have to use the post detail endpoint to get that.
Story Highlights
Get story highlights from an Instagram user
Highlights Details
Get details about a specific highlight
Reels using Song
Get reels using a song
YouTube Tutorials
Channel Details
Get comprehensive channel information including stats and metadata. Can pass channelId, handle, or url.
Channel Videos
Get all videos from a channel with detailed information. Can pass channelId or handle.
Channel Shorts
Get the shorts from a channel. If you need more details about the short like description, publish date, etc, you'll need to use the 'Video/Short Details' endpoint.
Channel Shorts (we handle the pagination)
Convenience endpoint to get the latest shorts from a channel. We handle the pagination for you. This will cost you more credits because under the hood we're using the 'Channel Shorts' endpoint, just like you would. But making it easier for you. If you need more details about the short like description, publish date, etc, you'll need to use the 'Video/Short Details' endpoint.
Video/Short Details
Get complete information about a video or short including the transcript! There is a 'transcript' field and a 'transcript_only_text' field.
Transcript
Get transcript of a video or short
Search
Search YouTube and get matching videos, channels, playlists, shorts, lives, etc. Video explaining the response format: https://www.tella.tv/video/explaining-youtube-search-results-payload-353a
Search by Hashtag
Search YouTube and get matching videos, channels, and playlists
LinkedIn Tutorials
Person's Profile
Get a person's public profile (including recent posts) *Note this is for a persons public profile. This only returns what's publicly available, ie what you see in an incognito browser. So unfortunately LinkedIn doesn't return work history or job title publicly anymore :(
Company Page
Get a LinkedIn company page
Facebook Tutorials
Profile
Get public Facebook profile information
Profile Posts
Get public Facebook profile posts. This only returns the posts you would see from an incognito browser session. You can pass a url or a pageId. pageId is faster. pageId is the id of the profile page. Only returns 3 posts at a time.
Facebook Group Posts
Get a public Facebook groups posts. Only can get 3 posts at a time :( (Facebook API limit)
Reel
Get public reel details
Post
Get the post details from Facebook. Not much on the response payload right now. Just let me know what you want to see added: adrian@thewebscrapinguy.com
Facebook Ad Library Tutorials
Ad Details
Pass the Ad ID and get back details about the ad. Be careful that if an ad has multiple versions, you're actually going to want to get the title from the 'cards' object.
Search
Search the Facebook (Meta) Ad Library By Keyword. This endpoint will tap out around 1,500 results, beacuse the cursor becomes too big for a GET request. If you need more results, I can give you a POST endpoint.
Company Ads
Get all the ads a company has running
Google Ad Library Tutorials
Company Ads
Get the ads for a company. *This only gets the public ads. Some ads you need to log in for and sadly we can't get those.
Ad Details
Get the details for an ad. Since there are so many variations, the return types might not all be 100% the same, email me for suggestions. Also this endpoint is pretty new, if you find any errors or irregularities, please email me: adrian@thewebscrapingguy.com
LinkedIn Ad Library Tutorials
Twitter Tutorials
Profile
Get Twitter profile information including stats and metadata
User Tweets
Get tweets from a user's profile. *These aren't the users latest tweets. Sadly, Twitter publicly only returns 100 of the users most popular tweets.
Tweet Details
Get detailed information about a specific tweet including engagement
Reddit Tutorials
Subreddit Posts
Get recent posts from a subreddit with engagement metrics
Post Comments
Get comments from a Reddit post
Simple Comments
Convenience API to get the number of comments you wantfrom a Reddit post.
Truth Social Tutorials
Profile
Get Truth Social profile info
User Posts
Get public Truth Social posts by user
Post
Get Truth Social post
Webhook
Webhook to immediately get notified when Trump posts something. Email me to get this setup: adrian@thewebscrapingguy.com. The sample response is the response we'll send to your webhook_url. Make sure your webhook accepts a POST request.
Threads Tutorials
Profile
Get Threads profile info
Posts
Get Threads posts by user. Sadly, Threads only allows you to see the last 20-30ish posts of the user :(
Post
Get a post by url. Also returns comments and related posts.
Search by Keyword
Search for posts by keyword. Sadly Threads only returns 20-30 results at a time.
Bluesky Tutorials
Profile
Get Bluesky profile info
Posts
Get Bluesky user posts. Use user_id for faster response times.
Pinterest Tutorials
Search
Search Pinterest
Pin
Get Pinterest pin
User Boards
Get a user's boards
Google Tutorials
Twitch Tutorials
Kick Tutorials
Snapchat Tutorials
Age and Gender Tutorials
