Learn how to scrape social media data using various programming languages and tools
Scrapes a public TikTok profile
Get the audience demographics of a TikTok user. Right now you can only get the audience countries. π¨ Costs 26 credits per request.
Scrapes videos from a TikTok profile. Pass cursor to get more videos.
Scrapes videos from a TikTok profile. Just pass handle and amount you want.
Scrapes data from a TikTok video
Scrapes transcript from a TikTok video
Scrapes a TikTok user's live stream
Scrapes comments from a TikTok video
Scrapes accounts that a TikTok user follows
Scrapes followers of a TikTok account
Scrapes TikTok users matching a search query
Scrapes TikTok videos by hashtag
Scrapes TikTok videos matching a keyword
There is a 'Top' Search in TikTok and this is that. You can get Photo Carousels and Videos. Whereas with just the Keyword Search, you can only get Videos.
This endpoint can take up to 30 seconds. Get popular songs from TikTok. If you want to check out the page this is actually scraping, check out: https://ads.tiktok.com/business/creativecenter/inspiration/popular/music/pc/en
Scraping details from the song detail
Get the TikToks using a song
Get the trending feed from TikTok.
Scrape TikTok Shop Products from a search! When I was testing it out, you can only get around 500 products at a time. NOTE: This costs more than 1 credit! Since we are doing the paginating for you, it costs 1 credit per page (TikTok returns 30 products per page). Also this endpoint takes a while and is new, so please be patient. Email me if you have any issues with it: adrian@thewebscrapingguy.com
Get the products from a TikTok Shop. NOTE: This costs more than 1 credit! Since we are doing the paginating for you, it costs 1 credit per page (TikTok returns 30 products per page). Also this endpoint takes a while and is new, so please be patient. Email me if you have any issues with it: adrian@thewebscrapingguy.com
Get the details of a TikTok Shop Product! Get the exact amount of stock the product has, related tiktok videos promoting the product, and more!
Gets public Instagram profile data, recent posts, and related accounts
Get a basic profile by user id. This endpoint is actually free right now also.
Get a public profile's public posts.
Get the public detailed information about a specific post or reel
Get the transcript of an Instagram post or reel. This is a little slow since I run it through an AI. You should get results in 10-30 seconds. If no one is spreaking it should return null. Since some posts are carousel, it will return a transcript for each item in the carousel.
Get comments from an Instagram post or reel. Note! that this costs more than 1 credit! This costs 1 credit per 15 comments. This won't return *all* the comments, but a good number of them.
Get all public reels from a profile. Can provide a user_id or handle, but for faster response times, use user_id. This won't include pinned reels right now. And I'm not sure why, but it looks like IG doesn't return the description of the reel on this endpoint :( You'll have to use the post detail endpoint to get that.
Get the public reels from a user's profile. Can provide a user_id or handle, but for faster response times, use user_id. This won't include pinned reels right now. And I'm not sure why, but it looks like IG doesn't return the description of the reel on this endpoint :( You'll have to use the post detail endpoint to get that.
Get story highlights from an Instagram user
Get details about a specific highlight
Get reels using a song
Get the HTML embed for a user's profile
Get comprehensive channel information including stats and metadata. Can pass channelId, handle, or url.
Get all videos from a channel with detailed information. Can pass channelId or handle.
Get the shorts from a channel. If you need more details about the short like description, publish date, etc, you'll need to use the 'Video/Short Details' endpoint.
Convenience endpoint to get the latest shorts from a channel. We handle the pagination for you. This will cost you more credits because under the hood we're using the 'Channel Shorts' endpoint, just like you would. But making it easier for you. If you need more details about the short like description, publish date, etc, you'll need to use the 'Video/Short Details' endpoint.
Get complete information about a video or short including the transcript! There is a 'transcript' field and a 'transcript_only_text' field.
Get transcript of a video or short
Search YouTube and get matching videos, channels, playlists, shorts, lives, etc. Video explaining the response format: https://www.tella.tv/video/explaining-youtube-search-results-payload-353a
Search YouTube and get matching videos, channels, and playlists
Get comments from a video. Can only get 1k top comments and about 7k new comments.
Get about 48 trending shorts
Get a person's public profile (including recent posts) *Note this is for a persons public profile. This only returns what's publicly available, ie what you see in an incognito browser. So unfortunately LinkedIn doesn't return work history or job title publicly anymore :(
Get a LinkedIn company page
Get a Linkedin post. This can be a post or an article. Thinking about separating posts and articles, let me know if you think I should do that: adrian@thewebscrapingguy.com
Get public Facebook profile information
Get public Facebook profile posts. This only returns the posts you would see from an incognito browser session. You can pass a url or a pageId. pageId is faster. pageId is the id of the profile page. Only returns 3 posts at a time.
Get a public Facebook groups posts. Only can get 3 posts at a time :( (Facebook API limit)
Get a public Facebook post or reel by url.
Get the transcript of a Facebook post. Can be a post or reel.
Get the comments of a Facebook post (or reel)
Pass the Ad ID and get back details about the ad. Be careful that if an ad has multiple versions, you're actually going to want to get the title from the 'cards' object.
Search the Facebook (Meta) Ad Library By Keyword. This endpoint will tap out around 1,500 results, beacuse the cursor becomes too big for a GET request. If you need more results, call this endpoint as a POST, and pass the query params in the body.
Get all the ads a company has running
Search for companies by name and get their ad library page id
Get the ads for a company. *This only gets the public ads. Some ads you need to log in for and sadly we can't get those. This might be a bit slow since we are OCRing all the ads to get the text.
Get the details for an ad. I'm using an OCR to get the text from the ad, so it might not be 100% accurate. If you find any errors or irregularities, please email me: adrian@thewebscrapingguy.com
Search the Google Ad Transparency Library for advertisers to get their advertiser id
Search the LinkedIn Ad Library
Get the details for an ad
Get Twitter profile information including stats and metadata
Get tweets from a user's profile. *These aren't the users latest tweets. Sadly, Twitter publicly only returns 100 of the users most popular tweets.
Get detailed information about a specific tweet including engagement
Get the transcript of a video tweet. Unfortunately this is a little slow because I'm using AI to get the transcript
Get the details of a Twitter(X) Community
Get the tweets from a Twitter(X) Community
Get recent posts from a subreddit with engagement metrics
Get comments from a Reddit post
Convenience API to get the number of comments you wantfrom a Reddit post.
Search Reddit for posts
Search the Reddit Ad Library. Can only get a max of 30 ads.
Get a specific ad by id
Get Truth Social profile info
As of 8/27/2025, Truth Social is only letting you view the public profile/posts of prominent users (like Trump and Vance), but requiring auth for everyone else. So sadly most other accounts will not work :(
Get Truth Social post
Get Threads profile info
Get Threads posts by user. Sadly, Threads only allows you to see the last 20-30ish posts of the user :(
Get a post by url. Also returns comments and related posts.
Search for posts by keyword. Sadly Threads only returns 20-30 results at a time.
Search for users by username
Get Bluesky profile info
Get Bluesky user posts. Use user_id for faster response times.
Get Bluesky post
Search Pinterest
Get Pinterest pin
Get a user's boards
Get the pins of a pinterest board
Search Google
Get Twitch profile
Get Twitch clip
Get Kick clip
Get Snapchat user profile
Linktree page
Komi page
Pillar page
Linkbio (lnk.bio) page
Get products and other details from a creator's Amazon Shop page
Get age and gender of creator. This uses AI to analyze the profile image. *The profile photo must have a clear face to get an accurate result.*
How many credits you have left