What You'll Learn
- • Setting up your development environment
- • Installing the required HTTP client
- • Authenticating with the ScrapeCreators API
- • Making requests to Twitter
- • Handling responses and errors
- • Best practices for production use
How to Scrape Twitter User tweets with PHP
Extract user-tweets data from Twitter
🐘 Using PHP
Overview
Learn how to scrape Twitter user tweets using PHP. This comprehensive guide will walk you through the entire process, from setup to implementation.
What You'll Get
- • Access to user tweets data
- • JSON formatted responses
- • Real-time data access
- • Scalable solution
- • Error handling patterns
- • Performance optimization tips
Prerequisites
1. API Key
First, you'll need a ScrapeCreators API key to authenticate your requests.
Sign up at app.scrapecreators.com to get your free API key with 100 requests.
2. Development Environment
Make sure you have the following installed:
- • PHP and its dependencies
- • A code editor (VS Code, Sublime, etc.)
- • Basic understanding of API requests
- • Command line interface access
Step 1: Install HTTP Client
Guzzle is a PHP HTTP client that makes it easy to send HTTP requests
composer
composer require guzzlehttp/guzzleStep 2: API Implementation
Now let's make a request to the Twitter API using PHP. Replace YOUR_API_KEY with your actual API key.
PHP
<?php
require_once 'vendor/autoload.php';
use GuzzleHttp\Client;
$apiKey = 'YOUR_API_KEY';
$client = new Client(['base_uri' => 'https://api.scrapecreators.com']);
function scrape() {
global $client, $apiKey;
try {
$response = $client->request('GET', '/v1/twitter/user-tweets', [
'headers' => [
'x-api-key' => $apiKey,
'Content-Type' => 'application/json'
],
'query' => [
'handle' => 'levelsio',
'trim' => 'false'
]
]);
$data = json_decode($response->getBody(), true);
echo 'Response: ';
print_r($data);
return $data;
} catch (Exception $e) {
echo 'Error: ' . $e->getMessage();
return null;
}
}
// Usage
$result = scrape();Step 3: Testing Your Code
API Parameters
This endpoint accepts the following parameters:
handleRequired(string)Twitter handle
Example: levelsio
trimOptional(boolean)Set to true for a trimmed down version of the response
Example: false
Run Your Code
Execute your script to test the API connection. You should see a JSON response with Twitter user tweets data.
✅ Success: You should receive a structured JSON response containing the requested data.
Expected Response
Here's an example of the JSON response you'll receive:
Sample Response
{
"tweets": [
{
"__typename": "Tweet",
"rest_id": "1828402665845322123",
"core": {
"user_results": {
"result": {
"__typename": "User",
"id": "VXNlcjo0NTIwMjQxMjA5",
"rest_id": "4520241209",
"affiliates_highlighted_label": {},
"is_blue_verified": true,
"profile_image_shape": "Circle",
"legacy": {
"created_at": "Fri Dec 18 02:48:59 +0000 2015",
"default_profile": false,
"default_profile_image": false,
"description": "Social Media Scraping API's: https://t.co/eSvJcfOZwF\n\nWeb Scraping Course: https://t.co/aTGlgWnvD7",
"entities": {
"description": {
"urls": [
{
"display_url": "scrapecreators.com",
"expanded_url": "https://scrapecreators.com/",
"url": "https://t.co/eSvJcfOZwF",
"indices": [
29,
52
]
},
{
"display_url": "theultimatewebscrapingcourse.com",
"expanded_url": "https://theultimatewebscrapingcourse.com/",
"url": "https://t.co/aTGlgWnvD7",
"indices": [
75,
98
]
}
]
},
"url": {
"urls": [
{
"display_url": "thewebscrapingguy.com",
"expanded_url": "https://thewebscrapingguy.com/",
"url": "https://t.co/gNUelkV9LA",
"indices": [
0,
23
]
}
]
}
},
"fast_followers_count": 0,
"favourites_count": 87301,
"followers_count": 17297,
"friends_count": 1139,
"has_custom_timelines": true,
"is_translator": false,
"listed_count": 162,
"location": "Austin, TX",
"media_count": 1152,
"name": "Adrian | The Web Scraping Guy",
"normal_followers_count": 17297,
"pinned_tweet_ids_str": [
"1628769691547074562"
],
"possibly_sensitive": false,
"profile_banner_url": "https://pbs.twimg.com/profile_banners/4520241209/1710267319",
"profile_image_url_https": "https://pbs.twimg.com/profile_images/1413647704161275904/1tTdl4v9_normal.jpg",
"profile_interstitial_type": "",
"screen_name": "adrian_horning_",
"statuses_count": 17751,
"translator_type": "none",
"url": "https://t.co/gNUelkV9LA",
"verified": false,
"withheld_in_countries": []
},
"professional": {
"rest_id": "1554172330263339015",
"professional_type": "Business",
"category": []
},
"tipjar_settings": {
"is_enabled": false,
"bandcamp_handle": "",
"bitcoin_handle": "",
"cash_app_handle": "",
"ethereum_handle": "",
"gofundme_handle": "",
"patreon_handle": "",
"pay_pal_handle": "",
"venmo_handle": ""
}
}
}
},
"unmention_data": {},
"edit_control": {
"edit_tweet_ids": [
"1828402665845322123"
],
"editable_until_msecs": "1724763740000",
"is_edit_eligible": false,
"edits_remaining": "5"
},
"is_translatable": false,
"views": {
"count": "493762",
"state": "EnabledWithCount"
},
"source": "<a href=\"https://tweethunter.io\" rel=\"nofollow\">Tweet Hunter Pro</a>",
"legacy": {
"bookmark_count": 1030,
"bookmarked": false,
"created_at": "Tue Aug 27 12:02:20 +0000 2024",
"conversation_id_str": "1828402665845322123",
"display_text_range": [
0,
216
],
"entities": {
"hashtags": [],
"media": [
{
"display_url": "pic.x.com/fh8xBzkrKI",
"expanded_url": "https://x.com/adrian_horning_/status/1828402665845322123/photo/1",
"id_str": "1828402662682841089",
"indices": [
217,
240
],
"media_key": "3_1828402662682841089",
"media_url_https": "https://pbs.twimg.com/media/GV_KiM-W0AEa31c.png",
"type": "photo",
"url": "https://t.co/fh8xBzkrKI",
"ext_media_availability": {
"status": "Available"
},
"features": {
"large": {
"faces": []
},
"medium": {
"faces": []
},
"small": {
"faces": []
},
"orig": {
"faces": []
}
},
"sizes": {
"large": {
"h": 117,
"w": 182,
"resize": "fit"
},
"medium": {
"h": 117,
"w": 182,
"resize": "fit"
},
"small": {
"h": 117,
"w": 182,
"resize": "fit"
},
"thumb": {
"h": 117,
"w": 117,
"resize": "crop"
}
},
"original_info": {
"height": 117,
"width": 182,
"focus_rects": [
{
"x": 0,
"y": 0,
"w": 182,
"h": 102
},
{
"x": 28,
"y": 0,
"w": 117,
"h": 117
},
{
"x": 35,
"y": 0,
"w": 103,
"h": 117
},
{
"x": 57,
"y": 0,
"w": 59,
"h": 117
},
{
"x": 0,
"y": 0,
"w": 182,
"h": 117
}
]
},
"media_results": {
"result": {
"media_key": "3_1828402662682841089"
}
}
}
],
"symbols": [],
"timestamps": [],
"urls": [],
"user_mentions": []
},
"extended_entities": {
"media": [
{
"display_url": "pic.x.com/fh8xBzkrKI",
"expanded_url": "https://x.com/adrian_horning_/status/1828402665845322123/photo/1",
"id_str": "1828402662682841089",
"indices": [
217,
240
],
"media_key": "3_1828402662682841089",
"media_url_https": "https://pbs.twimg.com/media/GV_KiM-W0AEa31c.png",
"type": "photo",
"url": "https://t.co/fh8xBzkrKI",
"ext_media_availability": {
"status": "Available"
},
"features": {
"large": {
"faces": []
},
"medium": {
"faces": []
},
"small": {
"faces": []
},
"orig": {
"faces": []
}
},
"sizes": {
"large": {
"h": 117,
"w": 182,
"resize": "fit"
},
"medium": {
"h": 117,
"w": 182,
"resize": "fit"
},
"small": {
"h": 117,
"w": 182,
"resize": "fit"
},
"thumb": {
"h": 117,
"w": 117,
"resize": "crop"
}
},
"original_info": {
"height": 117,
"width": 182,
"focus_rects": [
{
"x": 0,
"y": 0,
"w": 182,
"h": 102
},
{
"x": 28,
"y": 0,
"w": 117,
"h": 117
},
{
"x": 35,
"y": 0,
"w": 103,
"h": 117
},
{
"x": 57,
"y": 0,
"w": 59,
"h": 117
},
{
"x": 0,
"y": 0,
"w": 182,
"h": 117
}
]
},
"media_results": {
"result": {
"media_key": "3_1828402662682841089"
}
}
}
]
},
"favorite_count": 1722,
"favorited": false,
"full_text": "I just scraped 2.8 million companies from crunchbase 🤯\n\nName, website, semrush stats, etc.\n\nI'm giving the entire thing away in the next 24 hours\n\nComment \"crunchbase\" and I'll send it to you. Make sure DM's are open https://t.co/fh8xBzkrKI",
"is_quote_status": false,
"lang": "en",
"possibly_sensitive": false,
"possibly_sensitive_editable": true,
"quote_count": 12,
"reply_count": 3186,
"retweet_count": 64,
"retweeted": false,
"user_id_str": "4520241209",
"id_str": "1828402665845322123"
},
"quick_promote_eligibility": {
"eligibility": "IneligibleUserUnauthorized"
},
"url": "https://x.com/Austen/status/1935730646267158797"
}
]
}Verify Response Structure
Check that your response includes the expected fields:
- ✓
tweets(object)
Best Practices
1
Error Handling
Implement comprehensive error handling and retry logic for failed requests. Log errors properly for debugging.
2
Caching
Cache responses when possible to reduce API calls and improve performance. Consider data freshness requirements.
3
Security
Never expose your API key in client-side code. Use environment variables and secure key management practices.
Performance Tips
Batch Requests
When scraping multiple user tweets, consider batching requests to maximize throughput while staying within rate limits.
Async Processing
Use asynchronous processing in PHP to handle multiple requests concurrently and improve overall performance.
Common Use Cases
Market Research
Analyze Twitter user tweets to understand market trends, competitor analysis, and audience insights.
Content Analytics
Track performance metrics, engagement rates, and content trends across Twitter user tweets.
Lead Generation
Identify potential customers and business opportunities throughTwitter data analysis.
Troubleshooting
Common Errors
401 Unauthorized
Check your API key is correct and properly formatted in the x-api-key header.
402 Payment Required
You ran out of credits and need to buy more.
404 Not Found
The resource might not exist or be private.
500 Server Error
Temporary server issue. Implement retry logic with exponential backoff.
Need Help?
Frequently Asked Questions
How much does it cost to scrape Twitter user tweets?
ScrapeCreators offers 100 free API calls to get started. After that, pricing starts at $10 for 5k requests with volume discounts available.
Is it legal to scrape Twitter data?
Scraping publicly available data is fair game, and we only collect public data. So anything that you can see in an incognito browser is what we collect.
How fast can I scrape Twitter user tweets?
There is no rate limit! So you can scrape as fast as you want!
What data format does the API return?
All API responses are returned in JSON format, making it easy to integrate with any programming language or application.
Can I use this with other PHP frameworks?
Yes! This tutorial focuses on core PHP HTTP concepts that work with any framework. The API calls remain the same regardless of your specific PHP setup.
How do I handle large datasets?
For large datasets, implement pagination, use streaming responses where available, and consider storing data in a database for efficient querying.
Related Tutorials
Ready to Start Scraping?
Get started with 100 free API calls. No credit card required.