Latest Version on Packagist Total Downloads
Check the details at SharpAPI's Web Scraping API page.
- PHP >= 8.1
- Laravel >= 10.48.29
Follow these steps to install and set up the SharpAPI Laravel Web Scraping API package.
- Install the package via
composer:
composer require sharpapi/laravel-web-scraping-api
-
Register at SharpAPI.com to obtain your API key.
-
Set the API key in your
.envfile:
SHARP_API_KEY=your_api_key_here
- [OPTIONAL] Publish the configuration file:
php artisan vendor:publish --tag=sharpapi-web-scraping-api
- HTML Extraction: Extract the full HTML content of any webpage.
- Structured Data Extraction: Extract specific elements from webpages using CSS selectors.
- Screenshot Capture: Take screenshots of webpages.
- Link Extraction: Extract all links from a webpage.
- Image Extraction: Extract all images from a webpage.
- JavaScript Rendering: Properly render JavaScript-heavy websites.
- Proxy Support: Use proxies to avoid IP blocking.
You can inject the WebScrapingApiService class to access the web scraping functionality.
- Scrape a Webpage: Use
scrapeWebpageto get the full HTML content of a webpage. - Extract Structured Data: Use
extractStructuredDatato extract specific elements from a webpage using CSS selectors. - Take a Screenshot: Use
takeScreenshotto capture a screenshot of a webpage. - Extract Links: Use
extractLinksto extract all links from a webpage. - Extract Images: Use
extractImagesto extract all images from a webpage.
Here is an example of how to use WebScrapingApiService within a Laravel controller:
<?php
namespace App\Http\Controllers;
use GuzzleHttp\Exception\GuzzleException;
use SharpAPI\WebScrapingApi\WebScrapingApiService;
class ScrapingController extends Controller
{
protected WebScrapingApiService $scrapingService;
public function __construct(WebScrapingApiService $scrapingService)
{
$this->scrapingService = $scrapingService;
}
/**
* @throws GuzzleException
*/
public function scrapeWebpage(string $url)
{
$result = $this->scrapingService->scrapeWebpage($url);
return response()->json($result);
}
/**
* @throws GuzzleException
*/
public function extractData(string $url)
{
$selectors = [
'title' => 'h1',
'paragraphs' => 'p',
'links' => 'a',
];
$result = $this->scrapingService->extractStructuredData($url, $selectors);
return response()->json($result);
}
/**
* @throws GuzzleException
*/
public function takeScreenshot(string $url)
{
$options = [
'width' => 1280,
'height' => 800,
'fullPage' => true,
];
$result = $this->scrapingService->takeScreenshot($url, $options);
return response()->json($result);
}
/**
* @throws GuzzleException
*/
public function extractLinks(string $url)
{
$result = $this->scrapingService->extractLinks($url);
return response()->json($result);
}
/**
* @throws GuzzleException
*/
public function extractImages(string $url)
{
$result = $this->scrapingService->extractImages($url);
return response()->json($result);
}
}
All requests are managed by Guzzle, so it's helpful to be familiar with Guzzle Exceptions.
Example:
use GuzzleHttp\Exception\ClientException;
try {
$result = $this->scrapingService->scrapeWebpage('https://example.com');
} catch (ClientException $e) {
echo $e->getMessage();
}
You can customize the configuration by setting the following environment variables in your .env file:
SHARP_API_KEY=your_api_key_here
SHARP_API_BASE_URL=https://sharpapi.com/api/v1
{
"url": "https://sharpapi.com/",
"timestamp": "2025-03-15T08:56:04.946195Z",
"scraped_data": {
"title": "AI-powered Workflow Automation API",
"detected_language": "en",
"headers": {
"charset": "utf-8",
"contentType": null,
"viewport": ["width=device-width", "initial-scale=1"],
"canonical": "https://sharpapi.com/",
"csrfToken": "ju6SYrG2vXRDCBVCXYQ1RWH5dn1qxnFmHC7dqNc9"
},
"meta_tags": {
"author": null,
"image": null,
"keywords": ["SharpAPI", "AI", "automation", "API"],
"description": "Leverage AI API to streamline workflow in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more."
},
"open_graph": {
"og:title": "AI-powered Workflow Automation API",
"og:type": "website",
"og:URL": "https://sharpapi.com",
"og:image": "https://sharpapi.com/build/assets/sharpapi-website-preview-ARuIroBi.png",
"og:description": "Leverage AI API to streamline workflow in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more."
},
"twitter_card": {
"twitter:card": "summary",
"twitter:site": "@sharpapi",
"twitter:creator": "@a2zwebltd"
},
"content_structured": [
{
"tag": "h1",
"content": "Automate workflows with AI-powered API"
},
{
"tag": "h2",
"content": "Leverage AI API for automation in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more."
}
],
"links": {
"internal": ["https://sharpapi.com/register", "https://sharpapi.com/en/catalog"],
"external": ["https://github.com/sharpapi/", "https://chatgpt.com/g/g-4P3SFFu6Q-sharpapi-com-automate-with-ai"]
}
}
}
For issues or suggestions, please:
Please see CHANGELOG for a detailed list of changes.
- A2Z WEB LTD
- Dawid Makowski
- Enhance your Laravel AI capabilities!
The MIT License (MIT). Please see License File for more information.
Stay updated with news, tutorials, and case studies: