Subject
4 entries
Web Scraping
Bookmarks
Web Scraping Open Project
A GitHub repository collecting open knowledge about web scraping in Python — covering tools, techniques, anti-scraping countermeasures, and best practices. Community-built reference for practitioners building data collection pipelines.
Scrapism: Web Scraping as Art and Activism
Scrapism is Sam Lavigne's guide to web scraping as artistic and activist practice — using Python scraping tools to collect and repurpose web data for critical or creative ends. Treats scraping as a form of political speech as much as technical skill.
scrapy-history-middleware: S3 Historical Cache for Scrapy
A Scrapy middleware that stores every crawled response in S3, building a historical archive of web resources over time. Enables point-in-time replay of crawls and separates the concerns of fetching from processing.
Announcing Portia — Open Source Visual Web Scraper
Scrapinghub's Portia is an open-source visual web scraper — a point-and-click interface on top of Scrapy that lets you build scrapers without writing code. It made web scraping accessible to non-programmers while keeping Scrapy's power underneath.