Open Civic Data
latest
Writing Scrapers
Getting Started Writing Scrapers
Creating a New Scraper
Writing a Person Scraper
Writing an Events Scraper
Writing a Bill Scraper
Running the Scraper
Submitting a Pull Request
Common tips for writing scrapers
Open Civic Data Formats
Style Guidelines
Open Civic Data Enhancement Proposals
Open Civic Data
»
Writing Scrapers
Edit on GitHub
Writing Scrapers
¶
Getting Started Writing Scrapers
Python
Understanding GitHub
Scraping Basics
Getting Started
Creating a New Scraper
Writing a Person Scraper
Special notes about People scrapers
Target Data
Creating a New Person scraper
Committees and Memberships
Scraper Example
Special notes regarding Posts, Memberships and Districts
Writing an Events Scraper
Target Data
Creating a new Events scraper
Writing a Bill Scraper
Target Data
Overview
Scraping Votes
Running the Scraper
Validating Data
Submitting a Pull Request
Fork the repo you want to contribute to
Submit a pull request
Best Practice
Keeping your branch up to date
Checking what you’ve changed
Common tips for writing scrapers
Fetching a page and setting URLs to absolute paths
Getting the current session
Common XPath tricks
Writing “defensive” scrapers