This is the Django based scraper for the CRS reports.
pyenv activate flatgov # Assumes Python >=3.7 pip install requirements.txt
Collecting the data
Two models are involved:
CrsReport. The table
Bills assumed to exist and have a bills data.
To scrape the data to CRS table enter a Django console and do:
python manage.py shell
from crs.populate_crs_table import CrsFromApi crs_api = CrsFromApi() crs_api.populate()
Then, when running the application, to download a CSV report of the data, go to
/crs/csv-report/. This may take a long time, and can be a large file to download from a remote server.