Dumps
Parltrack publishes complete dumps of its database on a daily basis. These dumps are in JSON format, and they are compressed with the lzip[1] toolzstd tool (since with 2024-11-11).
Due to most of the dumps being between 400 and 800 megabytes (at the time of writing in mid 2019) they might not be suitable to load all at once since when loaded into RAM they might use significantly more memory. To facilitate a record-by-record stream processing of these dumps, they are formatted in the following way, each line is one record, each prefixed either with:
- '[' for the first record,
- ',' for the other records,
- ']' on its own for the last line
All data is available under the Open Database License v1.0 (ODBL) license.
| Table | Description | Dump | Size | Last Updated | Schema | Previous dumps |
|---|---|---|---|---|---|---|
| Scraper Log | This contains all the logs from the last scraping. | 2024-12-24.log.zst (summary) | 230.8 KiB | 2024-12-24 | ||
| MEPs | This dump contains all the basic information about the MEPs | ep_meps.json.zst | 7.4 MiB | 2024-12-24 | ||
| Dossiers | This dump contains all the basic information about the dossiers in the EP | ep_dossiers.json.zst | 51.6 MiB | 2024-12-24 | ||
| MEP Activities | This dump contains most of the activities of MEPs (such as plenary speeches, questions, interpellations, etc) | ep_mep_activities.json.zst | 41.3 MiB | 2024-12-24 | schema | |
| MEP Plenary Amendments | This dump contains most of the plenary amendments | ep_plenary_amendments.json.zst | 6.5 MiB | 2024-12-24 | schema | |
| MEP Plenary Votes | This dump contains most of the plenary roll-call votes | ep_votes.json.zst | 10.0 MiB | 2024-12-21 | schema | |
| Committee Amendments | This dump contains most of the committee amendments | ep_amendments.json.zst | 105.2 MiB | 2024-12-24 | schema | |
| MEP Committee Votes | This dump contains most of the committee roll-call votes | ep_com_votes.json.zst | 56.2 KiB | 2024-12-24 | schema | |
| Committee Agendas | This dump contains most of the committee agendas | ep_comagendas.json.zst | 1.1 MiB | 2024-12-24 | schema |
[1] previously parltrack used xz, but due to claims xz being inadequate for long-term archiving we switched to lzip.