How to recognise it
Every automated request from N84 sends this user agent:
N84-Archive/1.0 (+https://n84.dev/archive-bot; contact: data@n84.dev)
How it behaves
- It reads robots.txt first, keeps a copy for 24 hours, and does not fetch a path robots.txt disallows. The only exceptions are hosts whose publisher's own documentation invites programmatic access under an open licence; there it follows that documentation's rate and route.
- It checks each file at most once a day, and usually downloads it only when it has changed. Most files change between twice a week and once a month.
- Requests to one host are made one at a time and spaced: at least one second apart, five seconds for archive.org and API portals, ten seconds where a site asks for it (Crawl-delay).
- It never logs in, never submits forms and never uses a site's search.
- Files are stored unchanged, except that personal data (names, addresses, postcodes, meter numbers) in embedded capacity registers is removed before anything is stored.
What we publish
Dated comparisons of what each file printed, with the publisher's attribution and licence. The publisher's own file always prevails. We do not imply that any publisher endorses N84.
Contact, or ask us to stop
Email data@n84.dev. If you run a site the bot visits and want it to slow down or stop, say so and name the host: we stop fetching that host within one working day and confirm by email. You can also disallow the user agent N84-Archive in your robots.txt; the bot sees the change within 24 hours.