# Documents, observations and revisions

The current public contract and the boundaries of retained history.

## Current document contract
The documents dataset retains one record per source URL. id is the public record identifier. title can be translated; original_title preserves the retained original. title_lang and is_translated tell you how the title is presented. source and source_url provide provenance. published_at can be unknown; collected_at and first_seen are first collection and last_seen the latest. publisher and acquisition_route (registry_listing, rss_feed, news_proxy, api) say who published it and how we reached it; translated_title is null when the original is English.

## Deduplication
The public coverage contract says one retained record per source URL. It does not promise semantic deduplication across mirrors or publishers. Two URLs can describe the same event; repeated retrieval of one ID does not imply a new publication.

## Original text
text_status distinguishes available, source_only and not_extracted. Approved detail can contain original_text, text_truncated and rights_basis_url. It is an original-language extraction, not a promised full translation. Rights approval is bound to the current extraction; later changes can withhold it.

## Observations and revisions
Every collection of a source URL is an observation: observation_count on the list row, observations[] (observed_at, acquisition_route; capped, with observations_total) on the detail. revisions[] lists retained content versions by version_id, content_hash and observed_at, and version_count counts them; the private body of each version is never served. change_type is revised when a different content hash was observed after first_seen, else newly_discovered when the publisher date is more than two days before first_seen (Beijing date), else new. Rolling out: the change feed, the document observation fields and revisions[] answer once database migration 043 is applied on a deployment. Before that, /api/changes and mode=changes answer HTTP 503 with code unavailable ("not yet enabled"), and list rows return the pending fields as null, named in unavailable_fields.

Coming soon: the text of each historical version. [Indicator releases](/docs/data/indicators) and editorial topic versions are different objects.

## Fields
| Field | Meaning |
| --- | --- |
| id | Record ID |
| title | Title |
| title_lang | Title language |
| is_translated | Machine translation flag |
| source | Publisher |
| source_url | Original source |
| original_title | Original title |
| published_at | Publication date |
| collected_at | First collected |
| language | Original language |
| topic | Topic |
| text_status | Text availability |
| publisher | Publisher |
| acquisition_route | Acquisition route |
| translated_title | Translated title |
| first_seen | First seen |
| last_seen | Last seen |
| observation_count | Observations |
| change_type | Change type |
| version_count | Retained versions |
| last_revised_at | Last changed body observed |
| content_hash | Hash of the current extracted text |