Research Metadata and PIT Checklist

Before a public research record enters the bibliographic index, establish when it was published, when the system discovered it, which version is visible, and whether the source permits the intended use. This page covers research metadata only, not market data or trading implementation.

Four metadata gates

1. Source identity

• Does the page come from a verifiable author, publisher, or official catalog?
• Is the original link stable, and is there a DOI, arXiv ID, or another public identifier?
• Are aggregator pages distinguished from original publication pages?

2. Time semantics

Keep these timestamps separate:

Time	Meaning
published_at	The publication time claimed by the source
first_discovered_at	When the system first knew about it
fetched_at	When the system retrieved a particular content version
updated_at	The update time claimed by the source, when available

Historical backfills must not present published_at as first_discovered_at. If a source has no reliable date, record it as unknown rather than inferring one.

3. Versions and relationships

• Do the preprint, working-paper, and formally published versions represent the same work?
• Do title, author, or abstract changes create a new content version?
• Are corrections, retractions, and replacement versions linked clearly?

4. Permission and minimal retention

• Store only bibliographic fields permitted by the public source.
• Do not copy reports, PDFs, page text, screenshots, charts, or restricted abstracts.
• For login, subscription, or specially licensed content, retain only the public entry point and access status.

Research-index mapping

The Research Index separates published_at, first_discovered_at, and fetched_at, and records content versions and external identifiers.

The public source directory records entry points, content types, and access boundaries. The Research Index publishes bibliographic facts and original-source links only.

Public record example

work_id:
title:
source_url:
published_at:
first_discovered_at:
fetched_at:
external_ids: []
access_status: