Callimachus is an open-source tool that checks the citations in a manuscript: whether a cited source can be identified and matches the reference, is formatted correctly, and actually supports the claim. It hands a language model the one call only a reader can make, weighing a claim against a source, and nothing else. The model never fetches a source, rewrites a claim, or names a verdict; the pipeline around it stays deterministic and on the record.
$ python run.py --input manuscript.pdf --accuracy standard
Sample findingAttention Is All You Need
The claim in the manuscript
“RNN sequence-to-sequence models have not been able to attain state-of-the-art results in small-data regimes[37].”
↓ cites [37] Vinyals et al., 2015, fetched and read in full
What that source actually says
“… when training on [the small training set] only, a baseline LSTM does not achieve any reasonable score, even with dropout and early stopping.”
The verdict
Partially supported
The source shows one model struggling on limited data, not that these models can’t. Its own attention variant scores competitively, so the broad claim isn’t established.
[ Latest ] Desktop release
Download Callimachus.
Available on Microsoft Store for Windows, with standalone packages for Windows, Linux and macOS.
Microsoft Store manages updates for Store installations according to your Store settings. Standalone packages are updated separately. For installation instructions and checksums, see the release notes.
Most checkers answer one question, whether the reference exists or is formatted right, and hand you a number. Callimachus refuses to collapse identity, form, and support into a single score. A reference can be real but misidentified, correctly formatted but irrelevant, or on-topic yet unsupported by the text. Those stay separate, and every answer keeps its evidence.
01
Identity
Can the cited work be identified, and does it match the reference?An unresolved search is not proof that a source does not exist.
02
Citation form
Does the bibliography entry pass the deterministic checks for its style?Style findings stay independent of identity and support.
03
Claim support
What does this source establish about this exact claim?Supported, partial, contradiction, related, off-topic, or non-decidable.
[ 02 ] How it works
The model proposes. The system controls the record.
A deterministic Python driver owns every phase: it reads the manuscript, checks identifiers and bibliographic records, and fetches available source text before any judgement happens. A language model runs in exactly one phase, Verify, on one claim and one source at a time. It cannot admit a source, bypass grounding, write a terminal state, or compose the report. When required proof or integrity is missing, the run fails closed instead of guessing.
1.0
Parse
Read the manuscript (DOCX, LaTeX, PDF, Markdown, plain text, or HTML) into claims and references. The claims are extracted here, so the model is never free to invent one.
Deterministic
2.0
Resolve
Check identifiers and bibliographic catalogues, then compare returned metadata with the citation. The result may identify the work, record no match after completed searches, or remain unresolved; an unresolved result does not prove nonexistence.
Deterministic
3.0
Fetch
Retrieve and normalise the source text; identity and content hashes are retained as evidence.
Deterministic
4.0
Verify
The one place a model runs. It answers a chain of yes/no questions about a single claim and source; code composes the verdict, then a second judge re-checks the evidence.
Guarded jury
5.0
Report
Compose the deterministic report and audit ledger from admitted results only.
Deterministic
parse → resolve → fetch → verify → report
[ THE DESKTOP WORKFLOW ]
See the audit as it unfolds.
Callimachus 1.1.0 shows source-by-source progress, asks for an explicit operator decision when review is needed, and keeps findings navigable in the final report. These current desktop previews use demonstration data.
01 / ANALYSE
Start and track a run
Choose a paper and verification mode, then follow each source through the pipeline.
02 / PARSE REVIEW
Resolve an ambiguous citation
Choose the matching reference or claim, then resume the analysis with the operator decision recorded.
03 / GUIDED FETCH
Handle a source that needs attention
Guided Fetch presents the source and its review state; the run continues only after the operator explicitly proceeds.
04 / REPORT
Follow the recorded finding
Open the report to inspect outcomes, sources, evidence and the audit trail. The dark preview uses the report's own high-contrast theme.
[ 03 ] Inside Verify
The model answers. The system decides.
This is the only step where a language model runs. It is handed one exact claim from the manuscript and the passages of the source Callimachus already fetched, and it is never asked to grade the citation in a single shot. It answers a chain of narrow yes/no questions, and deterministic code turns those answers into the verdict.
Q1 Does the source give usable support for what it is cited for?
yes → Q2no → Q3
Q2 Does it support the claim fully, at the same scope and certainty?
yes · Supportedno · Partial
Q3 Does the source instead show the opposite?
yes · Contradictionno → Q4
Q4 Is it in the same or an adjacent field at all?
yes · Relatedno · Off-topic
[ 04 ] Runs on any model
Model-agnostic by design.
Because the model does such a narrow job, weighing one claim against one source through bounded yes/no questions, you are not tied to a single provider. Point Callimachus at a hosted API, at a model on your own machine through Ollama or FreeToken, or at any OpenAI-compatible endpoint.
Anthropic
OpenAI
Gemini
Mistral
OpenRouter
GLM
FreeToken local
Ollama local
Any OpenAI-compatible endpoint
Small model? Add the RAG.
Large-context models read the whole source. For smaller local models, an optional dependency (requirements-rag.txt) enables extractive RAG: a deterministic in-document ranking (BM25) that feeds the model only the most relevant passages, within a fixed character budget. If that dependency is missing, Verify stops instead of silently truncating the evidence.
example · C7 / jury1_guard
[ 05 ] No silent recovery
When no admissible result survives, the report says so.
Here the Jury 1 proposal failed the verification guard. The pair stayed operationally complete but non-crediting: no plausible replacement verdict was invented to fill the gap. Missing evidence is recorded as missing, not converted into success.
Semantic outcomeNone
Resolutionjury1_guard
Creditingfalse
Evidence scopefulltext_complete
[ 06 ] What you get
The result is a record, not a paragraph.
Read the report first, then follow any finding down to the retained evidence, the source identity, the model's attempt, and the deterministic finalisation that produced it. The run ledger is an authoritative, auditable record, with a report history and integrity seal.
report.htmlSelf-contained, reader-first companion
report.mdDeterministic final report
run.sqliteAuthoritative audit ledger
sources/Normalised evidence with identity and hashes
journal + sealReport history and signature status
On the name & the mark
Callimachus of Cyrene compiled the Pínakes at the Library of Alexandria, the catalogue that first answered which scroll was truly which. This tool asks the same of every source a manuscript cites, and keeps the evidence attached to the answer.
The mark writes it as [C]: at once a citation reference and a harness: the brackets that let the model weigh a claim but never let it write the record.
[ C ] Inspect before you trust
Citation verification with the evidence still attached.
