Zotero: group-wide export, CLI export, and per-collection routing - #3
Merged
Conversation
…onfig-less mode Closes #2. Implements the "one command from Zotero to refs.bib" flow for grant-proposal and paper workflows: - ZoteroClient.export_bibtex() now takes collection_key: str | None and format: {bibtex, csljson, ris}. Talks to api.zotero.org directly via urllib to sidestep pyzotero's BibDatabase-for-bibtex return type and to read the Total-Results header for pagination (Zotero caps bibtex/ris responses at 100 items). - `docflow zotero export [--collection] [--format] [--output]` CLI wrapper; stdout by default, respects .docflow/config.yaml's zotero.collection_key with `--collection ""` as the whole-group escape hatch. - `docflow zotero add-dois [FILE] [--collection] [--dry-run]` — one DOI per line from FILE or stdin, `#` comments/blank lines skipped, bioRxiv (10.1101/*) DOIs classified as BIORXIV so version-suffix normalisation applies on dedup. - Per-collection routing on `push`: input JSON may carry a per-citation `collection` field (highest precedence) or `tags`, plus a `--rules RULES.yaml` for tag/doi_prefix matchers with default_collection fallback. Precedence: per-citation > matching rule > rules default > --collection flag > None. - Push loop assigns the destination collection at create_items time (item["collections"] = [...]), not via addto_collection — the latter would need the server-assigned key/version/data.collections fields. - `push` and `add-dois` exit non-zero when any citation failed to push, so downstream Makefiles/CI can detect partial failure. - Config-less operation: all `docflow zotero` subcommands run without a .docflow/config.yaml when ZOTERO_API_KEY is set. If ZOTERO_GROUP_ID isn't set, the library is auto-discovered via /keys/current — used directly when there's exactly one accessible library, or via per-group collection probing when a --collection hint disambiguates across multiple groups. - Makefile template gets refs.bib target (guarded by ZOTERO_API_KEY), a `bib` phony alias, and `bib` in the `all:` recipe. Spec at .specify/specs/zotero-export-and-routing.md captures design, precedence rules, config-less mode, push mechanics, and exit-code invariants. 73 new/updated tests, all AI-generated marked. Co-Authored-By: Claude Code 2.1.210 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follows #2. Adds a design proposal (no code changes) for splitting a Zotero library into one .bib per top-level collection. Motivation: grant proposals frequently cite from multiple curated subcollections (e.g. BIDS group has "BIDS and BEPs", "Related", "biosketch-refs" — https://www.zotero.org/groups/5111637/bids/library). Users want a matching per-section .bib without hand-mapping collection names to keys and re-invoking `export --collection` for each. Proposed CLI additions: - `docflow zotero export-collections [--parent] [--output-dir] [--format] [--recursive/--no-recursive] [--naming slug|preserve|key] [--include/--exclude GLOB] [--skip-empty] [--manifest] [--dry-run]` - `docflow zotero list-collections [--parent] [--depth] [--format]` Design highlights (see .specify/specs/zotero-export-per-collection.md): - Orthogonal `--depth` (how many .bib files) vs `--recursive` (how much each contains) — Zotero collections don't own items, so both dials matter. - Filename schemes: slug (default), preserve, key; collisions resolved deterministically by appending -<KEY>. - Manifest (bibs/index.yaml) as single source of truth for downstream tooling — records library_version stamps for a future --only-changed. - Descendant fetches use one API round-trip per output unit via Zotero's comma-separated collectionKey filter (URL-length fallback: per-collection with client-side dedup). - Makefile: adds `bibs` and `bibs-list` phony targets, env-var-guarded like refs.bib. NOT in `all:` by default (round-trip scaling). Open questions flagged inline for review: item-in-multiple-collections dedup, list-collections vs. --dry-run overlap, --only-changed staleness. Co-Authored-By: Claude Code 2.1.210 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements the v1 subset of .specify/specs/zotero-export-per-collection.md.
Enables the "one .bib per Zotero (sub)collection" workflow: users can
regenerate section-scoped bibliographies with a single `make bibs`.
New:
- ZoteroClient.list_collections(parent_key=None): assembles pyzotero's
flat all_collections() response into a CollectionNode tree.
- ZoteroClient.export_collection_tree(output_dir, parent_key, format,
naming, recursive, skip_empty, manifest_path): writes one file per
top-level collection under parent_key (or root), plus a YAML manifest
mapping collection keys → names → filenames → item counts. When
recursive, OR-unions descendant keys via Zotero's collectionKey filter
(comma-separated); URL-length fallback to per-collection fetches with
client-side dedup for very wide subtrees.
- slugify_name(name, mode, key): 'slug' (Unicode NFKD, lowercase,
non-alnum → '-'), 'preserve' (case kept, path-unsafe chars → '_'),
'key' (raw collection key). Deterministic collision resolution via
`-<KEY>` suffix.
- CLI:
* `docflow zotero list-collections [--parent] [--format text|json|yaml]`
* `docflow zotero export-collections [--parent] [--output-dir]
[--format bibtex|csljson|ris] [--naming slug|preserve|key]
[--recursive/--no-recursive] [--skip-empty/--keep-empty] [--manifest]`
- Makefile template: `bibs` and `bibs-list` targets, env-var-guarded
like refs.bib. Deliberately NOT in `all:` (round-trip scaling with
collection count).
Spec updated with a v1 scope table listing what shipped vs. what's
deferred (--include/--exclude, --parent-name, --depth>1, --dry-run,
--only-changed, --prune, --sleep-between, zotero.bibs config block).
21 new tests, 226 total pass. ruff clean; mypy clean.
Co-Authored-By: Claude Code 2.1.210 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.