Modernize postprocess#1
Draft
yangyzs wants to merge 113 commits into
Draft
Conversation
…e test classes during sample extraction for README.md
…ent newline stripping
…c and fix linter/compilation issues
… configs strictly
…e 6313 (googleapis#6521) The temporary build-step workarounds and associated TODO comments for the symlink extraction issue are removed from the Node.js installer tool configuration and its unit tests. Since native symlink extraction is now supported by the fetch package, the manual symlink creation steps in the generator build configuration are no longer necessary, and the test mock setup is simplified. Fixes googleapis#6313
…mplate (googleapis#6519) The product documentation link in the README template is corrected by removing the redundant trailing "/overview" path suffix. For googleapis#6442
…gleapis#6520) The new pure Go-based README generator is fully integrated into the Node.js post processing generation pipeline, completely replacing the old external script-based generator. The generator now creates the package `README.md` from the embedded Go text template during the post-processing phase, unless the `README.md` is explicitly listed in the library's keep configuration to preserve hand-written versions. This feature will not bring changes to `README.md` in google-cloud-node since `README.md` is in the default keep list. We will gradually rollout this feature after the partial migration is done. For googleapis#6442
…ependency (googleapis#6503) Include the `google-cloud-lro` dependency in a generated crate's `Cargo.toml` if any of its methods are configured as LRO pollers (`IsLroPoller`).
…s#6522) Sidekick needs to generate a custom `encode()` and a custom initializer for messages when any field is url-safe encoded.
In discovery-based services the requests will be nested messages. We need to generate code that fully qualifies the request type, like we do for fields. This requires new annotations for the messages.
🤖 I have created a release *beep* *boop* --- ## [0.22.0](googleapis/librarian@v0.21.0...v0.22.0) (2026-06-22) ### Features * **internal/librarian/java:** remove redundant keep items in librarian.yaml ([googleapis#6291](googleapis#6291)) ([2965478](googleapis@2965478)) * **internal/librarian/java:** support alternate_headers for monolithc libraries ([googleapis#6481](googleapis#6481)) ([4165a09](googleapis@4165a09)) * **internal/librarian/nodejs:** add release level markdown generation ([googleapis#6476](googleapis#6476)) ([1d1281f](googleapis@1d1281f)) * **internal/librarian/nodejs:** add support for readme partials ([googleapis#6505](googleapis#6505)) ([eca8e3d](googleapis@eca8e3d)), closes [googleapis#6442](googleapis#6442) * **internal/librarian/nodejs:** extract sample metadata for node readme ([googleapis#6454](googleapis#6454)) ([00e5e0d](googleapis@00e5e0d)), closes [googleapis#6442](googleapis#6442) * **internal/librarian/nodejs:** generate README in Node library ([googleapis#6520](googleapis#6520)) ([68c0a20](googleapis@68c0a20)) * **internal/librarian/nodejs:** implement README generation without partials ([googleapis#6488](googleapis#6488)) ([44e6954](googleapis@44e6954)) * **internal/postprocessing:** implement Java method deprecation ([googleapis#6497](googleapis#6497)) ([289a385](googleapis@289a385)), closes [googleapis#6298](googleapis#6298) * **internal/postprocessing:** implement Java method duplication ([googleapis#6484](googleapis#6484)) ([0c5959c](googleapis@0c5959c)), closes [googleapis#6298](googleapis#6298) * **nodejs:** support per-API version mixin configuration([googleapis#6462](googleapis#6462)) ([71cd24e](googleapis@71cd24e)) * **postprocessing:** implement Java method deletion ([googleapis#6436](googleapis#6436)) ([820646f](googleapis@820646f)), closes [googleapis#6298](googleapis#6298) * **sidekick/rust:** add condition to include `google-cloud-lro` as dependency ([googleapis#6503](googleapis#6503)) ([7a89172](googleapis@7a89172)) * **sidekick/rust:** bigquery query metadata ([googleapis#6407](googleapis#6407)) ([6989ebc](googleapis@6989ebc)) * **sidekick/swift:** `bytes` for discovery docs ([googleapis#6433](googleapis#6433)) ([d20f64c](googleapis@d20f64c)) * **sidekick/swift:** generate method signature overloads ([googleapis#6473](googleapis#6473)) ([27a72be](googleapis@27a72be)) * **sidekick/swift:** qualified names for requests ([googleapis#6506](googleapis#6506)) ([9489715](googleapis@9489715)) * **sidekick:** parse method signatures ([googleapis#6451](googleapis#6451)) ([7a433e7](googleapis@7a433e7)) * **sidekick:** parse method signatures ([googleapis#6461](googleapis#6461)) ([16aa2e6](googleapis@16aa2e6)) ### Bug Fixes * **internal/librarian/nodejs:** correct product doc link in readme template ([googleapis#6519](googleapis#6519)) ([9cd8ee9](googleapis@9cd8ee9)), closes [googleapis#6442](googleapis#6442) * **internal/librarian/nodejs:** path leak during generte_readme ([googleapis#6470](googleapis#6470)) ([d3e7c16](googleapis@d3e7c16)) * **internal/postprocessing:** support deleting multiple methods and extract boundary finder ([googleapis#6471](googleapis#6471)) ([20442d8](googleapis@20442d8)), closes [googleapis#6298](googleapis#6298) * **librarian:** print errors on failure ([googleapis#6458](googleapis#6458)) ([37e4f91](googleapis@37e4f91)) * **sidekick/rust:** disable docs/clippy warning for BQ generated files ([googleapis#6498](googleapis#6498)) ([0a6a4d8](googleapis@0a6a4d8)) * **sidekick/rust:** use struct initializer for QueryMetadata ([googleapis#6504](googleapis#6504)) ([2bdb3b5](googleapis@2bdb3b5)) * **sidekick/swift:** UrlSafe requires custom serialization ([googleapis#6522](googleapis#6522)) ([09c74f6](googleapis@09c74f6)) * **surfer:** print errors on failure ([googleapis#6465](googleapis#6465)) ([d91bf4c](googleapis@d91bf4c)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Joe Wang <106995533+JoeWang1127@users.noreply.github.com>
… layout (googleapis#6494) This PR bumps `pnpm` version to `11.7.0` and `Node.js` to `22` in the Dockerfile, and updates the Node.js tool installer to support `pnpm` v11+ global bin directory layout. We assume `pnpm` v11 is used. ### Changes * **Dockerfile (pnpm & Node bump):** Bumps `NODEJS_PNPM_VERSION` to `11.7.0` and `NODEJS_NODE_VERSION` to `22` in `cmd/librarian/Dockerfile`. Node 22 is required for `pnpm` v11. * **Dockerfile (Build Fix):** Sets `PNPM_CONFIG_DANGEROUSLY_ALLOW_ALL_BUILDS=true` during the Docker image build (`librarian install nodejs`) to bypass interactive build approval prompts for dependencies (fixing `[ERR_PNPM_IGNORED_BUILDS]`). * *Reference:* Starting with pnpm v10, build scripts are blocked by default for security. See the [pnpm v10.0.0 release notes](https://github.com/pnpm/pnpm/releases/tag/v10.0.0) for details on why this is needed. * **PNPM_HOME Adjustment:** Updates `getPNPMEnv` in `internal/librarian/nodejs/install.go` to use the parent directory of Node bin directory as `PNPM_HOME` (`filepath.Dir(globalBin)`). This ensures `pnpm` (which installs to `PNPM_HOME/bin` in v11+) places the binaries back in the Node bin directory, which is in `PATH`. ### Scope Notes * The use of `LIBRARIAN_INSTALL_DIR` is outside the scope of this pull request. This pull request does not make it harder to use `LIBRARIAN_INSTALL_DIR`. Fixes googleapis#6480
For discovery-based APIs the generator produces structs corresponding to the synthetic request messages. To avoid name clashes,these structs are generated in extensions to the client struct.
…ration (googleapis#6538) This utility splits CamelCase Java class names into space-separated words to generate human-readable titles for the README code samples table. For googleapis#6515
Filenames must be unique on case-insensitive filesystems. Swift also requires the basename to be unique, so I removed the `Clients/` intermediate directory because it was solving the problem of filename clashes, but not Swift source clashes.
…is#6546) Adds 'extractTitle' to extract 'sample-metadata:' titles. It returns an error if the title is missing or empty. For googleapis#6515
…ogleapis#6548) The version of gapic-generator in the embedded librarian.yaml (used by librarian install) is updated to v1.36.0. This adds Python 3.15 pre-release testing to generated libraries to prepare for the [Python 3.15 release in October](https://peps.python.org/pep-0790/#schedule). Signed-off-by: Anthonios Partheniou <partheniou@google.com>
go/github-zizmor-help The 1st commit was an automated fix (`--fix`). The 2nd commit was a manual fix. ``` suztomo@suztomo:~/librarian-2026/librarian$ zizmor .github/workflows INFO zizmor: 🌈 zizmor v1.25.2 INFO audit: zizmor: 🌈 completed .github/workflows/ci.yaml INFO audit: zizmor: 🌈 completed .github/workflows/create-issue-on-failure.yaml INFO audit: zizmor: 🌈 completed .github/workflows/dart.yaml INFO audit: zizmor: 🌈 completed .github/workflows/go.yaml INFO audit: zizmor: 🌈 completed .github/workflows/java.yaml INFO audit: zizmor: 🌈 completed .github/workflows/linter.yaml INFO audit: zizmor: 🌈 completed .github/workflows/multi_approvers.yaml INFO audit: zizmor: 🌈 completed .github/workflows/nodejs.yaml INFO audit: zizmor: 🌈 completed .github/workflows/python.yaml INFO audit: zizmor: 🌈 completed .github/workflows/sidekick.yaml No findings to report. Good job! (1 ignored, 43 suppressed) ```
…oogleapis#6552) Update generate commit type to `feat`. Fixes googleapis#6529
…l-schema (googleapis#6551) Rename `doc/api-allowlist-schema.md` to `doc/sdk-yaml-schema.md`. Fixes googleapis#6540
The legacylibrarian part of this project is being turned down. This change deletes all associated code, including legacy automation, CLI commands, config checkers, and infrastructure build configurations. Internal Bug: b/518756763
) The `node` and `pnpm` version are updated in googleapis#6494, we should update these versions in workflow.
) Update nodejs generator to [v4.12.1](https://github.com/googleapis/google-cloud-node/releases/tag/gapic-generator-v4.12.1).
…oogleapis#6574) Add helper functions collectSampleFiles and parseCodeSample to scan samples/src/main/java and extract sample metadata. Add codeSample struct and unit tests. For googleapis#6515
Remove legacylibrarian from workflow.
…lts (googleapis#6571) The applyDefaults function is updated to safely handle a nil defaults parameter to avoid a runtime nil pointer panic when attempting to access defaults.Output. A test case is added to verify this safety.
🤖 I have created a release *beep* *boop* --- ## [0.23.0](googleapis/librarian@v0.22.0...v0.23.0) (2026-06-29) ### Features * **internal/librarian/java:** add decamelize utility for README generation ([googleapis#6538](googleapis#6538)) ([a6d950a](googleapis@a6d950a)), closes [googleapis#6515](googleapis#6515) * **internal/librarian/java:** Add JSpecify dep to proto module template ([googleapis#6564](googleapis#6564)) ([660ab2b](googleapis@660ab2b)) * **internal/librarian/java:** add production sample filter ([googleapis#6545](googleapis#6545)) ([df213ec](googleapis@df213ec)), closes [googleapis#6515](googleapis#6515) * **internal/librarian/java:** add README sample extraction helpers ([googleapis#6574](googleapis#6574)) ([500ba22](googleapis@500ba22)), closes [googleapis#6515](googleapis#6515) * **internal/librarian/java:** add title override extractor ([googleapis#6546](googleapis#6546)) ([0986d6a](googleapis@0986d6a)), closes [googleapis#6515](googleapis#6515) * **internal/librarian/python:** update gapic-generator to 1.36.0 ([googleapis#6548](googleapis#6548)) ([a0930f4](googleapis@a0930f4)) * **nodejs:** bump pnpm version to 11.7.0 and support v11+ global bin layout ([googleapis#6494](googleapis#6494)) ([1373628](googleapis@1373628)), closes [googleapis#6480](googleapis#6480) * **sidekick/rust:** track recording error info for discovery LROs ([googleapis#6304](googleapis#6304)) ([feb2ef2](googleapis@feb2ef2)), closes [googleapis#6286](googleapis#6286) * **sidekick/swift:** generate synthetic messages ([googleapis#6530](googleapis#6530)) ([7303648](googleapis@7303648)) * **sidekick/swift:** handle clashing names ([googleapis#6543](googleapis#6543)) ([56bf365](googleapis@56bf365)) * **sidekick/swift:** swap client vs. protocol ([googleapis#6566](googleapis#6566)) ([14e186c](googleapis@14e186c)) ### Bug Fixes * **internal/librarian:** gracefully handle nil defaults in applyDefaults ([googleapis#6571](googleapis#6571)) ([3ee938a](googleapis@3ee938a)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
… librarian.yaml" (googleapis#6512) Reverts googleapis#6291 due to regression in which tidy removes too many files, many of which are necessary. Fixes googleapis#6510
…on (googleapis#6578) Add extractSamples main function to locate Java sample files and return parsed codeSample structs. Add unit tests in readme_test.go. For googleapis#6515
…6588) For discovery-based services, most methods get a method signature with all the path parameters. This does not work for methods without any path parameters. Such methods typically require one or more query parameters, and there is no way to tell which ones are required.
I got tired of refactoring code only to be told I was not following the guidelines in `howwewritego.md`. Change all the `t.Errorf(, diff)` to use the prescribed format.
…etadata (googleapis#6582) Remove the Java-specific mapping in RepoMetadataTransport that translated transport values to "http" or "both". Now it returns the standard values ("grpc", "rest", "grpc+rest") directly, aligning Java with other languages. After merging, we will merge googleapis/google-cloud-java#13586, once we update the librarian pseudo-version and regenerate the repo-metadata values. Fixes googleapis#4854
…leapis#6576) A new `debug` command with an `env` subcommand is added to the CLI to print the librarian environment. This includes resolved paths for `LIBRARIAN_CACHE`, `LIBRARIAN_BIN`, and language-specific tool installation directories. To support accessing the tool installation directories from the new command, the internal `getInstallDir` functions in the golang and java packages are exported as `InstallDir`. __Local Testing__ run `go run ./cmd/librarian debug env`, result: ``` LIBRARIAN_CACHE=/usr/local/google/home/hongdaj/.cache/librarian LIBRARIAN_BIN=/usr/local/google/home/hongdaj/.cache/librarian/bin Language-specific tool installation directories: golang: /usr/local/google/home/hongdaj/.cache/librarian/bin/go_tools java: /usr/local/google/home/hongdaj/.cache/librarian/bin/java_tools ``` Fixes googleapis#6374
…apis#6569) We had to manally add Release Please entry in the configuration files when we onboard a new library to google-cloud-node. googleapis/google-cloud-node#8693 is the example. Let's have Librarian to add the entries in the Release Please files. Python and Go already have the feature with the "bulk" files. Let's make the logic work for google-cloud-node's configuration files. They use the default names: release-please-config.json and .release-please-manifest.json. Note that when Librarian starts to touch the Release Please files, it sorts the JSON keys. googleapis/google-cloud-node#8777 is the outcome of an example invocation for the agentregistry package using the source tree before the package was introduced. The JSON keys are sorted. (The irrelevant changes in the mixin fields are due to recent change in Librarian for NodeJS.) Fixes googleapis#6528
…r README rendering (googleapis#6593) Add collectSnippetFiles and extractSnippetsFromFile helpers to scan Java and XML sample files for code snippets. Add unit tests in readme_test.go. For googleapis#6515
…bom when generating gapic-libraries-bom/pom.xml (googleapis#6601) Exclude google-cloud-bom and libraries-bom when generating gapic-libraries-bom/pom.xml. This is needed because googleapis/google-cloud-java#13498 added new modules, among them are these two BOMs. Fixes googleapis#6600 For googleapis/google-cloud-java#13609
…pretty_override support (googleapis#6603) This pull request adds metadata_name_override and name_pretty_override configuration fields to NodejsPackage in librarian.yaml . Currently, Librarian infers name and name_pretty in `.repo-metadata.json` directly from the backend service config ( api.ShortName and api.Title ). When multiple packages share a single service authority (e.g., @google-cloud/dialogflow and @google-cloud/dialogflow-cx both generating from dialogflow.googleapis.com ), Librarian overwrites both `.repo-metadata.json` files with identical values ( "dialogflow" / "Dialogflow" ), causing config collisions. Adding these override fields aligns Node.js with Java ( name_pretty_override ) and Python ( metadata_name_override ), allowing packages with shared service authorities to explicitly preserve their unique metadata names. For googleapis#6453
Use the discovery configuration to generate `getOperation()` mixins in discovery-based APIs.
…om JavaModule (googleapis#6584) Finalizes the migration to JavaDefault by removing the deprecated libraries_bom_version field from the JavaModule struct, its merging logic, and its library-level fallback check in the generator. This is backwards compatibile because it is only defined once at the global default.java level in librarian.yaml in google-cloud-java. Follow up to googleapis#6447. Fixes googleapis#5171
…ts (googleapis#6554) The library configuration loader is updated to automatically populate Java Maven coordinates (GroupID and ArtifactID) based on API path prefix mappings defined in default configuration. When a library's Java module configuration is missing or its GroupID is empty, the loader scans the library's API paths against the default api_path_to_group_id map to resolve the appropriate GroupID and sets the ArtifactID using the standard "google-<library>" naming convention. Add default mappings in follow up PRs. For googleapis#6513
🤖 I have created a release *beep* *boop* --- ## [0.24.0](googleapis/librarian@v0.23.0...v0.24.0) (2026-07-01) ### Features * **add:** handle Release Please config for google-cloud-node ([googleapis#6569](googleapis#6569)) ([1f8ee00](googleapis@1f8ee00)) * **internal/librarian/java:** add code snippet extraction helpers for README rendering ([googleapis#6593](googleapis#6593)) ([96f6925](googleapis@96f6925)), closes [googleapis#6515](googleapis#6515) * **internal/librarian/java:** add extractSamples for README generation ([googleapis#6578](googleapis#6578)) ([b5e3d45](googleapis@b5e3d45)), closes [googleapis#6515](googleapis#6515) * **internal/librarian/nodejs:** add metadata_name_override and name_pretty_override support ([googleapis#6603](googleapis#6603)) ([3f6cfed](googleapis@3f6cfed)), closes [googleapis#6453](googleapis#6453) * **internal/librarian:** add debug command with env subcommand ([googleapis#6576](googleapis#6576)) ([027103b](googleapis@027103b)), closes [googleapis#6374](googleapis#6374) * **internal/librarian:** populate Java Maven coordinates from defaults ([googleapis#6554](googleapis#6554)) ([accb8ad](googleapis@accb8ad)), closes [googleapis#6513](googleapis#6513) * **librarian/swift:** use discovery config ([googleapis#6604](googleapis#6604)) ([5a44ed7](googleapis@5a44ed7)) * **sidekick/discovery:** signatures without path params ([googleapis#6588](googleapis#6588)) ([bb40e83](googleapis@bb40e83)) ### Bug Fixes * **internal/librarian/java:** exclude google-cloud-bom and libraries-bom when generating gapic-libraries-bom/pom.xml ([googleapis#6601](googleapis#6601)) ([b8e50a5](googleapis@b8e50a5)) * **internal/serviceconfig:** normalize transport name for Java repo-metadata ([googleapis#6582](googleapis#6582)) ([e20f77a](googleapis@e20f77a)) * **sdk.yaml:** allow rust for many non-cloud apis ([googleapis#6598](googleapis#6598)) ([0efb6e7](googleapis@0efb6e7)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.